TANGLEWOOD MEMO TO MARILYN ANCHLEY AND DONALD PENCHIALA FROM

Tanglewood

Memo

To: Marilyn Anchley and Donald Penchiala

From: Jordan Spieker, Shane Grove, Natalie Busse, Jason Farris

CC: Jon Werner

Date: 21-Dec-21

Re: Validation Procedures

After reviewing the data given from Tanglewood we came to many conclusions about the current predictors Tanglewood is using and have been experimenting with many proposed methods. Tanglewood currently uses traditional predictors in the fields of work experience, education and interview score to predict whether or not a candidate can excel in citizenship, absence, performance and promotion potential. Out of the data given we noted that education has the best correlation score with promotion potential at .14 and a p-value < .01, however, education is seen as statistically significant but not practically significant. Work experience has a .16 correlation to performance and a .18 correlation to promotion potential with a p-value <.01 for both. This indicates that work experience is statistically significant for both performance and promotion potential. Interview score has a correlation of .16 and a p-value of <.01 for promotion potential which tells us that it is statistically significant when based on promotion potential. However, it is the worst statistical predictor of performance with a low correlation of .01 and a high p-value <.26. None of our comparisons from the traditional method are going to be practically significant since none of the correlations are moderate or high. Based on these numbers we can note that all three predictors are proven to predict a future employee’s promotion potential and that work experience is the only predictor to predict performance. Citizenship, absence and performance are not statistically significant when predicted by the rest of the selection tools.

We have been experimenting with proposed methods in the Seattle area for the past year to compare our scores with the traditional methods previously discussed. Out of these proposed methods the Marshfield Customer Service Biodata Questionnaire and Essay turned out to be the best predictor for citizenship with a correlation score .22 and p-value <.01. The personality tests which measure conscientiousness has good correlation scores with citizenship and absence with scores of .18 and -.33 respectively and both have a p-value <.01. This makes the scores for conscientiousness and absence statistically and practically significant. Performance and the applicant exam were closely related with a correlation score .34 and p-value <.01 also making it statistically and practically significant. Based on the proposed data given we can see that biodata and conscientiousness have better correlations for citizenship and absence than the traditional methods used. The proposed methods also raised the scores of education, work experience and interview score for promotion potential. Even though biodata proved to be the most statistically significant we don’t believe it is a very practical tool due to cost and other factors. Some applicants may not have the experience due to their age or other limitations and they also may find the questions difficult to answer. The personality traits, consciousness and extraversion, are believed to be the most relevant for the position of a retail clerk. This will be beneficial for Tanglewood since they are currently struggling with the performance of their retail clerks.

When reviewing the validity of each scale, we have come to the following conclusions: education seems to be one of the best predictors when measuring job performance and promotion potential. However, it does not measure citizenship and absence well. We suggest looking at the potential employees’ attendance record in their last school and analyze how many group projects they were involved in while during school. Education is important to employers because it can see how an employee will manage their job responsibilities and if they can problem solve and answer other kinds of questions that are important to the organization.

Work experience measures citizenship, job performance, and promotional potential very well. The only scale it did not measure well on was absence. Thus, we suggest looking at their sick days on their previous employment records or call and ask their previous employer to compare how many days they were unexcused to how many days they were excused from work. Work experience is especially important to a new employer because they would like to see what their employment history was like and if the employee has a great record or a bad record could depend if they receive the job or not.

Interview score scored high on citizenship and very high on promotional potential. To measure absence and job performance better, we would suggest asking questions pertaining to those topics such as asking their previous absence record, reasons they think is acceptable to leave work, what rating would they give themselves on their job performance at their last job, and other similar questions. Like the score indicated, the interview can tell the employer how well they will fit in with their organization because they can get a feel of the potential employees’ personality and can ask them questions about themselves.

Retail knowledge measured well on job performance and promotional potential. Retail knowledge is important to know when applying for a sales position at the company but obviously does not measure important factors on the job such as citizenship and absence. To make retail knowledge a better predictor, we suggest that questions be asked about how important it is to be to work everyday and how important it is to get along with coworkers to get a better feel for the potential employee.

Biodata scored very well on citizenship, absence, and job performance. Although it is a good indicator of three of the four scales, the worry that we have with this test is the cost and the time. Applicants are asked to fill out essays about their life experience which are difficult to answer and to score. Also, some applicants may not have a lot of life experiences due to their age or the environment they grew up in. It would be good, however, in a sense of getting to know someone and seeing how they would handle certain situations.

Applicant exam scored well on job performance and potential promotion. We would suggest asking questions on the retail knowledge exam like ‘how important it is to be to work everyday’ and ‘how important it is to get along with coworkers’ to better assess how they feel about absences and how they feel about getting along with co-workers. The applicant exam would be good to see how a potential employee will handle problem situations, but it takes a lot of time to go through each applicant.

Conscientiousness is a great indicator for all four of the scales. We do not have many suggestions to improve this predictor due to its significance in all four scales. This is about the potential employees’ personality, which has very little to do with the job.

Extraversion seemed to be one of the worst predictors. We suggest removing this category all together. This is also about the potential employees’ personality, which will not tell the employer about how well they will do on the job or any other scale factor. Also, both of these tests were not made by a professional so it could just be asking the wrong questions but it was poor indicator on all scales.

Going through the data provided to us, we believe we have successfully narrowed-down the best predictors which Tanglewood should utilize when determining good candidates to three: biodata (i.e. questions about an individual's significant life and work experiences, which could potentially be associated with his/her job performance), conscientiousness (to be guided by or in accordance with the dictates of conscience), and extraversion (the act of directing one’s interest outward or to things outside the self, which also goes along with conscientiousness in this case). Our reason for choosing these specific three predictors over the others offered was simply due to the results, which were shown in the data we were presented with. To be a bit more specific, we noted how the p-value scores for each of the three chosen predictors showed great reliability all across the board for each of the given categories (citizenship, absence, performance, and promotion potential).

The problem with using these three predictors together is that they would not necessarily reveal any sort of actual work-related information (examples of ones that would show this information include work experience and applicant exam). As such, you could consider removing extraversion from the three chosen predictors, and replace it with applicant exam. Our reasoning for choosing this particular predictor over the others as the replacement is that while applicant exam doesn't have the best p-value scores in regards to citizenship and absence, it still maintains great p-value scores for performance and promotion potential. As such, when paired-up with conscientiousness and biodata (which both have great scores in all four categories), the low scores found with applicant exam won't matter, as the other two predictors will basically “cancel them out” thanks to the high scores of citizenship and absence found in those two predictors .

Finally, based on the description of the experimental validation procedure, we do not believe that the observed validity estimates will generalize to other stores. This is because validity is situation specific. Also, it has been indicated that data provided wasn’t complete so out of the 25,000 employees, there is only [incomplete] data for 12,510. Your traditional method for collecting validation evidence yields different results from your experimental procedure because there is much subjectivity in regards to your managers’ rating promotion potential for your employees. Also, the data from the 832 employees from the experimental validation procedure isn’t even comparable to the 12,510 employees from the traditional method.

We, therefore, find that your proposed method of validation is more accurate because there are more predictors, most of which have strong statistical significance, to make better hiring decisions. However, it is important to keep in mind a few factors into our reasoning of why it will not generalize to your other stores. One factor is the different levels of subjectivity between the interviewers your organization provides. Similarly, as previously stated, the level of subjectivity in the managers’ assessment of promotion potential plays a factor and thus, cannot be generalized. Two other factors are region specific: the quality of weather in the Seattle region and the economy of the various regions. If, for example, Seattle experienced high accumulations of rain throughout the past year, the level of absence would be higher than those absence levels in your other stores, thus, making it difficult to ever accurately predict absenteeism. The economy of the different regions also comes into effect because the performance of the employees will vary based on customer/traffic flow.

We, therefore, suggest implementation of your experimental validation procedure in the future.

Tags: anchley and, marilyn, anchley, tanglewood, donald, penchiala