By Dr. Cynthia A. Hedricks, Chief Analytics Officer, SkillSurvey
Virtually every best practice guide or blog with a focus on reference-checking and job interviewing includes the advice that you should ask a job candidate about their areas of strengths and the areas where they could improve. These have become commonly asked questions because it is always interesting to learn how well a candidate knows himself or herself and whether these perceptions align with what others say about him or her in their reference check.
Strangely enough, there haven't been many studies done on the content provided by references or on the impact of this information on hiring decisions and subsequent success on the job. SkillSurvey has decided to fill that knowledge gap. At SkillSurvey, we believe references’ comments can be among the most informative and critical parts of the candidate feedback report. Now, thanks to a vast database of reference comments, and the use of text analytics software, we have been able to initiate the industry’s most extensive text analysis to date of reference feedback on job candidates.
SkillSurvey’s analytics team looked at verbatim comments provided by references during confidential reference checks—a sample that included 44,941 words or short phrases that were provided by references. The team identified these top work-related areas of improvement and strengths:
- Dependable/Meets Deadlines
- Team Orientation
- Attention to Detail/Accuracy
Top Areas of Improvement:
- Time Management and Prioritizing
To conduct this study, the analytics team used stratified, random sampling to select 3,200 candidates who were reference-checked on one of 16 job-specific, representative surveys. These 16 surveys spanned a variety of job roles such as customer service representative, registered nurse, software developer, store manager, and engineer, to name a few. Four references (two managers and two coworkers) per candidate were randomly chosen for analysis, yielding feedback from 12,800 references (3,200 candidates x four references each). You can see more details on our research here.
The insights provided by references align well with the applied or soft skills deficits of candidates in a report recently published by the Society for Human Resource Management (SHRM).
We look forward to continuing our research in the future and to investigate the relationships between the results of the text analytics and outcome variables such as hiring decisions, job performance and turnover, as well as a study of the differences in quantity and type of feedback provided by managers vs. coworkers. We have already proven that quantitative reference feedback (behavior ratings and response rates) are significantly related to work outcomes. Our ultimate goal is to integrate the quantitative and qualitative feedback provided by the references, thus enabling companies to make even better hiring decisions.