Job Interview - Validity and Predictive Power

Validity and Predictive Power

There is extant data which puts into question the value of job interviews as a tool for selecting employees. Where the aim of a job interview is ostensibly to choose a candidate who will perform well in the job role, other methods of selection provide greater predictive power and often lower costs. Furthermore, given the unstructured approach of most interviews they often have almost no useful predictive power of employee success.

While unstructured interviews are commonly used, structured interviews have yielded much better results and are considered a best practice. Interview structure is defined as "the reduction in procedural variance across applicants, which can translate into the degree of discretion that an interviewer is allowed in conducting the interview". Structure in an interview can be compared to a typical paper and pencil test: we would not think it was fair if every test taker was given different questions and a different number of questions on an exam, or if their answers were each graded differently. Yet this is exactly what occurs in an unstructured interview; thus, a structured interview attempts to standardize this popular selection tool. While there is debate surrounding what is meant specifically by a structured interview, there are typically two broad categories of standardization: 1) content structure, and 2) evaluation structure. Content structure includes elements that refer to the actual content of the interview:

• Base questions on attributes that are representative of the job, as indicated by a job analysis
• Ask the same questions of all interviewees
• Limit prompting, or follow up questions, that interviewers may ask
• Ask better questions, such as behavioral description questions
• Have a longer interview
• Control ancillary information available to the interviewees, such as resumes
• Don’t allow questions from applicants during interview

Evaluation structure includes aspects that refer to the actual rating of the interviewee:

• Rate each answer rather than making an overall evaluation at the end of the interview
• Use anchored rating scales (for an example, see BARS )
• Have the interviewer take detailed notes
• Have more than one interviewer view each applicant (i.e. have panel interviews)
• Have the same interviewers rate each applicant
• Don’t allow any discussion about the applicants between interviewers
• Train the interviewers
• Use statistical procedures to create an overall interview score

It is important to note that structure should be thought of as a continuum; that is, the degree of structure present in an interview can vary along these various elements listed above.

In terms of reliability, meta-analytic results provided evidence that interviews can have acceptable levels of interrater reliability, or consistent ratings across interviewers interrater reliability (i.e. .75 or above), when a structured panel interview is used. In terms of criterion-related validity, or how well the interview predicts later job performance criterion validity, meta-analytic results have shown that when compared to unstructured interviews, structured interviews have higher validities, with values ranging from .20-.57 (on a scale from 0 to 1), with validity coefficients increasing with higher degrees of structure. That is, as the degree of structure in an interview increases, the more likely interviewers can successfully predict how well the person will do on the job, especially when compared to unstructured interviews. In fact, one structured interview that included a) a predetermined set of questions that interviewers were able to choose from, and b) interviewer scoring of applicant answers after each individual question using previously created benchmark answers, showed validity levels comparable to cognitive ability tests (traditionally one of the best predictors of job performance) for entry level jobs.

Honesty and integrity are attributes that can be very hard to determine using a formal job interview process: the competitive environment of the job interview may in fact promote dishonesty. Some experts on job interviews express a degree of cynicism towards the process.