Journal Article on Personality Tests Leads to Rebuttals

When an article suggesting that employers reconsider the use of personality tests when selecting employees appeared in the fall issue of Personnel Psychology, the responses came quickly from several SIOP members.

“It struck a nerve and generated a great deal of reaction, certainly more than most pieces that appear in the journal,” said Jeff W. Johnson, an associate editor. “We received many requests to write responses to that article. Because of space and timing considerations, we allowed just two sets of authors to respond, as well as running a rebuttal from the authors of the original article."

The original article, which questioned the validities of self-report personality tests used for personnel selection, was authored by Frederick P. Morgeson, Michael A. Campion, Robert L. Dipboye, John R. Hollenbeck, Kevin Murphy, and Neal Schmitt and, according to Johnson, was “seen by many SIOP researchers and practitioners as counter to their experience and needed some balance.”

In their rebuttal, Deniz S. Ones and co-authors Stephan Dilchert, Chockalingam Viswesvaran, and Timothy A. Judge maintain that personality constructs have proven ability to explain and predict work attitudes, behaviors, job performance, and outcomes.

“Personality includes a wide range of characteristics that people possess; many of them have an impact upon the ways people behave in the workplace,” Ones said.

Job-related characteristics can be isolated and measured with the results providing data that are useful in selecting people to hire, she points out.

For example, conscientiousness, one of the Big Five personality measures, is quite predictive of job performance, says Ones, who is widely known for her meta-analyses of the personality literature. “In my data base, there are more than 300 studies on conscientiousness that show validity levels useful for personnel selection.”

Other studies, focusing on different personality dimensions, have also yielded positive results and useful validities for various occupational groups and job families.

“There is nothing in the literature to suggest that validities are poor when looking at the validities of relevant personality measures,” she said.

Ones says there is no “one size fits all” personality assessment. “You can’t lump all the personality traits together or all criteria because that will dilute the validities. So care has to be taken not to do that. It is important to look at one thing at a time in order to achieve accurate estimates of how useful these tests are in applied settings.”

Leaetta Hough agrees. "The relationship between personality variables and criteria are complex. There is considerable evidence supporting the usefulness of self-report personality scales for predicting work-related criteria. However, not every personality scale predicts every criterion – the relationships are more complex than that. When personality scales are included that are appropriate for the criteria of interest, meaningful relationships emerge."

A person should not take some over-the-counter medications for serious conditions simply because they think it might make them feel better without first going through a thorough examination by a qualified medical practitioner. Nor should personality tests be given without a thorough review of what needs to be learned about the candidates. Then the proper tests can be prescribed and they will produce reliable results, Ones said.

Robert P. Tett and Neil D. Christiansen advocate personality testing as well. “Meta-analyses have demonstrated that published personality tests, in fact, yield useful validity estimates when validation is based upon confirmatory research using job analysis and taking into account the bi-directionality of trait performance linkages,” they wrote, noting further that prediction is better for narrow over broad traits, when multiple traits are used in combination, and when traits are identified using appropriate theory.

Tett and Christiansen stress the importance of situational specificity in assessing personality test validity. That is, whether or not a given trait predicts job performance varies widely between jobs even within the same job family.

In order to identify the traits most relevant in a given job, they recommend use of personality-oriented job analysis. Such confirmatory strategies yield validity much stronger than that for irrelevant traits.

An exploratory study might measure 20 or more different traits and find only two or three that predict job performance. “Averaging all the validity coefficients dilutes validity found using the more accurate confirmatory approach,” Tett and Christiansen say, adding that "targeting job-relevant traits for use in personnel selection is professionally and legally mandated.”

Bi-directionality is a special case of situational specificity. “Consider agreeableness. Research shows that, depending upon the job, agreeableness can be a good thing or a bad thing. This means that validity coefficients for agreeableness can be positive or negative, and averaging validity across jobs will yield a weak overall mean. This doesn’t imply that agreeableness is unimportant or useless as a predictor.”

Tett and Christiansen argue that failing to account for situational specificity leads to gross underestimates of the value of personality for use in hiring.

Like Morgeson et al., Ones agrees that applicant faking likely occurs among personality test takers and can be a cause for concern. The degree of concern is a different question, though. “I would worry if faking ruined the validity of self-report tests. However, a review of some 30 studies among actual job applicant samples shows that validity was not destroyed. Actually the validities were quite good. Undoubtedly some of the answers from job applicants were faked, but validity didn’t come down to the level that the tests were useless.”

Tett and Christiansen believe that “research on job applicants suggests that personality test validity in actual selection settings does indeed suffer from faking,” and the fact that some applicants distort answers more than others “poses practical, theoretical and ethical challenges to those relying on personality tests to make employment decisions and who ignore intentional distortion.”

They also dismiss as premature claims that faking might contribute positively to job performance in some situations.

Both Ones and Tett advocate further investigation into the impact of applicant faking on personality test validity.

Steven Hunt, the director of business transformation at SuccessFactors in Portland, OR, has been involved with personality based testing for the past 15 years.

He says that personality assessments, if used correctly, can make hiring decisions more efficient and effective, but if used incorrectly, they can quickly do some serious damage to an organization.

The key to using these powerful but potentially risky tools is to approach them with a clear sense of their strengths and limitations, Hunt maintains.

“On the one hand, researchers and practitioners should avoid making sweeping statements about the generalized validity of personality assessments. On the other hand, we should not overly focus on the limitations of personality tests and lose sight of their advantages. Self-report tests often provide the best solution available to companies that need to sort through large number of candidates in an efficient manner,” he said.

"Until we develop better and more affordable methods for measuring personality, the option to use personality tests with relatively low validity levels often remains one of the better selection methods available to many organizations. Personality assessments with even small levels of validity provide significant financial value when applied to large numbers of hiring decisions. It is true that these measures are not as predictive as we would like them to be, but in many hiring situations such personality tests are clearly better than many other forms of assessment currently in use," he said.

“We know that personality-based selection tests work, but they can work even better,” he conceded. “As scientists it is incumbent upon us to develop more effective methods of personality assessment. And as practitioners, it is incumbent upon us to support and encourage their appropriate use as long as they continue to be the best method available for evaluating candidates based upon job relevant personality traits.”

Morgeson acknowledged that the original article he and his colleagues authored was “likely to cause some controversy within the field. “ However, if the article stimulates more research and discussion, then “that is a good thing.”

And, it seems to have done just exactly that.