On the Validity of the Selection and Assessment Interview
Andrea Castiello d’Antonio
European University of Rome, Italy
Most of the research concerning selection and assessment interviews reveals a low degree of validity, relevance, ecological validity, and impact. We all have read in the course of the years the classic papers on the validity of the selection and assessment interview (Arvey & Campion, 1982; Eder, Kacmar, & Ferris, 1989; Hollingworth, 1923; Mayfield, 1964; Schmidt & Hunter, 1998; Ulrich & Trumbo, 1965). The results regularly show little validity of these selection methods—and seldom strong reliability. Instead, there is great prominence afforded to the psychological test, particularly aptitude and intelligence tests (psychometric test), and more structured, or behavioral, interviews and the behavioral assessment center (see, e.g. Farr & Tippins, 2010). The psychometric approach and techniques tend to demonstrate greater validity (and reliability) than the “clinical” psychosocial and psychodynamic approach, in which the human being is viewed as a whole.
In my opinion, a large part of this kind of research has little or no real and technical relevance, and no practical evidence, because of the confusion made on a central matter: the professional identity of the interviewer, or assessor. “Interviews are the most frequently used procedures in personnel selection across all countries. Practically, 100% of selection processes use one or more interviews. However, not all types of interviews are used to the same degree” (Salgado, Viswesvaran & Ones, 2001, p. 180).
In the studies published over the years, indeed, and also in the major reviews and meta-analyses, the interviewers, judges, or assessors are random. They may be personnel managers, personnel professionals, psychologists (junior or senior), line managers, line professionals, and university students. In many cases they are students of the courses in psychology. In some cases the professional identity of the interviewer or assessor is not clearly indicated, and in other cases the researchers have utilized interviewee paper people (Gorman, Clover, & Doherty, 1978; Murphy, Herr, Lockhart, & Maguire, 1986). Moreover, the interviewee to be assessed was only viewed by the “judges” in a videotape for a few minutes. “Audio- and videotapes, as a form of indirect observation, do not adequately capture the dynamics that invariably occur during the verbal and nonverbal exchanges in the interview” (Buckley & Weitzel, 1989, p. 301).
In the studies in which psychologists were engaged, we can see a great variety of professional and academic identities, not only work and organizational psychologists but also clinical psychologists, researchers in psychology, teachers working in the universities, or “psychologists” with academic or professional qualifications.
Selection and assessment interviews, like all interviews in all settings (organizational, clinical, educational, forensic, and social), are not “instruments” or devices (like aptitude or intelligence tests) but are psychosocial and psychodynamic events in which the personal and the professional skill of the interviewer are fundamental features. To study the real validity of the selection and assessment interviews, we should define very strictly the professional identity and requirements of the interviewer or assessor.
We could really say that the core of the selection and assessment interviews is the interviewer or assessor. I strongly agree with Dreher and Maurer (1989, p. 262) who stated that “the most pressing need is for studies of interviewer validity, not the validity of ‘the interview’—therefore we need to examine individual differences in assessor validity” (Morris, Kwaske, & Daisley, 2011, p. 324).
I suppose that, in other words, it should be impossible to evaluate the validity (for example) of the medical interview if we use as “interviewers” care assistants and professional junior and senior doctors, social workers, care managers, and all kinds of students in medicine or in other fields of study. Why, in the field of work, industrial, and organizational psychology, and in the specific area of the selection and assessment of the psychological characteristics of candidates or employees, do we still have to stay in a similar, absurd situation?
There is also great confusion regarding the format and the setting of the selection and assessment interview. The labels that are used in the studies to identify the “not structured” selection and assessment interviews are many: conventional interview, standard interview, or unstructured interview. Actually, there is not any scientific evidence or standardization regarding these labels, and I would ask the researchers: What does traditional, conventional, or classical interview mean? Further, the so-called “standard interview” does not in fact exist. The world of work is full of a strange kind of interview that I have called “interviste general-generiche,” in other words, “general-generic interviews.” These interviews are conducted by any type of person and are not controlled in any way. In my opinion, all the studies that compare the validity of the structured interviews against the big family of the “nonstructured” interviews, with no other definitions, are of little or no utility, including meta-analyses that mix them all (see e.g., McDaniel, Whetzel, Schmidt, & Maurer, 1994; Orpen, 1985; Wiesner & Cronshaw, 1988). Something similar occurs with psychological questionnaires. “Meta-analyses assume that personality scales from different inventories with similar names are equivalent and measure the same thing. However, this assumption is not justified conceptually or empirically” (Hogan & Kaiser, 2010, p. 102).
“Logically, researchers should not aggregate interview research which is confounded by purpose, nor that which reflects different operationalization of ‘the interview’…There is a need to focus on proper phenomena and address the validity of data collection and evaluation procedures as represented in natural settings” (Buckley & Weitzel, 1989, p. 296). The interview process is seen as influenced by a large number of specific factors (as the type of questions asked, the number of interviewers, and the amount of information gathered). “Among these factors, the degree of structure is generally considered to be the most important, not only because of its effects on the interview process itself but also because of its impact on reliability and validity” (Huffcutt, Conway, Roth, & Stone, 2001, p. 900). In this statement we can observe the total disregard for the professional identity of the interviewers. The degree of “structure” of the interview seems to be more important than the person who interviews (!), and one of the four criteria that Huffcutt, et al. (2001) used to select the studies represented was that these studies had to concern the “typical” interview.
I think that a highly structured interview is more similar to an attitudinal test—this kind of interview has been named spoken questionnaires (Cook, 1998)—than to an assessment interview based for example on the theoretical and technical ideas of Carl R. Rogers (Kirschenbaum, 2007; Rogers & Russell, 2002). I could assert the same regarding a long list of features of the interviews used in the organizations. So, a highly focused behavioral interview lasting 20–30 minutes has no similarity with a psychodynamic in depth interview lasting 2 hours or more (Castiello d’Antonio, 2003), and to compare these different kinds, formats, and settings has no meaning at all. (See also the wide range of observations regarding the strengths and weakness of the employment/selection interview in Eder & Ferris, 1989.)
These kinds of matters are, in my view, not specific of the field of the I-O psychology; we observe similarities in the field of clinical psychology. For example, a psychiatric interview conducted on the basis of the DSM-IV (First, Spitzer, Gibbon, & Williams, 2002) is very far from a psychiatric interview based on Harry Stack Sullivan’s ideas (Conci, 2010). Are we sure that we can seriously compare techniques, settings, and human encounters that are so different? We know—from a constructivist point of view—that the methods that we use partially “construct” the “reality” of the human being that is in front of us. In the organizational context, the interview is not a static technique or an instrument but a process, a social process (Dipboye, 1992). In summary, I propose that we need to stop the “refrain” concerning the low validity and utility of the unstructured interviews (behavioral, targeted, situational, and so on) and start to implement more specific, relevant, and meaningful studies (Castiello d’Antonio, 1994).
In conclusion, the research on the validity of the selection and assessment interviews should be very specific and clear about the professional identity of the interviewer and/or the assessor, and the type (not only the format or the structure) of interview that is realized. “A general shortage of solid research needs to be remedied by showing how real-life interviewers…differ from each other” (Guion, 2011, p. 500). Interviewers differ not only in their approach to the interview and the interviewee, in cognitive and social processes, as Robert Guion says in a recent release of his famous work, but they also differ in emotional, psychological, psychodynamic status, and in theory-of-the-mind. The selection and assessment interview should be posed in the context of individual psychological assessment (IPA), that is “a core competence of I/O psychologists” (Silzer & Jeanneret, 2011, p.291; see also Jeanneret & Silzer, 1998). In situations in which the interviewers and/or the assessors are psychologists (Castiello d’Antonio, 2006), we have to know not only what kind of psychologists they are but also understand their theoretical and methodological orientation and which technique, format, and setting they use.
Arvey, R. D., & Campion, J. E. (1982). The employment interview: a summary and review of recent research, Personnel Psychology, 35, 281–322.
Buckley, M. R., & Weitzel, W. (1989). Comment on the research methodologies used in interview research. In R.W. Eder & G. R. Ferris (Eds.), The employment interview. London, UK: Sage.
Castiello d’Antonio, A. (1994). Interviste e colloqui in azienda. [Interviews in organizations]. Milan, Italy: Raffaello Cortina.
Castiello d’Antonio, A. (2003). Psicodiagnosi manageriale. [Managerial psychodiagnosis]. Milan, Italy: Franco Angeli.
Castiello d’Antonio, A. (2006). La selezione psicologica delle risorse umane: l’intervista individuale. [The psychological selection of human resources: The individual interview]. Milan, Italy: Franco Angeli.
Conci, M. (2010). Sullivan revisited—Life and Work. Harry Stack Sullivan relevance for contemporary psychiatry, psychotherapy and psychoanalysis. Trento, Italy: Tangram.
Cook, M. (1998), Personnel selection and productivity (III Ed.). New York, NY: Wiley.
Dipboye, R. L. (1992). Selection interview: Process perspective. Cincinnati, OH: South-Western.
Dreher, G. F., & Maurer, S. D. (1989). Assessing the employment interview: Deficiencies associated with the existing domain of validity coefficient. In: R. W. Eder & G. R. Ferris (Eds.), The employment interview. London, UK: Sage.
Eder, R. W., & Ferris, G. R. (1989). The employment interview. London, UK: Sage.
Eder, R. W., Kacmar, K. M., & Ferris, G. R. (1989). Employment interview research: History and synthesis. In R. W. Eder & G. R. Ferris (Eds.), The employment interview. London, UK: Sage.
Farr, J. L., & Tippins, N. T. (2010). Handbook of employee selection. New York/London: Routledge.
First, M. B., Spitzer, R. L, Gibbon, M., & Williams, J. B.W. (2002). Structured clinical interview for DSM-IV-TR axis I disorders, research version, non-patient edition (SCID-I/NP). New York, NY: Biometrics Research, New York State Psychiatric Institute.
Gorman, C. D., Clover, W. H., & Doherty, M. E. (1978). Can we learn anything about interviewing real people from “interviews” of paper people? Two studies of the external validity of a paradigm. Organizational Behavior and Human Performance, 22, 165–192.
Guion, R. M. (2011). Assessment, measurement, and prediction for personnel decisions (2nd Ed.). New York/London: Routledge.
Jeanneret, R., & Silzer, R. (1998). Individual psychological assessment: Predicting behavior in organizational settings. San Francisco, CA: Jossey Bass.
Hogan, R., & Kaiser, R. B. (2010). Personality. In J. C. Scott & D. H. Reynolds (Eds.), Handbook of workplace assessment. San Francisco, CA: Jossey Bass.
Hollingworth, H. L. (1923). Judging human character. New York, NY: Appleton.
Huffcutt, A. I., Conway, J. M., Roth, P. L., & Stone, N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews. Journal of Applied Psychology. 86, 897–913.
Kirschenbaum, H. (2007). The life and work of Carl Rogers. Ross-on-Wye: PCCS Books.
Mayfield, E. C. (1964). The selection interview: A re-evaluation of published research. Personnel Psychology, 17, 239–260.
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of the employment interview: A comprehensive review and meta-analysis, Journal of Applied Psychology, 79, 599–616.
Morris, S. B., Kwaske, I. H., & Daisley, R. R. (2011). The validity of individual psychological assessment. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4, 322–326.
Murphy, K. R., Herr, Lockhart, M. C., & Maguire, E. (1986). Evaluating the performance of paper people. Journal of Applied Psychology, 71, 654–661.
Orpen, C. (1985). Patterned behavior description interviews versus unstructured interviews: A comparative validity study. Journal of Applied Psychology, 70, 774–776.
Rogers, C. R., & Russell, D. E. (2002). Carl Rogers. The quiet revolutionary. An oral history. Roseville, CA: Penmarin.
Salgado, J. F., Viswesvaran, C., & Ones, D. S. (2001). Predictors used for personnel selection: An overview of constructs, methods and techniques. In N. Anderson, D. S. Ones, H. Sinangil, & C. Viswesvaran (Eds.) Handbook of industrial, work and organizational psychology. Volume I. Personnel psychology. London, UK: Sage.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology. practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.
Silzer, R., & Jeanneret, R. (2011). Individual psychological assessment: A practice and science in search of common ground. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4, 270–296.
Ulrich, L., & Trumbo, D. (1965). The selection interview since 1949. Psychological Bulletin, 63, 100–116.
Wagner, R. (1949). The employment interview: A critical summary. Personnel Psychology, 2, 17–46.
Wiesner, W. H., & Cronshaw, S. F: (1988). A meta-analytic investigation on the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275–290.