Applicant Faking: A Look Into the Black Box
Humboldt-Universität zu Berlin, Berlin, Germany
Applicant faking behavior has been a concern probably for as long as personality questionnaires have been used for selection purposes. By now, the impact of faking on test scores, more specifically, their means, construct validity, and criterion validity is more or less agreed upon. Many researchers have put their efforts toward developing methods that prevent faking or help to correct for it. However, models describing the actual faking process are scarce. The models that do exist primarily focus on describing antecedents that lead to faking or moderate the amount of faking. The cognitive process itself mostly remains a black box. Based on existing literature from survey research as well as qualitative and quantitative research, the present paper introduces a process model of applicant faking behavior. Using this model as a starting ground, possible future research areas are discussed.
Many different definitions for applicant faking behavior on a personality questionnaire exist. Ziegler, MacCann, and Roberts (2011) recently wrote in an edited book on faking: “Faking represents a response set aimed at providing a portrayal of the self that helps a person to achieve personal goals. Faking occurs when this response set is activated by situational demands and person characteristics to produce systematic differences in test scores that are not due to the attribute of interest” (p.8). In the same book MacCann, Ziegler, and Roberts (2011) reviewed different definitions of faking that exist in the literature. They concluded: “It thus seems that most experts view faking as a deliberate act, distinguishing this from other forms of response distortion that may not be conscious and intentional. Faking is thus a deliberate set of behaviors motivated by a desire to present a deceptive impression to the world. Like most other behavior, faking is caused by an interaction between person and situation characteristics” (p. 311). There also seems to exist a rising agreement that, as a conservative estimate, about 30% of applicants fake (Converse, Peterson, & Griffith, 2009; Griffith, Chmielowski, & Yoshita, 2007; Griffith & Converse, 2011; Peterson, Griffith, & Converse, 2009) and that this rate might be moderated by such aspects as employability (Ellingson, 2011). Furthermore, it is hardly controversial that faking impacts mean scores of personality questionnaires in applicant settings (Birkeland, Manson, Kisamore, Brannick, & Smith, 2006; Ziegler, Schmidt-Atzert, Bühner, & Krumm, 2007). Less agreement exists as to the effect of faking on construct and criterion validity (Ellingson, Smith, & Sackett, 2001; Ziegler & Bühner, 2009; Ziegler, Danay, Schölmerich, & Bühner, 2010). Furthermore, a lot of research is dedicated to developing questionnaires or warnings that prevent faking (Bäckström, Björklund, & Larsson, 2011; Dilchert & Ones, 2011; Stark, Chernyshenko, & Drasgow, 2011) or methods that allow correcting for faking (Kuncel, Borneman, & Kiger, 2011; Paulhus, 2011; Reeder & Ryan, 2011). Even though this research has come up with a lot of encouraging results, its aim might seem a bit hasty to people outside of the faking community. Transferring the research path taken so far to medicine, for example, would mean that scientists try different cures for a new virus or different prevention strategies without having fully understood the virus’ nature. In other words, despite the amount of research being done, little is known about the actual faking process. What do people think about when faking a personality questionnaire? Even though there are some models of faking (Mueller-Hanson, Heggestad, & Thornton, 2006; Snell, Sydell, & Lueke, 1999), most of these models are concerned with the antecedents of faking behavior but not faking itself. Therefore, this paper introduces a process model based on existing literature as well as presents some new research to further elicidate the nature of faking.
The Process of Answering Personality Questionnaires
The actual thought process that takes place when answering a personality questionnaire item has been examined in detail in survey research. Krosnick (1999) summarized different models and ideas and proposed a four-step process of responding that consists of comprehension, retrieval, judgment, and mapping. In this sense, test takers first have to encode the item and form a mental representation of its content (comprehension). During the next step, information is retrieved that is of value with regard to the item content (retrieval). This information is then compared with the mental representation of the item (judgment) and the result mapped onto the rating scale (mapping). Obviously, this is an optimal process that occurs when people are motivated to answer the items of the questionnaire in a sincere manner. Krosnick used the term optimizing for this strategy. Depending on factors such as motivation, cognitive ability, and fatigue, some test takers might not undergo this optimal process. In that case, a satisficing strategy is used. Here, respondents adjust the effort put forward relative to their expectation of the importance of the test. Tourangeau and Rasinski (1988) suggested a similar process model that included an additional step during mapping called editing. This editing is supposed to reflect social desirable responding or faking. Thus, one could conclude that a rather detailed model of faking behavior already exists. However, some results from faking research raise doubts. Birkeland et al. (2006) in their meta-analysis demonstrated that, depending on the job, applicants fake personality traits differently. Moreover, there is evidence that some people fake in the wrong direction (Griffith, 2011). In addition, there is growing agreement that faking cannot be regarded as a unitary process that is comparable for each and every test taker. This idea is supported by empirical evidence showing that there are at least two different forms of faking: slight and extreme faking (Zickar, Gibby, & Robie, 2004; Zickar & Robie, 1999). Thus, it seems worthwhile to further investigate the cognitive process people undergo when faking on a personality questionnaire.
A Qualitative Study of Faking
We conducted a qualitative study using the cognitive interview technique (Dillman & Redline, 2004; Willis, 2004) to elucidate the cognitive process underlying faked personality questionnaire responses. Participants first filled out a personality questionnaire with a specific faking instruction (see Ziegler & Bühner, 2009) that they should assume they are applying for a position as a student in undergraduate psychology. In order to get a place in the program, they had to first take the personality test, the NEO-PI-R (Ostendorf & Angleitner, 2004). However, they were told a test expert would examine the results for faking. All participants were asked to think out loud, to voice all their thoughts as they filled out the personality questionnaire. The entire time a test administrator was present but not visible to the participant. The administrator took notes and in case participants fell silent, reminded them to speak all thoughts out loud. Finally, participants were administered a semistructured interview. Participants were asked how they tried to achieve the given goal, whether they applied that technique to all questions, and finally if they could name the strategy they had used. There were 50 participants (34 women and 16 men). All were undergraduate students enrolled in different studies (27 were psychology students). The average age and semester were 22.26 (SD = 1.91) and 1.89 (SD = 2.01), respectively. Two faking experts independently analyzed the combined information with the goal of developing a cognitive process model of faking. The approach chosen to analyze the qualitative data was based on the grounded theory.
After analyzing all the information gathered, both judges independently identified the two main strategies for intentionally distorting a questionnaire suggested by Zickar et al. (2004): slight faking and extreme faking (see also Robie, Brown, & Beaty, 2007). Consequently, rater agreement was compared using Cohen’s kappa (= .77). A third expert was consulted for the cases of nonagreement. Based on this, about 20% of the participants were categorized as extreme fakers and 80% as slight fakers.
Of more interest though was the actual thought processes that were verbalized. We first looked at the strategies participants reported using to fake. In most cases this provided little insight because participants could not really name a strategy. Of more help were the participants’ responses to how they had tried to achieve the goal of faking good. Here two strategies emerged. Most of the people said something like they took the answer they would have given under normal circumstances and pushed it a little in the “right” direction. These responses were confirmed by the actual thought protocols. To give an example, one participant said when pondering his answer to the item “I keep my things clean and tidy:” “Well, I guess I don’t really do that, but if I want to be selected, I better endorse a four.” Only a few participants, those labeled as extreme fakers, stated that they endorsed the highest possible category. However, one important aspect that needs to be mentioned is that neither extreme nor slight fakers faked all items regardless of their content. Before considering an answer, participants judged whether the question would reveal information important to select psychology students. If that was the case, they faked. If it was not the case participants did one of two things. They either answered honestly or they answered neutrally using the middle category, some even did both alternatingly. The middle category was often chosen in order to avoid a wrong answer or keep information believed to be unnecessary for the position applied for a secret.
Another interesting finding was that students currently enrolled in psychology faked in different ways from students in other academic programs. The differences were rather severe. Although psychology students tended to endorse Conscientiousness items and portray themselves as low in Neuroticism, other students faked Openness and portrayed themselves as more neurotic.
Developing a Process Model of Applicant Faking Behavior
Our findings can be integrated into the general models by Krosnick (1999) and Tourangeau and Rasinski (1988) described above. The resulting model is displayed in Figure 1. We did find evidence for the four-stage process model consisting of comprehension, retrieval, judgment, and mapping. However, there were also several noteworthy differences or extensions.
Figure 1: A cognitive process model of applicant faking behavior
One noteworthy aspect is that the interaction between person and situation characteristics could be observed in each of the stages. It seems that people first evaluate the importance of an item in terms of the situational demand (e.g., application for a certain job or student program). If the test taker judges the item as unimportant in regard to the demand of the situation, no faking occurs. Thus, immediately after forming a mental representation of the item content, its importance is classified. Therefore, the general model was extended to include this classification. As a consequence, even the use of the maximum response strategy does not necessarily result in a maximum score because not all items are faked. The results support the idea that specific knowledge and implicit theories about the position applied for are used for evaluation of importance. Further, the stakes of the situation may also impact the importance decision.
Following this initial importance classification are retrieval and judgment. This process is similar to that described by Krosnick (1999). Personality traits such as self-understanding or the ability to reflect might be named as influencing this process more towards optimizing or satisficing. On the situation side, the presence of the administrator clearly increased the likelihood of optimizing.
The mapping stage happened so quickly that it was virtually impossible to gather information from the thought protocols. Of course, the “true” standing regarding the item poses a natural limit for the amount of faking possible. Personality traits such as honesty or a high perceived employability can be assumed to affect mapping as well. Drawing a distorted picture of one’s self in an applicant setting certainly requires high self-efficacy beliefs (Ziegler, 2007). This means the test taker may believe he or she can live up to this standard. Otherwise, so-called dark personality traits such as narcissism or a general tendency to overclaim might be implied. Finally, continued lying requires a certain cognitive effort. One has to keep up with the lies. Consequently, general mental ability (McGrew, 2009) was included in our model.
Implications and Future Research
At first glance, knowing more about the actual thought process does not seem to bear immediate practical implications. An important issue though is the importance of classification and its influence on the way uninformative items are answered. The increased use of the middle category is a potential problem. Thus, rating scales without such a middle category might be advisable. Moreover, the fact that prior knowledge as well as implicit theories about the job are used suggests that providing job information before the test might level the playing field. This might be particularly necessary if the applicant pool contains people new to the job as well as experienced workers. Otherwise, job experience might help applicants to fake in the right direction leaving people new to the job at a disadvantage. Because the personality questionnaire is used to assess individual personality differences but not job knowledge, this would endanger the test score’s construct validity.
The methods used here only give insight into conscious faking efforts. However, as Paulhus has outlined (Paulhus, 2002), there are also unconscious processes (e.g. self-deceptive enhancement and self-deceptive denial) influencing the answering process. Their influence as well as their interaction with other personality and situational aspects of the model suggested here will be important to study. The occurrence of two qualitatively different ways to use a rating scale has already been reported for low-stakes settings. Here, people preferring either middle or extreme categories could be distinguished (Rost, Carstensen, & von Davier, 1997, 1999). The extent to which these response styles relate to slight and extreme faking should be examined using large samples. Otherwise, we might be talking of the same phenomena using different labels.
This study also contradicts one of the dogmas of faking research, which is to always use real applicant samples. In order to investigate basic cognitive processes it sometimes is necessary to go back to the laboratory. This insight might also be true for other faking research questions.
The model introduced here has many white spots on the situation side. This clearly underscores a need to broaden the research perspective and place more emphasis on situational factors associated with faking. This idea is anything but new. Nevertheless, systematic research on how situations are perceived and how this perception influences behavior is rare.
Finally, the importance classification of items took place so naturally and quickly that it seems unlikely that such a classification does not occur in low-stakes settings as well. Thus, an important next step would be to further elucidate the cognitive process of item answering under normal conditions.
The model introduced here must be considered as preliminary. As the many incomplete lists imply, the enumeration of person and situation aspects influencing the thought process is by no means exclusive. However, the model should be understood as a starting point for more research aimed at understanding applicant faking behavior more fully. To this end, the hypotheses put forward here based mainly on qualitative research should be tested applying quantitative methods and large samples. Finally, the proposed model shows that to understand the complexity of faking behavior, more elaborate models including person and situation variables, as well as their interaction, are necessary. Assumptions that a general editing process or the differentiation between slight and extreme faking suffice to explain individual differences in actual faking behavior are definitely premature.
This paper’s purpose was to elucidate the black box that is applicant faking behavior. Using existing cognitive models for the item answering process as well as qualitative analyses, a model was introduced consisting of five stages: comprehension, importance classification, retrieval, judgment, and mapping (including editing). The importance classification based on knowledge and implicit theories, the handling of supposedly neutral items (i.e. uninformative with regard to the faking goal), as well as clear evidence for the person situation interaction provide additional insight into the cognitive process taking place during faking.
Bäckström, M., Björklund, F., & Larsson, M. R. (2011). Social desirability in personality assessment: Outline of a model to explain individual differences. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 201–213). New York, NY: Oxford University Press.
Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14, 317–335.
Converse, P., Peterson, M., & Griffith, R. (2009). Faking on personality measures: Implications for selection involving multiple predictors. International Journal of Selection and Assessment, 17, 47–60.
Dilchert, S., & Ones, D. S. (2011). Application of preventive strategies. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 177-200). New York, NY: Oxford University Press.
Dillman, D. A., & Redline, C. D. (2004). Testing paper self-administered questionnaires: Cognitive interview and field test comparisons. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin & E. E. Singer (Eds.), Methods for testing and evaluating survey questionnaires: Hoboken, NJ: Wiley.
Ellingson, J. E. (2011). People fake only when they need to fake. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 19–33). New York, NY: Oxford University Press.
Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86, 122–133.
Griffith, R. L. (2011, April). Can faking ever be overcome in high-stakes testing? Paper presented at the Debate conducted at the 26th Annual Conference for the Society for Industrial and Organizational Psychology, Chicago, IL.
Griffith, R. L., Chmielowski, T., & Yoshita, Y. (2007). Do applicants fake? An examination of the frequency of applicant faking behavior. Personnel Review, 36, 341–357.
Griffith, R. L., & Converse, P. D. (2011). The rules of evidence and the prevalence of applicant faking. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 34–52). New York, NY: Oxford University Press.
Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567.
Kuncel, N. R., Borneman, M., & Kiger, T. (2011). Innovative item response process and Bayesian faking detection methods: More questions than answers. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 102–112). New York, NY: Oxford University Press.
MacCann, C., Ziegler, M., & Roberts, R. D. (2011). Faking in personality assessment: Reflections and recommendations. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 309–329). New York, NY: Oxford University Press.
McGrew, K. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37, 1–10.
Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C. (2006). Individual differences in impression management: an exploration of the psychological processes underlying faking. Psychology Science, 3, 288–312.
Ostendorf, F., & Angleitner, A. (2004). NEO-PI-R. NEO Persönlichkeitsinventar nach Costa und McCrae. Revidierte Fassung. [NEO-PI-R. NEO Personality Inventory]. Göttingen, Germany: Hogrefe.
Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 49-69): Mahwah, NJ: Erlbaum.
Paulhus, D. L. (2011). Overclaiming on Personality Questionnaires. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 151-164). New York, NY: Oxford University Press.
Peterson, M., Griffith, R., & Converse, P. (2009). Examining the role of applicant faking in hiring decisions: Percentage of fakers hired and hiring discrepancies in single- and multiple-predictor Selection. Journal of Business and Psychology, 24, 1–14.
Reeder, M. C., & Ryan, A. M. (2011). Methods for correcting for faking. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 131–150). New York, NY: Oxford University Press.
Robie, C., Brown, D. J., & Beaty, J. C. (2007). Do people fake on personality inventories? A verbal protocol analysis. Journal of Business and Psychology, 21, 489–509. doi: DOI 10.1007/s10869-007-9038-9
Rost, J., Carstensen, C. H., & Von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J. In Rost & R. E. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 324–332). New York, NY: Waxmann.
Rost, J., Carstensen, C. H., & von Davier, M. (1999). Are the Big Five Rasch scalable? A reanalysis of the neo-ffi norm data. Diagnostica, 45, 119–127.
Snell, A. F., Sydell, E. J., & Lueke, S. B. (1999). Towards a theory of applicant faking: Integrating studies of deception. Human Resource Management Review, 9, 219–242.
Stark, S., Chernyshenko, O. S., & Drasgow, F. (2011). Constructing fake-resistant personality tests using item response theory: High-stakes personality testing with multidimensional pairwise preferences. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 214-239). New York, NY: Oxford University Press.
Tourangeau, R., & Rasinski, K. A. (1988). Cognitive-Processes Underlying Context Effects in Attitude Measurement. Psychological Bulletin, 103, 299–314.
Willis, G. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin & E. E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 299–317). Hoboken, NJ: Wiley.
Zickar, M. J., Gibby, R. E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 168–190.
Zickar, M. J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551–563.
Ziegler, M. (2007). Situational demand and its impact on construct and criterion validity of a personality questionnaire: State and trait, a couple you just can’t study separately! Dissertation, LMU München: Fakultät für Psychologie und Pädagogik.
Ziegler, M., & Bühner, M. (2009). Modeling socially desirable responding and its effects. Educational and Psychological Measurement, 69, 548.
Ziegler, M., Danay, E., Schölmerich, F., & Bühner, M. (2010). Predicting academic success with the Big Five rated from different points of view: Self-rated, other rated and faked. European Journal of Personality. doi: 10.1002/per.753
Ziegler, M., MacCann, C., & Roberts, R. D. (2011). Faking: Knowns, unknowns, and points of contention. In M. Ziegler, C. MacCann & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 3-16). New York, NY: Oxford University Press.
Ziegler, M., Schmidt-Atzert, L., Bühner, M., & Krumm, S. (2007). Fakability of different measurement methods for achievement motivation: Questionnaire, semi-projective, and objective. Psychology Science, 49(4), 291–307.