From Phrenology to Fraud: The Breakdown of Science in the Practice of
I-O Psychology
Brent Holland, Robert Hogan, and Dana Shelton
Hogan Assessment Systems
In the last issue of TIP, Janine Waclawski (1998)
asked why psychology has such an ambivalent, even unattractive, popular image. This
question greatly concerns us, and we didn’t find the TIP explanation
persuasive. Waclawski suggests that psychologists are public figures of fun because of
"extremists" in the field and media portrayals of psychologists as incompetent.
This interpretation projects the blame outward and ignores our own actions that contribute
to the field’s poor reputation. We see two themes in modern psychology that probably
contribute to our poor public image. The first is a lack of concern for definitions, which
leads to widespread terminological confusion, and the second is an anomalous disregard for
data.
Successful communication with the public depends on clear definitions
of key concepts; without common definitions, it is impossible to establish convincing
generalizations and meaningful relationships between variables. Yet psychologists tend to
ignore this fact. Consider, for example, the concept of motivation, which is used in at
least three different and mutually contradictory ways. There is motivation in the sense of
underlying needs that persist over time; such as, the need for achievement. Then, there is
motivation in the sense of a person’s reasons or intentions; such as, the motive for
a decision. And finally, there is motivation in the sense of arousal; such as, Is the
workforce motivated? This terminological confusion makes discussions about motivation
almost incoherent—because the terms are never defined.
Similar confusion surrounds the concept of "situation." The
hoary person-X situation debate concerns the degree to which behavior is function of
personal characteristics or situational factors. However, there is no agreement whatsoever
on a definition or a taxonomy of situations, which makes the debate insoluble in principle
(Hogan & Roberts, in press). These examples point out problems that occur when
definitions are either ambiguous or lack consensus and they contribute to the
public’s misunderstanding of what we are about.
As for the claim that the profession, protestations notwithstanding,
doesn’t pay attention to data, the claim is easy to demonstrate with two examples
from the field of assessment. Specifically, in our view, few people pay attention to the
notion of validity, despite the fact that validity is the bottom line in assessment.
Consider first the widespread use of the Myers-Briggs Type l indicator (MBTI) in
organizational interventions and ask yourself about the status of validation research with
the MBTI. Consider secondly the thriving 360 appraisal industry, and ask yourself where
the dimensions on the various appraisal forms came from. Were they chosen on the basis of
known correlations with managerial effectiveness or do they reflect the biases of the test
authors? The answer is obvious. The bad news is that we can provide many more examples of
this sort.
The reputation of the field probably will not improve if we continue
using vague concepts and ignoring data. To correct the problem, we need to accept
responsibility for our own inadequacies and begin clearly defining what we are measuring
and making recommendations based on data. Otherwise, the utility of our services and
assessments remains a mystery, which supports stereotypes about what we do.
References
Hogan, R., & Roberts, B. W. (in press). A socioanalytic perspective on
person/environment interaction. In W. B. Walsh & R. H. Price (Eds.), New directions
in person-environment psychology. Mahwah, NJ: Erlbaum.
Waclawski, J. (1998). The real world: Psychologists on celluloid
and pulp fiction "shrinks." TIP, 36 (2), 51–60.
TIP
Vol. 36/No. 3 January, 1999
Return to Table of Contents