The Mismeasure of I-O Psychology Programs
Illinois Institute of Technology
Each year the number of graduate programs in Industrial-Organizational psychology
increases and the choice of which graduate programs to apply to becomes more difficult for
prospective students. For an unconscionably long time students have coveted a resource to
aid them in their search for the program befitting their interests and experience. Winter,
Healy & Syvantek (1995) may be commended for addressing the need for an objective
measure of program quality. However, Winter et al..s contribution cannot be regarded as a
panacea for the ills of survey-like measures (e.g., U.S. News & World Report, March
1995) of graduate program quality until some important concerns have been considered.
First, in the multidimensional space of performance/productivity, it is important to
note that Winter et al. (1995) only consider one dimension. Despite the fact that research
in I-O psychology has provided ample evidence of the multidimensional nature of
performance (e.g., Campbell, McCloy, Oppler, & Sager, 1993), Winter et al..s
methodology is based on an obsolete theory that the best possible measure of performance
is an objective measure of individual accomplishment, such as number of publications.
While we concur with Winter et al. (1995) in their critique of the U.S. News and World
Report (March, 1995) methodology, and in their assessment of "subjective"
criteria, we disagree that number of publications is an absolute or even adequate proxy
for program quality.
Second, the results of Winter et al. (1995) are based on the number of publications
found in five journals "unanimously chosen by a group of I-O psychologists" as
likely places for I-O psychologists to publish their research findings. It is arguable
that this list of journals is limiting in that it neglects research contributions that
I-O psychologists make in domains of psychology rarely addressed in these five publications.
For example, I-O psychologists have made several important contributions in the areas of
item response theory, structural equation modeling, and meta-analysis. These methods for
analyzing psychological data represent significant developments for our field despite the
fact that they do not always receive treatment in the more mainstream I-O journals.
Further, the source for Winter et al.'s justification to use the chosen five journals
(Howard, Maxwell, Berra, & Sternitzke, 1985) was published over a decade ago. Few can
deny that the research pursuits of I-O psychologists have, since then, diversified. To be
sure, Katzell and Austin (1992) list a total of 38 areas of research in which I-O
psychologists have become immersed.
Third, and perhaps most importantly, advocating that a graduate program's reputation be
determined solely on the basis of the number of publications is, in essence, stating that
I-O programs that train students to apply I-O interventions in the work place (rather than
to conduct basic research) are without merit. This declaration exacerbates the already
substantial separation between the I-O scientist and practitioner. By advocating a
uni-dimensional measure of program quality, Winter et al. disregard other important
criteria such as number of internships received by graduate students, job placements, or
even number of technical reports written. This would indicate that a good measure of
program quality is much more difficult to derive than a convenience variable such as
number of publications.
The changing face of I-O psychology is reasonably well understood (Dunnette, 1990). The
fact that large numbers of graduate students are being trained primarily for applied
positions rather than academic-research positions does not mean that I-O psychology will
be less able to develop as a scientific discipline. The fact is that graduate programs are
currently producing more academic-research Ph.D.s than there are positions to be filled.
This changing face does mean, however, that those who discourage the application of
I-O psychology to the workplace, by filtering information about graduate program reputations,
will be doing a great disservice to the advancement of the field. As we have seen so
resplendently, the intrinsic differences between science and practice often result in
discord. What our field needs now is not individuals bent on proving that science
overshadows practice, or practice overshadows science, but rather individuals who affirm a
cooperative and synergistic relationship. Instead of fragmenting we can unite and make our
combined knowledge base work for the benefit of humankind. We submit that a good first
step towards this goal is recognizing the contributions of all I-O programs, not just
those that publish the most.
America's Best Graduate Schools. (1995, March 20). U.S. News and World Report,
Campbell, J.P., McCloy, R.A., Oppler, S.H., & Sager, C.E. (1993). A theory of
performance. In N. Schmitt & W.C. Borman (Eds.), Personnel selection. Jossey-Bass: San
Dunnette, M.D. (1990). Blending the science and practice of Industrial and
Organizational psychology: Where are we and where are we going? In M.D. Dunnette &
L.M. Hough (Eds.), Handbook of Industrial and Organizational Psychology (Vol. 1). Palo
Alto, CA: Consulting Psychologists Press, Inc.
Howard, G.S., Maxwell, S.E., Berra, S.M., & Sternitzke, M.E. (1985). Institutional
research productivity in industrial/organizational psychology. Journal of Applied
Psychology, 70, 233-236.
Katzell, R.A., & Austin, J.T. (1992). From then to now: The development of
industrial-organizational psychology in the United States. Journal of Applied Psychology,
Winter, J.L., Healy, M.C., & Svyantek, D.J. (1995). North America's top I-O
psychology doctoral programs: U.S. News and World Report revisited. The
Industrial-Organizational Psychologist, 33(1), 54-58.