What I Learned Along the Way
Frank Landy
Landy Litigation Support Group
For this issue’s column, you have the pleasure of the ruminations of two very heavy hitters—Bob Guion and Paul Sackett. Bob has been telling everyone for the past 10 years or so that he just doesn’t “do this stuff” anymore. And then he writes a book, a paper, or scolds us at a convention. So I guess I just don’t know what “this stuff” is. He keeps doing the stuff I like, and as he will tell you, it is always about me. Paul is a bit different. He wishes there were 27 hours in the day but only if the day is between Monday and Friday (or so he would have us believe). Bob is telling you to take the business of applied psychology very seriously and Paul is telling you to take it less seriously. But Bob chooses not to mention that at various times, he has frittered away his time making candy, blowing glass, playing the oboe, and curling (not up in a ball, on the ice like the Canadians do!). So at the end of the day (particularly if it is a Saturday or Sunday), Bob and Paul may be more alike than different. I know I like them both a lot.
Still Learning
Robert M. Guion
Bowling Green State University
I’ve never learned to say no to Frank Landy, no matter what he asks. It’s an old habit. When he was still a graduate student and I was department chair, he would meet me at the parking lot as I came to work in the mornings and tell me what I had to do that day. And, meekly, I did it. I still do, even when he’s asking me to share opinions under the guise of acquired knowledge. But I have learned a few things, so here’s a baker’s half-dozen of things I’ve learned (or am trying to learn):
1. Psychologists are not all enthusiastic about the same things, and that can be a good thing. It’s obvious but true: We have different special interests, and sometimes we don’t even talk to those whose interests are different. During WWII, the Army in its omniscient wisdom sent me to school. One of the courses supposed to teach me something was an introduction to psychology. It met first thing in the morning. I thought the stuff about synapses and reflexes and defense mechanisms was incredibly boring. Army discipline in that setting was sorely lacking, so I rarely went to class—a fact reflected in my test scores. Later, during a dull period in Italy, I thought an Armed Forces Institute correspondence course would be a pleasant way to spend some time, but the only one I could get a book for was general psychology. It was duller than the camp-life episode, which ended before the course did. A longer dull period came between the end of the war in Europe and the end in Japan, and I tried again. Again, the only course for which books were available was general psychology. I tried to get interested—I really did—but I soon quit the pretense and took up piano lessons at the local Red Cross instead.
After my discharge, I found an interesting job chasing down parts in an electronics plant and wondered whether I was rehabilitated enough for college. What better way to find out, I thought, than to try another psychology class. I signed up for the available psychology class at the local extension center—a course called applied psychology. It seemed to have answers to the things that were plaguing me at work. I got quite enthusiastic about applied psychology, maybe as enthusiastic as the text book writers and course instructors in general psychology had been about synapses and conditioned reflexes. To each his own.
I learned that topics I had thought were dull were in fact relevant to my real-world job, topics like attention, perception, learning, and even defense mechanisms and conditioning! As I went on to major in psychology, I discovered that topics I wasn’t enthusiastic enough about to make a career of them were nevertheless interesting and useful enough to learn about. Years later, as a department chair, I used Likert’s idea of the link pin between organizational entities and found or hired people who could link (or bridge) various subdisciplines within the department. It worked well, and the effect was more collegiality within the department, despite diverse individual enthusiasms, yielding a far less parochial and far more collaborative research agenda in the department.
2. It is not always necessary to share my opinions with others. To be sure, that is much of what one does as one who professes. But I eventually learned that gratuitous sharing is not always necessary or helpful—especially when one is out of sorts in general. For the first half-dozen years or so of my tenure at Bowling Green, I alternated between being too mad to stay and too mad to leave. Many of the things that made me mad are things I’ve long since forgotten, but I do know I acquired a reputation for firing off explosive letters to the deans, the effect of them now reminding me of Garner’s famous (and only partially and usually euphemistically misquoted) comment on the impact of the vice presidency in FDR’s time.
Some of it, however, was effective. What was then called griping (now called assertiveness) resulted in the first full-scale curriculum revision in our department, a revision that created the program in industrial psychology. It has prospered. If I had been less assertive, or if I had actually left, the program we now have may never have been started. Nevertheless, my assertions have been wrong or carelessly stated often enough that honesty and rare humility require me to admit that hindsight, if nothing else, shows that sharing opinions is not always a good thing. I’m thinking of assertive opinions on such topics as synthetic validity (which seems unwilling to die despite the lack of much evidence to support it after more than 40 years) or my early attempt to define “unfairness” in employment practices. A better practice seems to me to be waiting for second thought and, more than that, for data before sharing opinions—even though that’s what I’m doing now.
3. Tacit knowledge—knowledge acquired through experience rather than intentional learning or training—may be false. (Like Sportin’Life’s Bible, “it ain’t necessarily so.”) I learned that long before I ever heard of tacit knowledge, or of Sternberg, either. When I was a counselor for the Schools and Departments of Engineering at Purdue, I acquired the tacit knowledge that good bets for engineering success were those with high Q-to-L score ratios on the ACE test. When a research project came along, I had the opportunity to correlate that ratio to actual grades. It was a strong negative correlation.
4. Writing is a good way to clear one’s thinking, but it does little good unless it also clears one’s prose. What one writes is not necessarily what someone else reads. I have a personal but classic example of unclear writing that has haunted me unnecessarily for many years. It is the often-cited and usually misinterpreted article by Guion and Gottier (1965). After a review of the literature (nonquantitative, of course, because Schmidt and Hunter had not yet been invented), we concluded that there was no good evidence that personality measures (except ad hoc ones developed for specific situations) had yet shown any usefulness for employment purposes—and that there would be no such evidence until more targeted and competent research had been done; ours was call for more and better research on personality tests and their validities. But hardly anyone heard that call. What most people read (and what many still do) was that we said personality tests have no validity. To decry the lack of validity evidence is not the same thing as concluding a lack of validity, but the article obviously did not clearly make that distinction.
I have recently begun reading the journals again, after a few years of total retirement. I am appalled by the formula-like, copycat writing style and even more by the number of ambiguities in many scientific articles in major journals. Some ambiguities occur because of poor word choices, some because of inappropriate punctuation (editors should require authors to read a book called Eats, Shoots and Leaves by Lynne Truss). Most of it, I suspect (sharing my opinions again), is due to unclear thinking about the data and what they mean, and that may be due to a desperate effort to make interpretations fit the “theory” some journals mandate for the early pages of manuscripts.
5. Career choices are often more the result of circumstances than of careful planning. I chose industrial psychology over my prewar major in chemistry because I walked into the postwar chemistry lecture room and saw what had happened to the periodic chart during my absence; if I was going to have to start a major from scratch, it seemed a good idea to start something new, with no unlearning to do. I chose my first job (at Bowling Green) because there were only two jobs available when I got my degree, and I didn’t want the other one. I spent most of my career on selection and measurement (instead of my earlier interest in work motivation) because (a) I wrote stuff that later became a book for my class on personnel testing, (b) I was appointed to the APA Committee on Testing and subsequently to a Standards revision committee, and (c) I got deeply involved, largely at the request of a former colleague, in EEO matters. For those who believe as strongly in planning as I used to when I taught courses in career development, I emphasize my total lack of regret for any of these career turns.
6. There are more ways, Horatio, to solve organizational problems than are dreamt of in your philosophies and theories. I-O psychologists are often guilty of a self-serving, arrogant belief that we, like Father, know best. We establish theories (with or without prior data), develop studies, and peddle them. Because of one of those circumstantial events, I found myself one summer working for the state personnel services in a lovely vacation area. Their state legislative auditor had severely criticized their civil service testing program. As a result, a high official in the personnel department was sent to a series of workshops, one of which I led. He invited me to spend the summer finding out whether the criticisms were justified (they were) and to tell them what to do to return to the auditor’s good graces. I ultimately offered the director a choice among three research programs of varying comprehensiveness. He looked them over, rather carefully and thoughtfully, and finally said, “I think I’ll play a game of golf with the auditor.” He did, the auditor won, and in the ensuing conversation, an organizational problem was solved. The solution was not, and still is not, one I approve, but it solved the organizational problem at hand.
7. Good organizational research is not finished when the report is turned in. Life would be so pleasant if every research project had a clear beginning, pathway, and ending. I’ve been in many projects that did, usually because of the time limits on a grant or in a contract with an organization. These often include requirements for a final “deliverable” that defines the end of the project. What I’ve learned is that such a clear endpoint is great for getting a publication, but it is not so great for the organization that paid for the work. If our project is going to do an organization any good, we had better plan for long-term follow-up to make sure that organizational changes, personnel changes, loss of interest, the pressures of newer concerns, or just temporal decay have not left our deliverables gathering dust on a storeroom shelf with none of the organizational benefit our results had promised. I wish I had learned this lesson much, much earlier. This is one reason I have always believed that major organizations should have in-house, research-oriented psychologists who can keep an eye on the progress of implementing research findings, maintain the necessary follow-up schedule, maintain ties with the researcher, and recognize the signs of faltering implementation before they become fatal.
What the Hell, It’s Only a Hobby!
Paul Sackett
University of Minnesota
Looking back at earlier columns, I see that contributors often take a biographical perspective. But as I was cajoled into writing an autobiography for the SIOP past president Web page (http://www.siop.org/Presidents/Sackett.htm), I won’t repeat that. I found myself thinking about advice I received in graduate school. Much of it went in one ear and out the other. But there are some insights that have stayed with me. Here are a few of them:
Buy low, sell high. The question of the choice of a research topic (e.g., for a dissertation, for a program of research) is one that causes agony for many. The safe option is a “next step” study in a well-defined area. Although more straightforward to accomplish, this is also less likely to be of high impact than a study in a new or emerging domain. Bob Sternberg expressed this idea nicely with the admonition that heads this paragraph: One is most likely to have impact by investing in a novel topic that then comes to be recognized as important. The trick, of course, is that such a strategy requires two things: identifying a novel topic and accurately gauging that the topic will indeed be recognized as important. I just spent a sobering hour looking up the citation rates to articles I’ve published over the years. On a number of occasions I’ve initiated research on relatively novel topics; some have proven influential and some not. I wrote about integrity testing when the topic was novel, and that work is widely cited. Twenty-four years ago George Dreher and I questioned the construct validity of assessment centers, starting a line of research that continues to this day. On the other hand, Ann Marie Ryan and I observed that individual psychological assessment was underresearched, given its prominent role in psychological practice. We carried out a series of studies that we hoped would help establish this as a thriving research topic; to our disappointment, few followed our lead. But all in all, although work on a new or understudied topic is no guarantee of high impact, I do find that my highest impact articles tend to be in this category.
What the hell, it’s only a hobby. In my first term in graduate school, Bob Billings related to us the story, possibly apocryphal, of a symposium in which a young researcher presented strong empirical data that were devastating for a key proposition of a leading scholar’s theory. The leading scholar was to serve in a discussant role and sat on stage impassively as his ideas were sliced and diced. Finally, it was his turn. The audience leaned forward eagerly, hoping for bloodshed. Said the great man: “What the hell, it’s only a hobby.”
I’ve repeated that mantra to myself many times over the years. I give it several layers of meaning. First, it’s a great feat to be able to smile in the face of criticism of your own work. It’s something I aspire to and admittedly sometimes fail at. Second, it’s critical to avoid the trap of becoming ideologically committed to a particular position, such that you are not open to new evidence, or that you take disagreements personally. I’ve published articles that are critical of my own earlier work: If new theory, methods, and/or data shed new light, reassess your view of the world and move on.
Psychometrics and statistics are your friends. Sitting on the bottom shelf of an office bookcase near my desk are 24 spiral bound notebooks, one from each course I took in graduate school. I’m sorry to say that most haven’t been touched since then. The exceptions are the notebooks from core courses in measurement and in correlation and regression; they have been pulled out at least once a month for the last 30 years.
I’m not sure I believed it when I was told that these would be the most important courses I would ever take. But I do now. I try to convince our incoming students that their methodological toolkit, and most specifically their knowledge of measurement and statistics, would see them through just about anything.
But here’s the critical thing: Although I pull those notebooks off the shelf regularly to obtain or verify a computational formula, most of the time when I draw on my psychometric knowledge I do so without access to text or notes. The true value of this body of knowledge is not the ability to compute various indices when called upon to do so, but as an internalized way of making sense of data. A table of means, standard deviations, correlations, and reliabilities is just an imposing array of numbers to the newcomer; it reveals a very rich story once one has an integrated perspective on how all these indices interrelate. Robert Abelson’s wonderful book Statistics as Principled Argument has a chapter with the great title “On Suspecting Fishiness.” I’d guess that I find an impossible or implausible result in roughly one out of every five manuscripts I review, which often turn out to be transcription errors in preparing a table or computational errors in carrying out analyses. But one needs to have internalized central principles of measurement and statistics in order to have a sense that something is fishy when reading research results.
Save some fun for tomorrow. This is a line that my wife, Pat, likes to use when suggesting that there’s more to life than work. Early on I struggled with questions about work–life balance. I wanted to be a good psychologist and a productive researcher. Surely 10 more hours of work per week would help. And then why not 10 more? Things got out of control when I tried adding a 20-hour per week journal-editing job without subtracting anything. It took me a while to realize that a career is a marathon, not a sprint, and that pacing is important. I concluded that I’d be more productive in the long run with a more balanced life, and I believe that has proven to be true. I’m convinced that a reason I’m still passionate about psychology 30 years into a career is that it’s only one of my passions. Others include getting through a Beethoven sonata with relatively few mistakes, meeting the qualifying standard for entry into the Boston Marathon (19 consecutive years!), traveling the world with family and friends, and organizing a 24-hour running event to raise scholarship funds for inner-city kids. And tomorrow I get to be a psychologist again: There’s an early morning meeting with students, a midday talk to give at a local university, and an afternoon flight to meet with colleagues about a research project. As Pat also likes to say, it’s all part of life’s rich pageant.