Getting to Know SIOP’s Award Winners

Liberty Munson and Garett Howardson

A few months ago, I opened an email from Tara, the new TIP editor. In it, she shared some thoughts on possible new features for TIP and was looking for people who might be interested in writing some of these articles. One idea grabbed my attention—feature stories on SIOP’s award winners.

I have always been curious about what it takes to win one of these awards (what is the secret sauce?) and what we, as an industry, can learn from the research and practice that led to the award. Further, I imagined that the people who won these awards would be passionate about the work that they had done, and it’s always fun to talk to someone who loves what they do. No matter what the topic, talking to someone who is truly jazzed about something will get me excited, too. It reminds me of all there is to love about the work that we, as I-O psychologists, do every day, about the difference we make in the lives employees, how we really have the power to change the course of the employee experience within organizations. Are you starting to get as excited as I was?!

I’m hoping these feature stories will provide you with valuable insights into not only the award process but also into how to be successful and effective in your role, even if you don’t think the work that you do is award worthy… But, what I’m starting learn is that you actually might be doing something that is!  

As I was trying to decide where to start, I noticed that Nathan T. Carter (University of Georgia), Dev K. Dalal (University at Albany, SUNY), Anthony S. Boyce (Aon Hewitt), Matthew S. O’Connell (Select International, Inc.), Mei-Chuan Kung (Select International, Inc.), and Kristin Delgado (Select International, Inc.) won two awards in 2015—the Hogan Award for Personality and Work Performance and the Jeanneret Award for Excellence in the Study of Individual or Group Assessment for their work on personality measurement that was published in the Journal of Applied Psychology. I didn’t even know that was possible! It is—Nathan checked multiple times with the Awards Committee and the SIOP Administrative Office.

So, what are these awards all about? The Hogan Award for Personality and Work Performance is awarded for a published or unpublished paper or chapter that shows the highest potential to further the understanding of personality as it relates to work performance; similarly, the Jeanneret Award for Excellence in the Study of Individual or Group Assessment is awarded for work (published or unpublished) that shows the highest potential to further the understanding of individual or group assessment, especially when such assessment supports the creation of a diverse workforce.

Without further ado, let’s meet our award winning authors (unfortunately Dev was unable to join our call but he did send his responses via email) and learn what insights they have to share about the importance of blending science and practice to generate breakthroughs in our field that drive both forward in unexpected ways.

I started by asking what they were working on now. All are still working in areas related to personality and are looking for ways to extend their award winning research in other directions. Nathan is also currently interested in the moderators of the personality–performance relationship and the application of psychometric network analysis to measures of individual differences. Dev is now at University at Albany, SUNY, focused on judgment and decision making, measurement theory and application, and research and quantitative methods. Matt is exploring how to improve the accuracy of identifying faking as well as the prediction of safety behaviors. Anthony (aka Tony) is developing adaptive employee personality tests and is focused on researching and developing the next generation of cognitive ability tests. Kristin is working with Matt to develop scoring algorithms for identifying faking and creating new scales that reduce social desirability; she is also exploring different ways to test cognitive ability using simulations to reduce adverse impact. Mei-Chuan (or Mavis) is working on projects similar to Matt and Kristin and is working on expanding the taxonomy of counterproductive work behaviors and exploring the uses of big data.

How It All Started…

A summer reading group started by Mike Zickar on the MySIOP website. An interesting article (Le et al., 2011) suggested by Nathan found mixed results when testing for curvilinear personality–performance links although there is a strong theoretical rationale for such a relationship between conscientiousness and job performance. It questioned the inconsistent results in the literature and asking “what’s wrong with the research” rather than “what’s wrong with the theory.”

It turns out that 47% of studies that explored this curvilinear relationship found it, whereas 53% had not found it. Stark et al. (2006) proposed ideal point IRT as more accurate measurement model for assessing this relationship because it allows for curvilinearity. However, even with this measurement strategy, the authors did not find an increase in criterion-related validity.

Here’s where it became important that Nathan took the approach of asking “what’s wrong with the research” rather than “what’s wrong with the theory.” He likens this to physicists who, when confronted with results that do not match theory, first ask “what is wrong with my experiment?” rather than “what is wrong with the theory?” Thinking through the logic of ideal point, he realized that applying it would only change the rank order for the top 10–15%, but, in terms of testing for curvilinearity, that is really where we need the order to be correct. Could this be tested empirically using job incumbents? Thinking about the connections he made at SIOP and presentations he attended, he saw an opportunity to validate this measurement approach. He reached out to Tony at Aon Hewitt, who demonstrated a similar interest in this area through his presentations at SIOP, and Tony thought that there could be a way to leverage some work that he was doing to test this theory. Through a series of “someone who knows someone who knows someone” conversations, Nathan was then connected to Matt at Select International who was also interested in testing this relationship with job incumbents.

The “Aha” Moment

This project solidified that measurement models/IRT are a method to index how much of a trait someone has in the minds of the authors. Once you realize this, you begin to understand how we can use these scores to answer some really interesting questions about work behaviors. Many people think that IRT or measurement theory is a separate area of research from “substantive” research, but scoring of psychological scales has major implications for our research findings, and they can also be used to directly test substantive hypotheses.

Potential to Better Our Understanding

The lasting contribution to science is that they were able to demonstrate that the rank-order changes that result from using an ideal point model expose a curvilinear link between conscientiousness and job performance 100% of the time. This has direct implications for practice, which they were also able to show—selection based on ideal point scores results in more favorable objective hiring outcomes.

Further, these findings have spurred additional research and helped set the path for redefining personality item development and how we might approach identifying fakers and scoring related to conscientiousness differently.

What Made This Collaboration Successful?

They all agree that the success of this research is largely based on Nathan’s passion and persistence to testing this idea on job incumbents and his willingness to do as much of the legwork as possible to move the project forward. Many practitioners want to do everything they can to support building the science of our field but the lack the time to add it onto their daily work activities. Nathan’s commitment to making this as easy as possible for Tony, Matt, Kristin, Dev, and Mavis to contribute to this research. This was a perfect example of how science and practice can work together to solve an interesting problem—an academic who had an awesome research idea and reached out to practitioners who shared similar interests to see if they have data that could be leveraged to test the idea and/or interest in exploring it further.

Keys to Award Winning Research: Words of Wisdom from the Authors

1. Review the awards that are available. Many are self-nominations. If you’re doing something in one of those areas, nominate yourself!
2. Make connections at SIOP. If someone is presenting on something that you’re interested in or researching yourself, introduce yourself.
3. Leverage your network to find data! Reach out to practitioners to get data to test your hypotheses; when practitioners are willing to share data, everyone wins.
4. Make it easy for practitioners to support your research. Be willing to do the hard work; one reason practitioners don’t share data is that it’s time consuming, and it doesn’t affect the jobs of most practitioners, so academics need to be willing to jump in and do the hard work.
5. Be persistent in keeping conversation going with practitioners; the more that practitioners and academics continue to share ideas and learn from each other, the better.

Something Fun and Some Last Advice

Finally, I thought it might be fun to end the article with some interesting facts about the authors and their keys to success.

Nathan was just made SIOP historian and plays guitar to relieve stress. He called this project “me-search” because he is (self-admittedly) not very high in conscientiousness. When asked about publishing this research, he noted that this experience was formative, making him realize what it truly takes to successfully publish in a top journal. His advice: Build good relationships with colleagues because first and foremost you will make new friends and develop yourself intellectually. Plus, you never know when those relationships will pay off in terms of fruitful collaborations.

Dev and his wife got married in the faculty lounge of the Psychology Department at Bowling Green State University, officiated by the department head, Mike Zickar. (I went to grad school with Mike. It is a seriously small world!). His advice: For graduate student: Take class research proposals seriously. These proposals can lead to thesis, dissertation, and/or publishable projects if you put in good work. Capitalize on the fact that it’s an assigned paper to build a research idea. For everyone: Be attentive to reviewer’s and editor comments. Reviewers and editors want to improve the paper, so approaching comments as constructive and helpful rather than combative really can make your paper better.

Matt has just published his first fiction novel (available on amazon.com), The Painter of Time! Color me curious, but I will be checking it out! In his remaining spare time, he plays golf and paddle boards. His advice: Sample as much of the field as possible; explore everything, and try to learn as much as possible about the different discipline areas. It will only make you better at the area you decide to pursue professionally.

Tony was a factory worker, pizza delivery person, house builder, and data entry clerk at a bank during one very busy summer, and he brews beer. His advice: Learn effective project management skills because effective execution is one of the major contributors to success as a practitioner

Kristin is currently living in Dubai, United Arab Emirates. She does Olympic-style lifting and has clean and jerk lifted 67 kilos, which is quite a bit more than she weighs! Her advice: Start networking as early as possible in your career, and never underestimate the value of statistics. If you have strong knowledge of statistics, you can go almost anywhere, and you can certainly have a significant amount of influence over decisions being made within your organization.

Mavis plays many different instruments—her favorite being the yangqin (pictured here), and she was trained as an opera singer and sings karaoke to relieve stress. Her advice: Learn how to talk as a “normal” person—that is learn to translate your work for a nontechnical audience.

In the future, we hope to showcase multiple award winners with a focus on those that won 2015 and 2016. If you have thoughts or suggestions for award winners to feature or have something that you’re specifically interested in learning about award winning research and practice, please let me know.  

Oh, by the way, this is probably the perfect time to remind you that the call for award applications is now open! For more information, visit: http://www.siop.org/siopawards/

References

Carter, N. T., Dalal, D., Boyce, A. S., O’Connell, M. S., Kung, M., and Delgado, K. M. (2014). Uncovering curvilinear relationships between conscientiousness and job performance: How theoretically appropriate measurement makes an empirical difference. Journal of Applied Psychology, 99(4), 564–586.

Le, H., Oh, I., Robbins, S. B., Ilies, R., Holland, E., and Westrick, P. (2011). Too much of a good thing:

Curvilinear relationships between personality traits and job performance. Journal of Applied Psychology 96(1), 113–133.

Stark, S., Chernyshenko, O. S., Drasgow, F., and Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology 91(1), 25–39.

Liberty Munson is currently the principal psychometrician and Assessment and Exam Quality lead at Microsoft. She is responsible for ensuring the validity and reliability of Microsoft’s certification and degree programs. Her passion is for finding innovative solutions to business challenges that balance the science of assessment design and development with the realities of budget, time, and schedule constraints.

Liberty loves to bake, hike, backpack, and camp—basically, if the sun is shining you’ll find her enjoying the great outdoors; if not, she’s in her kitchen tweaking some recipe just to see what happens. She has also been actively involved in editing Microsoft’s Cookbook to raise money for a local charity, FareStart, as part of Microsoft’s Give Campaign. She also just got a new mini schnauzer puppy, Apex!

Garett Howardson is currently an assistant professor of Psychology at Hofstra University in Hempstead, NY, and an adjunct professor of I-O Psychology at The George Washington University in Washington, DC. In addition to his academic responsibilities, he regularly consults with the American Council on Education and the U.S. Army Research Institute (among others) on a variety of topics. 

Most of his work focuses on quantitative, psychometric, and/or computational issues to better understand the psychology of modern, technical work writ-large (e.g., aerospace technicians, computer programmers). 

Garett is also an avid computer geek. In fact, he has a degree in computer science, which he avidly applies to his research and consulting in pursuit of one deceivingly simple goal: better integrate I-O psychology and the data/computational sciences to understand work.