What is I-O?

Industrial-organizational (I-O) psychology is the scientific study of the workplace. Rigor and methods of psychology are applied to issues of critical relevance to business, including talent management, coaching, assessment, selection, training, organizational development, performance, and work-life balance.

Practice Perspectives: Science–Practice Gaps in
Industrial-Organizational Psychology: Part I: Member Data and Perspectives

Rich Cober, Rob Silzer, Anna Erickson¹

¹ Author affiliations:  Rich Cober–Marriott International, Rob Silzer-HR Assessment and Development & Baruch-CUNY, Anna Erickson–Questar

Executive Summary

The recent SIOP Practitioner Needs Survey explored the possible “gaps” that might exist between the science and practice of industrial-organizational psychology.  Survey responses suggest that gaps do exist in a number of areas.  Possible reasons for these gaps include:

• practice may underutilize available science 
• science may undervalue innovations in practice 
• science may not produce research findings that are relevant to practice 
• practice might not provide sufficient opportunities to research relevant issues 

This article (Part I of a two-part article) presents member survey results related to science–practice gaps and explores the details around those perceived gaps.  In addition, a group of experienced SIOP members provide their perspectives on the survey results.  Part II (in the next TIP) will summarize member recommendations on the steps that can be taken to address these gaps and to increase science–practice collaboration.  

Introduction

The gap between I-O science and practice has long been discussed as a significant issue in our field, and SIOP tried to bridge the gaps by regularly encouraging conference forums that bring researchers and practitioners together.  Successful advances in other disciplines often depend on an initial incubation and testing of ideas in either a research environment or in practice efforts before they become widely studied and applied.  To explore this topic, the Practitioner Needs Survey included a question that asked where such “gaps” actually exist.

In 2008 the SIOP Professional Practice Committee conducted a membership survey to better understand practitioner views and needs on a variety of professional issues (Silzer, Cober, Erickson, & Robinson; 2008).  The survey was sent to all members, with an overall response rate of 36%.  Respondents were divided into four practitioner groups based on self-reported percent of work time devoted to being a practitioner (as opposed to time being an educator or scientist/researcher):

• Full-time practitioners (n = 612, indicating 70% or more time as a practitioner)
• Part-time practitioners (n = 101, indicating 21%–69% of time as a practitioner)
• Occasional practitioners (n = 193, indicating 1%–20% of time as a practitioner)
• Nonpractitioners (n = 99, indicating 0% of time as a practitioner)

Survey Results: Perceptions of the Science–Practice Gap

Survey respondents were asked to indicate: In which areas do you find the biggest gap between the available science/research on a topic and actual organizational practice in your work? Respondents evaluated the gap between science and practice in 26 content areas identified during the survey development process to reflect both research and applied interest areas in our field.  Respondents were asked to indicate whether they felt that a gap existed in the area by identifying whether (a) practice was ahead of science/ research, (b) science/research was ahead of practice, or (c) little or no gap exists.  Respondents were also allowed to indicate do not know if they did not have the knowledge or experience for answering in a particular area. 

Table 1 summarizes the responses to this question. The percent of total survey respondents that selected the do not know, found in the fourth data column of Table 1, provides some insight into which content areas are more or less relevant to SIOP member activities.  

As seen in Table 1:

• In 14 of the 26 areas, practice is seen as ahead of science/research by more than 50% of respondents (who chose one of the first three alternatives). 
• In another five areas, practice is seen as ahead of science/research by smaller percent of respondents (36%–49%).
• In five areas, science is seen as ahead of practice (by slight to substantial margins). 
• In just two areas the largest proportion of respondents indicate that little or no gap exists. 
• In seven areas (from organizational culture through assessment), responses suggest potential convergence of science and practice given the balance of responses (and with most responders having opinions, suggesting high familiarity in these areas). 
• In five areas more than 40% of our respondents indicated did not know with regard to a gap. It is likely that these areas, such as employment branding, labor relations, and litigation support, are not widely part of either research or practice activities.  

An evaluation of the content areas receiving the highest percentage of practice ahead responses (toward the top of the list) suggests that these areas tend to be:

• Hands on practice areas such as consulting, coaching
• On the organization side of I-O psychology such as strategic planning, organizational development
• Core areas of human resource practice such as succession/workforce planning, talent management, employment branding, HR technology, labor and employee relations, and employee recruitment  

An evaluation of the content areas receiving the highest percentage of science/research ahead responses (toward the bottom of the list) suggests that these areas tend to be:

• Measurement oriented such as measurement and statistics
• On the industrial side of I-O psychology such as job and work analysis and selection/staffing

Finally, there was the group of seven content areas that receive more balanced responses. These areas include organizational culture, performance management, cross-cultural issues, competency modeling, training and development, employee engagement and attitudes, and individual assessment/ assessment centers. In these areas a mutually beneficial connection or convergence might exist between science and practice.

In many of the areas found in Table 1, there is some response agreement across the four practitioner groups on which sector is “ahead” in an area.  In these areas:

• In practice ahead areas, practice knowledge, experience, and innovation) might have the most influence on handling an issue in organizations (though this does not necessarily mean that practice innovations are adequately researched).
• In science ahead areas, science (laboratory studies, empirical field research, meta-analyses) might have the greatest influence (though this does not necessarily mean that scientific findings are put into practice).
• If many respondents choose little or no gap, it might mean that science is being utilized in practice and that practice innovations are being researched. 

Table 2 reports response distributions from only those areas where differential response patterns exist across practitioner groups.  Such patterns were found in 9 of the 26 areas.  Data illustrating the differences are highlighted in bold font. The biggest response pattern differences are found in performance management, organizational culture, and competency modeling. 

Perspectives of SIOP Members

To further understand the implications of these results, we invited 12 SIOP members, whose professional experience bridges science and practice, to respond to several questions related to the survey data. Here is a summary of their responses to the first question (Part II of this article will provide a summary of responses to other questions):

Based on your experience, do the results in this area of the Practitioner Needs Survey surprise you? Why or why not?

Generally, the findings were not surprising to our SIOP members. Below are their reasons: 

• Topics with measurement foundation (i.e., methods per se, selection, attitude measurement) are topics where academics historically have made and continue to make central contributions. Note that those are areas where academics have a good chance of doing work that accomplishes the dual goals of (a) contributing to scientific psychology and (b) contributing to practice. Both goals loom large for academics.
• A significant issue in our field is that one can become an I-O psychologist and not practice. Those that have only learned about topics, and never really done a job analysis, developed a test, or dealt with a hostile client, will have divergent perspectives from those practicing in organizations. This data provides another data point for the need of some clear sort of certification, which includes a knowledge and practice component, for both practitioners and academics in our field. 
• Reward systems affect this issue significantly. Scientists can afford to study what they are interested in as long as it is publishable.  Publishable, quick turn-around work may not be the kind of work that will truly benefit practice.  However, there is little incentive for scientists to tackle some of the more nebulous applied topics (areas at the top of Table 1) unless they have an intrinsic interest.  Practitioners survive based on management’s willingness to pay.  Unfortunately, this drives work that may not have optimal scientific rigor. 
• Although grounded firmly in our field’s scientific principles and body of research evidence, much of professional practice requires a degree of art to sufficiently address business problems.  The practice areas that received the highest percentages for “practice ahead” are those that are the least “studyable” with I-O methodologies (e.g., large sample statistics with highly coveted small p-values); and the culture of SIOP related to professional practice is reflected in another section of the survey that found SIOP leadership does not fully understand the issues and context within which our practitioners operate. The gap ultimately exists because full-time practitioners and nonpractitioners have very different understandings of what professional practice is/entails.
• If there was any surprise, it was the relatively high level of agreement among the different response categories.  One may have thought there would have been some more entrenched positions on this topic, but these results are a testament to the applied nature of our field.  As more I-O psychologists join organizations, there is a growing realization and respect for the advances that are made in applied settings that may drive or in many cases outpace research. 
• We give ourselves credit for more collaboration between science and practice than typically gets noted. There are a number of studies that are born from applied data sets or whose implications directly affect the way a practitioner may choose to design an intervention/project. As Anthony Rucci stated in his 2008 SIOP Keynote “It is only where science and practice converge that I-O psychology really makes its full contributions.” We have many examples of that: Our struggle as a field may be in effectively sharing and disseminating those examples.

Thoughts on What These Results Tell Us

Why Is There a Science–Practice Gap?

To fully understand any science–practice gap, we need to ask why the gap exists.  There are several possibilities.

1.  Different reward systems.   One hypothesis is that limited connection between the practice and science may be due to the differential reward systems for scientists and practitioners. Those that pursue the science must focus on building research programs that can yield a large number of studies publishable in top-tier journals. Those who practice must focus on building useful and feasible solutions for organizations that are valued (and paid for) by the organization. 

2.  Normal evolution of the field.  The gap might just reflect the current state of our field and suggest an opportunity for the further evolution of I-O psychology. Areas such as job analysis and selection are foundational for much of the work done in organizations.  As I-O psychology (and human resource management for that matter) evolves and innovates in practice, new areas will emerge for our science to investigate. 

3.  Limited organizational resources.  Perhaps organizations are unwilling to pay for interventions and solutions that require adherence to research principles and findings. These approaches may be perceived as too expensive or unnecessary to address a problem. Organizational constraints and resources often dictate what a solution will look like, even when the I-O practitioner makes cogent arguments about the ROI and effectiveness of more rigorous approaches.  Key decision makers in organizations often do not value the benefits of scientifically sound interventions.

4.  Lack of relevance.  Practitioners may not be leveraging our science because of the nongeneralizability of research findings, a lack of relevance to real-world problems, and a lack of access to literature summaries by topic. Practitioners often face complex contextual issues, strategic objectives, and executive demands that require uniquely tailored solutions that are not addressed in the literature. On the other hand, researchers may not sufficiently value the innovative ideas and leading-edge efforts by practitioners.  They may not see relevance of practice activities to their research interests or efforts. 

5.  Science is hard to apply.   In some areas the science may have evolved in an area beyond what practice is able to absorb or apply.  Management may think that research approaches require unnecessary steps and delays.

6.  Insufficient time or motivation by researchers.  Researchers may not have the time to focus on key issues faced by practitioners in organizations, and those issues may not be of personal interest to the researchers.

7.  Insufficient time or motivation by practitioners. Practitioners may not take sufficient time to discover the relevant research on an organizational issue or may not be interested in trying to see the relevance of key research findings, particularly when they are under significant demands to add value and quickly produce work products and services.  

8.  No need to close the gap. There are some areas, such as measurement, where there is, and may always be, a profound gap between the methods used by science and the methods employed by practitioners. In a sense, the gap provides a healthy opportunity for science to advance the profession by experimenting with new methodologies or creating nonintuitive insights. Similarly, practice may continue to serve as an innovation lab for generating new approaches to emerging issues. 

Moving Forward

The time for moving the field to greater collaboration is now.  Economic downturns provide opportunities for innovation and entrepreneurship. We are living and working in a time where partnerships between practitioners and researchers can be mutually beneficial for both product/service quality and economic reasons.  We should seize this opportunity. In Part II of this article SIOP members provide recommendations on steps that can be taken to address these gaps.  

At our 2008 SIOP conference, Tony Rucci said the core purpose of I-O psychology today is “to support the dignity and performance of human beings, and the organizations they work in, by advancing the science and knowledge of human behavior.” It is incumbent on our entire professional community to capitalize on these ideas and work toward shared goals in order to provide lasting value and support the continuous evolution of our profession and its noble purpose.

Part I lays the groundwork of where gaps are perceived to exist today. Part II will present recommendations on how scientists and practitioners can increase their collaboration to facilitate science–practice convergence.

References

     Rucci, A. (2008, April). I-O psychology’s core purpose: Where science and practice meet. Keynote address presented at the 23rd Annual Conference of the Society for Industrial and Organizational Society, San Francisco, CA.
     Silzer, R. F., Cober, R.T., Erickson, A., & Robinson, G.  (2008, October).  Practitioner needs survey: Final survey report.  Society for Industrial and Organizational Society.  Bowling Green. OH.  (See full report at http://www.siop.org/Practitioner%20Needs%20Survey.pdf)

Acknowledgements

We would like to acknowledge the contributions of the SIOP members  who provided their reactions and insights including (listed alphabetically) Cristina Banks (Lamorinda Consulting LLC); Dennis Doverspike (The University of Akron); David Dye (Booz Allen Hamilton); Scott Erker (Developmental Dimensions International); Kurt Kraiger (Colorado State University – current SIOP President); Cindy McCauley (Center for Creative Leadership); Matt O’Connell (Select International); Sara Rynes-Weller (University of Iowa); Paul Sackett (University of Minnesota); Nancy Tippins (Valtera); William Shepherd (Huntington Bank); Vicki Vandaveer (Vandaveer Group).