The Bridge: Connecting Science and Practice

Tracy Kantrowitz and Eden KIng

The Bridge: Connecting Science and Practice

Exploring the Gap  Between I-O Trends  and the State of Research

Column Editors: Craig Wallace, Oklahoma State University; Lynda Zugec, The Workforce Consultants; and Mark L. Poteet, Organizational Research & Solutions, Inc.

The purpose of the “Bridge” column is to provide an additional conduit, building upon SIOP’s current efforts, for connecting science and practice. The column strives to accomplish this by publishing various types of article content on the subject of science and practice integration; for example, case studies of effective practice; discussions between scientists and practitioners on a relevant topic, reviews of the key scientific and practical implications of a topic area; summaries of latest research findings and their implications for practice; summaries of key practice issues and their implications for needed research; and/or, calls for research to help practitioners overcome challenges associated with effective practice (please see Poteet, Zugec, & Wallace, 2016, for more background information on the column).

 Rapid developments in science and practice make it difficult to stay informed. In the past year, a variety of articles have been published in mainstream news outlets on topics such as mobile games for selection, the use of big data in HR, the state of performance management, and whether high-potential talent presents risks for organizations, without a firm perspective on the state of the science.  Without this, mainstream articles like these leave us questioning whether they are informed at all by I-O research.If asked about these topics by your CEOs, how equipped do you feel to speak to the science and research?

Despite our best intentions to stay informed, I-Os are often too busy doing our jobs to pick up or contribute to the latest scientific journals. This article fulfills these best intentions by boiling down some of the most important findings of recent years on trending topics in I-O. We highlight some examples of research/benchmarks that represent the state of the science and provide some commentary on how robust the research is on these topics.  We also provide some ideas for how to stay current and thoughts on research/practice partnerships to shore up research on under-researched areas.

The Intersection of Trends and Research

We presented a preconference workshop at the annual conference this year on the intersection between trending topics in I-O and the state of the research. The top 10 workplace trends lists that SIOP has published for the past 2 years (http://www.siop.org/article_view.aspx?article=1467, http://www.siop.org/article_view.aspx?article=1343) provided us with a starting point for understanding what’s top of mind for SIOP members and examining the extent to which these trends are supported by current research. In this regard, these lists are useful barometers for measuring ourselves as a science/practice community on how effectively we have pursued research programs that keep pace with trending topics. When we examined these lists, we noted practice trends and research findings that have synergies and misalignments. Here we offer a few examples of each to illustrate ways in which the relationship between science and practice might be maximally leveraged.

Opportunities

Mobile assessment. Mobile technology pervades nearly all aspects of existence and mobile-delivered assessment is no exception.  In addition to minimizing costs and increasing speed and convenience for administrators and test takers, mobile assessment may also expand the size and composition of candidate pools. From a scientific standpoint, several questions have loomed large in terms of the psychometric considerations of mobile assessments. In general, research indicates that non-cognitive assessments show little evidence of score degradation for tests completed on mobile devices (e.g., Illingworth Morelli, Scott, & Boyd, 2014). Less research has been done on cognitive assessment and the research that exists has not produced consistent results. The evidence is also mixed regarding perceived fairness of mobile assessments. Kinney, Lawrence, and Change (2014) and Gutierrez and Meyer (2014) found no meaningful differences in perceived fairness across device type on noncognitive assessments. King Ryan, and Kantrowitz (2015) indicated candidates found it easier and felt they were given a better chance to perform when completing assessments on a computer as opposed to a mobile device. Early mobile assessments attempted to replicate computer based designs on smaller screens. The current momentum is pointing to strategies to redesign test questions that make better use of mobile screen real estate, use alternate item types outside of multiple choice questions, and allow for alternate ways of inputting responses (e.g., swiping).

Telework. Questions about telework pervade daily and strategic personnel decision making. Managers need to know for whom, how, and when telecommuting might be effective. Luckily, scientists have studied this issue in depth. Allen, Golden, and Shockley (2015) recently summarized this literature and offered specific recommendations. For example, Allen and colleagues concluded that there are both positive and negative outcomes of telework, and that some people (e.g., those skilled in self-regulation) engaged in some tasks (e.g., those tasks that are independent, rather than dependent) can telework effectively.

Other trending topics with burgeoning research programs include work-family integration, the use of social media for employment decisions, and multi-generation research. Much more work remains but it is promising to see studies published in prominent outlets on these topics.

Gaps

Assessing for potential. Organizations are increasingly interested in assessing for long term potential particularly for leader roles. To date, however, assessing for high potential has been fraught with more questions than answers. Most notably, defining high potential has been challenging as the "potential for what" question has been challenging to define. There is also a gap with understanding the most effective tools and methods of identifying and developing high potential talent. The literature provides some helpful benchmarks for understanding how organizations approach these issues, but little empirical research shows the effectiveness of various methods for achieving intended outcomes of preparing a new generation of leaders. Even more troubling is some evidence indicating that many organizations misidentify high-potential talent because they focus on which employees are performing well today. Shoring up the research on this topic and conducting longitudinal studies to track high-potential talent over time is clearly needed to address the gap between practice and science.

Diversity management. Another area where there is a gap between what practitioners need and what researchers are publishing is in the area of diversity management. Indeed, in her role as acting director of the Office of Personnel Management, Beth Cobert asked federal employees to combat unconscious bias. One of the primary strategies federal and private companies use to increase diversity and inclusion is diversity training. Yet a quick search for the term “diversity training” in the Journal of Applied Psychology yields a single hit in the 100 years of the journal’s publications. Clearly, there is a misalignment between what organizations are doing and what is being published in I-O.

Ideas for How to Stay Current

Several resources are available to help stay connected to the latest research topics and findings. Most journals offer table of contents alerts, which are useful for keeping tabs on research topics and gaining access to the specific articles of most interest to you. SIOP also offers a number of research resources. For a nominal fee, the SIOP Research Access (SRA) makes multiple EBSCO databases available to members. SIOP committees also publish several excellent white paper series. For example, the SIOP/SHRM Science of HR series, published by the Professional Practice Committee, has been very active in bringing the science of I-O to the HR community. These papers distill the latest and most impactful findings in clear and useful ways. The preconference workshops at the annual conference also bring the most current and pressing topics of interest to members to life through interactive learning experiences. On the flip side, several resources exist for researchers to become familiar with topics of interest to practitioners. The annual SIOP conference and Leading Edge Consortium highlight the latest thinking and issues facing practitioners. Practitioners are also very tuned into world and economic events that impact the organizations they work with, so staying plugged into business journals and periodicals like Harvard Business Review and Wall Street Journal can provide inspiration for timely research that should have direct application in practice.

Research/Practice Partnerships to Bridge Gaps and Exploit Opportunities

An ideal solution to build synergies between science and practice is through direct partnerships. This might take the form of sharing archival data, collaborating on new data collection opportunities, or directly engaging academic experts to inform practitioner needs. For instance, we have forged partnerships oriented around mutually interesting and beneficial research. Some research is better suited to data collection in academic settings that can’t be feasibly done with organizational research partners (e.g., multisession studies to examine the reliability of a new measure over time). Likewise, academic–practitioner partnerships can help provide access to field samples that academic researchers may not normally have access to; these opportunities can help academic researchers generalize lab findings to the field and present improved opportunities for publication.  Research partnerships can also be done exclusively in the practice domain. Consulting organizations may work with client organizations to participate in product development research that results in information to the organization about the efficacy of a new product while also providing the consulting organization with necessary data to develop and refine its new product. The precise way that partnerships are built is less important than building them in the first place; only by talking to each other can we learn where gaps and opportunities exist.

Calling Potential Contributors to “The Bridge: Connecting Science and Practice”

As outlined in Poteet, Zugec, and Wallace (2016), the TIP Editorial Board continues to have oversight and review responsibility for this new column. Members of the Professional Practice Committee (PPC) and Scientific Affairs Committee (SAC) will identify content areas and format, secure authors and column participants, and assist with and review members’ contributions to the column. Although PPC and SAC members will actively recruit column contributors, we invite interested potential contributors to contact us directly with ideas for columns. If you are interested in contributing, please contact either Lynda (lynda.zugec@theworkforceconsultants.com) or Craig at (craig.wallace@okstate.edu).

References

Allen, T. D., Golden, T. D., & Shockey, K. M. (2015). How effective is telecommuting? Assessing the status of our scientific findings. Psychological Science in the Public Interest, 16, 40-68.

Gutierrez, S. L. & Meyer, J. M. (2014, May). The mobile revolution: Measurement equivalence and mobile device administration. In T. Kantrowitz & C. M. Reddock (Chairs), Shaping the future of mobile assessment: Research and practice update. Symposium presented at the 29th Annual Conference of the Society for Industrial and Organizational Psychology, Honolulu, HI.

Illingworth, J., Morelli, N., Scott, S., & Boyd, S. (2014). Internet-based, unproctored assessments on mobile and non-mobile devices: Usage, measurement equivalence, and outcomes. Journal of Business and Psychology, 30, 25-34.

King, D., Ryan, A. M., & Kantrowitz, T. M. (2014, May). Mobile and PC delivered assessments: Comparison of scores and reactions. Paper presented at the 29^th Annual Conference of the Society for Industrial and Organizational Psychology, Honolulu, HI.

Kinney, T.B., Lawrence, A., & Change, L. (2014). Understanding the mobile experience. Data across device and industry. In T. Kantrowitz & C. M. Reddock (Chairs), Shaping the future of mobile assessment: Research and practice update.  Symposium presented at the 29th Annual Conference of the Society for Industrial and Organizational Psychology, Honolulu, HI.

Poteet, M. L., Zugec, L., & Wallace, J. C. (2016, April). The bridge: Connecting science and practice. The Industrial-Organizational Psychologist, 54(4), 18-23. Retrieved from http://www.siop.org/tip/april16/pdfs/bridge.pdf