Home Home | About Us | Sitemap | Contact  
  • Info For
  • Professionals
  • Students
  • Educators
  • Media
  • Search
    Powered By Google

Fostering Ethical, Relevant, and Rigorous Research

6/1/2016-

by SIOP Administrative Office

Members of Alliance for Organizational Psychology Agree on Solutions for Combating Unethical Scientific Conduct and Rebalancing Academic Rigor and Relevance

A Memorandum of Understanding on fostering ethical, rigorous, and relevant research was recently developed during the first Alliance for Organizational Psychology (AOP) Small Group Meeting.

The meeting, held in Zürich, Switzerland February 12 and 13, was chaired by Gudela Grote (representing EAWOP) and Jose Cortina (representing SIOP). More than 20 representatives of the three founding organizations of the AOP—International Association of Applied Psychology (IAAP) Division 1, EAWOP, and SIOP—were in attendance. The meeting was made possible through funding from SIOP, EAWOP, and ETH Zurich. .

Click here to read the Memorandum of Understanding.

The attendees of the small group meeting wanted to focus attention on the importance of improving practices in research. There are currently two very active debates in academia concerning (un)ethical scientific conduct on the one hand and (re)balancing academic rigor and relevance on the other. Agreement was reached that the most pressing problems that impede the conduct of ethical, rigorous, and relevant research in work and organizational psychology specifically, and in psychology and possibly other sciences more broadly, are the following:

1. Research quality, impact, and reputation are compromised by publication policies that

  • Place too little emphasis on cumulative theory building and testing, and by contrast place too much emphasis on empirical studies having to test novel theory
  • Place too little emphasis on real-world importance of the problems addressed in research
  • Encourage authors to produce Results sections that are flattering to Introduction sections

2. Common academic systems share some of the blame by incentivizing quantity (rather than quality) of individual output, which leads to an overloaded review and publication process and encourages unethical behavior (among those so inclined).

3. Academic systems promote theory novelty rather than major innovation and the use of a small range of scientific methods, both of which hamper research relevance.

4. There are few if any institutional mechanisms for influencing policy makers.

The group also reached agreement that some effective solutions to these problems already exist:

  1. Regarding publication policies, we can begin by redefining theoretical contribution and empirical contribution so that there is room for theory trimming, theory refining, constructive replications, intervention studies, null findings, and multidisciplinary and multimethod research.
  2. The burden on reviewers can be reduced by more desk rejections and by standardized reporting methods for empirical results.
  3. Academic departments should use multiple quantitative and qualitative assessment criteria, going beyond publication/citation counts based on fixed journal lists.

The group then agreed upon a detailed action plan, outlined fully in the Memorandum of Understanding.

"The Memorandum of Understanding marks an important step towards improving research practices in our field, leading to more meaningful and relevant scientific contributions and the necessary changes in how the academic system evaluates and rewards such contributions," Grote explained.

This topic continues the effort launched by Dr. Cortina during his term as SIOP president (2014-2015).  Cortina’s presidential theme, titled “The Revolution With a Solution: Culling the Madness From Our Methods,” focused on the scientific review process. Cortina argued that I-O psychologists have a review process that has been too ad hoc and driven by the wrong forces. (Watch Jose Cortina’s incoming and outgoing presidential addresses on SIOP’s YouTube page: incoming address at 46:27, outgoing address at 51:26)

"By requiring papers to make theoretical contributions, we guarantee that models never get tested more than once. By defining ‘theoretical contribution’ as ‘counterintuitive’, we incentivize the search for the bizarre rather than the search for the useful or the true. By defining ‘empirical contribution’ as ‘results that are flattering to hypotheses’, we incentivize HARKing, p-hacking, and what Ernest O’Boyle has termed the Chrysalis Effect whereby papers transmogrify from being unsupportive of most hypotheses to being supportive of them,” Cortina said, discussing the need for the small group meeting. 

A second, more practice-focused Alliance small group meeting is planned for late 2016 or early 2017.  AOP has also established the AOP Small Group Meeting Committee to identify future topics and meeting organizers. The committee includes Alex Alonso (Chair; SHRM, USA), Ioannis Nikolaou (University of Athens, Greece), and Sharon Parker (University of Western Australia).

The organizers of the Small Group meeting will also set up a mechanism for regular communication among the participants of the Small Group Meeting and progress reports on the implementation of the action plan.