Home Home | About Us | Sitemap | Contact  
  • Info For
  • Professionals
  • Students
  • Educators
  • Media
  • Search
    Powered By Google

Good Science - Good Practice

Jamie Madigan
National Archives

Tom Giberson
Oakland University

As with many other columns in this issue, this edition of Good Science–Good Practice will focus in part on some of what went on at the annual SIOP conference in Atlanta, Georgia. Unfortunately only one of us (Jamie) was able to make it to the conference, so you’ll still get a smattering of our traditional examination of research from other outlets. Either way, we’ll once again focus on things that we think exemplify the scientist–practitioner model.

First up was a SIOP panel discussing diversity entitled “Let’s Talk: Bridging the Gap Between Diversity Researchers and Practitioners” hosted by Bernardo Ferdman and featuring Ny Mia Tran, Matthew J. Dreyer, C. Douglas Johnson, Kecia M. Thomas, Matthew S. Harrison, Wendy R. Reynolds-Dobbs, and Melanie Harrington as panelists. Diversity and inclusive climates are, judging by this year’s SIOP program, still popular among employers, but as the members of this panel pointed out, the research that’s happening in the labs and universities sometimes lags a bit behind practical issues or even follows a separate tangent. Several of the discussants noted that the goals of the practitioners in diversity often stretched into areas different from researchers, and they often make calls (implicit or otherwise) for new areas of research that academics don’t immediately pick up on. Often the latter are too busy doing things methodically and at the traditional researcher’s pace in order to work the academic reward system. Some reasons for this disconnect included the fact that academics often try to wring as much as they can from a given dataset instead of constantly pursuing new data and the fact that academics and practitioners aren’t connected to each other via the same journals and conferences. Instead of wading through (not to mention waiting for) scientific journals, many diversity specialists are forced to reach for whatever aid is at hand because their problems are immediate and need satisfaction right now.

As many of you are probably thinking, this is not exactly an issue unique to the diversity field. The same criticisms (and justifications) can be leveled at almost any area of I-O psychology. Academics in any area often favor models and theories, whereas their practitioner counterparts seek out checklists, as one presenter said. But the feeling at this panel was definitely that the issue is exacerbated because diversity initiatives are often so high profile and so political in nature. It doesn’t help that in the context of business, diversity is often viewed as a problem to be solved not a business strategy. “Oh, we don’t have a problem,” one presenter said, quoting a skeptical client, “our numbers are good.”

So what needs to be done? Beyond the obligatory call for more collaboration, the panel members mentioned that practitioners need to stand up and take their share of the responsibility, too. They need to look for opportunities to stop oversimplifying out of habit and expediency, and they need to address problems that academics are interested in solving—things beyond those that rely on simple headcounts as the dependent variable.

Speaking of which, also at SIOP was one of the more entertaining and practical presentations I’ve been to in a while. Entitled “Successful Field Experiments: Getting In, Getting the Data, and Getting Published,” the panel was chaired by Susan M. Kochanowski and featured entertaining and practical lectures by such luminaries as Charles Seifert, Gary Yukl, Dov Eden, and Gary Latham. And true to its premise, the talk offered pithy but dramatic pointers on how you can meld good experimental design with an applied workplace setting.

Several common themes spread across all the presentations. First, don’t compromise on your experimental design. You may not be able to keep it as tight as you could in a lab, but that’s no excuse to throw up your hands and settle for garbage. The way to hedge your bets in the uncertain world of field research is to keep things as simple as you can and collect as little data as possible in order to test the model or hypotheses at hand. Then stop, at least for that study.

A second theme for the panel discussion dealt with how to handle the stakeholders and decision makers. Some good advice was given in how you frame the experiment—namely that you don’t call it an “experiment” or “manipulation” at all, but rather refer to it as a bit of consulting and intervention. Offering the manipulation to the control group after the meat of the experiment is done is a good way to reinforce this idea, keep the client happy, and pacify concerns from human subjects boards.

Third, communication of a reliable and understandable nature was also said to be critical. Avoid jargon at all costs, but prepare simple explanations of things like randomization and effect sizes in case you need them. And always make sure the client understands what benefits they are getting out of the deal. The dependent variable (e.g., absenteeism, performance) is what they’re interested in not your nifty experimental manipulation.

Finally, there was some lively conversation around how to get the data published, which is the end goal of many if not all such projects. Unfortunately you’re often at the mercy of journal reviewers and/or editors who turn up their noses at field research, which points to the previously stated priority on true experimental designs over quasi-experiments. The presenters recommended targeting outlets that you know are willing to consider field research and to anticipate common criticisms like small samples, questionable external validity, and focusing on hypotheses more than overarching theory.

After reading Jamie’s account of the diversity session at SIOP, I was interested in finding a recent diversity-oriented study that might have some practical applications. One such piece from the Journal of Business and Psychology by Herdman and McMillan-Capehart (2010) brings together several key variables including diversity initiatives, organizational-level outcomes, as well as the notion of creating and managing a “diversity climate.”

Herdman and McMillan-Capehart (2010) suggest that the extant diversity research focuses primarily on the main effects of diversity and various performance outcomes without much attention toward intermediary variables such as diversity climate. Gonzalez and DeNisi (2009) define diversity climate as the “aggregate perceptions about the organization’s diversity-related formal structure characteristics and informal values” (p. 24). Thus, the idea of diversity climate goes beyond the formal diversity policies, practices, and interventions within an organization and gets to employee perceptions of the organization’s true support of diversity. Why care about diversity climate? Past research has demonstrated a wide variety of important outcomes, including turnover intentions (McKay, Avery, Tonidandel, Morris, Hernandez & Hebl, 2007), commitment, job satisfaction (Hickes-Clarke & Iles, 2000), and others.

The authors investigated five hypotheses, each testing potential antecedents to a diversity climate. The first two hypotheses tested potential tangible antecedents of diversity climate: the existence of diversity programs and management-team heterogeneity. Thinking about these first two potential antecedents, the first suggests what an organization says it does and believes, and the second indicates what it actually does and believes with regard to diversity. The third hypothesis suggests that managerial team heterogeneity will moderate the relationship between the existence of diversity programs and diversity climate. The fourth and fifth hypotheses test the relationship between “managerial relational values” and the existence of diversity programs (the greater the relationship orientation of management, the more likely the existence of diversity programs), as well as managerial relational values as a moderator of the relationship between diversity programs and diversity climate (i.e., the more managers actually value relationships, the more likely they are to reinforce in practice the practices and values underlying diversity programs).

This study was conducted with 163 hotels within the United States and relied on a balance of self-report data (e.g., diversity climate, relational values) and objective measures (e.g., race/ethnicity within levels of management). The results suggest several important considerations for practice. First, simply providing diversity programs does impact perceptions of a diversity climate. Interestingly, management diversity did not have a main effect on perceptions of a diversity climate. However, both management team ethnic composition and supervisor relational values moderated the relationship between diversity programs and perceptions of a diverse climate. In other words, greater management diversity and higher levels of supervisor relational values both increase the effect of diversity programs and perceptions of a diverse climate. Finally, managerial relational values were positively associated with an organization’s adoption of diversity programs.

This study is interesting and useful for organizations interested in making positive change in support of diversity and efforts to reap the benefits of a diverse workforce. First, organizations that provide diversity programs can make some gains in terms of perceptions of a diversity climate. Second, in order to enhance the creation and development of a diversity climate, organizations should ensure that higher levels of the organization reflect the diversity its programs purport to support. Finally, hiring and developing managerial relational values provides yet another opportunity to better support the creation of a diversity climate.

American Psychologist recently dedicated an entire issue to diversity and leadership. I will briefly review a piece by Ayman and Korabik (2010), in which the authors identify several areas where leadership theories seem to ignore or avoid issues of diversity. Hopefully, some of the authors’ suggestions overlap with practitioners’ call for more practical and timely diversity-related research.

Ayman and Korabik (2010) suggest that in today’s global, diverse organizations, theories of leadership that do not include diversity are of limited value. In the quest to understand universal leadership truths, gender and culture are generally ignored; yet assuming simply a “leader” and ignoring important factors such as gender and the unique experiences and assumptions of individuals having various cultural/ethnic identities limits the generalizability of leadership theories.

The authors contend that most leadership theory and research utilizes an etic-oriented approach built upon the gender- and culture-based assumptions of the scholar. Given the historical dominance of White men in academia (and business), it is likely that most theories and models of leadership implicitly assume the White male leader. The authors further contend that an emic-oriented approach, built upon the gender and cultural identities of diverse leaders, is necessary to truly build relevant leadership theories.

Ayman and Korabik (2010) summarize a variety of research that challenges the assumption that leadership is a universal construct. For example, some scholars have found that the traits identified as “leadership traits” vary across cultures. Similarly, gender-based differences within the Big Five personality factors impact the perception of leadership between men and women (Eagly & Carli, 2007). Some research has suggested that women perceived as more androgynous (high instrumentality, high expressivity) are less likely to suffer cultural biases to which men are less susceptible.

The authors suggest that the bias toward (White) men in leadership positions furthers the gender and cultural biases inherent not only in leadership research but also in practice. As an example, competency-based selection systems typically start with developing competency profiles from existing employees and leaders. To the extent that an organization’s current leaders and workforce are composed of men, for example, the more the competency profile will reflect that group’s implicit assumptions about leadership. As the authors point out, simply because an organization’s leaders have been historically male says more about the organization’s assumptions about leadership and less about the potential for women to be as (or more) effective in the same roles.

Similar to their survey of the trait approach to leadership, the behavioral school suffers from the same biases. For example, women who exhibit more male-oriented behaviors are likely to be rated more negatively than their male counterparts (Eagly, Karau, & Makhijani, 1995). Meanwhile, the contingency approaches to leadership have not been evaluated across gender or cultures, calling into question the effectiveness of their use in managerial decision making in a diverse workplace.

Overall, this particular article reinforces several of the issues that practitioners raise with regard to those of us in higher education. As I-O psychologists, we are typically trained as “scientist–practitioners,” and that continues to apply whether we choose to work professionally primarily as scholars or practitioners.

We have different roles to fill in society that are often complimentary; however, many in higher education view the discipline of I-O psychology,  rather than practitioners in the field, as our “client.” In general, practitioners are interested in “what is” or “what works,” whereas most scholars are interested in understanding “why” things are the way they are or “why” things work the way that they do. Although not necessarily the immediate goal, understanding “why” often leads to deeper understanding, new lines of research, and practical applications.

All disciplines and fields of knowledge must continue to push the boundaries of knowledge without regard to potential short-term economic or practical gains. Compatibly, practitioners are in the best position to deploy emic-oriented applied research studies to solve diversity-related issues unique to their organization and have the training to do so. To bridge the gap goes beyond sharing journals—as journals by-and-large necessarily target different audiences. The best way to close the research–practice gap is within individual scholar–practitioner partnerships to solve issues at the organizational level—the proverbial “win–win.” The Herdman and McMillan-Capehart (2010) article summarized above is a great example of an applied study that brought scientific integrity to an issue of both theoretical and practical value to the host organization and, assuming generalizability, many other organizations. As scientist–practitioners we need to do more of this type of applied research while also respecting the different roles that scholars and practitioners play in society.


     Ayman, R. & Korabik, K. (2010). Leadership: Why gender and culture matter. American Psychologist, 65(3), 157–170.
     Eagly, A. H., & Carli, L. L. (2007). Through the labyrinth: The truth about how women become leaders. Boston: Harvard Business School Press.
     Eagly, A. H., Karau, S. J., & Makhijani, M. G. (1995). Gender and the effectiveness of leaders: A meta-analysis. Psychological Bulletin, 117, 125–145.
     Gonzalez, J. A. & DeNisi, A. S. (2009). Cross-level effects of demography and diversity climate on organizational attachment and firm effectiveness. Journal of Organizational Behavior, 30(1), 21–40.
     Herdman, A. O. & McMillan-Capehart, A. (2010). Establishing a diversity program is not enough: Exploring the determinants of diversity climate. Journal of Business and Psychology, 25, 39–53. 
     Hickes-Clarke, D. & Iles, P. (2000). Climate for diversity and its effects on career and organizational attitudes and perceptions. Personnel Review, 29, 324–345.
     McKay, P. F., Avery, D. R., Tonidandel, S., Morris, M. A., Hernandez, M., & Hebl, M. R. (2007). Racial differences in employee retention. Are diversity climate perceptions the key? Personnel Psychology, 60, 35–62.