Does Globalization Change I-O Research? Not That Much, So Far
Lauren J. Mondo and Allen I. Kraut
Baruch College
The field of I-O psychology is undergoing some major changes. For one, SIOP today has much more of a global membership than it did just 20 years ago, a fact that was reported in TIP recently (Kraut & Mondo, 2009). At the same time, we saw a dramatic increase in the share of journal articles authored by non-U.S.-based I-O psychologists. This shift raises major questions about the kind of research that would be done by this globalized base of I-O psychologists.
In this report we will look at some of the changes in the topics and samples used in the published research that have accompanied that shift. Are there more cross-cultural studies or perhaps more testing of traditional topics among cross-cultural populations? What topics would be most different? The results we found are not quite what we expected.
- In our prior report we noted that the total number of SIOP members has increased during the last 2 decades from 2,556 in 1991 to 3,945 in 2007, a gain of 54%. During that same time, the number of non-U.S. members has risen 10 times as fast, from 86 to 479, which is a gain of 557%. The trend continues, and in 2009 more than 26% of SIOP’s new professional members had addresses outside of the United States. (T. L. Vanneman, personal communication, January 22, 2010).
- At the start of 2010, 15% of SIOP’s nearly 4,000 members, or almost one in every seven SIOP members is based outside of the United States. (Student Affiliates are not included, only Fellows, Members, Associates and International Affiliates.)
- This rise in the number of non-U.S.-based SIOP members is accompanied by a large increase in non-U.S.-based authors and their articles in the leading journals in our field, Personnel Psychology and Journal of Applied Psychology (JAP).
- The number of articles in JAP with at least one non-U.S.-based author rose from 10% in 1967 to 43% in 2007. In Personnel Psychology, the comparable number rose from almost 4% to about 21% in 2007.
- There was also a sharp rise in the number of authors who are non-U.S. based. In JAP the number of authors goes up from 8% in 1967 to 33% in 2007. In Personnel Psychology the comparable number rises from 5% to 14% over that period.
Thinking about these findings made us curious about shifts we might see in the kind of topics researched and the samples used in these studies. Would there be a shift to different issues, perhaps more of cross-cultural research, and even refutations of earlier American-based findings? We had already noted (Kraut & Mondo, 2009) there was more collaboration among I-O psychologists from different countries. In JAP, articles by U.S.-only authors declined from 90% in 1967 to 57% in 2007. Collaborative articles, involving both U.S.-based and non-U.S.-based authors, rose from 0% to 20% over that time. In Personnel Psychology over the same period, articles by U.S.-only authors went down from 96% to 79%, and collaborative articles rose from 0% to 7%.
Furthermore, there was a greater use of samples that included non-U.S. groups and even some cross-national groups. In JAP, studies with samples that included or were made up of non-U.S. subjects went from 10% in 1967 to 36% in 2007. In Personnel Psychology over the same period, studies with non-U.S. samples rose from 0% to 28%. Still, the nagging and important topic that remains with us is what kinds of topics and areas are studied in the research published by non-U.S.-based I-O psychologists?
This concern is highlighted as extremely important by the recent research of Cascio and Aguinis (2008). They did a content analysis of all articles published in JAP and Personnel Psychology from 1963 to 2007 and found that most of the topics and areas studied over those 45 years had not changed very much in relative popularity. More important, perhaps, was their judgment that the research being published was not changing to make I-O more visible or relevant to current societal or organizational issues (such as human capital issues).
Although Cascio and Aguinis (2008) noted the increased number of non-U.S.-based authors in the top two I-O journals, they did no analysis of differences in the topics and areas studied by these two sets of authors. We felt such an analysis, even with our more limited sampling, would be a helpful follow-up to our earlier findings of an increased non-U.S.-based membership of SIOP.
Methodology
We decided to examine the topic distribution just for JAP because there were too few publications in Personnel Psychology (PP) by non-U.S.-based authors. More specifically, there were only two (12.5%) non-U.S.-based articles in 1997 and four (20.7%) non-U.S.-based articles in 2007 in PP.
Using the coding taxonomy and methodology developed by Cascio and Aguinis (2008), we content-coded all of the 210 articles from the 1997 and 2007 issues of the Journal of Applied Psychology.
To facilitate reporting, we used the top six categories as identified by Cascio and Aguinis (2008) along with a seventh category for all the articles not included in the top six categories. These six most frequent categories found by Cascio and Aguinis were (1) methodology/psychometrics issues, (2) work motivation and attitudes, (3) predictors of performance, (4) performance measurement/work outcomes, (5) human factors/applied experimental psychology, and (6) leader influences. In cases where an article fit with more than one category, we chose the category with the best perceived fit for the article’s content.
Table 1 displays the percentage distribution of topics for 1997 and 2007. Because the 1997 and 2007 JAP distributions were so similar we combined them in the following charts to gain stability. A chi-square test of independence confirmed that the topic distributions for 1997 and 2007 were not significantly different (
2 = 11.83, p > .05), which provides empirical support for combining the 2 years.
Findings and Discussion
Although we might expect the topic researched to vary with the geographic location of the authors, the results of our study do not support that prediction. Table 2 displays the topic distribution of the articles in our sample by author group.
By inspection, there does not seem to be any substantial difference in the topic distributions of the three author groups. (A chi-square test of independence was not possible because the necessary statistical assumptions were not met.) If anything, the topic distributions of the U.S.-based and the collaborative (both U.S. and non-U.S.-based) author groups seem more alike than the non-U.S.-based-only authors’ topic distribution. Compared to these two groups, the data for the non-U.S.-based-only authors suggest less interest in studies of methodology and psychometrics and predictors of performance and more interest in work motivation and attitudes as well as human factors and applied experimental psychology.
Nationality of Samples Does Shift
The results for the nationality of samples used in the studies we looked at are presented by author location in Table 3. Authors seem to favor “local” samples. That is, articles by non-U.S.-based authors use more non-U.S. samples, but articles by U.S. authors use more U.S. samples. Articles by collaborative (both U.S. and non-U.S.) authors have more cross-national samples than either U.S.-based-only authors or non-U.S.-based-only authors. (Though the pattern is clear, a chi-square test of independence could not be conducted due to unmet assumptions.)
One interesting finding is that collaborative authors (both U.S. and non-U.S.) use more U.S. samples (48%) than non-U.S. (32%) or cross-national (20%) samples. This could be due to the fact that U.S. samples are more easily available for research. It might also reflect greater influence by U.S.-based authors who collaborate with non-U.S.-based authors.
Out of all the articles by non-U.S.-based authors, more than 8 out of 10 include non-U.S. samples. Out of all the articles by collaborative (both U.S. and non-U.S.-based) authors, just over half used either cross-national or non-U.S. samples. Finally, out of all the articles by U.S.-based authors, less than 5% used samples that included people from outside of the U.S.
As shown in Table 4, the topic distribution for studies involving non-U.S. samples (including both non-U.S. samples and cross-national samples) is not vastly different than that for studies involving U.S. samples. A chi-square test for independence was conducted to examine whether the distribution of topic categories differed significantly for U.S. samples and non-U.S. samples. The Pearson chi-square value was 5.647 (p > .05), leading us to conclude that the distribution of topic categories did not differ significantly depending on the nationalities of the samples. However, it is worth noting that studies involving non-U.S. and cross-national samples seem twice as likely to investigate work motivations and attitudes than studies involving U.S. samples.
A Caveat on Generalization
Table 5 displays the distribution of topics for our smaller sample of 210 articles and that of Cascio and Aguinis’ (2008) larger set of over 1,200 articles. A chi-square test of independence found that these two topic distributions are significantly different (2 = 35.9, p < .01). So, although the 2 years of studies we examined are a substantial number, they may not be safely generalized to all of the 15 recent years in JAP covered by Wayne and Cascio.
Concluding Thoughts
Based on the studies we have described in this review, the growth of non-U.S.-based I-O psychologists has not made for much change in the topics studied in our field. This was not a finding we expected, and the conclusion may be illusory, as there could be many reasons for this seeming stability.
For example, the coding system we used for the content analysis may obscure differences in the work done by researchers from different nations. The broad “umbrella” topics used may mask important shifts in narrower subtopics.
It is also important to note that JAP, the premier I-O journal we looked at, publishes only about 5% of all the manuscripts submitted to it. Many of the articles not accepted by JAP wind up in other journals, and we did not look at those.
In fact, many non-U.S.-based I-O psychologists may prefer to publish in their own national journals, especially if they can do so in a native language that is not English. Such choices may make it easier to get local readers and make it more likely their submissions will be published. Anecdotally, we have heard that academics out of the U.S. would like to be read in JAP because of the higher prestige it carries in academic circles than do less well-known journals. However, the majority of SIOP members are not academics and may not seek to be published in JAP.
Of course, it is possible that there is stability in the topics studied due to the influence of U.S.-based authors, even in collaborative studies. It seems reasonable that non-U.S.-based researchers who join SIOP might first be interested to replicate or extend the work done by U.S. scholars before heading off into new areas.
Also, because more than half of the non-U.S.-based SIOP members come from the “Anglo” set of countries outside the U.S., like Canada and Australia, there may be a common cultural set of values and interests, as well as a common language, that perpetuates the inertia to continue working on similar topics as U.S.-based scholars.
A sign of change does exist, though, in the greater use of non-U.S. samples by researchers who are non-U.S. based or are working collaboratively with I-O psychologists not based in the U.S. In the research studies we analyzed, there is a preference for non-U.S. samples in collaborative research.
Over the long term, the use of cross-national and non-U.S. samples holds promise for testing the universal power and validity of findings and theories developed mostly in the U.S. This would be a positive development for a truly global I-O psychology. A good bit more research is needed to assess meaningful shifts. In summary, while we see a rise in the globalization of I-O psychology membership, there is not much change visible in the research being done, so far.
References
Cascio, W. F., & Aguinis, H. (2008). Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology, 93(5), 1062–1081.
Kraut, A. I., and Mondo, L. (2009). “SIOP goes global. Or is it the other way around?” The Industrial-Organizational Psychologist, 47 (1), 33–40.