Information
Site Tools

 


Measuring Prestige of Journals in
Industrial-Organizational Psychology 

Michael J. Zickar and Scott Highhouse1
Bowling Green State University

1 Author Note: We are grateful to Margaret Brooks-Laber and Steve Russell for their assistance on this project.

The quality or prestige of journals is often used as a surrogate for evaluating the quality of individual publications. It has been our experience that faculty search committees and tenure evaluation committees often make judgments about the research productivity of candidates without reading the individual articles listed in the research vita. Instead, the quality of articles is judged by the prestige of the journal in which the article has been published. This reliance on journal prestige has been noted in management (e.g., Extejt & Smith, 1990; Johnson & Podsakoff, 1994; Kirkpatrick & Locke, 1992) as well as other fields such as sociology (Teevan, 1980). This over-reliance on journal prestige as a measure of journal article quality mandates that alternative measures of journal prestige be explored.

There have been several different indicators of journal quality proposed. One of the most popular indicators of journal prestige has been the Social Science Citation Index (SSCI) impact factor. The logic behind the impact measure is that journals with high prestige will contain articles that are cited more frequently by subsequent articles. One criticism of impact factors is that articles on topics with mass appeal (e.g., training and development) will tend to be cited more frequently than articles in areas of more limited appeal (e.g., psychometrics). Similarly, journals with larger audiences, such as those catering to both business school and psychology faculty, will necessarily have higher impact factors than journals with smaller audiences such as those catering only to psychologists. Other measures such as rejection rates, library circulation, or years in press are also confounded by factors such as audience size and topic popularity.

Another method of assessing journal prestige is to ask members of the field to judge the quality of particular journals. Although surveys of faculty impressions of management journals exist (e.g., Extejt & Smith, 1990; Gomez-Mejia & Balkin, 1992), it is unclear whether rankings of business journals can serve as rankings of I-O journals. One issue that has received attention over the years is the identity of I-O psychology relative to related disciplines, such as human resource management, organizational behavior, or organization development (e.g., Highhouse & Zickar, 1997; Naylor, 1966). Indeed, a considerable number of leading academics in I-O reside in business schools, raising the questions of whether I-O psychologists in psychology departments view journals differently than I-O psychologists in business schools.

To begin to address these questions, as well as to provide a specific measure of journal prestige in I-O psychology, we surveyed a sample of academic members of SIOP concerning their impressions of journal quality in I-O. One advantage of this method is that it focuses on the judgments of constituents of one area thereby avoiding the problem of area size dictating journal impact. We were also able to identify the academic home of these members, in order to examine differences in impressions of journal prestige for psychologists in business schools versus departments of psychology. 

Methods 

We selected 23 journals that, in our judgment, are common outlets for research by I-O psychologists. Journals were selected from previous lists of management journals (e.g., Extejt & Smith, 1990), excluding journals that are exclusively associated with business research (e.g., Sloan Management Review, Journal of Labor Economics) or with psychological research (e.g., Journal of Personality and Social Psychology, Psychological Bulletin). Although we recognized that some of these excluded journals can and do serve as occasional outlets for research by I-O psychologists, we were ultimately concerned with having judges compare I-O journals directly against one another. We also added journals that have not been included in previous lists, either because they are newer (e.g., Human Performance; Human Resource Management Review; International Journal of Selection and Assessment; Organizational Research Methods) or because they have been previously overlooked (e.g., Applied Psychology: An International Review; Journal of Business and Psychology; Journal of Organizational Behavior). We also included space for judges to add journals that they felt should be included on the list.

We sampled from the membership roster of SIOP. We received mailing labels from the SIOP database of all members who reported an academic setting as their primary place of employment. This list included 1,003 members in alphabetical order. We used a spaced sampling technique to arbitrarily sample SIOP members to reach our target sample of 500 members. We excluded people who were sent surveys in an earlier small-scale pilot study. Our sample represented approximately 50% of the academic members of SIOP as of 1999.

A questionnaire was sent to each member of the sample, and responses were made anonymously. Part I of the questionnaire instructed judges to place the 23 journals into one of three tiers and to indicate (a) whether they have published in the journal, and (b) whether they read the journal on a regular basis. The tiers were defined as follows:

First Tier: This ranking should be reserved for journals that present uniformly high quality research and/or review articles. Articles in top tier journals should be both methodologically sound and important in advancing our knowledge base.

Second Tier: Journals in this tier should routinely have high quality articles. However, the quality of content is uneven.

Third Tier: Journals in this tier routinely publish articles with suspect methodology.

The judges were instructed to circle the option labeled ? if they were unfamiliar with the journal.

Part II of the questionnaire instructed judges to choose and rank the top 10 journals (from the list) that in their opinion publish the most important research for I-O psychologists. The purpose of this section was to force judges to make relative distinctions among the higher quality journals. We also recognized that although a journal could be considered first-tier, it may not be seen as publishing research of direct relevance to I-O.

The final part of the questionnaire asked judges to indicate their primary place of employment (i.e., business school, labor/industrial relations department, psychology department, other) and years since they received their PhD. Judges were also asked to indicate areas in which they have conducted research, using the competency areas identified in the SIOP guidelines for education and training.

Results

Two-hundred seventeen usable responses were returned. Nine questionnaires were returned uncompleted or because of a change of address. This represented an effective response rate of 44.2%. The sample was evenly split between those employed in psychology departments (46.9%) and those employed in business schools (43.1%); a smaller number of respondents were employed in labor/industrial relations (LIR) departments (3.8%) and other departments (6.2%). For analyses based on department we combined business school and LIR faculty. The average respondent had completed their PhD 13.9 years prior to completing the questionnaire.

Table 1 reports the average tier ratings for each of the 23 journals. These ratings range from 1 to 3 with lower ratings corresponding to higher prestige. These ratings are presented for the overall sample as well as separated by business school and psychology department respondents. The rank ordering of journals on this index is also presented in the table.

Table 1. Tier Ratings of I-O Journals

___________________________________________________________________

 

Overall Tier

Psych Tier

Business Tier

AMJ

1.15 (2)

1.17 (2)

1.06 (2)

AMR

1.21 (3)

1.28 (5)

1.10 (4)

ASQ

1.27 (5)

1.482(6)

1.082 (3)

APM

1.82 (9)

1.69 (7)

2.00 (13)

AP:IR

2.33 (19)

2.37 (18)

2.24 (19)

BASP

2.24 (18)

2.22 (17)

2.29 (21)

EPM

1.91 (12)

1.88 (11)

1.92 (11)

GOM

2.33 (20)

2.40 (19)

2.26 (20)

HP

2.08 (15)

2.01 (14)

2.19 (17)

HR

2.21 (17)

2.442 (21)

2.032 (14)

HRMR

2.33 (21)

2.53 (23)

2.23 (18)

IJSA

2.48 (22)

2.43 (20)

2.59 (23)

JAP

1.07 (1)

1.07 (1)

1.05 (1)

JASP

1.87 (10)

1.83 (9)

1.89 (91)

JBP

2.52 (23)

2.49 (22)

2.57 (22)

JM

1.76 (8)

1.85 (10)

1.65 (7)

JOOP

2.07 (14)

2.04 (15)

2.09 (15)

JOB

1.90 (11)

1.91 (12)

1.89 (91)

JVB

1.94 (13)

1.94 (13)

1.93 (12)

LQ

2.18 (16)

2.21 (16)

2.16 (16)

OBHDP

1.28 (6)

1.23 (4)

1.29 (6)

ORM

1.75 (7)

1.82 (8)

1.69 (8)

PP

1.22 (4)

1.20 (3)

1.20 (5)

Note. 1 Indicates a tie; 2 indicates that a significant difference exists between business school and psychology department faculty on this index.

___________________________________________________________________

 

Table 2 presents the number of points that the 23 journals received in the top 10 rankings. Journals were given 10 points if they were ranked in the first position, 9 if they were in the second, and so forth. As before, rankings and breakdowns by psychology departments and business schools are presented in this table.

 

Table 2. Rankings of I-O Journals

 

Overall

Top Ten Points

Psychology

Dept.

Business Dept.

AMJ

1337 (3)

551 (3)

693 (2)

AMR

911 (5)

440 (5)

436 (4)

ASQ

546 (6)

189 (7)

321 (6)

APM

70 (18)

42 (16)

28 (18)

AP:IR

46 (22)

24 (20)

21 (20

BASP

59 (21)

36 (19)

14 (21)

EPM

188 (11)

106 (12)

62 (13)

GOM

69 (19)

14 (22)

51 (16)

HP

238 (10)

146 (9)

59 (131)

HR

106 (15)

39 (18)

52 (15)

HRMR

63 (20)

4 (23)

59 (131)

IJSA

38 (23)

23 (21)

11 (23)

JAP

1884 (1)

898 (1)

860 (1)

JASP

178 (13)

109 (11)

48 (17)

JBP

93 (16)

69 (14)

13 (22)

JM

438 (7)

180 (8)

234 (7)

JOOP

187 (12)

193 (6)

212 (8)

JOB

425 (8)

100 (13)

65 (11)

JVB

284 (9)

126 (10)

123 (9)

LQ

72 (17)

41 (17)

26 (19)

OBHDP

980 (4)

516 (4)

409 (5)

ORM

161 (14)

66 (15)

86 (10)

PP

1484 (2)

718 (2)

668 (3)

_________________

Note. 1Indicates a tie; this index was formed by weighting inversely each mention of the journal in the respondents top ten most influential journals list (e.g., if a journal was ranked first it received 10 points and if it was ranked in the 10th spot, it received 1 point).

___________________________________________________________________________

 

These rankings were highly correlated (r = --.87), so we decided to combine these two indexes of journal quality (after standardizing and reverse-coding prestige) to come up with an overall index. Table 3 presents the overall top 10 index based on this combination.

 

Table 3. Top Ten Overall Journals

___________________________________________________________________________

 

1. Journal of Applied Psychology

2. Personnel Psychology

3. Academy of Management Journal

4. Academy of Management Review

5. Organizational Behavior and Human Decision Processes

6. Administrative Science Quarterly

7. Journal of Management

8. Journal of Organizational Behavior

9. Organizational Research Methods

10. Journal of Vocational Behavior

____________________________________________________________

 

As a final index of interest, Table 4 presents the percentages of respondents who reported that they read the 23 journals in the study. The majority of I-O psychologists read AMJ, AMR, JAP, and PP, whereas most journals included in this study were read by less than 20% of the sample.

 

Table 4. Reading Frequencies of I-O Journals

 

Overall % who Read

Psychology

Dept.

Business Dept.

AMJ

79.2

77.8

83.9

AMR

72.3

66.7

80.9

ASQ

37.3

26.31

49.51

APM

12.9

19.2

8.1

AP:IR

13.3

19.2

9.1

BASP

10.6

14.1

7.0

EPM

20.8

24.3

17.2

GOM

16.6

12.1

23.3

HP