Home Home | About Us | Sitemap | Contact  
  • Info For
  • Professionals
  • Students
  • Educators
  • Media
  • Search
    Powered By Google

Building a Better Literature Review:
Reference and Information Sources for I-O Psychology

Travis Tubr1
The University of Southern Mississippi

Paul R. Bly, Bryan D. Edwards,

Robert D. Pritchard, and Sharon Simoneaux
Texas A&M University

1 Author Notes: Correspondence concerning this article should be addressed to Travis Tubr, Department of Psychology, The University of Southern Mississippi, Box 5025, Hattiesburg, MS 39406-5025. Electronic mail may be sent to Travis.Tubre@usm.edu.

The literature review is an important component of any research endeavor. Good literature reviews reduce unnecessary duplication of efforts and help to put papers in context by building on the previous work of others. The identification and use of appropriate resources is of primary concern when conducting literature reviews. The number of reference and information sources available for conducting literature reviews has increased significantly in recent years. Traditional printed sources of information (e.g., Psychological Abstracts) have been joined by an array of other media including CD-ROM, online databases, and ever increasing World Wide Web (www) technologies. The purpose of this paper is to provide a rudimentary introduction to several search techniques and a brief guide to some of the numerous information sources which are relevant to I-O psychology. The knowledge and effective use of these available resources can greatly improve the efficiency and quality of literature reviews.

Issues in Doing Literature Reviews

Recall and precision are key issues in the literature search process. Recall refers to the percentage of all relevant sources that are actually located during the literature search, while precision is the percentage of all located sources that are actually relevant to the researchers interests. As expected, recall and precision are often inversely related. Recall is enhanced by using more sources and spending more time on the search, but this process is likely to decrease precision, resulting in a larger body of irrelevant material that must be sorted through in order to find useful material (White, 1994). A good literature review will not necessarily find every document that is related in some way to the research topic; however, it will find the important articles, including those that are outside the researchers normal scope of reading materials (White, 1994).

Strategies for Performing Literature Reviews

Wilson (1992) outlined five major search strategies which encompass a variety of techniques for locating references. These include (a) consultation, (b) searches in subject indexes, (c) browsing, (d) citation searches, and (e) footnote chasing. The following paragraphs provide a brief description of each technique along with a representative source for each technique.

Consultation. Consultation involves locating references by corresponding with others. It tends to be a low-recall strategy, since it is limited by the knowledge, memory, and biases of correspondents. Conversely, it is generally high in precision because the references provided have been pre-screened for relevance. A major advantage of consultation is the ability to locate unpublished materials. Internet technology (e.g., e-mail, discussion groups) has greatly increased the utility of consultation as a search technique. For instance, RMNET (see the full description in the source list that follows) is an e-mail list which provides a forum for discussion on a variety of topics related to I-O psychology. Unlike many e-mail lists, RMNET also provides a searchable subject index which allows the user to access previous discussions on a topic of interest.

Subject indexes. Most researchers are familiar with retrieving unknown publications by searching subject indexes (e.g., PsycINFO, ABI/INFORM). Subject indexes are used to search bibliographic descriptions which generally include the title, abstract, and authors of a document in addition to controlled-vocabulary terms (e.g., subject headings, index terms) which are added by the database producer. Subject index searching tends to be high-recall and low-precision since a large amount of irrelevant information is often retrieved. The precision of subject index searching can be improved by combining keyword searches of abstracts and titles with controlled-vocabulary searches (e.g., locating documents by subject heading). In addition, many databases offer search-refining capabilities such as the ability to combine multiple searches and limit searches by type of publication, year, language, or other characteristics.

Browsing. A third search technique identified by Wilson (1992) is browsing, which is literally defined as browsing through materials (e.g., library shelves, journal indexes) to find relevant information. Browsing is a high-recall strategy, since the amount of information collected is limited only by the researchers willingness to continue searching. However, it generally results in low-precision retrievals, since large volumes of information must be evaluated for relevance (White, 1994). As with other search strategies, technology is improving the efficiency and effectiveness of browsing. For instance, Current Contents, an electronic database, simplifies and enhances the browsing strategy by indexing the tables of contents of selected journals.

Citation searching. This extremely powerful, and potentially neglected, search technique focuses on locating references that cite known articles published previously. The citation search begins by identifying an influential article on a given topic. Next, a citation index (e.g., Social Science Citation Index, SSCI) is used to locate all of the articles that were the original sources. For instance, a researcher doing a literature review on organizational socialization could use SSCI to locate articles that cite Scheins (1990) seminal American Psychologist article on the topic. Once additional materials have been located, the process can be repeated using the new materials as the starting point for additional citation searches (White, 1994). As such, citation searching is extremely useful for tracing the orderly progression of a body of literature. In contrast to most other search strategies, both recall and precision tend to be high for well-constructed citation searches. Unfortunately, Cooper (1985) reported that only 9% of authors surveyed reported using computer searches of citation indexes as a means of gathering resources.

Footnote chasing. A closely related strategy, footnote chasing, is the process of locating useful information by searching the reference sections of other papers (e.g., topical bibliographies, meta-analyses) which focus on the topic of interest (Wilson, 1992). Footnote chasing tends to have high precision, since other authors have reviewed the material and found it to be pertinent to the content domain. However, recall is dependent on the quality of the literature review in the source materials. SSCI provides the capability to conduct electronic footnote chasing, since it indexes the cited references for database entries. The combination of citation searching and footnote chasing using SSCI is an extremely powerful search technique. 

Reference and Information Sources for Doing Literature Reviews

The Appendix presents a selected list of reference and information sources that may be of use to researchers in the I-O field. Most local libraries can provide information on accessing and using these sources. In addition, major vendors such as OVID and ISI have Web sites that provide information and access to databases. It is important to note that this is not in any way an exhaustive list of reference and information sources relevant to the I-O field. The total number of potentially useful reference and information sources is enormous. It is our hope that providing representative examples of potential sources will be of practical value by increasing the scope of reference materials considered for use by researchers in the I-O field.

References

Cooper, H. (1985). Literature searching strategies of integrative research reviewers. American Psychologist, 40, 1267--1269.

Schein, E. H. (1990). Organizational culture. American Psychologist, 45, 109--119.

White, H. (1994). Scientific communication and literature retrieval. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis, 41--56. New York: Russell Sage Foundation.

Wilson, P. (1992). Searching: Strategies and evaluation. In H. D. White, M. J. Bates, and P. Wilson, For information specialists: Interpretations of reference and bibliographic work, 153--181. Norwood, NJ: Ablex.

Appendix
Selected Reference and Information Sources for I-O Psychology
2

2 A more comprehensive list of available reference and information sources is available from the first author upon request.

ABI/INFORM (University Microfilms International). This database contains over 550,000 citations, providing complete bibliographic information, abstracts, and selected full-text articles from more than 1,000 national and international business and management periodicals. ABI/INFORMs coverage of human resources management and organizational behavior is particularly relevant. Coverage: 1971 to the present.

Current Contents: Social and Behavioral Sciences (Institute for Scientific Information). Current Contents indexes the tables of contents, with complete bibliographic information, for over 1,375 journals in the social and behavioral sciences. Full-length author abstracts are available for approximately 85% of the articles and reviews. One unique and very useful feature is the inclusion of reprint author addresses. The database is available in multiple electronic formats. Coverage: 1969 to the present.

ERIC (U.S. Department of Education; Educational Resources Information Center). ERIC is the most comprehensive database covering the literature in education. The database, available in a variety of electronic formats, corresponds to two print indexes: Resources in Education (RIE) and Current Index to Journals in Education (CIJE). The RIE file covers the document literature in education including such sources as conference proceedings and technical reports, while CIJE provides comprehensive indexing of nearly 800 periodicals. The ERIC database is an indispensable resource for researchers in the training and development field. Coverage: 1966 to the present.

ERIC/AE Test Locator (ERIC Clearinghouse on Assessment and Evaluation). The Test Locator is a joint project of the ERIC clearinghouse, the Library and Reference Services Division of the Educational Testing Service, the Buros Institute for Mental Measurements, and Pro-ED test publishers. The Test Locator consists of several components providing a variety of information including (a) descriptive information, including availability, for over 10,000 tests and research instruments; (b) citations to reviews of educational and psychological tests and measures; and (c) names and addresses of over 900 major commercial test publishers. The Test Locator can be found at the following URL: http://www.ericae.net/testcol.htm.

NTIS (National Technical Information Service). NTIS delivers comprehensive coverage to unclassified technical reports from research sponsored by the U.S. government, foreign governments, federal agencies, and other sources. Records include bibliographic information and abstracts with coverage of such relevant areas as management practices, human factors, and public policy. NTIS is available in multiple electronic formats and is the sole provider for numerous items located in its database. Coverage: 1964 to the present.

ProQuest Digital Dissertations (formerly Dissertation Abstracts, University Microfilms International). This database contains over 1.6 million citations to dissertations and selected masters theses covering approximately 3,000 topics. Abstracts are included for records indexed since 1980. Coverage: 1861 to the present.

PsycINFO (American Psychological Association). PsycINFO provides unparalleled access to journal articles, books and book chapters, dissertations, technical reports, and other materials from psychology and related disciplines. PsycINFO corresponds to the printed volume Psychological Abstracts and the CD-ROM product PsycLIT, although its coverage is more extensive than either of its companion sources. As of June 2000, the database contained over 1 million records and indexed over 1,400 scholarly journals. Coverage: 1967 to the present.

RMNET (Research Methods Network, Research Methods Division, Academy of Management). This e-mail list is an excellent resource for current discussions on an extensive variety of primarily methodological topics. In addition, RMNET allows the user to conduct keyword searches of its rich archive files. The RMNET coordinator is Jeff Edwards (jredwards@ unc.edu), and the e-mail address is rmnet@listserv.unc.edu. RMNET access is limited to members of the Research Methods Division of the Academy of Management.

Social Science Citation Index (SSCI, Institute for Scientific Information). Arguably the most powerful social science literature searching aid currently available, the SSCI indexes 1,725 social science journals and selected social science related articles from an additional 3,300 journals. The SSCI distinguishes itself by providing the option of cited reference searching. Using this strategy, the researcher can identify a seminal article in a given area and locate every article published since that cited the original source. As additional articles are uncovered, the procedure can be repeated in a process called cycling. Thus, the SSCI provides unparalleled access to the progression of a given body of literature. The SSCI is published in print and multiple electronic formats. Coverage: 1972 to the present (weekly updates).


April 2001 Table of Contents | TIP Home | SIOP Home