Home Home | About Us | Sitemap | Contact  
  • Info For
  • Professionals
  • Students
  • Educators
  • Media
  • Search
    Powered By Google

Is Job Analysis Doing the Job?  Extending Job Analysis with Cognitive Task Analysis

Rosemarie Reynolds and Michael T. Brannick
University of South Florida 

Today's job analysis methods have their origins in the industrial revolution. Demands for efficiency brought about an increased interest in the division of labor as a method of insuring that employees were easy to train and to replace and job analysis as an aid to time and motion studies. Over the years, job analysis methods have evolved to serve such purposes as job description (e.g., functional job analysis), selection (e.g., job element method, ability requirements scales), training (e.g., the task inventory), and job evaluation (e.g., the position analysis questionnaire).

The question addressed in this paper is how the changing nature of work and working in the post-industrial information age may affect job analysis. As many authors have pointed out, todays world is characterized by changes in technology, transportation, and communication that have created flatter hierarchies, an increasing reliance on teams, and the dominance of high-tech industries. Two aspects of the changing nature of workcognitive labor and teamworkare discussed below. As a result of these changes, we suggest that industrial psychologists begin to incorporate cognitive task analysis (CTA) into some aspects of job analysis. A brief description of CTA, as well as some applied examples, follows the discussions of cognitive labor and teamwork.

Cognitive Labor

As Goldstein (1993) pointed out, the increasing use of technology paradoxically leads to morenot lesscognitively demanding jobs. Jobs that were formerly routine and predictable now involve diagnosis, monitoring, and decision making. The increased emphasis on the cognitive nature of tasks has spread beyond professional and service jobs. The information portion of the information age is extending to the shop floor as well, where initiatives such as just-in-time inventories and total quality management are adding cognitive demands to jobs that were once considered primarily physical (Wall & Jackson, 1995). 

Team Work

In the current economy, competitive advantage comes from knowledge, quality, speed, and flexibility. The result is an increasing move from traditional work design to a design emphasizing teams. With this movement comes the need for team selection, team performance appraisal, team reward structures, and team training. All of these needs are based on job analysisbut job analysis with a twistnamely team task analysis. However, as several researchers have pointed out, little research exists on team task analysis (Landy, Shankster-Cawley, & Moran, 1995; Salas & Cannon-Bowers, 1997).

Cognitive Task Analysis

CTA has been used in the learning, human-computer interaction, and human factors fields. A large and varied number of methodologies have been developed: card-sort techniques, verbal protocols, time tags of events, and others. The methods share the common aim of understanding the knowledge, thought processes, and mental aspects of goal attainment at work (Schraagen, Chipman & Shalin, 2000). Yet despite the increasing popularity of CTA, it has been virtually ignored by industrial psychology. A recent computer search uncovered 90 articles on CTA, but none were in journals of primary interest to I-O psychologists.

This neglect of CTA is astonishing when one considers that CTA has its roots in industrial psychology. Annett (2000), for example, traced the historical and theoretical development of CTA from the classical methods of time and motion studies, through WWI and studies of fatigue and working conditions based on psychology, up to the modern days methodologies for performing CTA.

Examples of CTA 

Some specific examples of CTA taken from the literature may help to clarify both the methods used and the potential usefulness of CTA to I-O psychology. The first example described below focuses on the analysis of a cognitive task, while the second example concerns teamwork.

Cognitive labor. Schaafstal and Schraagen (2000) described work done for the Dutch navy on the job of weapons engineering service technician. A major function of this job is troubleshooting systems that are malfunctioning. The authors were asked to investigate the increasing number of complaints from the Dutch fleet regarding the timeliness and effectiveness of the troubleshooters and performed a series of studies, the first of which focused on radar systems technicians. Radar systems technicians at different levels of expertise were asked to solve four different radar problems while thinking aloud. From the results of these verbal protocols, the authors concluded that (a) the radar theory instructor was not necessarily a good troubleshooter, indicating that there was a gap between theory and application, (b) there was little transfer of knowledge from one radar system to another, (c) the troubleshooters were unsystematic in their approach to troubleshooting, and (d) that problems, if solved, were solved because of their similarity to previous problems experienced by the technicians.

The second study focused on inexperienced computer systems technicians, using a similar methodology. For this sample, only 40% of the four test problems were solved. An even more interesting finding was that the correlation between scores on a knowledge test and actual problem diagnosis was only .27, indicating that system knowledge and theory were not enough to make for effective troubleshooting.

The authors then conducted a formal CTA of the troubleshooting process. As a result of the CTA, they determined that troubleshooting consisted of four subtasks, each with its own cognitive skill requirements. The first subcomponent, problem description, requires the technician to be able to identify both normal and abnormal system behavior. The second subcomponent involves generating possible causes for the abnormal behavior, while keeping the possibilities within reason. The third component, testing, requires the technician to compare the results of testing to the hypotheses. A number of skills are involved, such as the ability to choose and execute the right tests, knowledge about setting up conditions for measuring test outcomes, and how to develop accurate expectations regarding the possible outcomes of the test. Finally, the troubleshooter must evaluate what needs to be done in order to fix the problem.

As a result of this CTA, several recommendations were made regarding training. One suggestion was to teach a systematic approach to troubleshooting in the training. In order to evaluate the new training, one group of subjects took a 1-week course incorporating the new training in addition to the regular training course, while another group took only the regular training course. At the end of the training, both groups were scored on a theoretical knowledge test, as well as on their ability to solve four unfamiliar systems problems. There was no difference between the two groups on the test of theoretical knowledge, but the experimental group performed significantly better on actual problem solving, solving twice as many problems as the control group.

Teamwork. Klein (2000) used CTA to examine both team processes and team knowledge in an emergency response organization (ERO) at a nuclear power plant. Although the plant was operating efficiently, the ERO was consistently running into problems during drills, and the Nuclear Regulatory Commission (NRC) was threatening to increase the number of drills at the plant each year. Plant management called in a CTA team in the hopes of avoiding this response by the NRC.

The CTA team began by observing emergency exercises, during which they traced the number of steps required by each team action. They also identified the decisions made and the decision makers in the team. Finally, the CTA team also wrote down major events and time tagged them.

This information was then used as the basis of interviews with those identified as decision makers, in which subjects were asked about incidents, asked to define what happened in the incident, and exactly why it happened. As a result of these interviews, it became obvious that some of the procedures that had accumulated over the years were irrelevant or outdated. Lags in the system, unnecessary handoffs to others, the lack of a shared mental model, and too many people in the ERO room were some of the problems identified.

Over 50 recommendations were implemented as a result of the CTA, including a reduction in the number of people from 80 to 35. During the next exercise, the NRC reduced the number of required drills to one every 2 years.

Conclusion

This paper suggests that the changing nature of work in the post-industrial era may require changes in the conduct of job analysis, and that CTA may help in designing those changes. This is not to suggest that CTA is a substitute for job analysis. CTA is a highly labor intensive method and should be limited to those tasks that promise sufficient return for the effort (Schraagen et al., 2000). Jobs or tasks most suited for CTA are those with a large cognitive component and aspects of teamwork. Traditional job analysis methods are not particularly well equipped to investigate the cognitive processes underlying tasks. A job analysis is often based on observation, interviews, or self-report. However, cognition is difficult to observe, and interviews and self-reports would be based on the assumption that people are consciously aware of their thought processes, an assumption that cognitive psychology has repeatedly shown to be inaccurate. Conversely, in the first example presented here, verbal protocols were used to identify the cognitive processes underlying troubleshooting, which enabled the researchers to develop training that focused on these specific cognitive processes.

Traditional methods of job analysis typically focus on the accomplishment of tasks and not the interconnections of tasks. Thus, in the analysis of team tasks, traditional job analysis methods may fall short of organizational needs. In the team CTA sudied in the second example, the authors used time tagging, interviews, and a form of critical incidents to observe and record the actions among team members. As a result of their methods they were able to identify lags in the system, unnecessary handoffs to others, and the lack of a shared understanding of the problem.

In both of the above examples, CTA was able to provide information beyond that provided by traditional job analysis. However, this does not mean that CTA will be useful for all organizational purposes. The examples presented here show the application of CTA in training and job design, areas in which CTA has historically been used. There are hints, however, that CTA will be useful for more than training and job design. CTA has also been effectively used to develop a selection device for public safety dispatchers (Hunt & Joslyn, 2000). Further, Baker and Salas (1997) suggested that CTA may be useful for performance appraisal, especially in a team context.

The purpose of this article was to introduce CTA to I-O psychologists. We described the application of CTA to training and to teamwork and showed that CTA is particularly applicable to understanding the representation and organization of knowledge by workers. Although CTA is labor intensive, it is useful in areas in which traditional methods of job analysis fail to provide detailed information or insight. 

References

    Annett, J. (2000). Theoretical and pragmatic influences on task analysis methods. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin, (Eds.), Cognitive task analysis (pp. 2537). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Baker, D. P. & Salas, E. (1997). Principles for measuring teamwork: A summary and look toward the future. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and application (pp. 331356). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Goldstein, I. L. (1993). Training in organizations, (3rd Ed.) Pacific Grove: Brooks/Cole.
   
Hunt, E. & Joslyn, S. (2000). A functional task analysis of time-pressured decision making. In J. M. Schraagen, S.F. Chipman, & V. L. Shalin, (Eds.), Cognitive task analysis (pp. 119134). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Klein, G. (2000). Cognitive task analysis of teams. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin, (Eds.), Cognitive task analysis (pp. 417430). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Landy, F. J., Shankster-Cawley, L., & Moran, S. K. (1995). Advancing personnel selection and placement methods. In A. Howard (Ed.), The changing nature of work (pp. 252289). San Francisco: Jossey Bass.
   
Salas, E., & Cannon-Bowers, J.A. (1997). Methods, tools and strategies for team training. In M. A. Quiones and A. Ehrenstein, A. (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp. 249280). Washington, DC: APA.
   
Schaafstal, A. & Schraagen, J. M. (2000). Training of troubleshooting: A structured, task analytical approach. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin, (Eds.), Cognitive task analysis (pp. 5770). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Schraagen, J. M., Chipman, S. F., & Shalin, V. L . (2000) Introduction to Cognitive Task Analysis. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin, (Eds.), Cognitive task analysis (pp. 323). Mahwah, NJ: Lawrence Erlbaum Associates.
   
Wall, T. D., & Jackson, P. R. (1995). New manufacturing initiatives and shopfloor job design. In A. Howard (Ed.), The changing nature of work (pp. 139174). San Francisco: Jossey Bass. 

July 2001 Table of Contents | TIP Home | SIOP Home