Put Your Survey on a Diet: How to Develop, Deploy, Analyze, and Justify Brief Measures of Organizational Constructs
Presenters: Fred Oswald, Rice University
Jeff Stanton, Syracuse University
Coordinator: Tim McGonigle, SRA International
Survey researchers often face the dilemma of “too many items and not enough time:” numerous constructs to measure but limited testing time and respondents who dislike long surveys. We offer a set of conceptual considerations, research designs, and statistical methods for addressing this common dilemma. Presenters of this workshop will (a) lead an interactive discussion of the settings under which reducing survey length is recommended as well as those where reduction is not recommended; (b) provide an in-depth review of important considerations when shortening a measure (e.g., the goals of measurement, content representativeness, reliability and validity, adverse impact); (c) conduct analyses on longer surveys that provide critical statistical guidance on how to create shorter ones; (d) review options for short-survey deployment in a variety of organizational and research settings; (e) conduct analyses on short-survey data that provide information on item refinement and survey quality; (f) discuss implications of this approach for organizational research and practice.
Participants who enroll in this workshop will be able to:
1. Describe the challenges of survey measurement under circumstances that constrain survey length
2. Analyze long-survey data to inform short-survey development
3. Create systematic procedures to reduce survey length with minimum loss of quality
4. Design and administer surveys in a manner that preserves psychometric integrity and face validity
5. Analyze survey data to cross-check scale reduction results
6. Describe the use of shortened measures and caveats on their interpretation
Target Audience: Researchers and practitioners with less than 5 years of experience and the ability to conduct and interpret typical reliability and validity studies. Audience members should be interested in delivering shorter measures while retaining solid psychometric characteristics that preserve their measurement goals.
Dr. Fred Oswald is an associate professor of Rice University in the industrial-organizationalprogram. His research is concerned with personnel selection issues in organizational, education, and military settings. Specifically, his work deals with defining, modeling and predicting performance outcomes from psychological measures based on cognitive and motivational constructs (e.g., cognitive abilities, personality traits, situational judgment tests, job knowledge and skill, and biographical data). His methodological work in adverse impact, test score banding, meta-analysis and structural equation modeling also informs personnel selection issues and psychological testing. Dr. Oswald publishes his research in collaboration with the graduate students he mentors, and he has a history of large-scale grant-funded projects. He is Associate Editor of the Journal of Business and Psychology, and he serves on the editorial boards for Journal of Applied Psychology, Personnel Psychology, Journal of Management, Organizational Research Methods, and International Journal of Selection and Assessment. Dr. Oswald received his PhD from the University of Minnesota in 1999.
Dr. Jeffrey Stanton is associate dean for Research and Doctoral Programs in the School of Information Studies at Syracuse University. Dr. Stanton’s research focuses on organizational behavior and technology. He is the author of more than 40 peer reviewed journal articles as well as two books, The Visible Employee: Using Workplace Monitoring and Surveillance to Protect Information Assets – Without Compromising Employee Privacy or Trust and Information Nation: Educating the Next Generation of Information Professionals. Stanton’s methodological expertise is in psychometrics including the measurement of job satisfaction and job stress, as well as research on creating abridged versions of scales; he is on the editorial board of Organizational Research Methods and is an associate editor at Human Resource Management. Dr. Stanton's research has been supported through 15 grants and supplements including the National Science Foundation’s CAREER award. Dr. Stanton received his PhD in industrial-organizational psychology from the University of Connecticut in 1997.