Generalizing Validity Evidence: How Is It Done and Is It Right for My Situation?
Presenters: Calvin C. Hoffman, Los Angeles County Sheriff’s Department
Piers Steel, Haskayne School of Business, University of Calgary
Coordinator: Cheryl Paullin, HumRRO
A cornerstone of I-O psychology practice is validating the assessments that we use to measure KSAOs, such as in personnel selection. To do this, we have traditionally relied on simulations of actual job requirements (content validity) or correlational relationships between scores on an assessment and some outcome measure of interest within a particular organization (local criterion-related validity). These methods can be extremely expensive and the results can quickly become out of date. As Frank Landy noted in 2007, “As the nature of work changes from the stable to the volatile, the luxury of orthodox criterion- and content-related studies is often no longer available.” (p. 423). This workshop addresses alternatives to criterion- and content-related studies that take advantage of validity evidence that already exists. Specifically, the workshop will provide an overview of: transportability, meta-analysis, and synthetic validity, reviewing the basic requirements of each technique as well as its pros and cons. The presenters will also provide insights on the situations in which each technique may be most viable and their personal knowledge and experiences regarding the legal defensibility of each technique. The goal is to provide practitioners with faster, less costly, easier to use, and more accurate ways to validate selection systems.
Landy, F.J. (2007). The validation of personnel decisions in the twenty-first century: Back to the future. In S.M. McPhail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence. San Franciso, CA: John Wiley & Sons.
This workshop is designed to help participants:
1. Describe the major features of transportability, meta-analysis, and synthetic validity, with emphasis on practical implementation.
2. List at least one advantage and one disadvantage associated with each technique.
3. Describe the type of situations for which each technique is well-suited.
4. Describe at least one legal defensibility implication for each technique.
Target Audience: The target audience is mid-level to advanced practitioners (5-10 years of experience or more) in the areas of assessment development and talent management who have at least a basic understanding of the concept of validity as defined in the Principles for the Validation and Use of Personnel Selection Procedures, Standards for Educational and Psychological Testing, and Uniform Guidelines on Employee Selection Procedures.
Calvin C. Hoffman, PhD, earned his doctorate in industrial and organizational psychology from the University of Nebraska in 1984. The majority of his professional experience has been in the private sector, where he has been involved in conducting a variety of interventions, including job analysis, job evaluation, job family development, selection system design and validation, management assessment and development programs, customer satisfaction research, and HR system redesign. He has published on a variety of topics including job analysis, test validation strategies, utility analysis, and job component validation. He has also consulted extensively with both public-sector and private-sector organizations. Dr. Hoffman is currently a civilian employee with the Los Angeles County Sheriff’s Department and an adjunct faculty member of Alliant International University. He was elected a Fellow of SIOP in 2008 and was the recipient of the 2010 Innovations in Assessment award granted by the International Personnel Assessment Council.
Piers Steel, PhD, earned his doctorate in industrial and organizational psychology from the University of Minnesota in 2002. He is a professor of Human Resources and Organizational Dynamics at the Haskayne School of Business at the University of Calgary. Dr. Steel’s particular areas of research interest include personnel selection, culture, and motivation, particularly procrastination. He has published in the premier journals in the social sciences and business (e.g., Academy of Management Review, Psychological Bulletin, Journal of Applied Psychology, Personnel Psychology). He has received several teaching awards, including the Dean's Award for "Outstanding Leadership in Teaching and Learning" and the Academy of Management "Best Paper in Management Education." Dr. Steel has received international recognition for his procrastination research, with widespread media coverage ranging from the New York Times, USA Today, Globe & Mail, National Post, and Scientific American. He has also received media attention related to research on synthetic validity and a project to create a universal, automatic, instantaneous selection system.