Broadening the Basis for Validation Evidence: Alternative Strategies and Their Implications
Presenters: S. Morton McPhail, CEB Valtera, Calvin C. Hoffman, LA County Sheriff's Department, Damian J. Stelly, CEB Valtera
Coordinator: Ryan O’Leary, PDRI
Target Audience: Intermediate; experience in validation of selection procedures required
This workshop focuses on methods for developing validation evidence in situations when traditional criterion-related strategies are unavailable or impracticable. Topics include the theoretical and methodological underpinnings of various strategies, including transportability, job component validity, and construct validity. Attention will be given to applications, professional standards, regulatory guidance, and legal review.
A common issue that arises in development and validation selection procedures is the availability (or lack thereof) of suitable criterion-related validation evidence. Although research since the 1970s has consistently demonstrated the broad generalization of validity evidence for a number of constructs such as cognitive ability and conscientiousness, EEO enforcement agencies still require that adequate evidence be provided of the validity of selection procedures in particular situations if they result in disparate impact with respect to protected subgroups. For a wide variety of reasons (including costs, sample availability, and organizational constraints, etc.), direct evidence of selection procedure validity in the form of statistical relationships with relevant job outcomes may not be obtainable. There are, however, a number of appropriate and efficient alternative strategies to provide evidence of job relatedness and business necessity. This workshop will present several of these strategies in considerable depth. The presentations will include both the theoretical basis for the approaches and guidance on their practical application. The presenters also will discuss relative strengths and weaknesses of the alternatives and the existing legal implications to the extent that relevant case law allows.
This workshop is designed to help participants:
- Identify professionally and legally acceptable alternative strategies for development of validation evidence to meet the needs of particular situations
- Assess the relevance, efficiency, and risks of using alternative strategies
- Plan and implement alternative strategies with appropriate contingencies and follow-up research
- Prepare appropriate reports and documentation to support the strategy(ies) used
S. Morton (Mort) McPhail manages CEB Valtera’s (a part of The Corporate Executive Board Company [CEB]) assessment and selection practice in Houston. In his 34 years as a consultant, he has worked with a wide variety of public and private sector clients on issues ranging from selection, training and development, and performance assessment to termination. He has served as an expert in litigation involving such diverse issues as job analysis, test development and validation, violence in the workplace, equal employment opportunity, compensation, and reductions in force. He is a Fellow of SIOP and served as its Secretary/Financial Officer from 2009-2011. He has published research articles in scholarly journals and book chapters, edited a volume in SIOP’s Professional Practice Series (Alternative Validation Strategies, 2007), and presented on numerous topics at professional meetings. He recently (2012) co-authored a chapter in the Oxford Handbook of Assessment titled “Individual Psychological Assessment” reviewing its history, research, and practice. He serves as adjunct faculty for both the University of Houston and Rice University. He is a licensed psychologist and is currently a member of a committee of the Texas Psychology Board charged with development and validation of the State’s Jurisprudence and Ethics Examination for licensure.
Calvin C. (Cal) Hoffman is the Test Development Manager for Los Angeles County Sheriff’s Department, where he manages all test development and validation efforts for both sworn and civilian classifications. Additionally, he is a Senior Consultant with Easi*Consult. In his almost 30 years of applied experience, he has worked in the private sector, in academia, in consulting, and in the public sector. His applied experience covers a broad range of I-O interventions, including entry level selection programs, promotional programs, job analysis, job evaluation, training design and evaluation, employee and management development programs, developmental assessment, survey and customer satisfaction research, and process improvement. He has published on a variety of topics, including job analysis and job families, test transportability, job component validation, statistical corrections to validity, utility analysis, and physical ability testing, and has been a frequent presenter at SIOP and other professional meetings. He coauthored a chapter (Job Component Validity) for a book in SIOP’s Professional Practice Series (Alternative Validation Strategies, 2007), and coauthored a chapter (Lifecycle of Successful Selection Programs) in the Oxford Handbook of Assessment (2012). In 2010, he received the Innovations in Assessment Award from the International Personnel Assessment Council. He was elected as a Fellow of SIOP in 2008.
Damian J. Stelly is a Senior Consultant with CEB Valtera, a part of The Corporate Executive Board. Prior to joining CEB, he was Organization Development Director at JCPenney where his responsibilities included leading the company’s selection, assessment and performance management programs. In his over 15 years of applied experience, he has managed a broad range of projects in a variety of industries, including employee and customer surveys, selection and placement systems, executive assessment, employee development programs, performance management processes, organizational development initiatives, litigation consulting, and applied research and analytics. He has presented and published on a variety of topics, including validation methods, test development, and leadership behavior. Most recently, he coauthored chapters in two books in SIOP’s Professional Practice Series (Alternative Validation Strategies, 2007; Handbook of Workplace Assessment, 2010). He earned his Ph.D. in industrial-organizational psychology from the University of Missouri – St. Louis and is licensed as a psychologist in Texas.