The Art and Science of Selection: Loading Up for Implementation and Sustainability
Presenters: Rich Cober, Marriott International, Inc., and Nancy Tippins, Valtera Corporation
Coordinator: Rob Michel, Edison Electric Institute
Target Audience: All levels of experience
An effective selection program is based on a well-developed and validated test, or set of tests, that meets the goals of many stakeholders including line leaders, the staffing organization, and legal and labor advisors. Although the science of test development and validation is a central component of most I-O graduate programs, the process of designing sustainable selection programs is not always included. This workshop focuses on three groups of decisions that must be made for effective selection programs: (a) design decisions; (b) implementation decisions; and (c) sustainability decisions.
Design decisions take into account the purpose of the assessment program, the context in which the test will be used, the jobs for which the test will be used, and its stakeholders. Those responsible for a selection program must determine who the relevant stakeholders are and the related goals that influence the perspectives of each group. At times, goals are conflicting, and the selection expert must attempt to reconcile the differences or make decisions regarding which goal takes precedence. Similarly, the context in which the test will be administered shapes its design. A high volume, un-proctored internet testing program may require a different type of test than a program that must be face-to-face or used for a relatively small number of people. The number of jobs that are included in the selection program may also affect the design of the test; for example, a situational judgment test that reflects experiences found on the job may be less useful when a broad array of jobs is included.
Many implementation decisions contribute to the sustainability of the test. Policies and procedures related to topics such as retesting, waivers and exemptions, and ADA accommodations clarify who should be tested as well as the timing and conditions of the testing event. Training of administrators, availability of equipment, and suitable testing environments enhance the likelihood the test is administered appropriately. Procedures for record keeping and reporting ensure test information is accurately stored and available to decision makers. When implementing global testing programs, multi-cultural differences and country-specific practices and laws provide for rich contextual factors from which decisions regarding selection programs must be made.
To keep a selection program alive, the testing expert must monitor the test results and ensure the test is doing what it was intended to do. The I-O psychologist may monitor adverse impact data as well as measure the quality of hire. Although monitoring adverse impact data may be more germane for testing professionals responsible for programs inside the United States, more general monitoring of assessment results is also relevant to those providing oversight for multi-national programs. He or she may also want to re-validate a program that is in use despite the hazards of doing so (e.g., restriction of range issues). In addition, the expert must ensure that the organization remains prepared to execute the program by keeping stakeholders informed and including them in key decisions.
This workshop will cover the design of selection programs, their implementation, and ways to ensure their continued viability. The presenters will pose critical questions about each of the three broad topics; offer answers based on their own experiences, and solicit other approaches from the workshop participants.
As a result of attending this session participants will be able to:
- List the key questions to ask when designing a selection program and understand possible approaches to answering each.
- Discuss the decisions that must be made for a successful implementation and understand the pros and cons of different approaches.
- Identify the aspects of a selection system that must be continuously monitored throughout the life of the test.
- Describe the pros and cons of continuous improvement of existing selection programs.
- Depict the components of a robust test data storage system.
Rich Cober is Vice President responsible for the Talent Management Analytics and Solutions Team for Marriott International. In this capacity, Rich is responsible for the development and implementation of global selection and performance management systems, job analytic work, surveys, and workforce analytics that support Marriott’s family of brands (including The Ritz-Carlton, Marriott Hotels, Renaissance Hotels, Courtyard, and Residence Inn). His expertise and experience span across most major Human Resource Management processes and tools including competency modeling, employee recruitment and selection, employee performance management, and organizational survey design and analysis. In addition to his professional experience, Rich has designed and conducted a number of research projects in the areas of employee staffing and performance management. He has published articles in journals including the Journal of Applied Psychology, Public Personnel Management, Journal of Management, Human Resource Management Journal, and International Journal of Selection and Assessment, and he is a regular presenter and chairperson at national conferences (e.g., Society for Industrial Organizational Psychology Annual Meetings). Rich received his PhD in Industrial and Organizational Psychology from The University of Akron.
Nancy T. Tippins is a Sr. Vice President and Managing Principal of the Selection Practice Group of Valtera Corporation where she is responsible for the firm's development and execution of strategies related to job analysis, competency development, employee selection, assessment, and leadership development. Nancy oversees the teams that develop legally and professionally compliant tools, administrative processes, and delivery platforms to meet client staffing, assessment, and succession planning requirements. Nancy also conducts executive assessments and provides expert support in litigation. Nancy is active in professional affairs. She has a longstanding involvement with the Society for Industrial and Organizational Psychology (SIOP) where she served as President (2000-2001). She currently serves on the Joint Committee to revise the Standards for Educational and Psychological Tests. Nancy has made numerous presentations on tests and assessments and authored many papers on the subject. Recently, she co-authored Designing and Implementing Global Selection Systems, co-edited the Handbook of Employee Selection, and just completed an edited volume, Technology Enhanced Assessments, last year. Nancy received her PhD in Industrial and Organizational Psychology from Georgia Institute of Technology.