I Meets O: Implementing New Selection Systems as Change Management January 2008

There is abundant literature on designing and developing tests and assessments for use in business settings. However, as Muchinsky (2004) noted, little attention has been paid to the contextual factors that influence the success of newly designed selection processes in organizations. We want to provide a perspective on how O-level organizational factors impact on traditional I-level interventions such as the implementation of new selection procedures, performance management tools, or training programs. In this article we would like to raise awareness of this interaction, share a few illustrative experiences and “lessons learned” specifically in the selection context, and call on readers to submit their own experiences and lessons learned (see end of article for details). We hope to encourage our colleagues in academia and field settings to consider these contextual issues and strategies as they study, develop, and/or implement new selection procedures in organizations.

Background

Anyone involved in developing a selection process for an organization can attest to the presence of organizational pressures that often lead us to compromise the dictates of best practices and theory of how to design selection assessments. In fact, the newly revised SIOP Principles for the Validation and Use of Personnel Selection Procedures (2003, 4th edition) acknowledges how important the organizational context is when conducting selection and assessment in field settings. What is missing though is discussion and research addressing the specific factors and strategies that may help ensure high-quality selection procedures given these organizational constraints and pressures. Exploring these issues more systematically may hold a key to bridging the gap between theory and practice. The importance of these O-level forces can not be overstated. Our efforts in developing a technically sound selection, performance appraisal, or training system become futile if the organization decides in the end to not use the process or use it in ways not intended (as noted for performance appraisals by Murphy & Cleveland, 1995).

Illustrative Experiences

We have collectively had many experiences where O-factors influenced the design and implementation of selection procedures. To share a few real-world examples:

A rigorous criterion validation study demonstrates that the addition of a cognitive ability test to a selection battery would increase the overall multiple regression coefficient to a small (2% incremental variance explained) but statistically significant degree. However, this addition would also introduce some (but less than the 4/5ths threshold) adverse impact against Black applicants. The organization had invested in the design of a new selection procedure under a consent decree to settle a suit based on racial discrimination. As a result of a desire to change its public image, tainted by those earlier charges of discrimination, the organization’s leadership decides not to use the cognitive ability test in the selection battery, despite the validity evidence.
A large manufacturing organization with a long history and strong commitment to rigorous selection procedures is updating its programs and conducts a job analysis study on several entry-level jobs. Many senior plant leaders hired under the existing selection program are skeptical whether updating is really necessary. Instead of relying on sound, sampling procedures to identify a sample that would yield adequate statistical power, the organization oversamples and collects data from large numbers of respondents for each job at each plant so that every plant feels a stake in the selection procedures that will be designed on the basis of the job analysis.
Members of the subject matter expert panel are asked to write items to reflect different content domains for a job knowledge test. Some members are better item writers than others. The least skilled item writer is the member with the highest rank in the organization; this person’s support is critical if the test is to be implemented. This person’s items are included to assure his support even though their inclusion detracts from the overall quality of the test.
The director of human resources recognizes that the sales managers in his company are using questionable criteria for selecting new salespeople. He engages an I-O psychologist to design a new procedure and reaches out to allies in the sales organization to serve as subject matter experts and guide the new tool’s design and successful validation. When the procedure is ready for roll-out, he asks the I-O psychologist to present the new procedure—a simulation—to the executive committee. The committee is indignant that the HR director undertook this initiative without their prior buy-in and shelves the initiative altogether.

Lessons Learned

From these and other real-world experiences we have arrived at some lessons learned to maximize the likelihood that a rigorous selection process will be implemented and sustained. A few of those lessons learned:

Involve key stakeholders early and keep them involved, especially those who may be resistant; it’s never too early to involve key constituencies, especially IT
Re-engage key stakeholders at critical milestones and decision points
Whenever possible, meet key stakeholders face to face
Address resistance by proposing a partial implementation, aka “a pilot”
Present decision makers with alternatives that contrasts the right choice with a choice that would be unacceptable
Err on the side of inclusion when identifying the stakeholders whose buy-in is needed
Sustain the involvement of key people even after implementation
Over time, use the organization’s own norm in addition to, or instead of, raw scores or external norms.

Applying O-Level Theory

We believe a number of perspectives drawn from the organizational literature could be fruitfully applied to understand and address the impact of O-level factors on I initiatives. A few of these include:

Viewing selection systems as expressions of the organization’s vision and/or culture
Envisioning the design and implementation of new selection processes as a way to create organizational change (if “the people make the place,” different kinds of people will make the place different).
Understanding power and its application in shaping HR policies and practices
Recognizing the organization (or HR) is an entity that conducts external sensing and often imitates perceived leading-edge companies in the adoption of new selection procedures
Considering the expression of values other than equity/meritocracy as a basis for selection system design (need, nepotism, image)
Using communication strategies that facilitate or undermine the implementation of selection procedures

Muchinsky (2004) argued that it may be beneficial to view the implementation of a selection program in an organization from an organizational change perspective. In this vein, the organizational change literature can provide ideas of strategies that may aid during the design, development, and implementation of I-level processes and programs.

For example, we have outlined below the O-level factors that could affect the design and implementation of a new selection procedure using Kotter’s (1996) well-known eight-step model of change leadership. These steps, touching lightly on just a few of the issues, include:

Increase urgency: What is the burning platform used to stimulate an investment in time and resources to create and implement a new selection system? Is HR “rattling the chains” about the risk of lawsuit or does senior leadership feel that a different skill set is required in light of evolving business strategies?
Create the guiding team: Who owns the new process and its creation? What stakeholders are included in the Project Advisory team?
Get the vision right: Are the expected outcomes of the new procedures clear and measurable and easily explained?
Communicate: Who needs to know and what do they need to know? Do the implementers (e.g., recruiters, test administrators, hiring managers) know what’s coming?
Empower action: Who is empowered to make the final judgments? What decision criteria should they use? From which stakeholders must buy-in be secured?
Create short-term wins: How do you publicize initial success? Are there short-term operational metrics that are seen as short-term wins (e.g., reduced cycle time) pending completion of longer term validation studies against performance criteria?
Don’t let up: Keep focused on the ultimate criteria against which to demonstrate the value of the new procedures.
Make change stick: How does the new procedure become “the way we do things around here” and insulated from the whims of a new leader or of short-term economic ups and downs?

The intention is not to work out a comprehensive model here but to give you a sense of how we’ve been using O-constructs and perspectives to understand better the context within which our selection process interventions occur.

Call for Other Examples

Have you experienced any similar issues? We are calling on you, dear reader, to e-mail us your experiences that illustrate the impact of O-factors on I-interventions. Please e-mail all your experiences to the second author, Therese Macan, following the questions below. Her e-mail address is: Therese.Macan@umsl.edu.

In describing your own experience(s):

Briefly describe the organizational context and the new selection processes you were implementing.
What are the major contextual factors that influenced the success or failure of the selection program implementation?
What are some key strategies you applied in dealing with these contextual challenges?
What are your key lessons learned for dealing with these challenges?
In your opinion, what research is needed to advance theory and practice in this area?

Please share your examples with us. If we receive enough rich examples, we plan to examine any patterns within a more rigorous model of organizational change and share our results with you. Our goal is to build on this brief article at raising our field’s awareness and create a more detailed theoretical framework in a future research publication. Therefore, from your examples, we hope to provide a more comprehensive treatment of implementation strategies and to spark much-needed research studies in this area. Please send your examples to Therese.Macan@umsl.edu.

References

     Kotter, J. P. (1996). Leading change. New York: Free Press.
     Muchinsky, P. M. (2004). When the psychometrics of test development meets organizational realities. A conceptual framework for organizational change, examples, and recommendations. Personnel Psychology, 57, 175–209.
     Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
     Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.