Improving Hiring Decisions: 9 Insights from the 2025 SIOP Annual Conference

By SIOP Fellow Steven Hunt

I was honored to facilitate a panel at the 2025 SIOP Annual Conference called “Criterion Validation in the Digital Age: Perspectives on Hiring System Design.” The session discussed using data to improve quality of hire and included assessment experts who manage hiring processes at Amazon, Verizon, and Johnson & Johnson, build hiring technology at HireVue, Phenom , Bryq, and Crosschq, and research validation methods at Rice University[i]. These individuals collectively oversee technology systems that process millions of hiring decisions annually. The following are nine insights from the panel that I found particularly intriguing[ii]. Given the backgrounds of the panelist who shared these thoughts, they are likely to shape the future of hiring system design.

1) Poor “quality of hire” data is a major challenge to improving hiring.
The most important decision a company makes about employees is the decision to hire them. Every other decision is a consequence of the initial action to bring them into the organization. Despite the importance of making good hiring decisions, companies struggle to collect accurate and comprehensive quality of hire data measuring the value employees provide after they are hired. This data includes things such as post-hire retention, time to achieve competence in their new role, job performance levels, and career progression. Technology exists that allows companies to effectively measure quality of hire, but this technology has not yet been widely adopted.

2) AI/ML hiring solutions are only as good as the data they are trained on.
The panelists all use AI/ML in some form to support hiring, and several are pioneers in development of highly sophisticated AI/ML recruiting methods. They believe in the value AI/ML can provide to improve hiring decisions, but expressed concern about the proliferation of AI/ML recruiting solutions that may not actually predict post-hire performance. Many AI/ML applications currently used for recruiting are trained using pre-hire data such as fit between candidate skills and job descriptions. While these solutions can improve recruiting efficiency and reduce time to hire, the fact they are not trained using post-hire data raises the risk that they could enable companies to make poor hiring decisions quickly and at a massive scale.

3) There is a need to expand the predictor and criteria space.
Panelists noted that hiring quality is influenced by many things that go beyond traditional candidate selection measures. One panelist shared their company’s definition of an assessment as “any tool that is used to gather information about an individual during their candidacy or employment process, assigning or categorizing, ranking, or comparing them to aid in employment decisions.” Note the definition includes tools used to aid hiring decisions, which includes technology used to guide candidate sourcing decisions such as candidate search algorithms. Companies have the potential to access data across the entire employment lifecycle spanning candidate sourcing, employee selection, new-hire performance, and employee exit. One of the next major evolutions in hiring process design will be finding ways to fully leverage this data to improve quality of hire.

4) It is not about having data, it is about having access to data.
As one panelist put it, “our organization collects a lot of pre- and post-hire data we could use to improve hiring quality, but we are not sitting on a mountain of data. It is more accurate to say we have a lot of small hills of data strewn around the countryside.” A challenge to using data to improve hiring methods is getting access to data collected in different systems and owned by different groups within the company. This includes pre-hire data such as interview ratings, test scores, and applicant sources and post-hire data such as attendance, compensation bonuses and merit increases, sales quota attainment, customer conversion rates, learning course completion, performance ratings, employee engagement, job transfers and promotions. Companies need better tools to integrate this data into a single system that can be used to analyze the effectiveness of hiring methods. Equally important, the organizational leaders who control access to this data need to support using it to measure and improve quality of hire.

5) Manager quality of hire ratings are better than nothing but not ideal.
Comparing post-hire manager performance ratings to pre-hire assessment scores is a longstanding method for validating the accuracy of candidate selection tools. Companies still use this method but are finding it to be increasingly inadequate for several reasons. First, it requires managers to provide the ratings which is often seen as an unacceptable given their workloads. Second, manager ratings are often inaccurate and questionable, particularly when they are asked to evaluate whether a hiring decision that they made was a good one. Third, using ratings tends to be unsustainable due to its manual nature. This prevents companies from continuously monitoring and improving the methods used to hire employees. The panel felt the future of assessment validation will increasingly move beyond manager ratings to other forms of post-hire data that can be collected automatically from other systems.

6) Synthetic validation is better than no validation.
Many of the use cases discussed in this panel involved hiring systems that process hundreds of thousands of hires per year. These systems collect large data sets that enable use of sophisticated statistical scoring methods. But what about use cases where companies are hiring small numbers of people and there is not adequate data to use advanced statistical validation techniques? In these cases, panelists shared that synthetic validation methods can work provided three conditions exist: a) the assessment measures well-established constructs that are known to reliably predict job-relevant behaviors, b) there is a system to guide hiring managers through a structured job analysis to identify job relevant constructs, and c) hiring managers actually know what behaviors are critical for job success. Empirical validation is always the preferred approach particularly for high volume hiring processes, but using synthetic validation to guide selection system design is better than relying on unstructured, poorly defined hiring selection methods.

7) Hiring assessment validities change over time, often quite drastically.
Several panelists shared stories of hiring assessments losing their accuracy over time. One assessment that predicted job performance when it was initially implemented was found to negatively predict performance years later. Another assessment experienced a rapid decline in predictive validity over just a couple of years. The reasons for changes in validity remain unclear and might be a result of changes in the jobs themselves or changes in applicant characteristics and values. But the lesson learned was the importance of constantly measuring and monitoring assessment validity.

8) What defines quality of hire can change over time.
One panelist shared a story where a company that used selection measures to hire high performing employees shifted its focus to emphasize hiring employees who might be more average performers but who were less likely to quit. This shift was in response to changes in the labor market that made overall workforce stability a more critical business issue than individual employee productivity. This ability to monitor and adapt hiring methods to changing business needs is likely to become more important as we see increasing levels of economic and labor market volatility.

9) Better tools are needed to explain hiring assessment validity.
Convincing business leaders of the value of using scientifically designed, empirically based hiring methods is an ongoing challenge in the field of talent acquisition. The panelist expressed the need for better reporting tools that can show business leaders why they should invest time and resources into using more sophisticated hiring methods. One panelist noted business leaders have different standards for different types of selection methods. In her example, business leaders were fine using in-person interviews without any form of validation. Some did not even consider the interview to be a form of assessment. In contrast, automated interviews were seen as a questionable form of assessment that had to be extensively validated before they would accept it.

These insights are a fraction of the information about improving hiring that were shared on this panel and at the SIOP Annual Conference overall. My sincere thanks go to the panelists and the members of the broader SIOP community for their commitment to apply psychological science to improve the world of work, and their willingness to share what they have learned with others who are on the same journey. Nothing good comes from hiring people into jobs they are unsuited to perform, and great things happen when people are matched to career opportunities that fit their interests, skills and potential. In my view, there are no bad hires, just bad hiring processes. This is something we can and should fix, and it was an honor to share the stage with people who are working to fix it.

[i] Ashley A. Walvoord, Anne Hansen, Christine Norris-Watts, Eleonora Makarouna, Fred Oswald, Mike Hudy, and Noelle Frantz.

[ii] Thank you to Isaac Thompson who recorded and provided a transcript of this session that served as an invaluable source of content for this article.

—

About the Author

Steven T. Hunt, Ph.D., is the Founder of i3 Talent LLC. An expert on the intersection of human psychology, work technology, and business performance, Dr. Hunt has helped organizations around the world build stronger workforces through improving methods to hire, develop, and engage employees. He has written hundreds of articles and several books. Dr. Hunt was formally honored as a Fellow in the Society for Industrial and Organizational Psychology for advancing applied psychological science through using technology to improve the quality of work for millions of employees. He is driven by the belief that better work environments create better world environments, and that every person deserves the opportunity to have a rewarding, enjoyable, and healthy career.

—

The views expressed in this article are those of the author and do not necessarily reflect the views of the Society for Industrial and Organizational Psychology or its affiliates.

If you are interested in submitting an article for Thought Leadership for a Smarter Workplace, email SIOP Senior Brand and Content Strategist Amber Stark at astark@siop.org.