Home Home | About Us | Sitemap | Contact  
  • Info For
  • Professionals
  • Students
  • Educators
  • Media
  • Search
    Powered By Google

On the Legal Front: 
Remembering Frank and Reactions to Ricci: A Settlement in Bridgeport, CT
 

Eric Dunleavy
DCI Consulting

Art Gutman
Florida Institute of Technology

Additional Comments:
James L. Outtz
Outtz and Associates

The field lost a superstar when Frank Landy passed away on January 12, 2010 after battling cancer. Frank was an exceptional I-O psychologist who cared deeply about the field. He excelled as an academic at Penn State and other universities, as a practitioner at various firms including SHL, Landy, Jacobs & Associates, and Landy Litigation Support Group, and as past president of SIOP. Frank wrote some of the most insightful and influential material available on validation theory, performance appraisal, and the role of I-O psychology in employment discrimination litigation. He also wrote Work in the 21st Century, one of the most used introductory texts in I-O psychology. He was a passionate speaker whose participation in presentations at professional conferences ensured some entertainment as well as content expertise. If you missed the funeral and would like to pay respect to Frank, donations can be made to the Memorial Sloan Kettering Cancer Center (http://www.mskcc.org/) in his name.

A Settlement in Bridgeport

At the time this article was written, the legal and human resource management communities have had 8 months to consider the implications of the Supreme Court ruling in Ricci v. Destefano. We have spent two columns in TIP (April and October, 2009) deconstructing the case and presenting what we think the likely implications are (and aren’t). Many organizations continue to grapple with what decisions are legally defensible in light of Ricci. Employment decisions aren’t made in a vacuum, and hiring and promotion systems similar to the system evaluated in Ricci were developed long before the ruling and continue to be used after the ruling. It isn’t unreasonable to expect some of those systems to be put under the microscope now via “traditional” adverse impact claims or “reverse discrimination” disparate treatment claims.

Intuitively, in this scenario, anyone could be a potential victim of discrimination. This is particularly the case when the following factors are involved: 

  • the employer conducted disparity analyses after an assessment was implemented;
  • the adequacy of the assessment is challenged; and
  • after an assessment had been implemented, the employer “changed” assessment implementation characteristics like cut scores, weighting schemes, and so forth (or cancelled the results as in Ricci) based in part on the results of the disparity analysis and/or adequacy of the assessment.

We have discussed the insulation provided by conducting proactive adverse impact analyses and stringent validity research before a test is operational, but note that this is above and beyond what is required under the Uniform Guidelines on Employee Selection Procedures (UGESP). UGESP requires validation research only when adverse impact is identified. In some situations, this “a priori” work may not be feasible given test security concerns, budgetary constraints, and so on. What are I-O practitioners to do?

A recent settlement out of Bridgeport, CT is worth noting, particularly in light of Ricci. The settlement made local headlines just before the holidays in 2009. As reported by various media outlets, the settlement stemmed from a claim of discrimination in promotions made by 12 white firefighters. The claimants asserted that the city of Bridgeport discriminated against them by changing test weighting schemes postadministration in an attempt to reduce adverse impact against minority applicants. According to reports, after the test had been implemented, the city changed the original weighting scheme from 50% written test, 45% oral assessment, and 5% seniority to a scheme of 75% oral assessment and 25% written test. As we will witness below, the new weighting scheme did not alter the demographic representation of the applicants that were promoted. Nevertheless, the 12 plaintiffs in this case charged reverse discrimination, and the city of Bridgeport agreed to settle.

Unfortunately, this was a settlement and not an actual court ruling, and as such, few details of the settlement were made publically available. In theory, the city of Bridgeport viewed the facts of Ricci, interpreted the ruling, and was not optimistic of their particular circumstances. In other words, the city may have perceived that:

  • the weighting scheme of the promotion assessment was changed after implementation and based on the results of adverse impact analyses (i.e., “race-based” under Ricci standards);
  • the appropriateness of those weights may have been questionable; 
  • they did not have a strong basis in evidence to justify changing the weights.

SIOP Fellow Jim Outtz was mentioned in the local media reports as the testing expert who made recommendations for how to weight the different dimensions of the promotion assessment. The media falsely reported that these changes were made to increase minority promotions. Jim has been doing work for the city of Bridgeport for a long time, and we contacted him in the hopes of getting more detail on the situation and any specifics he could share on the settlement. As you will see, we are glad that we reached out to him; Jim was willing to write a guest piece on the case; he included some important details on the situation in Bridgeport that were not included in media reports. Jim’s summary of what happened in Bridgeport follows.

One View of the Bridgeport Settlement

I am offering this summary of what I know about the development of a fire lieutenant examination in the city of Bridgeport, Connecticut in the hope that it will provide a useful context in which to assess media reports about a Ricci-like settlement entered by that jurisdiction. I was not asked nor did I provide advice with regard to any aspect of the settlement. However I was directly involved in the development, implementation, and validation of the examination at issue, including weighting of the examination components and interpretation of the results. Elements of the information provided here are in the public record in the form of a letter submitted to the Bridgeport Civil Service Commission that became part of the record in a civil service hearing related to the examination.

Outtz and Associates was retained in 2007 by the city of Bridgeport, Connecticut to develop a promotion examination for the position of fire lieutenant. I had developed examinations for fire department positions for over a decade. The development process included the usual steps associated with sound professional practice including identification of important job tasks and areas of knowledge, skills, and abilities (KSAs) via a job analysis; ensuring that the assessment process was linked to the KSAs; and subsequent linkage of the selection process back to important job components. The resulting promotion process was an assessment center with a written multiple-choice test, work sample simulation, oral situational judgment test, and a form-completion exercise. A key factor in the implementation of the process as well as the subsequent legal dispute was weighting of the components.

The weighting of exercise components was a consistent problem in Bridgeport. For many examinations, particularly those used for public-sector (police and fire) selection, stakeholders such as member unions have sought to influence the weighting process. There also have been administrative rules requiring that changes to a selection process in Bridgeport be subject to negotiation with the union. These constraints often prove counterproductive because of legal obligations such as the Uniform Guidelines on Employee Selection Procedures (The Guidelines). The Guidelines require selection procedures be valid, if they have adverse impact. Note that stakeholders often have no legal obligations or culpability should a selection process run afoul of EEO laws. This leaves the jurisdiction caught in the middle should the selection process prove problematic. This is what happened in Ricci and appears to have occurred in the case of the Bridgeport settlement.

After the 2007 Bridgeport fire lieutenant examination was administered, Bridgeport Civil Service requested an assessment of the results from a number of perspectives, including whether (a) any appeals from individual candidates were valid and (b) exam results showed adverse impact. Note that the city had good reason to be concerned with regard to adverse impact because the weights for examination components were not derived directly from the job analysis (as they should have been) but rather were based on an agreement with a stakeholder and other administrative considerations. It should also be noted that civil service regulations require that examinations comport with principals of merit.

For the Bridgeport fire lieutenant examination, the weights called for by union agreement were 50% for the written component and 45% for the oral component. I recommended that the weights be based on the job analysis. Bridgeport Civil Service could then elect to accept or reject that recommendation. An additional complication arose for this examination however. The examination was unique in that it contained an oral exercise that measured the candidate’s ability to handle emergency situations (e.g., emergency fire incidents) safely. It had taken a number of years to perfect this component but once developed, feedback from subject-matter experts within and outside the Bridgeport fire department indicated that it had the greatest fidelity with the job of a fire lieutenant in an area of greatest importance to job success.

For this component of the examination, candidates were assessed not only with regard to their ability to apply job knowledge but also with regard to their safety performance at a fire scene. Specifically, the actions of the candidates were evaluated on whether they would result in serious bodily injury or death to citizens or fire department personnel. Such actions or omissions by a candidate represented grievous errors. Giving more weight to the written component of the examination than the oral component would in my opinion have significantly decreased the validity of the examination. I so informed the Bridgeport Civil Service Commission in writing (see Exhibit 1). The examination did not have adverse impact based on the projected number of promotions, but improperly weighting the most job-related component was of great concern. For example, using the administrative weights, of the 22 candidates eligible for promotion, 14% committed an act or omission during the fire incident simulation that would have resulted in death or serious bodily injury. I believed that this was unacceptable. Therefore given the job analysis results and the criticality of the KSAs they measured, I recommended that the oral component, which included the situational judgment component and the emergency fire incident simulation, be given a weight 75%, and the written component (the multiple-choice test and form-completion exercise) be given a weight of 20%. The weight for seniority was set at 5%.

This weighting would result in no candidate being promoted who committed an act or omission that would result in death or serious bodily injury based on the fire incident simulation. The demographic makeup of the candidates promoted would remain the same although the specific individuals being promoted would change. I also recommended that at a minimum, candidates who committed unsafe acts should be given additional training. The Civil Service Commission chose to accept my weighting recommendation. Twelve White firefighters subsequently filed a lawsuit claiming that the recommended weights constituted a change in the weighting process that prevented them from being promoted. The city of Bridgeport entered into a settlement (no doubt influenced by the Ricci decision) in which the weights were returned to those called for under the union agreement.

The Bridgeport settlement adds to the evolving body of case law in the aftermath of the Ricci decision. It raises a number of issues that should be of interest to I-O psychologists including (a) the ethical and professional obligations of the I-O psychologist when developing and implementing selection procedures, (b) the roles that stakeholders should play that process, and (c) the degree to which advances in selection procedure development should influence evolving case law.

Exhibit 1: Letter to Bridgeport Civil Service Commission

September 9, 2007
James L. Outtz, Ph.D.
Outtz and Associates
816 Connecticut Avenue, N.W.
Suite 800
Washington D.C. 20006

Mr. Ralph Jacobs
Personnel Director
Bridgeport Civil Service
City Hall, Room 325
Bridgeport, Connecticut 06604

Dear Ralph:

The purpose of this letter is to respond to your request for an assessment of the results of the Bridgeport Fire Lieutenant promotion examination. As you know, the weights for the written and oral components of the examination were set at 50% and 45% respectively. These weights are based on administrative considerations, and more specifically, weights that were used for prior promotion examinations. The weights will result in the promotion of 19 Whites, 1 African American and 2 Hispanic Americans based on the projected 22 promotions over the life of the promotion list. Although these figures show a disproportionate selection rate for Whites, there is no adverse impact as defined by the 4/5s rule of the Uniform Guidelines on Employee Selection Procedures or based statistical significance. There is a problem with the weights however with regard to the validity of the examination. This examination is unique in that it contains an oral exercise that directly measures the candidates’ ability to handle emergency situations (e.g., emergency fire incidents) safely. On this component of the examination, candidates are assessed not only with regard to their ability to apply job knowledge but also with regard to their safety performance at a fire scene. Specifically, the actions of the candidates were evaluated as to whether they would result in serious bodily injury or death to another person including fire personnel. Such actions or omissions by a candidate represent grievous errors. Giving more weight to the written component of the examination than the oral component that includes this specific fire incident simulation would in my opinion significantly decrease the validity of the examination. For example, of the 22 candidates who could be promoted, (this does not include the factor of seniority, but I don’t think that would affect the outcome) 14% will be individuals who committed an act or omission that would result in death or serious bodily injury. I believe that this is unacceptable. Therefore I recommend that the weights for the written and oral components of the examination be set at 20% (written) and 75 % (oral) with 5% remaining for seniority. This weighting would result in no candidate being promoted who committed an act or omission that would result in death or serious bodily injury. The demographic makeup of the candidates promoted would remain the same. At a minimum the candidates who committed unsafe acts should be given additional training. If you have any questions, please don’t hesitate to contact me.

James L. Outtz PhD


Implications

In our opinion (Eric and Art), the Bridgeport settlement is troubling for several reasons. First, it was misrepresented in the media. For example, as posted on December 3, 2009 by the Liberty Law (http://www.libertylawoffice.com/category/uncategorized/):

The 2006 test was rescored and reranked after Jim Outtz, a nationally known test designer, found the results have been weighted unfavorably against minority candidates.

As demonstrated in Exhibit 1, there was no adverse impact, and the rescoring had to do with safety not minority representation.

Second, the rescoring was the ethical and legal thing to do. Ethically, we have a responsibility to study and report any factor we believe affects the validity of a selection test. In this case, the issue was safety, and the fear, a legitimate one, was that selected firefighters could pose a danger to fellow firefighters and citizens. Legally, if a firefighter should be excluded for safety reasons and is not, there is liability in a negligent hiring suit if that individual causes injury or loss of life. As suggested by Jim, at the very least, individuals that fail the safety item should receive special training.

Third, the Bridgeport settlement is clearly an overreaction to the Ricci ruling. As we reported in October 2009 issue of TIP, Ricci is a disparate treatment case not an adverse impact case. The key question in Ricci was whether the city of New Haven had a strong basis in evidence for believing it would lose an adverse impact claim to minorities. A 5–4 Supreme Court majority ruled that New Haven had only a good faith belief it would lose, which is not good enough. No precedents were established relating to test validity. Indeed, in a recent release, the OFCCP gave the following answer to the question, “Does the Supreme Court’s decision in the Ricci case change how OFCCP will conduct compliance evaluations of contractors’ employment practices?”(See http://www.dol.gov/ofccp/regs/compliance/faqs/Ricci_FAQ.htm.)

No. The Ricci decision does not affect how OFCCP examines the use and impact of selection procedures, such as tests. OFCCP will therefore continue to assess whether a contractor’s use of its particular selection procedures complies with the Uniform Guidelines on Employee Selection Procedures (UGESP) at 41 CFR Part 60-3, available online at http://www.dol.gov/dol/allcfr/Title_41/Part_60-3/toc.htm.

Indeed, if anything, the Ricci ruling supports the rescoring of the test because Exhibit 1 provides a strong basis in evidence for believing the exam, as originally scored, had serious issues relating to its validity.

In summary, we are concerned by the ethical and legal implications of the Bridgeport settlement, particularly as it represents a false application of the Ricci ruling. This is troublesome regardless of the breakdown among us as to whether the Ricci ruling is good or bad. At stake here is the viability of selection testing. For example, on January 6, 2010, the Chicago Sun-Times (http://www.policeone.com/patrol-issues/articles/1986463-Chicago-police-may-scrap-entrance-exam/) reported that the city of Chicago is considering scrapping its entry exam. In the article, Fran Spillman and Frank Main suggest that scrapping exams would “bolster minority hiring, save millions on test preparation, and avert costly legal battles that have dogged the exam process for decades.” A similar article was written by Sun-Times columnist Neil Steinberg on January 8, 2010 (http://www.suntimes.com/news/steinberg/1979744,CST-NWS-stein08.article). We will continue to follow these and similar developments, and report what we find in future issues of TIP.

Reference

     Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. 1607 et seq. (1978).

Cases Cited

     Ricci v. Destefano 129 S. Ct. 2658 (2009).