On the Legal Front
Pattern-or-Practice Dressed as Disparate Impact? A Review of Chin v. Port Authority
Kayo Sady and Eric Dunleavy
From time to time this column will showcase work from a guest columnist with a story to tell or a ruling worth reviewing. This issue we welcome guest columnist Kayo Sady. Kayo is a colleague of ours at DCI Consulting Group, where he focuses on adverse impact, pay equity, and validation research. Before joining DCI, Kayo did his graduate work in I-O at the University of Houston and worked at Valtera in their Houston office. Kayo noted an interesting police promotion case that has been staying under the radar, and his article focuses on that. We hope you enjoy his work as much as we do.
Before turning the column over to Kayo we wanted to briefly note three other issues. First, in October the Supreme Court heard oral argument in Fisher v. Texas. A comprehensive recap of those arguments can be found at www.scotusblog.com/case-files/cases/fisher-v-university-of-texas-at-austin/. We previewed the case in the July 2012 column (http://www.siop.org/tip/july12/
18gutman.aspx) and predicted the likelihood of various outcomes. Based on the oral argument, the use of race in a small percentage of admissions decisions at Texas may be in trouble. The question remains whether the Supreme Court rules narrowly on whether the Texas admissions policy is narrowly tailored or whether they take on Grutter v. Bollinger as good law. We expect to have a Supreme Court ruling to review for this column sometime in 2013.
Second, readers interested in adverse impact measurement should review the 10th Circuit Appeals Court ruling in Apsley v. Boeing, decided on August 27, 2012. For more detailed reviews of the case readers can check out summaries at www.ofccp.blogspot.com/2012/09/10th-circuit-rules-practical.html and www.ofccp.blogspot.com/2012/10/some-follow-up-thoughts-on-recent-eeo.html. We have recently spent substantial space in this column on adverse impact measurement, most notably in reviewing the 3rd Circuit appeals court ruling in Stagi v. Amtrak. (http://www.siop.org/tip/april11/17gutman.aspx). In that case, the 3rd Circuit essentially endorsed statistical significance testing over measures of practical significance in a scenario where selection rate differences were small but statistically significant. In Apsley, the 10th Circuit endorsed the opposite perspective and gave more weight to practical significance measures (e.g., the shortfall relative to the expected number of selections) than to significance tests, supporting that small rate differences were likely not due to chance. The court essentially noted that statistical significance tests can be trivial when sample sizes are very large and that practical measures should also be considered to determine whether a disparity supports an inference of discrimination. We found the 10th Circuit ruling to be insightful. It is worth a detailed read.
Third, as this column was being written we were about 2 weeks away from the presidential election. This election will certainly influence the EEO landscape for the next 4 years and beyond. For example, there are a number of proposed regulations sitting at the Office of Management and Budget (OMB) related to pay equity enforcement standards, affirmative action for veterans, and affirmative action for persons with disabilities. The fate of these proposed regulations may depend on the election. Enforcement priorities may vary substantially across administration. We will likely consider the election and EEO enforcement implications in the April 2013 column. With that, we turn it over to Kayo. Enjoy!
Chin v. Port Authority of New York and New Jersey
The Appeals Court ruling in Chin v. Port Authority caught my eye midsummer, and I thought it worth analysis for the broader TIP audience. In some aspects it is a typical police promotion case, but other aspects stood out as extraordinary. In particular, the court’s (a) reasoning for disallowing pattern-or-practice claims, (b) interpretation of null hypothesis significance testing, and (c) view of the role of anecdotal evidence in disparate impact cases all stood out as novel.
The case focuses on police promotion decisions. Eleven Asian-American plaintiffs alleged discrimination based on race by the Port Authority of New York and New Jersey. The plaintiffs alleged that the sergeant promotional process was discriminatory and sued under Title VII of the Civil Rights Act of 1964. The plaintiffs sought back pay, compensatory damages, and equitable relief. The plaintiffs alleged three different theories of discrimination: (a) individual disparate treatment, (b) disparate treatment pattern or practice, and (c) disparate impact. That is, the plaintiffs alleged that the Port Authority of NY and NJ intentionally discriminated on an individual basis, intentionally discriminated against Asian-American officers as a standard operating procedure, and had a facially neutral promotional process in place that unfairly excluded Asian-American officers.
Facts of the Case
The 11 plaintiffs were Asian-American police officers in the Public Safety Department of the Port Authority. During the time period of interest, entry-level police officers were promoted to sergeant based on a multiple-hurdle process involving both compensatory and noncompensatory selection procedures. Although the promotion process followed a series of steps, a lack of structure and standardization at some steps allowed for significant individual discretion. The promotional process included four steps:
- Step 1: Officers were required to pass an examination to be placed on a promotion eligibility list. Once an officer passed the promotional exam, the candidate remained on the eligibility list for 3 years. Requalification was required once 3 years had passed.
- Step 2: The commanding officer of each Port Authority’s facility (typically a captain) periodically recommended eligible officers for promotion based on the captain’s discretion. The Port Authority did not provide criteria to guide recommendations.
- Step 3: For each recommended officer, a promotion folder was provided to the Chiefs’ Board who would vote on each of the recommendations. Notably, in addition to performance evaluations, record of absences and disciplinary history, commendations, and awards, a photograph of the officer was provided in each promotion folder. Any officer receiving a majority of votes was recommended by the Chiefs’ Board, although no criteria were specified for whether an officer should be recommended. At times during the period of interest, this step was skipped altogether.
- Step 4: The superintendent made the final recommendations for promotion, and at times, promoted those rejected by the Chiefs’ Board.
On January 25, 2005, the Department of Justice issued a right-to-sue letter to the Asian Jade Society, who had filed a charge of race discrimination with the EEOC on behalf of its members alleging that the Port Authority illegally denied Asian-American police officers promotion to sergeant.
The case was heard in the District Court for the Southern District of New York. Each side employed both statistical and damage experts, and the trial lasted 9 days. In the end, the jury found for seven plaintiffs under all three theories of discrimination: individual disparate treatment, pattern or practice, and adverse impact. Back pay and compensatory damages were awarded to each of the seven. Equitable relief was also awarded in the form of retroactive promotions, seniority benefits, and salary and pension adjustments.
The Port Authority filed a motion for a new trial, but the District Court denied it and held:
- Background evidence from outside the statute of limitations can support a timely claim.
- Under an individual disparate treatment claim, the Port Authority could be liable for only those “discrete acts” within the statute of limitations. However, “continuing-violation” doctrine applies to pattern-or-practice and adverse impact claims because they are premised on an ongoing discriminatory policy (effectively allowing recovery for discrete acts outside the statute of limitations insofar as they were part of an ongoing discriminatory policy extending into the statute-of-limitations period).
- Lack of statistical significance as evidence of discrimination does not matter if the jury has enough other evidence of discrimination to find for the plaintiffs under all three theories of discrimination.
- Compensatory damages determined by the jury were upheld because they were not dissimilar to awards upheld by other judges and because they were not excessive.
On appeal, the Port Authority argued for a new trial on the grounds that:
- Evidence predating the limitations period was unlawfully admitted and considered by the jury.
- The evidence presented did not support the jury’s decisions.
- The damages and equitable relief are excessive and are based on periods of time outside the statute of limitations.
In the July 10, 2012 ruling, the appellate court ruled in favor of the plaintiffs on some accounts and against them on others. The three most notable findings concerned:
- The viability of a pattern-or-practice claim.
- Whether evidence presented at trial was sufficient to justify the jury’s findings.
- The relevance of “continuing-violation” doctrine to individual disparate treatment and adverse impact cases.
The Pattern-or-Practice Claim
With respect the pattern-or-practice claim, the appellate court held the method of proof established in International Brotherhood of Teamsters v. United States, 431 U.S. 324 (1997) was unavailable to the “nonclass private plaintiffs in this case.”1 The court argued that allowing nonclass private plaintiffs to file under a pattern-or-practice theory of discrimination beyond the class action context would effectively allow individual disparate treatment claims to move forward using a lighter burden at Phase 1. That is, individual plaintiffs would have a less difficult time meeting the burden of a prima facie case under a pattern-or-practice method than an individual disparate treatment method, and the court noted that such application of the pattern-or-practice method extends beyond its original or current application. Indeed, they noted, “Outside the class context, however, private plaintiffs may not invoke the Teamsters method of proof as an independent and distinct method of establishing liability.” Thus, the court threw out the pattern-or-practice claims but held that the individual disparate treatment claims and adverse impact claims were still appropriate.
Sufficiency of Evidence
With respect to the sufficiency of evidence presented at trial, we focus on the Port Authority’s two defenses related to Phase 1 of a disparate impact claim:
- Plaintiffs’ statistical evidence of adverse impact was fatally flawed, thus failing to make a prima facie case.
- Plaintiffs failed to identify the specific promotion practice that caused disparate impact, thus failing to make a prima facie case.
The court’s response to the statistical argument provides perhaps the most interesting, and alarming, part of the case. The Port Authority argued that analyses relying on data predating the statute of limitations period were not admissible; the court disagreed. The Port Authority argued that analyses relying on officers overall rather than officers on the eligibility list as the available labor pool were not admissible; the court agreed. Neither ruling is surprising, but the court’s interpretation of available statistical results is surprising. At the center of Phase 1 were analyses submitted by the plaintiffs’ expert Dr. Christopher Cavanagh. The analyses evaluated promotion rate differences between Asian Americans and others based on the eligibility lists. Although the Port Authority presented eligibility lists from three time periods (1996–1999; 1999–2002; 2002–2005), Dr. Cavanagh limited analyses to the data available up until the EECO complaint was filed (January 31, 2001).Table 1 includes the sample sizes of those on the eligibility lists and those promoted to sergeant during the three time periods.
Comparison of Eligibility List and Promotion Numbers
Note. Between 1996 and January 31, 2001, zero of 12 eligible Asian Americans were promoted, whereas 36 of 259 White Americans were promoted.
The court notes that from 1996 until the EEOC complaint was filed (January 31, 2001), not one of the12 Asian Americans on the eligibility lists had ever been promoted to sergeant. Such cases of an “inexorable zero” do not typically bode well for defendants, and experienced EEO professionals might reliably predict the outcome of the case based on this one fact. However, it was the court’s interpretation of Dr. Cavanagh’s statistical analyses that should raise eyebrows. The results of the Fisher’s Exact Test (FET) that Dr. Cavanagh conducted on the data between 1996 and January 31, 2001 indicated a probability value of .13.
If the null hypothesis is true (i.e., Asian Americans are just as likely to be promoted as White Americans in the population), there is a 13% chance that the Asian-American promotion rate would be smaller than the White-American promotion rate by a value equal to or greater than the selection rate difference observed in the data. The Port Authority argued that such a probability value does not provide enough evidence to infer discrimination, as a commonly accepted convention for determining statistical significance is a probability value of .05 or less. This is where it gets interesting. The court noted that relevant case law suggests that courts should evaluate disparities on a case-by-case basis by considering not only statistics but also the facts and circumstances germane to the question of discrimination. Further:
In many (perhaps most) cases, if there is a 13-percent likelihood that a disparity resulted from chance, it will not qualify as statistically significant. In this case, the plaintiffs offered other evidence that reasonable jurors could have relied upon to find that an 87-percent likelihood that the disparity was not due to chance qualified as significant…the plaintiffs presented a substantial amount of evidence that reasonable jurors could have relied on to conclude that the plaintiffs were more qualified than some of the white officers who were promoted, including comparing length of service, attendance records, and disciplinary histories. In the context of this case, it would not be unreasonable for a juror to find Dr. Cavanagh’s statistics significant despite only being significant at the 13-percent level.
It looks like the court fundamentally misunderstood null hypothesis significance testing. Bifurcating the sampling distribution into “percent due to chance” and “percent not due to chance” is not only incorrect but also leads to some interesting logical hoops through which to jump. For example, under the court’s interpretation it is likely that:
- When the Asian-American selection rate is less than the White-American selection rate, the likelihood that the disparity is due to chance is less than the likelihood that it is not due to chance.
- When selection rates are equal, there is a 50% likelihood that the lack of disparity is due to chance and a 50% likelihood that it is not due to chance.
- When the Asian -American selection rate is greater than the White- American selection rate, the likelihood that the disparity is due to chance is greater than the likelihood that it is not due to chance.
Problems with dividing the sampling distribution into percentage due to chance and percentage not due to chance aside, it appears that the spirit of the court’s assertion was to suggest that higher Type I error rates than α = .05 may be acceptable (or even appropriate) given a substantial amount of evidence pointing to discriminatory practices. If that is the case, there are a number of notable implications. One, introduction of anecdotal evidence in Phase I is historically the domain of pattern-or-practice cases and not adverse impact cases, but the court’s ruling suggests that if statistical analyses are inconclusive, strong anecdotal evidence may meet requirements for a prima facie case under an adverse impact theory. Two, strong anecdotal evidence may act as a nonstatistical measure of practical significance that augments the interpretation of statistical significance on its own. It will be interesting to see whether this notion is used in other cases.
Failure to Identify Specific Promotion Practice
The court ruled that the three parts of the Sergeant promotional process could not reasonably be separated and evaluated individually because the decisions made at each step did not necessarily limit the applicant pool going forward. For example, the Chiefs’ Board recommendations were disregarded or not requested by the uperintendent at times. The Chiefs’ Board was similarly inconsistent in its treatment of commanding officer’s recommendations. No surprises here, and this scenario would have left the pattern or practice argument on the table had the plaintiffs appropriately established class status.
With respect to the Port Authority’s argument that continuing-violation doctrine does not apply to this case because promotion decisions are discrete acts, the court agreed and reversed the District Court’s ruling that continuing-violation doctrine did apply. The court noted that timely claims to an incident of discrimination that is part of an ongoing policy of discrimination invoke the continuing-violation exception to the Title VII limitations period and render timely all claims of discrimination under that policy. However, consistent with the Supreme Court’s ruling in National Railroad Passenger Corp. v. Morgan (2002), the court ruled that failure to promote decisions are discrete acts, and a series of discrete acts does not constitute an ongoing policy. The finding significantly reduced the awards available to the plaintiffs.
As this column has reiterated many times, it has become common for plaintiffs and/or federal agencies to allege multiple theories of discrimination at once. On its face, the distinction between intentional and unintentional discrimination is an intuitive one, yet the EEO community continues to struggle with both (a) the differences between pattern or practice scenarios and disparate impact scenarios and (b) the role of statistics to help determine whether an inference of discrimination is reasonable. Chin v Port Authority appears to be another ruling where pattern and practice and disparate impact scenarios were confused and where statistical significance tests were interpreted in ways that are inconsistent with social scientific recommendations. The I-O community is in a position to help inform EEO decision makers on these issues, and hopefully this review is another step forward on that journey.
Apsley v. Boeing. (2012). [U.S. App. Lexis 18161].
Chin v. Port Authority of New York & New Jersey. (2012). 685 F .3d 135 (2d Cir.).
Fisher v. University of Texas at Austin. (2010.) U.S. Dist. LEXIS 3478.
Grutter v. Bollinger. (2003). 539 U.S. 306.
International Brotherhood of Teamsters v. United States. (1997). 431 U.S. 324.
National Railroad Passenger Corp. v. Morgan. (2002). 536 U.S. 101.
Stagi v. Amtrak. (2009). U.S. Dist. LEXIS 71207, 8/12/09.
Stagi v. Amtrak. (2010). U.S. App. Lexis 17261, 8/16/10.