Bias audit and impact-ratio testing for automated decisions

Description

NYC Local Law 144 has the most operationally specific bias-audit obligation in any of the AI laws currently on the books: an independent third party must compute the four-fifths impact ratio across protected categories for each automated employment-decision tool, publish a summary on a public-facing URL discoverable from the application surface, and update it annually. The statute is precise about the methodology in a way other AI laws are not, with the result that LL144 has become the audit framework regulators elsewhere copy when they want a rule that survives an enforcement challenge. The accidental result is that an LL144-compliant audit covers most of what the EU AI Act Annex III bias-testing obligations require (note: Colorado AI Act SB 26-189 narrowed scope and no longer mandates the bias-testing parallel SB 24-205 had created), even though the three regimes do not cross-reference each other. A working bias-testing program has three operational layers. The data layer is the per-decision record set: outcomes by protected category, sampling cadence sufficient to support statistical inference, and exclusion criteria that survive an audit (the candidate pool for a hiring tool is not the population of people who applied; it is the population the tool actually scored). The methodology layer is the impact-ratio calculation itself, with documented sub-group definitions, statistical-significance thresholds, and the handling of intersectional categories where sample sizes get thin; this is where the four-fifths rule lives. The publication layer is the public-facing audit summary regulators expect to be able to find without a FOIA request, with the audit date, scope, methodology, results by category, and the auditor's identifying information. Independent-auditor selection sits across all three layers: the auditor's methodology document, conflict-of-interest disclosure, and engagement scope are themselves audit artifacts that examiners review alongside the impact ratio. The numbers are converging on a pattern. NYC LL144 requires the impact-ratio bias audit annually and within one year of any material change to the tool, published at a URL discoverable from the application surface; the four-fifths threshold flags any selection rate for a protected category that falls below 80 percent of the rate for the highest-selected category. Colorado AI Act under SB 26-189 (repealing and replacing SB 24-205 in May 2026; effective 2027-01-01) substantially NARROWED the original framework: the mandatory pre-deployment bias-testing + algorithmic-discrimination risk-management obligations were ELIMINATED. SB 26-189 retains adverse-decision notices for ADMT used in seven covered domains but no longer mandates the bias-testing cadence SB 24-205 had imposed. Operators may still elect bias testing as evidence of due care under generic Colorado consumer-protection authority, but it is not a Colorado-statute requirement. EU AI Act Annex III lists eight high-risk-AI use cases (employment, education, essential services, law enforcement, migration, justice, biometric categorization, critical infrastructure) for which Article 15 bias testing and accuracy obligations apply, with the bulk of high-risk-system obligations effective 2026-08-02. The audit windows are shorter than the deployment review cycles most operators are used to running, and the public-disclosure expectation makes the audit harder to bury than a typical internal-control failure. The genuinely uncertain piece is what counts as an independent auditor under each regime. NYC LL144's implementing rules define independence operationally (no consulting work for the employer on the same tool within the audit window, no shared ownership, no employee relationship), but Colorado has not yet published parallel guidance, and the EU AI Act's notified-body conformity-assessment regime is still standing up. An auditor that satisfies LL144 may or may not satisfy the EU's notified-body criteria once those criteria solidify; operators auditing across all three jurisdictions are running parallel engagements until the regimes harmonize.

Applicability

Applies when: features include automated-decisions.

How predicates are evaluated

Required by (5 regulations)

• NYC LL144

  Employers and employment agencies using an automated employment-decision tool to substantially assist or replace discretionary employment decisions in NYC must commission an independent bias audit within one year prior to use, publish a summary on a public-facing site, provide candidate notice at least 10 business days in advance, and update the audit annually. The audit computes the impact ratio (selection rate for each protected category divided by the highest selection rate among categories) against the four-fifths threshold.

  NYC Local Law 144 of 2021 (Automated Employment Decision Tools); enforcement effective 2023-07-05; DCWP rules at RCNY Title 6 § 5-300 et seq.

• Colorado AI Act

  SB 26-189 substantially narrowed the Colorado AI Act framework: the mandatory algorithmic-discrimination risk-management program from SB 24-205 was ELIMINATED. SB 26-189 retains adverse-decision notice obligations for ADMT used in seven covered domains but no longer requires the bias-testing cadence that SB 24-205 had imposed. Operators may still elect to conduct bias testing as evidence of due care under Colorado UCC consumer-protection authority, but no longer as a mandatory SB 26-189 compliance obligation.

  Colorado AI Act (SB 26-189, repealing and replacing SB 24-205); effective 2027-01-01; codification sections pending Governor signature and AG rulemaking

  Source →

• EU AI Act

  Annex III enumerates eight categories of high-risk AI systems (biometrics, critical infrastructure, education, employment, access to essential services, law enforcement, migration and border control, administration of justice). High-risk systems are subject to Article 9 risk management, Article 10 data governance covering bias testing on training and validation data, and Article 15 accuracy and robustness obligations, with conformity assessments before market placement.

  Regulation (EU) 2024/1689 of the European Parliament and of the Council (AI Act); high-risk-system obligations applicable 2026-08-02

• FTC Act

  Section 5 prohibits unfair or deceptive acts in or affecting commerce. The FTC has applied Section 5 to AI-enabled discrimination in advertising, hiring, and consumer-facing automated decisions, with consent decrees requiring algorithmic disgorgement, bias-testing programs, and ongoing reporting (Rite Aid 2023 facial-recognition order; Amazon Alexa 2023 order).

  Federal Trade Commission Act § 5; 15 U.S.C. § 45; in force since 1914 with subsequent amendments

• Illinois AIVIA

  Supports the fairness posture behind AIVIA's 2022 demographic-reporting amendment (820 ILCS 42/20).

  Illinois Artificial Intelligence Video Interview Act

  Source →

Evidence formats

• most-recent independent bias audit report with impact-ratio results by protected category
• annual impact-ratio publication on a public URL discoverable from the application surface
• independent-auditor engagement contract documenting scope and conflict-of-interest disclosure
• auditor methodology document covering sub-group definitions and statistical-significance thresholds
• remediation plan addressing prior-year audit findings with closure status and timeline
• per-decision record set supporting the audit (outcomes by category, sampling cadence, exclusion criteria)