On September 23, 2022, the New York City Department of Consumer and Worker Protection (DCWP) published proposed regulations under the city’s Automated Work Decision Tools (AEDT) Act, effective January 1, 2023. As previously reported on Cooley’s Cyber /data/privacy insights blog, the AEDT law requires employers and employment agencies in New York City to meet an extensive set of requirements before using an AEDT in the hiring and promotion process. Among other things, employers must:
- Conduct an independent bias audit of the tool.
- Provide a publicly available summary of the audit and tool distribution date.
- Notify candidates and employees who have applied for a job and live in the city that the tool will be used—as well as the job skills and characteristics that the tool measures when evaluating candidates—and allow candidates to choose an alternative to apply for competitive examinations or accommodation.
- Provide information about the source and type of data collected by the tool and the employer’s data retention policy, unless disclosure of this information would violate the law or interfere with a law enforcement investigation.
In passing the AEDT law, New York City joined other jurisdictions, such as Illinois and Maryland, in regulating AI-driven tools in the employment process due to concerns about bias or differential impact on certain groups.
key definitions
The DCWP will collect comments on the proposed rules ahead of a public hearing scheduled for October 24, 2022. Among other things, the proposed rules aim to define certain key terms and clarify requirements for bias testing, the publication of the results of the bias testing and the notices that employers and agencies must provide to employees and candidates. We have outlined some of these elements here.
Substantially assist or replace decision-making at its sole discretion
The AEDT Act defines an “automated job decision tool” as “any computational process derived from machine learning, statistical modeling, data analysis, or artificial intelligence that produces a simplified output, including a score, classification, or recommendation, that is used to make discretionary decisions about employment decisions, affecting, substantially assisting or replacing individuals.” The proposed regulations clarify that the phrase “substantially assist or replace discretionary decisions” means:
- Rely solely on a simplified output (e.g. score, tag, classification, ranking, etc.) without considering other factors.
- Use a simplified output as one of several criteria that weight the output more heavily than all other criteria.
- Use a “simplified output to override or modify conclusions derived from other factors, including human decision-making.” (A “simplified output” is further defined as a prediction or classification that takes the form of a score , a tag or a categorization, a recommendation or a ranking.)
candidate for employment
The AEDT Act applies to employment decisions affecting “applicants for employment or employees for promotion”. The proposed rules define the former as “a person who has applied for a specific job by submitting the necessary information and/or elements in the format required by the employer or employment agency”. Therefore, the proposed definition clarifies that candidates must have taken positive steps by submitting information or other items covered by the law.
Independent Chartered Accountant
The AEDT Act requires an independent auditor to conduct a bias audit on any AEDT tool before it is used. The proposed rules define an “independent auditor” as “a person or group not involved in the use or development of an AEDT and who is responsible for conducting a bias audit of such AEDT”. While this definition provides some guidance, it remains unclear what level of independence is required of the auditor, including whether the auditor may be otherwise employed by the provider of the AEDT.
Bias audit requirements
As discussed above, an AEDT cannot be used unless it has not been independently biased for more than one year prior to using the tool. The proposed regulations clarify the bias test requirements, which now include calculations of a “selection rate” and an “impact rate” for each category, to be reported to the Equal Employment Opportunity Commission (EEOC) according to the EEOC Employer Information Report EEO-1 (race, ethnicity and gender).
Under the proposed rules, “selection rate” means the “rate at which individuals in a category are either selected to advance in the recruitment process or are assigned a classification by an AEDT”. This rate is calculated by dividing the number of people in the category who were promoted or received a classification by the total number of people in the category who applied for a job or were considered for promotion. On the other hand, “impact ratio” is defined as either “(1) the selection rate for a category divided by the selection rate of the most frequently selected category, or (2) the average score of all people in a category divided by the average score of people in the category with highest score.” Therefore, an independent auditor conducting a bias audit must compare the selection rates for each EEO-1 category and compare those rates to the most selected or preferred category.
The types of calculations required in a bias audit depend on how the AEDT is used in the employment process. When an AEDT selects individuals to advance in the hiring process or groups them, the bias audit must calculate for each category the selection rate, impact ratio and, if an AEDT groups individuals, the calculations for the selection rate and impact ratio must be for any such classification shall be carried out. However, when an AEDT evaluates applicants or candidates, the audit must calculate the average score for individuals in each category and calculate the impact ratio for each category.
Publication of the results of a bias audit
Before using an AEDT, employers must post the date of the last bias audit of the AEDT, a summary of the results of the audit, including selection rates, “clearly and prominently” on the careers or job section of their website, impact ratios for all categories, and the distribution date of the AEDT or the date the employer began using a particular AEDT. The proposed rules provide that employers can meet this requirement by inserting a hyperlink to an external website that contains the required information. In addition, the required information must remain public for at least six months after the last time the AEDT was used to make a hiring decision.
Notice to Candidates and Employees
As discussed above, the AEDT Act requires employers to notify candidates and employees at least 10 business days prior to using the AEDT with specific information about the use of the tool and other requirements. The proposed rules will allow the necessary notices to be given to city-resident applicants by including the notice on the careers or jobs page of the employer’s website, in a job posting, or by email or post. An employer can meet termination requirements for current employees by including them in a written policy, procedure, or job posting, or by submitting them in person or by email or post. Although the notice must include “instructions on how to apply for an alternative selection process or alternative accommodation”, the proposed rules state ambiguously that an employer is not obliged to provide an alternative selection process.
The proposed regulations also clarify the legal requirement for employers to provide information about the type of data collected and their data retention policies. Employers can meet this requirement by posting the information on the careers or jobs section of their website, or by providing it in writing by post or email. If the notice is not included on the website, an employer must post instructions on how to request this information in writing on its website.
Next Steps
While the proposed rules are not final, employers can get ahead of the AEDT Act by reviewing any AI tools used in the hiring or promotion process to determine if those tools fall under the law. Employers using such tools should contact the tool providers regarding their compliance with the AEDT Act. Employers should also educate supervisors and human resources staff on the myriad requirements of the law, including its disclosure and disclosure requirements.
We will continue to monitor updates to the AEDT law after the October 24 hearing. If you have any questions about the AEDT Act, please contact a member of Cooley’s employment team.
