BABL AI: Conducting third-party audits for automated employment decision tools

BABL AI conducts independent, third-party, criteria-based audits and certifications for automated employment decision tools (AEDT), with a particular focus on disparate impact (bias), governance, and risk assessment. In the language of the CDEI Assurance Guidance these are simultaneously ‘Compliance Audits’, ‘Certifications’, and ‘Bias Audits.’

Before an audit takes place we walk the prospective auditee through a series of questions to determine if sufficient testing of tools has been conducted to qualify for an audit. In cases where more preparation is required, we provide guidance for the prospective auditee. Once these criteria are met, then the auditee is on-boarded onto a client portal for progression to the actual audit phase. During the core phase of the audit the auditee submits documentary evidence to the client portal, which our trained and certified auditors review to satisfy audit criteria. During the review process, BABL AI auditors might ask for more supporting documentation or interact with the auditee’s internal and external stakeholders, such as employees or other third-parties, to verify the truth of statements made in the submitted documentation. Within the scope of our engagement, we perform amongst others the following procedures:

• Inspection of submitted documents and external documentation;
• Interviewing company employees to gain an understanding of the process for determining the disparate impact and risk assessment results;
• Observation of selected analytical procedures used in company’s bias testing;
• Inspection of the select samples of the bias testing data and results;
• Inquiry of personnel responsible for governance and oversight of the bias testing and risk assessment.