Resaro’s Bias Audit: Evaluating Fairness of LLM-Generated Testimonials

A government agency based in Singapore engaged Resaro’s assurance services to ensure that a LLM-based testimonial generation tool is unbiased with respect to gender and race and in-line with the agency’s requirements. The tool uses LLMs to help teachers become more efficient and effective in writing personalised testimonials for their students. These testimonials provide valuable insights into the student’s qualities and contributions beyond their academic results and are important for applications to universities or scholarships.

LLMs trained on large corpus of historical documents may perpetuate gender and racial biases, which they may have learnt from large amounts of data from the Internet. To ensure that the testimonial generation tool was safe before putting it out for wider use, Resaro developed a method to quantify differences in the quality of generated LLM output with respect to input attributes. The solution identifies bias based on differences in language style (how it is written) and lexical content (what is written) of the generated testimonials across gender and race for students with similar qualities and achievements. The audit process also flagged up other issues with the approach – hallucination on a small proportion of generated testimonials and limitations of the model in understanding local context. These issues were subsequently addressed as part of product design and guardrails, giving the agency additional confidence on the safety and robustness of the product released.

Existing social science research has shown that there are significant gender biases in letters of recommendation, reference letters, and other professional documents. For a government agency, it is important that such biases are mitigated before the tool is released. A third-party audit provides additional independent checks at a broader scale and scope than might have been tested by an internal team. Since this evaluation was conducted at the pre-deployment phase, Resaro had to construct its own representative sample of input prompts that captures the variety and breadth of the expected tool usage. A broad statistical study also allowed the team to review the variation in responses and generate statistically significant results in the evaluation of bias.  

Resaro’s LLM bias audit helps provide additional assurance to organisations launching LLM based products to ensure that they are in compliance with AI regulations, industry standards, and business use case. An independent validation helps build trustworthiness of the system and uncover blind spots before the system is released into production.

A bias audit is limited by the ability of Resaro to access the underlying model or system, and may not be fully representative of the end-to-end deployment process that the model is being used. The audit was also done with the inputs of the agency based on representative student profiles. Differences in how users interact with the system and the quality of input data may vary and could affect the resultant output. Changes in the foundation model and drift in student profiles over time will also necessitate conducting such audits more regularly.

Link to the full use case.

This case study was published in collaboration with the UK Department for Science, Innovation and Technology Portfolio of AI Assurance Techniques. You can read more about the Portfolio and how you can upload your own use case here.