FairNow: Conversational AI And Chatbot Bias Assessment

More organisations are starting to use chatbots for many purposes, including interacting with individuals in ways that could result in harm from differential treatment in terms of the user's demographic status. FairNow's chatbot bias assessment provides a way for chatbot deployers to test for bias. This bias evaluation methodology relies on the generation of prompts (messages sent to the chatbot) that are realistic and representative of how individuals interact with the chatbot.  

In order to test for bias in a chatbot, FairNow’s platform populates a suite of relevant prompts with information that associates the prompt with an individual’s race or gender. The evaluation analyses differences in responses between demographic groups to understand if the chatbot treats members of a different group more or less favorably. The evaluation varies by prompt type in terms of the specific content being assessed. Where customers have logs of previous chatbot interactions and are able to share them, FairNow leverages those logs as context to ensure the bias evaluation reflects user queries and engagement in terms of content, style, and tone. 

The evaluation of bias in chatbots and large language models is a new and evolving space. Companies looking to deploy chatbots in a way that doesn’t favor individuals in certain demographic groups need a way to understand the risks their applications pose and the magnitude of potential issues. FairNow’s chatbot assessment methodology enables users to evaluate their models before they deploy and as part of ongoing monitoring.  

Organisations attain high-fidelity bias evaluations of their models that reflect the ways in which their customers use the chatbot. Compared with existing chatbot bias benchmarks – which are often not specific enough to reflect actual usages – FairNow’s chatbot bias assessment methodology enables organisations to pinpoint specific issues with bias in relation to the chatbot’s intended and realized use.  
The evaluation can be run at any point and does not require the organisation to share any protected data from customers or employees since the prompts are synthetically generated.

The field of chatbot and LLM evaluations is emergent, and FairNow is committed to ongoing research and development to stay at the forefront of LLM bias testing.

• First, the field doesn’t yet fully understand the sensitivity of evaluation results to changes in testing procedures. Research shows that evaluation outcomes can change unexpectedly due to slight changes in the wording or style of the input prompt.  
• Second, this evaluation is not comprehensive of all the different ways that a chatbot could display bias. The evaluation currently tests for bias by gender and race, and does not yet test for bias in terms of other relevant factors like age.
• Lastly, this bias assessment is focused on bias (and robustness of responses to individuals of different demographic groups), and is not designed to measure a chatbot’s safety or security posture.     

FairNow is committed to following the latest scientific literature on this topic and applying our own testing to reduce these limitations.

Related links: 

• Link to the full use case.
• Case study bias evaluation for recruiting chatbot.
• FairNow and bias assessment.
• Information on FairNow AI regulatory compliance tools.

This case study was published in collaboration with the UK Department for Science, Innovation and Technology Portfolio of AI Assurance Techniques. You can read more about the Portfolio and how you can upload your own use case here.