Armilla Verified: third-party AI product verification

Armilla Verified is Armilla’s third-party verification of AI-powered products. It empowers AI vendors and enterprises alike to assure the quality and reliability of their AI solutions. Armilla Verified is a holistic, socio-technical assessment that has been derived from global best practices and standards for AI testing and risk management. The system leverages proprietary model evaluation technology to comprehensively, efficiently and cost-effectively validate the data sets, performance, fairness, interpretability, robustness and security of a given AI solution considering its wider context and business requirements. Eligible clients receive the Armilla Verified quality assurance seal as well as an independent expert report containing the results of the assessment.
As AI capabilities continue to accelerate, so do the risks for enterprises. Our third-party verification for AI products provides organisations with the confidence they need to unlock the potential of AI while mitigating risk whilst meeting emerging compliance obligations.

Our third-party AI model verification is a powerful quality assurance and risk mitigation tool for vendors of AI solutions and the enterprises that procure them.  

• Vendors of AI-powered products leverage Armilla Verified to:
  • Assure their systems through independent validation and trustworthiness;  
  • Get ahead of evolving enterprise procurement requirements for AI;  
  • Prepare for compliance with AI regulations and industry standards.
• Enterprises procuring third party AI products leverage Armilla Verified to:
  • Evaluate AI risk specific to the business use case;
  • Quickly and efficiently weed out underperforming or immature vendors before they are onboarded;
  • Mitigate risk of downstream AI-related damages and liability;
  • Assure compliance with emerging AI regulations, and corporate policies and procedures.

Based on the results of the assessment, we provide expected limitations of the model. Assessment dependencies include the availability of a sufficiently complete, representative sample of training and test data supplied by the solution developer, which also forms part of Armilla's verification and has direct impact on its results. Another dependency consists in Armilla's ability to access the model in question, typically via an API endpoint.

Link to the full use case.

This case study was published in collaboration with the UK Department for Science, Innovation and Technology Portfolio of AI Assurance Techniques. You can read more about the Portfolio and how you can upload your own use case here.