These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Resaro AI Solutions Quality Index (ASQI)
The objective is to make evidence of quality rather than hype, a driver of AI usage across the world. Without the ability to compare AI solutions with respect to quality (including aspects of performance, safety and security), markets lack transparency and thus lack competition as a driver for innovation. Moreover, without the ability to describe levels of quality, it is hard to agree on when a solution is “good enough”.
The Resaro AI Solutions Quality Index (ASQI) reflects the following characteristics that are necessary to ensure it is meaningful and useful: :
1. Specific to the use case - measuring a customer service chatbot isn’t the same as measuring a drone landing system or a deepfake detection solution. The ASQI approach combines a standardised overall framework with flexible content tailored to each use case.
2. A shared language - a quality index needs to be meaningful to business, governance, and technical teams. an ASQI creates a common language for AI quality that all stakeholders can work with.
3. Non-binary - quality is not a yes/ no characteristic. Every ASQI distinguishes 5 levels for each indicator — from best-in-class to minimal concern.
4. Mapped to automatable technical tests - every quality indicator within an ASQI links to technical tests that can be automated, translating results into the ‘shared language’ of the index.
5. The right level of detail - a quality index that is too broad is meaningless, too detailed is overwhelming. For a given use case, an ASQI uses about one to two dozen indicators spanning key aspects of performance and risk handling — a practical balance for real-world decision-making.
6. Compatible with established AI governance frameworks - an ASQI can support established regulations and standards like the EU AI Act, ISO/ IEC 42001, AI Verify, and company policies. Many indicators of quality will help to support compliance with such established governance frameworks.
7. Compatible with standardised task catalogues - while a quality index is designed to be at the system or solution level, it will inevitably refer to the 2-3 core tasks that a solution is designed to address.Such references should be to broadly recognised task catalogues as they are currently being developed in various standardisation initiatives.
About the tool
You can click on the links to see the associated tools
Developing organisation(s):
Tool type(s):
Objective(s):
Impacted stakeholders:
Target sector(s):
Lifecycle stage(s):
Type of approach:
Maturity:
Usage rights:
Target groups:
Target users:
Stakeholder group:
Validity:
Enforcement:
People involved:
Required skills:
Technology platforms:
Tags:
- evaluation
- ai quality
- benchmarking
- ai assurance
- ai solutions quality index
Use Cases
Would you like to submit a use case for this tool?
If you have used this tool, we would love to know more about your experience.
Add use case
Partnership on AI
