Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

Shakers' AI Matchmaking System

Apr 2, 2024

Shakers' AI Matchmaking System

With over 3000 registered talents and the trust of more than 450 companies, Shakers represents one of the largest communities of the booming European freelancing landscape. As such, Shakers has pioneered a novel AI-based matchmaking system to connect freelance talents and corporate project needs. This AI system is deeply integrated within Shakers' bilateral framework while covering different aspects of the entire lifecycle of project development—from initial talent and client registration to the final stages of collaboration and project completion.

 

Our system is designed to find a small but highly specialized pool of up to three highly vetted candidates for each project. This level of precision is achieved through a comprehensive analysis of various factors, including professional and personal experiences, sector knowledge, and personal skills, as well as individual behavioural information such as feedback from meetings, applications and past proposals or recent searches and interests shown by the talents. All this information is conveyed in a metric called Match, which represents relevant and personalized recommendations for both, talents and clients. Most relevant aspects contemplated within this match are presented in-app to all parties, so all our users have a better understanding of how our recommendations work. As of today, Shakers’ Match represents the backbone of our application and the basis upon which talents and clients meet and collaborate in our community.


 

Benefits of using the tool in this use case

Achieving Adigital’s certification of algorithmic transparency in 2023 underscores Shakers’s commitment to deploying transparent, responsible and ethical AI systems. As such, the evaluation processes highlighted several areas of improvement, particularly regarding protocols and stakeholder communication. As a few of these areas had been scarcely covered before we collaborated with Adigital, having the opportunity to work on them was highly valuable for us.

 

Furthermore, having the opportunity to work with an external, expert referee was beneficial to understanding the limitations and future directions of our system, and to clarify how to align AI-systems capabilities and business goals. Being in the position to start considering challenges about transparency, and ethical and responsible use of algorithms will ultimately benefit us in the long term. 

 

Lastly, working side-by-side with Adigital’s team has provided us with crucial insights into how regulatory actions might impact the digital startup ecosystem. With many certifications being focused on large corporations, having the opportunity to shape how regulators approach startups and digital SMBs has led to an interesting exchange of experiences and viewpoints.

Shortcomings of using the tool in this use case

We could foresee two main challenges associated with this tool. First, as a scaleup, we believe that many similar companies will be highly challenged to dedicate the required resources to obtain this certification, particularly if lacking previous extensive documentation. Thus, even with all the support from Adigital, a clear understanding of the commitments should be made before starting this process.

 

Secondly, as far as future legislation translates many of the requirements into legal obligations, a more nuanced distinction between the overlap of this certification and the IA Act or GDPR compliance could be provided for organizations to have a better understanding of the areas contemplated in each topic. 

Learnings or advice for using the tool in a similar context

One crucial insight from obtaining this certification is to highlight the paramount importance of building AI tools that are not only effective but also transparent and trustworthy. Working on this certification has encouraged us to have a deeper understanding of how users can easily understand and trust the AI's recommendations, thus enhancing the overall acceptance and satisfaction with the tool. 

 

Bearing this in mind, we recommend future participants check the different areas of evaluation before starting the certification process, as it might require some adaptations in a production environment that could significantly impact their product roadmap and business goals.  For us, and due to our previous emphasis on explainability and transparency, it was easy to reflect our previous documentation into the certification requirements. However, it still required significant efforts from the team to analyze certain aspects of the same. Thus, it could be helpful for future participants to prepare in advance documentation of the covered areas to speed up the certification process.  

Comparison with other tools

To the best of our knowledge, we are unaware of any similar certification focused on algorithmic transparency in Spain. 

Modify this use case

About the use case



Impacted stakeholders:



Target sector(s):


Country of origin: