Please provide a consolidated description of your organisation's approach to risk management across the AI system lifecycle, including how the practices identified above operate together and any cross-cutting governance, review or escalation processes. Please also indicate how often these practices are reviewed and what types of events trigger reviews and updates. Where relevant, please distinguish between different types of AI systems (e.g. generative or agentic AI).
Malt distinguishes between recommendation systems, generative AI and agentic AI. Generative systems receive additional controls for prompt and output security, factuality, privacy, provider processing and human review. Agentic systems, where applicable, require additional safeguards such as least-privilege access, approved tool lists, action limits, human confirmation for consequential actions and an effective interruption or shutdown mechanism.
The lifecycle process is reviewed and updated under three circumstances: (1) when a new system version or model iteration is released, including consultation with the AI Deployment Committee for new algorithmic model baselines; (2) at least annually, including extensive independent penetration testing and a broader review of the effectiveness of governance, security, privacy, fairness and monitoring controls; (3) on an ad hoc basis, following an active model defect, vulnerability, operational exploit, material incident, significant performance or fairness degradation, major data or architecture change, new third-party dependency, regulatory development or material research finding.
For the plan and design stage, before developing or deploying an AI system, Malt defines in the design specifications its intended purpose, users, affected stakeholders, expected benefits, foreseeable misuse and potential risks. Systems are assessed in the risk mapping across privacy, security, fairness, transparency, accuracy, human rights, operational resilience and environmental considerations by the AI deployment committee. The review determines whether the system can proceed, requires additional safeguards or should not be deployed.
In the collect and process data stage, Malt uses professional and marketplace information relevant to the service, such as skills and experience for matching and search. Malt evaluates data quality, completeness, and representativeness. Data is limited, cleaned, and tested, including for potential bias, before use. Users can correct, update, or request deletion of their information. Third-party models and AI services are subject to AI, privacy, security and contractual review. Data shared with external providers is governed by controls such as confidentiality, zero-retention policy, limited access.
In the build, integrate, and interpret model stage, Malt’s matching systems use layered architectures combining lexical and semantic retrieval, ranking and business rules. Models, prompts, datasets, configurations and third-party dependencies are documented and versioned where relevant. Development and testing are separated from production environments, and access is controlled according to role and need. User-facing explanations focus on the purpose and limitations of AI-assisted search and recommendations rather than presenting the system as an autonomous decision-maker. Proposed candidates, and AI-generated briefs remain subject to human validation. Users can edit, reject or replace AI-generated recommendations and can use keyword search in the directory and type in manually their brief.
Through the verify and validate stage, Malt verifies AI systems before and after material changes through a combination of automated testing, offline/online evaluation, human review, comparative experiments, security testing and fairness analysis. Malt also evaluates user group performance and potential disparities by including group analysis and counterfactual testing in which relevant characteristics are changed while professional skills and experience are held constant. These evaluations help identify differences in ranking position, visibility and exposure.
Before release, at the deploy stage, Malt considers the evaluation results. Where appropriate, changes are introduced progressively and monitored against an approved baseline. Production systems maintain logging and traceability sufficient to investigate outputs, model versions, relevant inputs and human interventions, while respecting data-minimization requirements. Malt maintains human oversight for AI-assisted matching and recommendations. Fallback and rollback procedures are designed to allow Malt to isolate a faulty component, revert to a previously validated version or use a deterministic alternative when an AI component or external service is unavailable or behaving unexpectedly.
While in operation, Malt monitors AI systems across data quality, performance, fairness, security, reliability, user feedback and operational impact. Monitoring may include error rates, latency, anomalous behavior, user complaints, and external-provider failures. AI-related incidents may result in additional testing, changes to data or prompts, enhanced human review, rollback, temporary suspension, retraining or retirement of the affected system. Feedback from clients, freelancers, employees, researchers and other stakeholders contributes to continuous improvement.