How does your organization define thresholds at which severe risks, unless adequately mitigated, may be considered "unreasonable"? Please describe the framework(s) most relevant to your organization's role and context. If your organisation does not use formal thresholds, you may describe instead the policies, decision criteria, or governance processes you apply when assessing severe AI risks. You may also describe:
• Specific examples of contexts in which your models or systems could, in your assessment, pose unreasonable or intolerable risk;
• How mitigations are graduated by capability or propensity level;
• How your organisation considers the specificities of different AI technologies, such as generative AI and agentic AI, including any specific risk assessment or governance mechanisms applied to agentic AI capabilities.
Karavela develops AI models using functional MRI data to study brain function and support research into neurological and psychiatric conditions. In this context, the most relevant severe risks concern participant and patient safety, the confidentiality of brain and health data, mental privacy, and inappropriate reliance on model outputs.
Karavela applies the risk methodology of its ISO/IEC 27001 information security management system. Each risk is scored as probability × impact on a scale of 1 to 16. Scores of 1–4 are acceptable, scores of 5–11 are acceptable only with a treatment plan, and scores of 12–16 are unacceptable and require immediate action. AI-specific risk scenarios are included in our risk register and assessed on the same scale. This methodology is complemented by human-subject research ethics, data-protection requirements, and scientific and clinical validation. We assess the potential severity, likelihood, scale and reversibility of harm; the vulnerability of affected individuals; the intended use and foreseeable misuse; and the effectiveness of available safeguards.
We would consider a use unreasonable where a credible pathway to severe harm remains insufficiently controlled. Examples include:
- Using an insufficiently validated model to make consequential diagnostic or treatment decisions.
- Inferring or disclosing sensitive cognitive or health information outside the authorised research purpose
- Enabling coercive or discriminatory uses of brain-derived information, including employment or insurance decisions.
- Presenting probabilistic neural decoding outputs as reliable evidence of an individual’s thoughts, intentions or beliefs.
Safeguards should increase with demonstrated capabilities, access to sensitive data, deployment scale and decision-making authority. Uses with potential clinical consequences require stronger validation across participants, sessions and acquisition settings as well as assessment of failure modes and subgroup performance. Where residual risk cannot be adequately mitigated, the relevant use should be restricted.
For generative neural decoding, a particular concern is that plausible generated content may not faithfully reflect the recorded brain activity. Fluency must therefore not be treated as evidence of decoding accuracy. Evaluation and communication should distinguish measured predictive performance from interpretation.