Please briefly describe your organization’s decision-making process for deploying an AI model or system including: who has decision authority; what evidence and criteria are required (e.g., performance, safety/security, robustness, fairness, privacy, compliance, independent external testing, evaluation, validation and verification (TEVV) or assurance); contextual caveats or limitations required in interpreting risk evaluation metric results; what approvals/sign-offs are needed; and what safeguards (e.g., mitigations, monitoring, or phased release) must be in place to move into deployment.
Prior to deploying AI systems, we implement the following measures from a multifaceted perspective, including AI governance, quality control, and security.
1. Risk Assessment Based on AI Governance and Human Rights Policies
In accordance with the “NEC Group AI and Human Rights Principles,” we conduct risk assessments using risk checklists and implement countermeasures at every phase of the lifecycle—from planning and proposal through design and development to operation and maintenance. Additionally, based on the “AI Guidelines for Business,” we apply and update these guidelines within our internal AI governance framework.
2. Development Management and Quantitative Standards Based on the AI Quality Guidelines
To ensure the quality of AI-powered systems, we have been applying the “NEC AI Quality Guidelines,” established in December 2019, to development projects since April 2020. These guidelines establish specific standards and checklists for each of the four development phases (1. System Planning and Proof of Concept (PoC), 2. Data Collection and Processing, 3. Model Creation, Evaluation, and Testing, and 4. System Operation). By conducting checks when transitioning between phases, we proactively prevent risks in the subsequent phase. In particular, by setting quantitative standards for the volume of training data, as well as for outliers and missing values, we ensure clarity in these standards and enable objective assessments by third parties.
3. Evaluation and Safety Measures for the Generative AI Core Technology “cotomi” and AI Agents
Regarding “cotomi,” our in-house generative AI core technology, we verify the model’s safety and reliability under a comprehensive evaluation framework that utilizes multiple assessment tools—including general-purpose benchmarks, Japanese-language benchmarks, and RAG performance evaluations—while ensuring strict compliance with laws and regulations regarding training data. In addition, we have implemented high-precision AI guardrail functions, including technologies for detecting false and misleading information and proposing corrections. Furthermore, with our AI agent technology “cotomi Act,” we leverage our accumulated knowledge on risk mitigation to configure and operate agents in accordance with corporate rules and policies from a governance perspective, thereby enabling the safe and secure use of autonomous agents.
4. Security in Development and Operations
In accordance with the “Basic Information Security Rules (NEC Security Policy),” we conduct evaluations in a secure environment equipped with necessary security measures—such as isolation from external networks and authenticated access—as a prerequisite for deployment. Additionally, depending on the system’s criticality (security level), we require or recommend that our in-house security team perform vulnerability self-assessments and penetration tests. Furthermore, regarding security risks—including those associated with AI systems—we implement risk management based on a cybersecurity checklist at every phase, from the planning and proposal stages through design and implementation, to operation, maintenance, and lifecycle management. The project manager and organizational manager verify the status of these implementation efforts.