What a high-risk system must satisfy before market placement: compliance duty, risk-management system, data and data-governance quality criteria, technical documentation, automatic logging, transparency and instructions for use, human oversight, accuracy/robustness/cybersecurity.
High-risk AI systems must comply with the requirements in this Section, taking into account their intended purpose and the generally acknowledged state of the art on AI and AI-related technologies. The risk management system under Article 9 must be taken into account when ensuring compliance 32024R1689 Article 8@2024-06-13. Where a product is also subject to Union harmonisation legislation listed in Annex I, Section A, providers may integrate the necessary AI Act testing, reporting processes, information and documentation into documentation and procedures that already exist under that sectoral legislation 32024R1689 Article 8@2024-06-13.
Providers must establish, implement, document and maintain a risk management system as a continuous iterative process throughout the entire lifecycle of the system, requiring regular systematic review and updating. The four mandatory steps are:
Residual risk, per hazard and overall, must be judged acceptable. In selecting measures, providers must: eliminate or reduce identified risks as far as technically feasible through design and development; where appropriate, implement adequate mitigation and control measures for risks that cannot be eliminated; and provide information required pursuant to Article 13 and, where appropriate, training to deployers 32024R1689 Article 9@2024-06-13. Systems must be tested prior to their being placed on the market or put into service, against pre-defined metrics and probabilistic thresholds appropriate to the intended purpose. Providers must also consider whether, in view of the intended purpose, the system is likely to have an adverse impact on persons under the age of 18 and, as appropriate, other vulnerable groups 32024R1689 Article 9@2024-06-13.
Systems using model-training techniques must use training, validation and testing data sets subject to appropriate data governance. Mandatory governance practices include: assessment of the availability, quantity and suitability of required data sets; examination for possible biases likely to affect health and safety, have a negative impact on fundamental rights, or lead to discrimination prohibited under Union law — particularly where data outputs influence inputs for future operations; and appropriate measures to detect, prevent and mitigate identified biases 32024R1689 Article 10@2024-06-13.
Data sets must be relevant, sufficiently representative, and to the best extent possible free of errors and complete in view of the intended purpose. They must have appropriate statistical properties, including, where applicable, as regards the persons or groups of persons for whom the system is intended 32024R1689 Article 10@2024-06-13. Data sets must, to the extent required by the intended purpose, account for characteristics particular to the specific geographical, contextual, behavioural or functional setting of intended use 32024R1689 Article 10@2024-06-13. For systems not using model-training techniques, these requirements apply only to testing data sets 32024R1689 Article 10@2024-06-13.
Technical documentation must be drawn up before placement on the market or entry into service, and kept up to date. It must contain at minimum the elements set out in Annex IV and must enable national competent authorities and notified bodies to assess compliance. SMEs and start-ups may use a simplified form established by the Commission 32024R1689 Article 11@2024-06-13.
High-risk AI systems must technically allow for automatic recording of events (logs) over the system's lifetime. Logging capabilities must enable recording of events relevant for: identifying situations that may result in a risk within the meaning of Article 79(1) or a substantial modification; facilitating post-market monitoring; and monitoring the operation of high-risk AI systems as referred to in Article 26(5) 32024R1689 Article 12@2024-06-13.
Systems must be designed and developed to ensure sufficient transparency to enable deployers to interpret output and use the system appropriately 32024R1689 Article 13@2024-06-13. Accompanying instructions for use must include concise, complete, correct and clear information that is relevant, accessible and comprehensible to deployers. They must include at minimum: the identity and contact details of the provider and, where applicable, of its authorised representative; the level of accuracy, including its metrics, robustness and cybersecurity against which the system has been tested and validated, the levels that can be expected, and known or foreseeable circumstances that may affect those levels; human oversight measures and technical tools facilitating interpretation of output; the computational and hardware resources needed, the expected lifetime of the system, and maintenance and care measures including their frequency; and, where relevant, a description of mechanisms allowing deployers to properly collect, store and interpret logs in accordance with Article 12 32024R1689 Article 13@2024-06-13.
Systems must be designed and developed with appropriate human-machine interface tools to enable effective oversight by natural persons during use. Oversight aims to prevent or minimise risks to health, safety or fundamental rights, particularly where such risks persist despite other requirements 32024R1689 Article 14@2024-06-13.
The system must be provided to deployers in a way that assigned oversight persons are enabled, as appropriate and proportionate: to properly understand the system's capacities and limitations and to duly monitor its operation, including in view of detecting and addressing anomalies, dysfunctions and unexpected performance; to remain aware of the possible tendency of automatically relying or over-relying on system output (automation bias); to correctly interpret the system's output using available interpretation tools and methods; to decide, in any particular situation, not to use the system, or to otherwise disregard, override or reverse its output; and to intervene in the operation of the high-risk AI system or interrupt the system through a stop button or similar procedure that allows the system to come to a halt in a safe state 32024R1689 Article 14@2024-06-13.
For systems used for remote biometric identification (Annex III, point 1(a)), no action or decision may be taken by the deployer on the basis of an identification result unless it has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority. This requirement does not apply where Union or national law considers its application disproportionate for law enforcement, migration, border control or asylum purposes 32024R1689 Article 14@2024-06-13.
High-risk AI systems must achieve an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle 32024R1689 Article 15@2024-06-13. The levels of accuracy and the relevant accuracy metrics must be declared in the accompanying instructions for use 32024R1689 Article 15@2024-06-13.
Systems must be as resilient as possible against errors, faults or inconsistencies arising within the system or its operating environment. Technical and organisational measures shall be taken in this regard; robustness may additionally be achieved through technical redundancy solutions, which may include backup or fail-safe plans 32024R1689 Article 15@2024-06-13. Systems that continue to learn after deployment must be developed to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and any such feedback loops must be addressed with appropriate mitigation measures 32024R1689 Article 15@2024-06-13. Systems must further be resilient against unauthorised attempts to alter their use, outputs or performance, with technical solutions to address AI-specific vulnerabilities that shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks such as data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks or model flaws 32024R1689 Article 15@2024-06-13.
This block connects to your organisation's own AI governance policy. In the full product it shows, cited to your policy, how YOUR organisation implements the regulation above — private to your organisation. (Demo placeholder.)
No recent cases are cached for this topic.
1. For a high-risk AI system that does NOT use model-training techniques, which data sets are subject to the data governance requirements of Article 10?
Article 10 explicitly limits its governance requirements to testing data sets when the system does not use model-training techniques. 32024R1689 Article 10@2024-06-13
2. Under Article 9, when must a provider test a high-risk AI system against pre-defined metrics and probabilistic thresholds?
Article 9 states systems must be tested before being placed on the market or put into service, not retrospectively or on a rolling basis post-deployment. 32024R1689 Article 9@2024-06-13
3. Article 14 requires that for remote biometric identification systems, no action may be taken on an identification result unless it has been verified and confirmed by at least two natural persons. Under what condition does this two-person rule NOT apply?
Article 14 carves out the two-person verification rule specifically when Union or national law deems it disproportionate in the context of law enforcement, migration, border control or asylum. 32024R1689 Article 14@2024-06-13
4. Article 9 sets out an order of priority for selecting risk management measures. What must providers do first?
Article 9 requires risk elimination or reduction through design and development as the primary action, with mitigation/control measures and information/training applied only to risks that cannot be eliminated. 32024R1689 Article 9@2024-06-13
5. What specific obligation does Article 15 impose on high-risk AI systems that continue to learn after deployment?
Article 15 singles out post-deployment learning systems and requires them to address feedback loops — the risk that biased outputs feed back into future inputs — through appropriate mitigation measures. 32024R1689 Article 15@2024-06-13