Catalogue of layer 2 of the K0NSULT model: AI and agentic incidents. Every risk is described with its response priority, the applicable AI Act flag, its human-oversight status (AI Act art. 14) and a link to the relevant playbook. Below is a quick test to qualify a system as high-risk.
Injection of instructions into input data (direct or indirect — from a document or web page) that overrides the model's policy, exfiltrates data or triggers unwanted agent actions. Method reference: MITRE ATLAS / OWASP LLM01.
Human oversight: required for actions with external effect (payment, e-mail, configuration change).
Takeover of an autonomous agent — abuse of its privileges (APIs, tokens, system access) to act on behalf of the organisation outside the operator's intent.
Human oversight: CRITICAL — immediate agent cut-off (kill-switch) requires sign-off from the AI Safety Officer.
Poisoning of training / fine-tuning data or the RAG store to introduce a backdoor, bias or quality degradation in the model. The effect often surfaces with a delay.
Human oversight: validation of data sources and dataset versioning (data provenance).
Model theft via mass querying (query-based extraction) or exfiltration of weights/parameters — loss of intellectual property and a vector for further offline attacks.
Human oversight: query-pattern monitoring, rate-limiting, anomaly detection.
The model generates false, fabricated or unverified information presented as fact — risk of decisions made on a wrong basis, especially in financial and advisory services.
Human oversight: claim ≤ proof doctrine — a claim without proof = GAP status, not publication.
Synthetic voice / video / document used for fraud (e.g. a "call from the CEO"), identity-verification bypass or disinformation. AI Act art. 50: obligation to label generated content.
Human oversight: out-of-band verification (second channel) for financial instructions.
An event involving an AI system that causes (or threatens) serious harm: to health, fundamental rights, critical infrastructure or property. AI Act art. 73 — reporting obligation.
Human oversight: Legal/DPO + AI Safety Officer decision on reporting to the authority within the deadline.
A false or unverified agent identity: impersonation of a service, lack of agent-to-agent authentication, an agent operating without an assigned owner/accountability.
Human oversight: agent registry, assigned owner, authentication (DID/PKI, PQC layer).
| Risk class | Events (demo) | Base priority | AI Act flag | Human oversight |
|---|---|---|---|---|
| Prompt injection | 18 | P1 | AI_ACT_RELEVANT | Required |
| Agent hijack | 9 | P0 | AI_SERIOUS_INCIDENT | Critical |
| Hallucination / false claim | 8 | P1 | AI_ACT_RELEVANT | Required |
| Deepfake | 7 | P1 | AI_ACT_RELEVANT (art.50) | Required |
| Data poisoning | 5 | P1 | AI_HIGH_RISK | Required |
| Model extraction | 4 | P2 | AI_ACT_RELEVANT | Recommended |
| Agent identity risk | 3 | P1 | AI_ACT_RELEVANT | Required |
A preliminary, indicative qualification test following the logic of the AI Act (art. 6 + Annex III). It does not replace legal assessment. If a system meets any of the following and does not fall within the exemptions — treat it provisionally as high-risk and proceed to Compliance.
Does the AI system affect any of the following areas?
The K0NSULT system treats the absence or weakening of human oversight as a standalone risk factor — independent of the threat type. An agent with privileges to perform irreversible actions without an approval point = priority escalation.
A human approves actions with external effect. desired state
A human sees the log post-factum, does not block. medium risk
The agent operates fully autonomously in production. high risk
Ability to cut off the agent immediately. required for P0/P1