What is AI governance?

AI governance is the organisational framework that ensures AI systems are used responsibly, are documented and comply with regulation. It covers policies, approval flows, audit trail, data processing agreements, model selection, human oversight and ongoing risk assessment. Governance is broader than compliance: compliance is about meeting the rules, governance is about having a system that ensures it happens consistently.

What is an AI audit trail?

An AI audit trail is the documentation that allows you to reconstruct what the AI did, when, for which user, based on which prompt and which model. In practice it consists of at least four layers: AI system inventory, risk classification per system, log of each AI-generated decision or output, and human oversight procedure. EU AI Act Art. 12 requires automatic logging on high-risk systems.

What should an AI policy contain?

A minimal AI policy runs to 3-5 pages and covers: approved tools with concrete names (Claude, ChatGPT, Gemini, and so on), a data policy specifying which data types may be sent to which models, an approval flow for new tools, audit trail requirements, human oversight procedure, named owner and contact, and a quarterly review date. The policy is typically owned by the IT lead together with the DPO.

How do you log prompts without storing client data?

Three approaches: 1) Hash prompts cryptographically and only log the hash values (SHA-256) together with model name, timestamp and user ID. This gives an audit trail without persisting content. 2) Use structured prompt templates where variable values (client data) are excluded from the log. 3) For high-risk systems where full prompt logging is required, store prompts encrypted with a client-specific key so deletion on client request is technically possible. The chosen approach is documented in the policy.

How do you choose which AI model may be used for what?

Model selection policy is based on four factors: data sensitivity (which data is sent), business criticality (consequence of error), data residency (EU vs non-EU), and vendor maturity (DPA, audit reports, EU AI Act readiness). Typical structure: Claude or Mistral self-hosted for confidential client data, Claude API with EU residency for internal production, ChatGPT or Gemini for non-confidential tasks, Copilot only for code that does not contain secrets.

Is AI governance connected to GDPR?

Yes, closely. AI systems that process personal data require a GDPR legal basis (Art. 6) assessed per use case. AI vendors are processors (Art. 28) and require a data processing agreement. Automated decisions with legal effect require explicit Art. 22 consideration. The AI system inventory should cross-reference the GDPR record so a supervisory authority can see the connection during an inspection.

When should we choose ISO 42001 over ad-hoc governance?

ISO 42001 (AI Management System) makes sense when you sell to public sector or enterprise customers who require it, have multiple high-risk systems where scale can be used, or build AI as a product and want to signal maturity. It does NOT make sense if you have 1-2 limited-risk systems or are a small SME without enterprise sales. For most knowledge-intensive SMEs, focused ad-hoc governance is the first path; ISO 42001 is revisited after 6-12 months of operation.

What is the difference between governance in practice and governance on paper?

Governance on paper is a policy that sits in a folder no one opens. Governance in practice is where the policy is concrete enough to act on, where audit trail is automatic rather than manual, where model selection policy is enforced technically (approval flow before installation), and where quarterly reviews actually take place. The difference is clearest when a regulator or enterprise customer asks for documentation: governance on paper collapses, governance in practice is produced in an hour.

Do small organisations need AI governance too?

Yes, but proportionate. A 15-person law firm does not need the same apparatus as a 500-person bank. Minimum for small knowledge-intensive organisations: a 3-page AI policy, an AI system inventory (can be a Google Sheet), classification of each use against the EU AI Act, and a named AI owner. That takes 2-4 weeks of focused work and gives a basis to stand on with clients, auditors and regulators.

Pillar · Published 2026-05-26 · 16 min read

AI Governance and audit trail in practice

Q: How long should we keep audit trail records?

The EU AI Act requires at least 6 months of logging on high-risk systems (Art. 12, para. 2). For financial services it can be 5-7 years (capital markets legislation). For client-related advisory at law firms and accounting firms, typically the duration of the client relationship plus 5 years. Our recommendation: minimum 12 months for all AI systems in production, longer if outputs underpin decisions that can be appealed.

Q: How many hours does it take to put governance in place?

Minimum package (AI policy, AI system inventory, basic audit trail practice): 40-80 internal hours over 4-8 weeks plus external advice if desired. Full EU AI Act readiness for an organisation with 1-3 high-risk systems: 100-300 hours over 3-6 months. ISO 42001 readiness: 200-500 hours over 6-12 months. The pace depends on whether dedicated time is allocated to an AI owner or whether governance has to be squeezed in alongside other work.

The EU AI Act pillar described what you have to comply with. This pillar describes how. Concretely: which four documentation layers need to be in place, how to log prompts without leaking client data, which model may be used for what, and where governance on paper differs from governance in practice. Written for executives at knowledge-intensive organisations who want a foundation that holds up in front of both auditors and supervisory authorities.

Written by Jesper Sachmann, founder of EnterpriseIQ. 27 years of IT leadership from Oracle, Logica and Capgemini, combined with hands-on AI experience and 11 years of GRC background from Archer (Alliance Director Europe and Integrated Risk Management Lead Nordics).

TL;DR

→Audit trail consists of four layers: inventory, classification, output log and oversight procedure.
→The minimal AI policy runs to 3-5 pages and covers tools, data, approvals, audit trail, owner and review.
→Model selection rests on four factors: data sensitivity, criticality, data residency and vendor maturity.
→GDPR and the EU AI Act overlap on five concrete points: Art. 6, 22, 28, retention and inventory record.
→ISO 42001 is not mandatory. Assess only after 6-12 months of ad-hoc governance.

Why governance matters now

Two years ago, AI governance was something enterprise customers talked about in PowerPoint decks. Today, boards at mid-sized knowledge-intensive organisations are being asked to account for it. Two forces pull in the same direction.

First: the EU AI Act requires formal governance for high-risk systems from 2 August 2026. That is no longer best practice, it is the law. Second: enterprise customers, particularly in banking, insurance and the public sector, are starting to ask about AI practice in tender rounds and vendor reviews. Even if you do not have high-risk systems of your own, your customers may require documentation of your AI use as a supplier.

This dual pressure is what makes governance urgent. You are either on the way to having to meet a legal requirement, or on the way to losing tender rounds because a larger customer wants the paperwork in order. For most knowledge-intensive SMEs, it is both.

The four documentation layers in audit trail

An AI audit trail consists of four layers that build on each other. Each one is meaningful on its own, but the full value emerges when all four are in place and can be cross-referenced.

Layer 1: AI system inventory

A complete list of which AI systems are used, for what, by whom. Includes both formal systems (approved SaaS tools, internal AI platform) and shadow IT (staff using ChatGPT on personal accounts for work tasks).

Practical format: Google Sheet or dedicated GRC system. Fields: name, vendor, use, user groups, data types, EU AI Act classification, GDPR basis, owner, approval date.

Layer 2: Risk classification per system

Each AI use is classified against the EU AI Act categories: prohibited, high-risk, limited risk or minimal risk. Classification follows the use, not the technology: the same Claude can be minimal risk in one setting and high-risk in another.

Practical format: added to the inventory sheet together with a written rationale per classification (1-3 sentences). Validation by a lawyer or AI adviser is recommended for high-risk assessments.

Layer 3: Output log per AI-generated decision

For each material AI-generated output or decision, the following is logged: timestamp, user, model and version, prompt reference (hash or template ID), output reference, and the human review that took place afterwards. The requirement tightens for high-risk systems (EU AI Act Art. 12).

Practical format: application log in a structured format (JSON or database). For self-hosted systems, logging is integrated into application code. For SaaS tools, the vendor audit API is used where available.

Layer 4: Human oversight procedure

Written procedure for how people validate AI output before it leaves the organisation or feeds into decisions about people. The level of detail reflects the risk classification: minimal risk can be sample checking, high-risk requires documented review per output.

Practical format: a 1-2 page procedure per AI use, linked to the inventory. Includes who reviews, what is specifically checked, and how deviations are documented.

The four layers together give the foundation a supervisory authority, an auditor or an enterprise customer needs to see. A missing layer means a conversation about why. All four in place means that conversation never comes up.

How to log prompts without leaking client data

A recurring concern in knowledge-intensive organisations: if we log every prompt, are we not storing confidential client information in a new system with its own risk profile? It is a valid concern. There are three practical approaches.

Approach 1: Hash-based logging

Prompts are hashed cryptographically (SHA-256) and only the hash value is logged together with metadata: timestamp, user, model. The content itself is not persisted. This gives an audit trail to confirm that a specific prompt was sent, but not the content. Well suited to minimal-risk and limited-risk uses where you primarily need to document usage patterns.

Limitation: you cannot reconstruct what was asked. If output later turns out to be wrong and you want to understand why, you only have the output log to go on.

Approach 2: Structured prompt templates

Prompts are structured as templates with variable placeholders, where the template ID is logged but not the populated values. Example: a contract review prompt uses template "review_contract_v3" with variables [contract_text] and [client_context]. The log contains the template ID and which variables were populated, but not the content.

Advantage: strong audit trail on usage patterns and prompt evolution without data leakage. Limitation: it requires disciplined prompt engineering, because ad-hoc prompts are logged as free text and yield lower audit value.

Approach 3: Encrypted full logging

For high-risk systems where EU AI Act Art. 12 requires full logging, prompts are stored encrypted with a client-specific key. This gives full audit trail while making client deletion requests technically feasible: deleting the client key renders the log unreadable.

This approach is the most expensive to implement but the only fully EU AI Act compatible one for highly sensitive client data. We recommend it for compliance-bound use cases in financial advisory, lawyer-client work with confidential substance, and healthcare applications.

Which approach you choose is documented in the AI policy and the audit trail procedure. It is not just a technical choice, it is a governance decision that needs to be explainable.

Model selection policy: which AI for what

One of the most overlooked parts of AI governance is an explicit policy on which model may be used for which type of task. Without it, staff will typically choose the tool they know best, not the one that fits the data type. The policy rests on four factors.

Factor	Describes
Data sensitivity	Public, internal, confidential, secret. Follows the organisation's data classification
Business criticality	What is the consequence of incorrect output? Trivial, quality issue, client loss, legal case
Data residency	EU, UK, US, other. Determines which vendors are even possible
Vendor maturity	DPA in place, EU AI Act readiness, SOC 2, ISO 27001, audit reports available

A typical model selection policy for a knowledge-intensive SME looks like this:

Self-hosted Llama or Mistral

Confidential client data that must not leave the premises. Client strategy, contract substance, capital information. Hosted on own Proxmox or an Azure tenant.

Claude API with EU residency

Internal production: research, first drafts of client-facing material, analytical tasks. EU residency agreement via Anthropic Enterprise or AWS Bedrock in Frankfurt.

ChatGPT or Gemini with a DPA

Non-confidential tasks: publicly available content, marketing, general research. DPA signed, data not used for training.

GitHub Copilot or Cursor

Code assistance only. Filtered for secrets. If you build client deliverables where customer code must not leak, use a self-hosted alternative such as Codeium or self-hosted Continue.

Perplexity or other demo models

Research with publicly available sources. Should never receive client data or internal strategic information.

The policy is documented in the AI policy and enforced technically where possible: approval flow before new tools are installed, and blocking of non-approved AI services on company devices through MDM or network filtering.

The minimal AI policy

An AI policy that actually works runs to 3-5 pages. Longer and it does not get read. Shorter and it lacks the substance that makes it useful in concrete situations. The structure below has been tested with knowledge-intensive SMEs and works as a starting skeleton.

1. Purpose and scope (1/2 page). Why we have this policy, and which AI uses it covers.
2. Approved tools (1/2-1 page). Concrete list with Claude, ChatGPT, Gemini, n8n and others. Includes which versions and which account framework (enterprise vs personal).
3. Data policy (1 page). Which data types may be sent to which models, based on data sensitivity and the model selection policy.
4. Approval flow for new tools (1/2 page). Who approves, what criteria, what turnaround time.
5. Audit trail requirements (1/2 page). What logs are kept, what retention, who is responsible.
6. Human oversight requirements (1/2 page). For which use cases is documented review required, and how is it documented.
7. Owner and contact (1/4 page). Named AI owner plus deputy. Where to ask questions.
8. Review cadence (1/4 page). Next review date, who attends, what is covered.

The policy is formally approved by the executive team and shared with staff at a team meeting, not just by email. Without training a policy is just a document. The quarterly review date is put in the leadership calendar before the policy is finalised, so implementation does not depend on people remembering.

GDPR and AI: five overlap points

GDPR and the EU AI Act are separate regulations, but they overlap on five concrete points for knowledge-intensive organisations. The AI system inventory should cross-reference the GDPR record so that a supervisory authority can see the connection during an inspection without extra work.

1. Art. 6 lawful basis

Each AI use that processes personal data requires a lawful basis (consent, performance of a contract, legitimate interest, legal obligation, vital interest, public task). The basis is assessed per use case and documented in the inventory.

2. Art. 22 automated decision-making

Decisions taken solely on the basis of automated processing and which have legal effect or similarly significant impact require explicit Art. 22 consideration. Example: AI-based credit scoring, automatic client rejection. Human oversight must be real, not rubber-stamp.

3. Art. 28 data processing agreements

AI vendors that process personal data are processors and require a DPA. The agreement specifies purpose, duration, categories of data subjects, security measures, sub-processors and audit rights. Add AI vendors to the GDPR processor register.

4. Retention

GDPR requires that personal data is not kept longer than necessary. AI training data, prompt logs and output history need an explicit retention policy. The EU AI Act minimum of 6 months for high-risk systems is aligned with GDPR retention.

5. Inventory record

The GDPR record (Art. 30) and the AI system inventory cover parts of the same ground. Cross-link them to avoid duplicated administration. A supervisory authority typically looks at both during an inspection.

Where governance fails in practice

We have seen governance implementations that look fine on paper collapse the first time they meet a real customer request or supervisory question. Five common pitfalls recur.

Pitfall 1: AI policy without a named owner

A policy without a named AI owner is a policy that does not get maintained. Quarterly reviews depend on someone having it in the calendar. Consequence: the policy is outdated six months after it was approved.

Pitfall 2: Inventory that does not catch shadow IT

The official inventory shows 5 approved AI tools. Actual use in the organisation is 12 tools, of which 7 run on personal accounts without company approval. When a customer request comes in with "which AI tools have processed our data", the mismatch is revealed. Solution: an anonymous staff survey every quarter.

Pitfall 3: Audit trail without retention discipline

Logging is started ambitiously. After six months, log volume has grown to an unrealistic level, no one reads the logs, and storage quietly stops without a formal decision. When a supervisory authority asks for logs from six months ago, they are not there. Solution: define the retention period in writing on day one and automate either archival or deletion.

Pitfall 4: Human oversight as rubber-stamp

The procedure says "AI output is reviewed by a qualified staff member before it leaves the organisation". In practice it means the junior employee clicks approve without reading the output. When an error becomes public, formal compliance falls apart at the first inspection. Solution: document concrete check points per review and do quarterly sample checks of whether reviews are actually being performed.

Pitfall 5: Governance that lives in isolation from GDPR

The AI system inventory sits in one folder, the GDPR record in another, and the two are maintained independently. When the supervisory authority arrives, inconsistencies are discovered. Solution: one shared inventory that covers both regimes with tags for which regulation applies to each system.

ISO 42001 versus ad-hoc governance

ISO/IEC 42001 (AI Management System) was published in 2023 and provides a structured framework for AI governance on a par with ISO 27001 for information security. The standard is certifiable and covers many of the EU AI Act requirements.

The central decision for knowledge-intensive SMEs is whether to aim for ISO 42001 certification or whether ad-hoc governance is enough. The recommendation is clear: start ad-hoc, assess ISO after 6-12 months of operation.

ISO 42001 makes sense when:

You sell to public sector or enterprise customers who explicitly require certification
You have multiple high-risk systems where scale can be used
You build AI as a product and want to signal maturity to the market
You already have ISO 27001 and can extend the governance apparatus

ISO 42001 does NOT make sense if:

You have 1-2 limited-risk systems and no high-risk
You are a small SME without enterprise sales
You only want to meet the EU AI Act minimum
You have no existing ISO experience and would have to build the governance apparatus from scratch

For most knowledge-intensive SMEs, focused ad-hoc governance is the first path. ISO 42001 readiness typically costs 200-500 internal hours plus DKK 80,000-150,000 in external advice and the certification process itself. That investment is justified by concrete customer or market demand, not by general compliance considerations.

Three steps you can take this week

Step 1: Establish an AI system inventory

Create a Google Sheet with the fields: name, vendor, use, user groups, data types, EU AI Act classification, GDPR basis, owner
Ask staff to name every AI tool in use, including personal accounts
Carry out a first classification per system (prohibited, high-risk, limited, minimal)
Identify the top three systems where governance is most pressing

Step 2: Write the minimal AI policy

Use the 8-point structure above as the skeleton
3-5 pages, no more. Should be readable in a coffee break
Name an AI owner and a deputy
Set the quarterly review date in the calendar before the policy is approved

Step 3: Cross-link to the GDPR record

Identify which AI systems process personal data
Add them to the existing GDPR record with a reference to the AI inventory
Verify that data processing agreements are in place with AI vendors
If any are missing, contact the vendor and ask for a DPA

These three steps give the foundation for both EU AI Act readiness and practical governance. They do not need to be perfect at first delivery: they need to be in place and have an owner. Iterative improvement over the next three months beats a perfect policy that waits six months.

FAQ

How long should we keep audit trail records?

EU AI Act minimum 6 months for high-risk systems. Financial services typically 5-7 years. Client-related advisory at law firms and accounting firms normally the client relationship plus 5 years. Our recommendation: minimum 12 months for all AI systems in production.

How many hours does it take to put governance in place?

Minimum package (policy, inventory, basic audit trail): 40-80 internal hours over 4-8 weeks. Full EU AI Act readiness: 100-300 hours over 3-6 months. ISO 42001 readiness: 200-500 hours over 6-12 months.

Can we outsource AI governance?

Policy drafting and inventory build-out can be outsourced. Audit trail practice and ownership cannot. The external adviser delivers a skeleton and quality check, but an organisation without an internal AI owner loses governance discipline after 3-6 months. A hybrid approach works best.

What if we do not have a DPO?

Knowledge-intensive SMEs are often not required to appoint a DPO (under 250 staff and no high-risk processing). In that case, the IT lead or compliance officer takes the owner role for AI governance. If you have high-risk systems or process sensitive personal data systematically, consider a voluntary DPO or a shared DPO function with other SMEs in the same sector.

Do staff need training in AI governance?

Yes, but proportionate. All staff using AI tools should have 30-60 minutes of introduction to the policy (typically at onboarding plus annual refresh). The AI owner and deputy should have deeper training, such as the EnterpriseIQ Bootcamp or equivalent. Specialised roles (DPO, compliance) should have formal training in the EU AI Act and ISO 42001.

How do we know if our governance is good enough?

Three tests: 1) An enterprise customer asks for documentation of your AI practice. Can you deliver in 24 hours? 2) A supervisory authority starts an inspection focused on AI. Are inventory and logs ready? 3) A staff member asks "may I send this client agreement to Claude?". Can they find the answer in the policy themselves? If yes to all three, you have governance in practice, not just on paper.

Next step

Three paths depending on where you stand:

Free

Download the EU AI Act checklist

8-page PDF with a 12-point checklist covering both compliance and the governance foundation.

From DKK 15,000

Book a Quick Check

1-day assessment with complete AI system inventory, risk classification, AI policy stub and a 90-day roadmap.

Free

30-minute call

A non-binding screening conversation. We figure out whether a Quick Check fits or something else makes more sense.

About the author

Jesper Sachmann is the founder of EnterpriseIQ. 27 years of IT leadership from Oracle, Logica and Capgemini, combined with hands-on AI experience and 11 years of GRC background from Archer (Alliance Director Europe and Integrated Risk Management Lead Nordics).

AI attribution: This article is AI-assisted, produced with Claude Opus 4.7, human review by Jesper Sachmann. See our AI transparency policy for how we use AI across every deliverable.

Citing this article? Use "EnterpriseIQ: AI Governance and audit trail in practice (2026-05-26)" or link to enterpriseiq.dk/en/insights/ai-governance-audit-trail.