AI Consulting Toolkit

The template — 10 fields

• ☐System name and internal identifier
• ☐Business owner and technical owner — the role, and the person holding it
• ☐What it does, in one sentence a non-specialist understands
• ☐Built in-house, bought, or embedded in a purchased product
• ☐Your role per regime: provider, deployer, importer, distributor
• ☐Component objects governed in their own right: models, prompts, datasets, agents
• ☐Third-party models and services it depends on
• ☐Lifecycle stage: proposed, in development, live, suspended, retired
• ☐Date entered, date last reviewed, date retired
• ☐Link to the classification, assessment and documentation for this entry

What to retain

The register and its change history, showing when each system was added and by whom. Retain retired entries: a system switched off last year can still be the subject of a complaint, and several regimes set multi-year record-retention periods that outlive the system.

Where these fields come from

• Presupposed across regimes; explicit in the NAIC AI model bulletin's AIS Program and in US interagency model risk guidance (SR 26-2), both of which expect a model inventory covering models in use, recently retired, and under development

The template — 8 fields

• ☐System identifier, linked to the register entry
• ☐Regimes considered, and why each does or does not apply
• ☐Tier assigned under each applicable regime
• ☐The specific criterion, annex entry or use-case category relied on
• ☐Reasoning, including why adjacent tiers were rejected
• ☐Whether anything makes you a provider rather than a deployer
• ☐Residual uncertainty, and where legal advice was taken
• ☐Classifier, reviewer, and date

What to retain

The dated classification with its reasoning, and the trail of any reclassification.

Where these fields come from

• EU AI Act Article 6 and Annex III for high-risk classification; Article 25 for when a deployer becomes a provider

The template — 8 fields

• ☐The deployer's processes in which the system will be used, in line with its intended purpose
• ☐The period over which, and frequency with which, the system will be used
• ☐Categories of natural persons and groups likely to be affected
• ☐Specific risks of harm to those categories, drawing on the provider's Article 13 information
• ☐How human oversight measures will be implemented, per the instructions for use
• ☐Measures if those risks materialise, including internal governance and complaint mechanisms
• ☐Where a DPIA exists: which parts are incorporated rather than repeated
• ☐Consultation undertaken, decision, decision-maker, and date

What to retain

The completed assessment, records of consultation, and the decision to proceed or not. Where the FRIA duty applies, the results are notified to the market surveillance authority. Where a mitigation was promised, keep evidence it was implemented.

Where these fields come from

• EU AI Act Article 27(1)(a)–(f) sets the six required elements; scope is public bodies, private entities providing public services, and Annex III 5(b) creditworthiness and 5(c) life and health insurance pricing
• GDPR Article 35 for the DPIA; Regulation (EU) 2026/1744 permits cross-reference between the two

The template — 14 fields

• ☐General description: intended purpose, provider, version and its relation to previous versions
• ☐How it interacts with hardware, software and other AI systems not part of it
• ☐The forms in which it is placed on the market — embedded, download, API — and the hardware it runs on
• ☐Development process: methods and steps, including any pre-trained third-party models and how they were used, integrated or modified
• ☐Design specifications: general logic, key design choices and their rationale, what it optimises for, expected output and output quality, trade-offs accepted
• ☐System architecture, and the computational resources used to develop, train, test and validate
• ☐Training methodologies and datasets: provenance, scope, main characteristics, labelling and cleaning
• ☐Human oversight measures, and the technical measures helping deployers interpret outputs
• ☐Validation and testing: procedures, data, metrics for accuracy and robustness, potentially discriminatory impacts, and dated signed test logs
• ☐Cybersecurity measures
• ☐Capabilities and limitations, including accuracy for specific groups, and foreseeable unintended outcomes
• ☐Pre-determined changes and how continuous compliance is maintained
• ☐Harmonised standards applied, or the solutions adopted where none were
• ☐The declaration of conformity, and the post-market monitoring plan

What to retain

The versioned pack, tied to the release it describes, plus the separate instructions for use given to deployers. Documentation that does not correspond to the version actually running is worse than none.

Where these fields come from

• EU AI Act Annex IV, nine points (sub-point lettering omitted deliberately: sources conflict on the letters, not the content)
• EU AI Act Article 13(3) for instructions for use — not Annex IX, which covers registration for real-world testing
• Model Cards for Model Reporting (Mitchell et al., arXiv:1810.03993): Model Details, Intended Use, Factors, Metrics, Evaluation Data, Training Data, Quantitative Analyses, Ethical Considerations, Caveats and Recommendations

The template — 9 fields

• ☐Groups tested: sex categories, race/ethnicity categories, and the intersectional combinations
• ☐Selection rate per category — those selected to advance, divided by applicants in that category
• ☐Scoring rate per category, where the system scores rather than selects — the rate of scores above the sample median
• ☐Impact ratio per category, against the most-selected or highest-scoring category
• ☐Number of individuals assessed in each category, and the number in an unknown category
• ☐Any category under 2% of the data excluded from the impact ratio, with the justification
• ☐Data used: historical use data, or test data with an explanation of why and how it was generated
• ☐Disparities found, the explanation for each, and the action taken or the reasoned decision to accept
• ☐Auditor independence, tester, date, and next scheduled test

What to retain

Dated results with the threshold stated in advance, and the record of what followed a failure. Where NYC Local Law 144 applies, a summary of results is published on the careers section of the website and kept there for at least six months after latest use, and candidates get at least ten business days' notice.

Where these fields come from

• NYC Local Law 144, rules at 6 RCNY §§5-300 to 5-304: selection rate, scoring rate, impact ratio, intersectional categories, the 2% exclusion, published summary contents, and the ten-business-day candidate notice
• EU AI Act Article 10(2) on examining and mitigating bias in datasets

The template — 10 fields

• ☐Claimed performance, and the metric behind the claim
• ☐Metrics and probabilistic thresholds defined in advance, appropriate to the intended purpose
• ☐Test set, and how it differs from training data
• ☐Results against the claim, including the conditions where it does not hold
• ☐Performance for specific persons or groups the system is intended to be used on
• ☐Edge cases and out-of-distribution behaviour
• ☐Adversarial testing: prompt injection including indirect injection, jailbreak, model extraction, data poisoning
• ☐Testing under real-world conditions, where undertaken
• ☐Failure modes found, and whether each is mitigated or accepted as residual risk
• ☐Tester, date, version tested; test logs dated and signed

What to retain

Test reports tied to versions, and the record of issues found, triaged and resolved. Red-team findings retained even where no action followed, with the reasoning. Residual risk judged acceptable, per hazard and overall.

Where these fields come from

• EU AI Act Article 9(6)–(7) on testing against prior-defined metrics and probabilistic thresholds, and Article 60 on real-world testing
• EU AI Act Article 15 on accuracy, robustness and cybersecurity

The template — 9 fields

• ☐Oversight role, who holds it, and the competence, training and authority they have
• ☐What that person sees, and whether it is enough to judge an output
• ☐How they are made aware of the system's capacities and limitations, and how anomalies are detected
• ☐Measures addressing automation bias — the tendency to over-rely on the output
• ☐Interpretation tools and methods available to them
• ☐Their ability to decide not to use the system, or to disregard, override or reverse its output
• ☐The stop mechanism, and how the system halts in a safe state
• ☐Expected decision volume and the time available per decision
• ☐Escalation route and out-of-hours cover

What to retain

The plan, training and competence records, and a log of interventions actually made. A system with oversight designed in but never exercised should prompt a question about whether it is real.

Where these fields come from

• EU AI Act Article 14(4)(a)–(e), including automation bias and the stop function; Article 14(5) on two-person verification for remote biometric identification
• EU AI Act Article 26(2): the deployer assigns oversight to people with the necessary competence, training, authority and support

The template — 9 fields

• ☐Disclosure shown to people interacting with the system, its exact wording, and where it appears
• ☐Whether it is given at the latest at the time of first interaction or exposure, clearly and distinguishably
• ☐Whether the AI nature is instead obvious to a reasonably well-informed, observant person, and the basis for that view
• ☐Content types generated or manipulated, and the visible or audible label applied to each
• ☐Machine-readable marking method, and whether it survives normal editing
• ☐For deepfakes: the disclosure, and any limitation where the work is artistic, satirical or fictional
• ☐For text published to inform the public on matters of public interest: the disclosure, or the human review and named editorial responsibility relied on instead
• ☐Emotion recognition or biometric categorisation: how exposed people are informed
• ☐Accessibility and languages of the disclosure

What to retain

Screenshots or recordings of the disclosure as users actually see it, the technical specification of the marking, and verification that it is detectable.

Where these fields come from

• EU AI Act Article 50: (1) interaction disclosure, (2) machine-readable marking of synthetic content, (3) emotion recognition and biometric categorisation, (4) deepfakes and public-interest text, (5) form and timing
• California AB 2013 for generative training-data disclosure published on the developer's website

The template — 9 fields

• ☐Events logged: inputs, outputs, decisions, overrides, configuration and version changes
• ☐Fields captured per event, and what is deliberately excluded
• ☐Whether the logs identify situations that may present a risk or lead to a substantial modification
• ☐Retention period and the provision setting it — at least six months for high-risk systems, for provider and deployer alike
• ☐Storage, access control, and who may read the logs
• ☐Tamper-evidence measures
• ☐How one decision is reconstructed end to end from the logs
• ☐Deletion process at end of retention
• ☐For remote biometric identification: start and end time of each use, the reference database, the input data that produced a match, and the people who verified the result

What to retain

The specification, plus a periodic test that a specific past decision can in fact be reconstructed. Retention quietly exceeding what is lawful is its own exposure.

Where these fields come from

• EU AI Act Article 12 on automatic event logging, and Article 12(3) for the remote biometric identification minimum log set
• EU AI Act Article 19 (providers) and Article 26(6) (deployers): at least six months

The template — 10 fields

• ☐Metrics monitored, and the threshold that triggers action
• ☐How performance data is actively and systematically collected, documented and analysed across the lifetime
• ☐Data from deployers and other sources, and how continuous compliance is evaluated
• ☐Interaction with other AI systems, where relevant
• ☐Drift detection method and frequency
• ☐Who is alerted, how, and out of hours
• ☐Incident severity definitions, tied to the statutory categories
• ☐Per incident: what happened, when detected, when the causal link was established, who was affected, action taken
• ☐Reporting deadline met, to which authority, and whether an initial report preceded the complete one
• ☐Post-incident investigation, risk assessment, corrective action, and changes made

What to retain

Monitoring output over time, the incident log including incidents judged not reportable with the reasoning, and evidence of reports actually filed. The monitoring plan itself forms part of the technical documentation.

Where these fields come from

• EU AI Act Article 72: documented post-market monitoring system, proportionate, based on a plan that forms part of Annex IV
• EU AI Act Article 3(49) defines a serious incident; Article 73 sets the 15-day, 10-day and 2-day deadlines and permits an initial then complete report
• California SB 53: critical safety incidents reported to Cal OES within 15 days, or 24 hours where there is imminent risk of death or serious physical injury
• OECD AI Incidents Monitor: harm type, severity, scope and geographic scale as the structure of a harm record

The template — 15 fields

• ☐Dataset identifier and version, and the design choices behind it
• ☐Collection process and the origin of the data; for personal data, the original purpose of collection
• ☐Sources or owners of each constituent dataset, including scraped and purchased data
• ☐Whether datasets were purchased or licensed, and any restriction attached
• ☐Whether they contain material protected by copyright, trademark or patent, or are public domain
• ☐Whether they contain personal information or aggregate consumer information
• ☐Number and types of data points, and the labels used
• ☐Preparation: annotation, labelling, cleaning, updating, enrichment, aggregation, de-identification
• ☐Assumptions about what the data is supposed to measure and represent
• ☐Availability, quantity, suitability, and statistical properties relative to the people the system will be used on
• ☐Geographical, contextual, behavioural or functional setting the data reflects
• ☐Examination for biases likely to affect health, safety, fundamental rights or to cause unlawful discrimination, and the measures taken
• ☐Time period of collection, whether collection is ongoing, and dates of first use
• ☐Whether synthetic data generation was or is used
• ☐For third-party models: what the provider does and does not disclose

What to retain

The record itself, licences and contracts for acquired data, the published disclosure where one is required, and the trail linking a deployed model version to the dataset versions behind it.

Where these fields come from

• EU AI Act Article 10(2)(a)–(g) on data governance practices, 10(3) on relevance and representativeness, 10(4) on setting, and 10(5) on special category data for bias correction
• California AB 2013: twelve enumerated disclosures per training dataset, published on the developer's website
• Datasheets for Datasets (Gebru et al., arXiv:1803.09010): Motivation, Composition, Collection Process, Preprocessing, Uses, Distribution, Maintenance

The template — 9 fields

• ☐Access control over models, weights, prompts, training data and logs
• ☐Protection against data poisoning in training data and in retrieval sources
• ☐Defences against model extraction and inversion
• ☐Prompt injection handling, including indirect injection through retrieved content
• ☐Constraints on the tools and data an agent may reach
• ☐Secrets handling in prompts, logs and traces
• ☐Resilience to errors, faults and inconsistencies, and behaviour on failure
• ☐Dependencies on third-party models and their security posture
• ☐Penetration testing scope, and whether it included the model surface

What to retain

The control record, test results demonstrating the controls work, and penetration test results scoped to include the AI surface.

Where these fields come from

• EU AI Act Article 15 on accuracy, robustness and cybersecurity, including resilience to attempts to alter use, outputs or performance
• OWASP GenAI Top 10 for LLM applications as the working taxonomy of attack classes

The template — 9 fields

• ☐Supplier, product, and the AI inside it — including features added by vendor update
• ☐Your role and theirs under each applicable regime
• ☐Documentation the supplier provides, and what they refuse to provide
• ☐Instructions for use received, and whether they are sufficient to operate the system correctly
• ☐Contractual commitments on compliance, notification, audit rights and information access
• ☐Their sub-processors and upstream model providers
• ☐Whether anything you do makes you the provider: your branding, substantial modification, or a changed intended purpose
• ☐Obligations that remain yours despite the supplier relationship
• ☐Exit plan if the supplier becomes non-compliant, withdraws the product, or fails

What to retain

Completed assessments, the contract clauses relied on, and supplier documentation retained where given. Record refusals explicitly — an unanswered question is a finding, not a gap.

Where these fields come from

• EU AI Act Article 25 on when a deployer becomes a provider; Article 26(1) on using a system per its instructions
• US interagency model risk guidance (SR 26-2) and the NAIC AI model bulletin, both of which put third-party models and vendor due diligence expressly in scope

The template — 11 fields

• ☐Assessment route: self-assessment or notified body, and the basis for choosing it
• ☐Standards and specifications applied, and where harmonised standards were not applied in full, the alternative solutions adopted
• ☐For each control in the standard's control set: included or excluded
• ☐Justification for each inclusion, and for each exclusion
• ☐Implementation status of each included control, and the evidence supporting it
• ☐Evidence type per control: a quantitative test result, a document, or a human attestation
• ☐Index of the artefacts relied on and where each lives
• ☐Gaps identified, with an owner and a date for closing each
• ☐Declarations and markings issued, with dates
• ☐Registrations filed, and in which database
• ☐Surveillance schedule and re-assessment triggers

What to retain

The file, the declaration of conformity, certificates where issued, and the audit trail of findings and their resolution.

Where these fields come from

• EU AI Act Article 17 on the quality management system, Article 43 on conformity assessment, Article 47 on the declaration of conformity, Article 49 and Annex VIII on registration
• ISO/IEC 42001 requires a Statement of Applicability; we do not reproduce its clause text or control titles, which are paywalled and which we have not read

The template — 10 fields

• ☐Policy version, approval date, and approving body
• ☐Scope: which systems, which people, which suppliers
• ☐Accountable roles, named, with their decision rights
• ☐Who is covered beyond employees — contractors, service providers, and others operating systems on your behalf
• ☐Training delivered by role, with content, method and date
• ☐How competence is checked, not merely attendance
• ☐How the level was matched to each group's technical knowledge, experience and context of use
• ☐Route for staff to raise a concern about an AI system
• ☐Where workers are subject to an AI system at work: how they and their representatives were informed
• ☐Exceptions granted, by whom, and for how long

What to retain

The approved policy with version history, training and competence records by role, and the log of concerns raised and how each was handled.

Where these fields come from

• EU AI Act Article 4 (as amended by Regulation (EU) 2026/1744) and Article 3(56) defining AI literacy
• EU AI Act Article 26(7): employers inform workers' representatives and affected workers before using a high-risk system at the workplace

The template — 10 fields

• ☐System identifier and the register entry being closed
• ☐Reason for retirement, and the decision-maker
• ☐Date of last production use, and date of shutdown
• ☐What replaced it, if anything, and whether the replacement inherited the same obligations
• ☐Decisions made by the system that remain live or appealable, and who now answers for them
• ☐Logs, models, prompts and datasets retained; where they are held; who can read them
• ☐Retention period for each, and the provision or rationale setting it
• ☐Scheduled deletion date for each retained item, and who executes it
• ☐Notifications made: registration withdrawn, deployers informed, authority notified where required
• ☐Where documentation and evidence for the retired system now live

What to retain

The completed record, the retained artefacts at their stated locations, and eventually the evidence of deletion at end of retention. Keep the register entry, marked retired — deleting it removes your own ability to show what you once ran.

Where these fields come from

• EU AI Act Articles 19 and 26(6): logs retained at least six months, subject to data protection law
• Colorado SB 26-189: developers and deployers keep records demonstrating compliance for at least three years