Writing the User Requirements Specification for an AI Compliance Assistant

The User Requirements Specification (URS) is where most AI validation projects quietly go wrong. Teams inherit a template written for a LIMS or a label printer, paste in a few sentences about "the system shall provide accurate answers", and then discover during IQ/OQ/PQ that nothing in the document is testable. GAMP5 Second Edition is unambiguous on this point: requirements drive the risk assessment, the test strategy, and ultimately the acceptance decision. For a probabilistic system such as a retrieval-augmented AI assistant, a vague URS does not just create rework — it makes the system unverifiable.

This post sets out how QA, validation, and CSV teams at mid-size pharma manufacturers and CDMOs should structure a URS for an AI compliance assistant so that it survives both internal review and an inspection under EU Annex 11 and 21 CFR Part 11.

Start with intended use, not features

Annex 11 Clause 4.4 requires that user requirements be traceable throughout the system life cycle. That traceability is only meaningful if the top of the chain is a precise statement of intended use. For an AI assistant, intended use is the control that keeps scope from drifting into regulated decision-making.

A defensible opening statement looks like this:

The system provides retrieval-based answers to questions about approved internal SOPs, validation protocols, and referenced regulatory guidance. It supports human preparation of deviation investigations, change control assessments, and audit responses. It does not make quality decisions, does not approve or release, does not generate GMP records, and does not replace review by qualified personnel.

Write the exclusions as explicitly as the inclusions. Under the EU AI Act, intended purpose is the hinge on which classification turns — a general-purpose knowledge retrieval tool used to inform human work sits very differently from a system that determines batch disposition. Your URS is the first document an auditor or notified body will read to understand which of the two you built.

Requirements must be testable, not aspirational

The single most common failure in AI URS documents is the untestable requirement. "The system shall provide accurate answers" cannot be verified, cannot be traced to a test case, and cannot be assessed for residual risk. Rewrite every requirement until it specifies an observable behaviour with a measurable acceptance criterion.

• Weak: The system shall retrieve relevant documents.
• Testable: For a defined 100-question reference set derived from the approved SOP corpus, the correct source document shall appear within the top three retrieved passages in at least 95% of cases, verified by two independent SME reviewers.
• Weak: The system shall cite sources.
• Testable: Every generated answer shall display the document identifier, version number, effective date, and section reference for each source passage used. Answers with no retrieved passage above the configured relevance threshold shall return a defined no-answer response.
• Weak: Access shall be controlled.
• Testable: Authentication shall occur via corporate SSO with role-based entitlements mapped to existing document access rights; a user without read access to SOP QA-014 shall not receive content from QA-014 in any answer.

That last point deserves emphasis. Retrieval systems flatten document permissions unless explicitly designed not to. If your DMS restricts certain validation protocols to a named group, the URS must require that the assistant honours the same entitlement model at query time, not merely at ingestion time.

Requirement categories that belong in an AI URS

Structure the document so that each section maps cleanly to a risk assessment and a test type. A workable set of categories for a GxP AI assistant:

• Functional retrieval requirements — query handling, relevance thresholds, top-k behaviour, handling of ambiguous or out-of-scope questions, multilingual behaviour (critical in DACH environments where SOPs exist in German and batch records reference English regulatory text).
• Source and traceability requirements — citation format, version binding, behaviour when a cited document is superseded, prevention of answers built on obsolete or draft documents.
• Corpus governance requirements — which document classes are ingestible, how effective-date logic is applied, how withdrawn SOPs are removed and within what interval.
• Data integrity and record requirements — audit trail content per Annex 11 Clause 9 and 21 CFR Part 11(e), retention period, immutability, exportability in human-readable form.
• Security requirements — authentication, authorisation, prompt injection resistance, segregation of tenant data, encryption in transit and at rest.
• Performance and availability requirements — response time targets, concurrent user load, planned availability windows, degraded-mode behaviour.
• Data residency and privacy requirements — processing location, subprocessor disclosure, GDPR Article 28 obligations where prompts may contain personal data.
• Change and configuration requirements — notification lead time for model changes, ability to pin a model version, regression evidence supplied by the vendor.
• Operational requirements — backup and restore, business continuity, decommissioning and data return.

Requirements the vendor must meet, not just the software

Annex 11 Clause 3 places supplier assessment squarely within your quality system, and ICH Q10 extends the same expectation to outsourced activities. For cloud-delivered AI, a material share of your requirements are contractual rather than technical. State them in the URS so they flow into the supplier assessment and the quality agreement:

• Minimum notice period before any change to the underlying language model or embedding model.
• Right to receive vendor regression test evidence for each release affecting retrieval or generation behaviour.
• Right of audit, including remote audit of the vendor's development and release controls.
• Defined exit provisions: corpus return format, deletion confirmation, audit trail export.
• Commitment to processing within the EU/EEA where required by your data classification policy.

Handling non-determinism honestly

Do not write a requirement stating the system will produce identical output for identical input unless you have configured it to do so and can prove it. A more defensible approach is to specify bounded behaviour: the set of source documents retrieved for a given query shall be stable across repeated executions under unchanged corpus and configuration, and the answer shall be semantically consistent as assessed against defined acceptance criteria by qualified reviewers.

This distinction matters because it shifts verification from character-level equivalence — which no LLM will pass — to the property that actually protects patients and product: the answer must be grounded in the same approved source, every time.

Traceability and review

Number every requirement uniquely and maintain a requirements traceability matrix from URS through functional risk assessment to test case and test result. GAMP5 Second Edition's emphasis on critical thinking means you are expected to justify why some requirements receive scripted testing and others are covered by supplier evidence or unscripted testing. The URS is where that justification begins.

Finally, have the document reviewed by the people who will actually use the system — not only QA. A validation manager will catch the audit trail gaps; the deviation investigator will catch that you forgot to require search across attachment tables in annexes. Both signatures belong on the approval page.

A well-written URS turns an AI assistant from an interesting pilot into a system you can defend. It is the cheapest document in the validation package and the one that determines the cost of everything downstream.

See how ComplianceGxP handles user requirements specification for AI compliance assistants for pharma and CDMO teams: See how it works →