How do you validate an AI feature in a clinical trial system? You specify its intended use, set an acceptance criterion as a rate rather than a single correct answer, qualify the installation, the configuration and the performance against a scripted set, and bring every model version under change control.
That sequence is familiar from any computerised system. One thing changes it. The output of a probabilistic feature varies with its input, so a test script cannot assert a single expected result and call the feature qualified. This guide covers intended use, the acceptance criterion, what installation, operational and performance qualification each settle, what they leave open, model versioning, and the route a model update travels. The lifecycle view of where these features arise sits in how AI is used in clinical trials, and the buying tests sit in AI for compliance and inspection readiness.
- Validation covers the intended use of the feature, the configuration it runs in and the review step around it.
- Performance qualification of a probabilistic feature measures a rate across a scripted set, and the acceptance criterion is written as a rate before testing starts.
- A model version is part of the validated configuration, so a model update is a change to a validated system.
How Do You Validate an AI Feature in a Clinical Trial System?
The obligation comes from the general computerised systems rules. Section 6.19 of the MHRA GXP Data Integrity Guidance and Definitions, revision 1 of March 2018 and last updated on 27 September 2021, states that computerised systems “should be validated for their intended purpose”. Section 3.6 sets the effort “commensurate with the risk”. ICH E6(R3), adopted on 6 January 2025, states at Principle 9.3 that computerised systems should be fit for purpose “through risk-based validation”, and at section 3.16.1(x)(viii) that the assessment “should be performed before being used in the trial”.
Five deliverables carry an AI feature from a vendor release to a qualified function at a site, and each produces a document the site can hand to an inspector.
| Step | What it produces | The question it answers |
|---|---|---|
| 1. Specify the intended use | One statement per feature, naming the records it reads, the output it produces and the role that confirms it | What is this feature for, and where does its output land? |
| 2. Assess the risk | A risk rating tied to what the output touches, setting the depth of testing | How much testing does this feature earn? |
| 3. Write the acceptance criterion | A numeric threshold expressed as a rate, agreed before any test runs | What result counts as a pass? |
| 4. Qualify installation, operation and performance | IQ, OQ and PQ records, with PQ run against a scripted set of the site’s own cases | Does the feature behave as specified in our configuration? |
| 5. Place the version under change control | An inventory entry carrying the model version, and a notification clause in the agreement | How will we know when this behaviour changes? |
Vendor evidence covers part of this and site evidence covers the rest. Section 6.19 states that vendor-supplied validation data in isolation “is not acceptable” without regard to configuration and intended use. AQ supplies the vendor half: every release goes through validated release and change control, with installation and operational qualification evidence and an assessment against 21 CFR Part 11 and EU GMP Annex 11 on computerised systems, at its revision of January 2011. Performance qualification against the site’s configuration and intended use is the site’s half. The split between the two halves, and the paperwork each one lands in, is set out in the guide to the MHRA computerised systems dossier.
What Does Intended Use Mean for a Probabilistic Feature?
Intended use for a deterministic function is easy to write. A date field accepts a date in a stated format and rejects anything else. Intended use for a probabilistic feature has to be narrower than the feature itself, because the model accepts any input and returns something for all of it. The statement therefore bounds the input as tightly as the output.
- Name the records in scope. A classification feature validated on the essential document set has no qualified use on correspondence.
- State the output as a proposal. The statement says the feature proposes a classification and a named person confirms it, which places the regulatory act on the person.
- Name the confirming role. The delegation log evidences the authority of whoever confirms the output.
- State the consequence of a wrong output. A missed flag leaves a document unfiled, which sets the risk rating and therefore the test depth.
A worked example shows the difference. A hypothetical trust, Northgate General, writes this: the feature reads documents uploaded to the investigator site file, proposes one of 32 document types from the site’s own index, and the research governance lead accepts or amends each proposal before filing. That statement is testable. “The feature classifies documents” names no input set, no output set and no confirming role, so nothing can be measured against it.
The classification of the feature itself comes first. Features that apply a fixed rule carry a lighter validation burden than features that return a judgement, and the four tiers are set out in automation versus AI in a trial platform. What a model may and may not do once qualified is covered in the MHRA expectations for AI in GxP systems.
A statement that bounds nothing tests nothing.
What Do IQ, OQ and PQ Settle, and What Do They Leave Open?
Installation, operational and performance qualification remain the structure for computerised system validation of an AI feature. The method itself is covered in the guide to GAMP 5 and computerised system validation. The three stages answer three different questions, and the third stage is where a probabilistic feature parts company with a deterministic one.
| Stage | What it settles for an AI feature | What it leaves open |
|---|---|---|
| Installation qualification | The release is the one the vendor shipped, the environment matches the specification, and the model version in the build is recorded | Whether the feature behaves usefully on this site’s records |
| Operational qualification | The feature is switched on for the intended roles, the confirmation step cannot be bypassed, and the audit trail captures the proposal and the decision | The quality of the proposals the feature puts in front of a reviewer |
| Performance qualification | The measured rate of correct proposals across a scripted set of the site’s own cases, against a pre-agreed threshold | Behaviour outside the scripted set, and drift after go-live |
The human confirmation step carries weight precisely because of the two gaps in the right-hand column. A feature qualified at 94% on a scripted set still produces wrong proposals, and the reviewer is the control that stops a wrong proposal becoming a record. AQ’s AI proposes, flags and drafts, and a named person reviews and confirms before any output becomes a record.
Also Read: How to Validate an eSource System Before First Patient In
How Is Performance Measured When the Output Varies With the Input?
Performance qualification of a deterministic function asserts equality. The script states the expected output, the system returns it, and the test passes. Performance qualification of a probabilistic feature measures a rate across many cases, so the test design carries four decisions.
- The scripted set. A fixed, versioned collection of the site’s own cases with a known correct answer for each, large enough that one case cannot move the result.
- The ground truth. A named person establishes the correct answer for every case before testing, and that record is the reference for the measured rate.
- The acceptance threshold. A rate agreed in the validation plan before any test runs, which stops the threshold moving to fit the result.
- The failure that matters most. A wrong proposal a reviewer accepts is the one failure that reaches the record, so the plan sets a separate and tighter limit on it.
That last decision changes how the result is read. A feature can hit a 95% correct rate and still be unfit, because the question is which direction the 5% runs in. A wrong flag a reviewer rejects costs time. A wrong proposal a reviewer accepts enters the record and becomes a data integrity finding. Performance qualification therefore reports the two kinds of error separately, and ICH E6(R3) section 3.16.1(x)(ii) requires the requirements for computerised systems to be “proportionate to the importance of the computerised system”.
The joint EMA and FDA guiding principles of good AI practice in drug development, published on 14 January 2026, take the same position across their ten principles. Principle 4 requires a clear context of use. Principle 8 requires risk-based performance assessment of the full system, including the human and AI interaction. Principle 9 requires life cycle management with scheduled monitoring and periodic re-evaluation for issues such as data drift.
What Counts as a Model Version?
A software version is visible. A release number changes and the site assesses it. A model version is often invisible from the screen, and four separate things can change the behaviour of one AI feature without the interface moving.
- The model itself. A new model generation or a retrained version of the same model returns different proposals on identical input.
- The instruction and configuration. The instruction the feature runs under, and the settings around it, shape the output as much as the model does.
- The reference material the feature reads. A feature that compares a document against an expected document set changes behaviour when that set is edited.
- The thresholds. The confidence level at which the feature raises a flag decides how many flags a reviewer sees.
Those four parts together form the validated configuration, so the inventory entry records all four rather than the software release alone. The practical test is a question an inspector can ask: a proposal confirmed in May 2026 came from a specific configuration, and the site should be able to name it. Section 6.8 of the MHRA guidance makes the general point for automation, stating that electronic worksheets “should be version controlled”.
How Does Model Change Control Work After an Update?
Model change control keeps a qualification current. The difficulty is specific to AI features, because a vendor can improve a model with no change a user would notice. The site therefore learns of the change through its agreement rather than its screens, and routes it by what changed.
| What changed | How the site finds out | The route it travels | The evidence that closes it |
|---|---|---|---|
| The model generation | Vendor notification under the technical agreement | Re-run the full scripted set and compare the rate against the previous version | A PQ report for the new version, with the two rates side by side |
| The instruction or configuration | Release notes, or a site-side configuration request | Targeted regression on the cases the change touches | A test record covering the affected cases, and an updated intended use statement where the scope moved |
| A threshold | A site-side change request | Re-measure the flag volume and the two error rates at the new setting | A record of the rates before and after, with the reviewer workload stated |
| The reference material | A site-side edit to the expected document set or index | Spot-check the cases the edit affects | A dated note in the change record naming the cases checked |
| Nothing material | Vendor notification with no behavioural change | Documented impact assessment with no testing | A signed assessment recording the decision and its basis |
The last row is the one sites skip and inspectors ask about. A documented decision that no re-testing was needed is evidence of control. An undocumented one reads as an uncontrolled change. Each route belongs in the quality management system rather than an email thread, so the change record, the test evidence and the approval sit in one place.
One contract term makes the whole table workable: the technical agreement names model changes as notifiable. The wider terms to ask a supplier for sit in the platform evaluation guide.
A model that improves quietly is still a change to a validated system.
What Does Periodic Review Add After Go-Live?
Performance qualification evidences one date. Periodic review keeps the evidence current, and section 6.15 of the MHRA guidance already expects a “periodic audit of the data generated” and asks whether system maintenance has occurred and whether its impact was assessed. Three measures make that review meaningful for an AI feature.
- Re-run the scripted set on a schedule. The same cases on the current configuration give a rate comparable with the original PQ result, which turns drift into a number.
- Read the acceptance and rejection pattern from live use. A reviewer rejecting most proposals from one feature indicates a feature performing below its qualified rate in practice.
- Review the flags that were missed. Gaps found by a monitor or an auditor, rather than by the feature, measure the error that matters most.
A feature performing below its threshold becomes a deviation, and the corrective action is a configuration change, a threshold change or a narrower intended use.
Also Read: AI-Assisted Quality Risk Detection for UK Research Sites
How AQ Supports Validation of Its AI Features
AQ runs AI as Connected Intelligence, with AQ Assist as the in-platform assistant. The design decision that governs validation runs through this guide. The AI proposes, flags and drafts, and a named person reviews and confirms, so the qualification covers the proposal plus the review step around it.
- Release evidence arrives with each version. Every AQ release goes through validated release and change control, with installation and operational qualification evidence and a 21 CFR Part 11 and EU Annex 11 assessment, which supplies the vendor half of the qualification file. The site’s performance qualification supplies the other half.
- Every AI action is in the audit trail. The proposal, the decision and the user sit in the same trail as manual entries, which gives performance review a source other than memory.
- Flags route into the quality system. A feature performing below its threshold becomes a deviation in AQ QMS with a CAPA action against it, which keeps the finding and the fix in one thread.
- The assurance position is documented. AQ holds NHS Data Security and Protection Toolkit Standards Met for 2026 and Cyber Essentials certification, operates ISO 27001-aligned controls, and the platform is UK-hosted.
AQ has been in continuous daily NHS use for six years, supports MHRA compliance, and was built by Phase 1 trial doctors. A live demo walks through a proposal, a confirmation and the audit trail entry the pair leaves behind.
Frequently Asked Questions
How do you validate an AI feature in a clinical trial system?
You write an intended use statement naming the records the feature reads, the output it produces and the role that confirms it. You set an acceptance criterion as a rate before testing. You then run installation, operational and performance qualification, with performance qualification measured against a scripted set of your own cases, and you place the model version under change control.
Does an AI feature need validating separately from the system?
The AI feature needs its own intended use statement and its own performance evidence, inside the validation of the system that hosts it. Section 6.19 of the MHRA GXP data integrity guidance requires validation for intended purpose, and a classification feature has a different intended purpose from the document store around it.
What does performance qualification measure for an AI feature?
It measures a rate rather than an equality. A fixed scripted set of cases with known correct answers runs through the feature, and the result is the proportion of correct proposals against a threshold agreed in advance. The report separates the wrong proposals a reviewer rejected from those a reviewer accepted, because only the second kind reaches a record.
Is a model update a change to a validated system?
Yes. A model update changes the behaviour of a qualified function, so it travels the change control route any other change travels. The route depends on what changed. A new model generation earns a full re-run of the scripted set, and a notification with no behavioural change earns a documented impact assessment recording that decision.
What should the system inventory record for an AI feature?
The model version beside the software version, the instruction and configuration in force, the reference material the feature reads, and the thresholds it runs at. Those four parts form the validated configuration, so a record created under a past configuration can be read against the behaviour that produced it.
How often should an AI feature be re-validated?
On every material change, and on a schedule between changes. The schedule follows the risk rating of the feature, and the review re-runs the original scripted set so the new rate is comparable with the qualified one. Principle 9 of the EMA and FDA good AI practice principles asks for scheduled monitoring and periodic re-evaluation for data drift.
