Who will review the Audit Trail Review?

Seriously, who will review the Audit Trail Review? First who will execute the Audit Trail Review? An internal Quality role – in the course of the batch by batch execution and review of records, during internal inspections or during an regulatory inspection preparation? Or weekly, bi-monthly or every third batch – or simply just all audit trails created by a system?

And who will analyze in detail and  finally judge the results of the Audit Trail Review?

Scenario 1: The user must insert the material number into a production machine for each produced batch. The manual entry of the material number is underpinned by an Audit Trail function, so that the user can do corrections if his/her initial entry of the material number was accidently wrong. This material number is important for the batch and lot traceability down to the machine in production. But also in the next production step the material number will be verified and cross-checked by another production area.

  • One result of the Audit Trail Review shows that about 3 % of the initial entries were  entered incorrectly (reason for change: transcription error)- NO WONDER, this is a  realistic statistical probability.
  • Second result shows that the entries were corrected always on the same or next day when the initial entry was done. It seems that the probability of detection is quite very good.

Now, what would be the result of the Audit Trail Review? We found out what we already have known. But is it really a good process? Not really….

We have not investigated if all cases of wrong entries were corrected and if it has a high incidence of unreported / undiscovered wrong entries. If we would have a rate that 6 % of the initial entries were entered incorrectly and just have not been detected, or additional 3 % were detected but not corrected (because of missing time, lack of interest) it would be still a realistic scenario. Let’s assume you have mainly 2 persons working on that machine. The first person has a rate of wrong entries of 10 %; the second person has a rate of 1 % – the first person works in the company since 20 years; the second person is not very happy with its job – but maybe the first person just needs new glasses – ….

No matter what you think or judge now – the process of manual entries is simply error-prone and nearly out-of-control. Even a second verification by a person will never reduce the error rate to zero.

Was it useful to execute this Audit Trail Review? No – it is and it will always be a weak process. Any manual entry should be eliminated (here: full automated scanning with double check against master data and material routing).

Scenario 2:  In the QC laboratory the analyst must make 3 test runs of one single sample. You find out in the Audit Trail Review that for several batches the analyst have executed more than 3 test runs – e.g. for one batch the analyst has done in total 9 test runs and “deleted” the test runs 1 to 3 and 4 to 6.  In the analysis report there are only the runs 7,8, and 9 recorded. The audit trails show also that the runs from 1 to 6 were out-of-spec. – 7 to 9 were within the limits.

Wow – a judgement is not easy in the first moment – without knowing how often this happened and the real reason for such “changes” (or modifications), which could make sense to such a scenario. Let’s make it even worse – the audit trail of the system does not have the capabilities to record the reason for change, because it is just a simple log function and not a real GMP audit trail. If so, and no other records like instrument logbooks or laboratory notebooks indicate the reason for that changes (or behaviors), it leaves the judgement finally to a worst case of data manipulation. Even interviews or statements would not really rescue this scenario.

It is more or less simple: The entire (even declared as validated) process is so weak, disgraceful and error-prone, which is not acceptable at all. The sample identification, management and tracking is simply not in place and systems do not restrict the multiple usage of samples, which is basically technically an easy task. Again, here the Audit Trail Review is just a retrospective risk approach – the main problem is still that a process was designed which is not fit-for-purpose.

Summary: Some Audit Trail Reviews will just show that the process is not GMP compliant, which is normally easier to be found during general assessments. And the Audit Trail Review execution won’t be the solution to such scenarios – process changes must be initiated and executed.

Read more about different types of the Audit Trail and a rational and approach for meaningful Audit Trail Reviews  – LINK  

Contact us: talk@comes-services.com