Embracing the Risk-Based TMF

Introduction

The release of the 2017 ICH GCP E6(R2) guidelines marks a paradigm shift in clinical research. Risk is mentioned no fewer than 30 times in the guidance. ICH GCP E6(R2) section 5.1 requires that the sponsor of a trial, “implements and maintains systems for quality assurance and quality and control.” ¹ The EMA recognizes, that although these basic systems are generally ‘good enough’ to ensure the quality of a trial, “current practice can… be expensive and there are too many trials in which avoidable quality problems arise.” ² Their renewed focus on risk is meant to realign current clinical practices with the ultimate goals of clinical research. At the core of this desire to encourage risk-based thinking is the notion of proportionality: ensuring that precious resources are allocated where they can increase the likelihood of positive outcomes in the trial while reducing the negative effects as much as possible.

These advances in decision making and risk management are powered by new electronic systems - allowing instant insight into the innerworkings of a clinical trial. Now, enabled by emerging technology allowing for the creation of an Electronic Trial Master File (eTMF), riskbased quality management systems have the potential to revolutionize the creation and delivery of the most important clinical trial deliverable: the TMF.

Risk-Based Thinking

FDA guidance defines risk as “as the combination of the probability of occurrence of harm and the severity of that harm.”³ A quality system is defined as, “The sum of all aspects of a system that implements quality policy and ensures that quality objectives are met.”³ Combined as one, they are defined as “a systematic process for the assessment, control, communication, and review of risks to the quality of the drug product across the product lifecycle.”³

On their own, these definitions can seem overly-reductive, confusing, and frustrating to apply. Upon the release of the new guidelines many in the industry held or attended ‘R2 impact’ meetings introducing this concept of Risk-Based systems. These meetings often began with a brainstorming session to identify risk, and most often ended with stakeholder pushback, or a lengthy list of complaints.

Risk-based thinking, however, does not require the upending of your documentation processes or day-to-day experience, but rather the ability to reflect on existing systems and the willingness to make adjustments when presented with new information. As part of being compliant to ICH E6(R1), your organization already has quality systems in place. According to the EMA, the key elements of a quality system are:

• Documented procedures and validated methods being developed, implemented and kept up-to date;
• Documentation system that preserves and allows for the retrieval of any information/documentation (quality records/essential documents) to show actions taken, decisions made and results;
• Appropriate training of sponsor personnel as well as of the personnel in affiliates, at the Contract Research Organizations (CROs), vendors or other service providers and at trial sites;
• Reflection paper on risk-based quality management in clinical trials
• Validation of computerized systems;
• Quality control, for example monitoring of trial sites and central technical facilities on-site and/or by using centralized monitoring techniques;
• Quality assurance including internal and external audits performed by independent auditors⁴

The bullets above are the rationale behind the authoring of SOPs, the creation of a document hierarchy (like the TMF Reference Model), the requirements applied to documents and filing systems, the training of staff, and the Quality Control (QC) of TMF documents. Risk-based quality management takes the framework of the quality system one step further by identifying risks on a continuous basis.

Most of the changes implemented in ICH GCP E6(R2) are found in Section 5 – where the need for a quality management system that, ‘identifies, evaluates, controls, communicates, reviews, and reports’ risk’1 is stipulated. These ‘steps’ in the cycle of risk-based quality management can be best summarized as a series of questions:

1. Identify: What can go wrong (or has gone wrong already)?
2. Evaluate: What is the chance it will go wrong? Will the subject’s safety or the quality of the data be impacted? What are metrics can we use to define what is/isn’t ok?
3. Control: Are the risks acceptable? What can be done?
4. Communicate: Who needs to know our plan? How can changes be shared?
5. Review: How will we know if new risk appear? How we know if our plans are working?
6. Report: How can we learn from our decisions at the end of the trial? How will we share them with others?²

As part of their Reflection paper on risk-based quality management in clinical trials, the EMA provides a process map of their template risk-based quality management system, but acknowledges “all quality management processes are dynamic”. Responsibility rests with the sponsor to build a true quality system around this framework of questions.

Identify and Evaluate

In a recent poll conducted during a webinar presented by LMK Consulting and Medidata, industry attendees were asked to identify their biggest TMF challenges. The top three answers were:

1. Completeness: missing and/or incomplete documentation; 38% of participants face this as their biggest challenge
2. Collaboration: challenges working with internal and external partners; 25% of participants face this as their biggest challenge
3. Integration: multiple non-integrated processes and system; 17% of participants face this as their biggest challenge⁵

By introspectively inventorying and ranking their TMF challenges, these webinar participants had already embarked on the first two steps of creating a risk-based quality system for their TMF: risk identification and evaluation.

The EMA, in discussing risk assessment, the process of identifying and evaluating risk, focuses on leveraging information already gathered as part of ongoing trial conduct. Risk factors that are inherent to a trial should be mined from, “compliance metrics, performance measurements, quality audit and/or inspection outcomes”² as well as the electronic systems, like the ones used to construct the eTMF, or eCRF and IVRS systems and other CTMS systems. At this stage of developing a risk-based quality system for the TMF, all stakeholders should be represented and willing to dive into the structure and function of a trial: its design, its investigational product and its administration/operation.

Of course, if these TMF risks are not identified in the planning stages, regulatory agencies like the FDA or MHRA, are ready to identify them once the trial has begun. These agencies expect an inspection ready TMF not just at the end of a trial, but during every part of its lifecycle. As part of the FDA’s Bioresearch Monitoring Program (BIMO), highlevel inspection metrics are published each year. Commonly observed deficiencies are listed, broken down by the type of site inspected (CRO, Investigator, IRB, etc.). For the fiscal year 2016, and for the last several years, a common item on the list of deficiencies remains “inadequate record keeping”,⁶ specifically at the site level, and the Sponsor or CRO’s “failure to bring the investigators into compliance.”⁶

MHRA also publishes a more detailed report of GCP inspection metrics. Of three critical findings issued between 2015 and 2016, one finding centered on an inadequate TMF. According to MHRA, this “Trial Master File (TMF) presented as a hybrid paper and electronic TMF to the inspectors did not meet the requirements of the legislation.”⁷ The finding was considered critical because the structure of the TMF would not allow for accessible inspection and, “generally impeded inspectors from assessing GCP and legislative compliance.”⁷ Strikingly, the issues identified by the inspectors closely mirror the concerns expressed by the webinar participants:

Completeness

• TMF provided for inspection were incomplete (missing records)
• The TMF table of contents was found to be unreliable as the location of documents was not accurate.
• Previous versions of documents were not present.
• There was evidence that the uploading of documents was not being undertaken in a timely manner

Collaboration

• There was a lack of Quality Control (QC) process of the TMFs
• The DIA TMF Reference model had been implemented (although with modifications) via a table of contents for each trial to identify the location of TMF essential documents in defined sections however the structure of the actual TMF electronic folders had not been changed to reflect the model.
• There was extensive duplication of scanned documents.

Integration

• There was a lack of integration with other TMF systems i.e. there were no links or placeholders directing the inspectors to the relevant repository or system for the relevant documents and data.
• The TMF was not defined (i.e., no quality system record to confirm all the systems that held TMF records, e.g. regulatory document system)
• Documents were not consistently held in the TMF.
• The document date was in the file name and the date in the system was either the upload date or finalization date. For this reason it was not possible to order documents in document date order.⁷

Once the risks have been identified and quantified, ideally before they contribute to risk in the TMF, “The first step is to clearly understand the processes and outcomes which really matter”.² This is also where the concept of proportionality is most important. Not all risks identified need to be mitigated, nor is an exhaustive catalog of all perceived risk particularly useful. A risk-based quality management system should not distract but rather focus. It is through the process of engaging the data and consulting with all stakeholders that priorities can be developed – priorities that are aligned with the overall goals of the TMF or the trial. From these priorities comes the next step: making decisions and acting to align the resources for the TMF with the areas of greatest need.

Control and Communicate

For most organizations, risk identification and evaluation are the first tentative steps. Past experience can provide some insight into where mistakes might occur in the next trial’s TMF. ICH E6(R2), however, instructs us not only to identify risk, but take prospective mitigating actions to control risk. “Risk mitigation activities may be incorporated in protocol design and implementation, monitoring plans, agreements between parties defining roles and responsibilities, systematic safeguards to ensure adherence to standard operating procedures, and training in processes and procedures.”¹ In the process of controlling or mitigating risk we are asking ‘Is the risk acceptable?’ and ‘Are the potential upsides of eliminating a risk balanced with the resources needed?’

Communication is a simple-sounding objective that can be difficult to implement. Once a decision has been made and an action agreed upon, these measures must be communicated to all stakeholders. Stakeholders must be identified and provided with the documents, tools, and training necessary to fulfill their obligations to the mitigation strategy.

Let’s examine some real-world examples of risk mitigation and acceptance that could be implemented in response to the TMF challenges of the webinar participants:

1. Completeness: Missing and/or incomplete documentation

Clinical trial assistants (CTAs) have asked to collect ICF templates from the site for filing. The start-up team is behind on their filing so the CTA’s are not sure what version of the ICF the site was approved on. When reviewing the sites with local IRBs, the CTAs do not know which ICF templates are missing. As a strategy to located them, CTAs contact the sites and request ‘all’ the ICF templates from the site’s coordinator. The coordinator forgets about a few and they never are filed in the TMF.

In this case, the potential risks are unacceptable. Confusion about ICF versioning can become a serious area of non-compliance if a subject is consented with the wrong version. If the new version of the ICF contained important safety changes, an auditor might wonder if the patient made a truly informed decision.

During the next kick-off meeting the team might allocate resources to the start-up and CTA groups. A new workflow or checklist may be proposed to ensure the handoff between the start-up team and the CTA is not missing critical information, like the initial ICF version. New stakeholders, like CRAs, might be included to perform a QC step to compare which templates are in use at the site with the templates that have been filed in the TMF. The local IRB is also in the process of launching an online portal so that all stakeholders can track approvals.

2. Collaboration: Challenges working with internal and external partners

At the end of the study, 3,000 documents are left on the study’s team shared drive. Nobody knows if they have been filed so they are all moved to the TMF. Many of the documents were already filed so now there are thousands of duplicates. An auditor is frustrated because the duplicates make it difficult to review the necessary documents.

The team in this scenario, especially faced with a frustrated auditor, would find this risk unacceptable. Once the duplicates are listed as an audit finding, it will take hundreds of re-work hours to remove the duplicates, and there is a risk that originals could be deleted in the process. As a mitigation step for future studies, the team wants to move toward ‘a single source of truth’, meaning adopting an eTMF system that can replace the many document repository systems used across the study. The team has also implemented a formal rule that TMF documents are not allowed on the shared drive.

3. Integration: We have multiple non-integrated processes and system

Site visit correspondence is supposed to be generated by the clinical trial management system (CTMS) and automatically filed in the TMF once sent and finalized. The CRAs don’t like working with the CTMS template because it is hard to use. Most of the CRAs send an email instead. At close-out review of the investigator site file, many sites are missing confirmation and followup letters or have two letters (one from CTMS and one in an email). The sites are frustrated when they are contacted for copies of these letters.

In this scenario, the potential harm is less significant. Although significant rework is required, the CRAs are contacted and most have the correspondence in their sent mail folders. It is decided that, in the short term, all CRAs will send confirmation letters via email and then file them immediately, as the company is already working on an update to CTMS that will make working with follow-up and confirmation letters more convenient. In fact, the CRAs will have an iPad, and can begin drafting the letters while at the visit.

In all these scenarios, a common theme is a failure to identify the potential risks ahead of time. In the absence of a documented procedure, the path of least resistance is often taken. Risk factors, like the problematic letter template or a growing and neglected shared-drive folder, were ignored. When a mismatch exists between a process and the resources available to satisfactorily complete that process, harm to the trial results.

Review and Report

A risk-based quality system is a cycle. It is not enough to propose and implement a solution in a vacuum; decisions must be continually reevaluated to ensure they are producing the desired outcome. In every step of the risk-based quality cycle, access to the metrics of a trial is essential. Metrics can assist in identification of risk. They can also aid in quantifying and evaluating risk by giving fuller visibility to the current status of the TMF. Most importantly, setting these quality tolerance limits provides actual verification that risk mitigation strategies are working. Metrics for the TMF usually fall into three key categories:

Timeliness: Duration of time between document finalization and availability in the TMF

Quality: Acceptability, completeness, and accuracy of content, metadata, and filing location of document content

Completeness: Measure of filed documents in relation to expected documents at the time when the assessment is conducted

With the adoption of electronic clinical trial systems, there is usually an existing pool of data from which to generate metrics. In many cases, these systems have built in reporting functionality that is not being fully utilized and the eTMF is no exception. Disregarding the near-term challenges of transitioning from paper files to an eTMF, the benefits of moving to an eTMF are significant and numerous:

Efficiency

• Documents do not need to be physically collected and can be filed immediately from anywhere
• Documents can be distributed instantly
• Documents can be more easily versioned and tracked
• Reconciliation and review do not require the collection of physical documents
• Personnel resources can be deployed in a more flexible manner
• Proper metadata allows for search functionality and allows for consistency and portability between organizations

Cost

• Labor related to physically manipulating documents is eliminated
• Physical space requirements are drastically reduced and the remaining need can be outsourced
• Moving costs are eliminated
• Accelerated timelines reduce overall costs

Compliance

• Inspections and audits are simplified
• Access controls and security can be standardized
• Meeting retention and archiving requirements
• Greater visibility of the quality and completeness

These benefits of an eTMF help stakeholders act more efficiently, potentially resolving quality issues resulting from unsustainable practices. Together, the benefits afforded by eTMF technology and the data that the technology compiles, can form the basis of a robust QC system, where risks are continually evaluated and identified, and mitigation decisions are followed through to completion to ensure their intended outcome. TMF QC is not a spreadsheet in your shared drive or an inventory of received documents. TMF QC is the full completion of the cycle of risk-based quality management.

Conclusion

The goal of a Risk-Based Quality Management System, for any aspect of a trial, including the TMF, is to balance conflicting priorities. The desire to efficiently and inexpensively complete a trial must be balanced with regulatory compliance and the wellbeing of subjects. The desire for completeness in the TMF must be balanced with limited resources and innate human factors. Although a Risk-Based Quality Management System cannot prevent every instance of harm, it ensures stakeholders focus on what truly matters. Above all, risk-based thinking balances the human tendency to be inconsistent by leveraging the power of reason; replacing ‘gut instinct’ with informed decisions.

References

1. Integrated Addendum to ICH E6(R2): Guideline for Good Clinical Practice. https://www.fda. gov/downloads/Drugs/Guidances/UCM464506.pdf
2. Reflection paper on risk-based quality management in clinical trials. http://www.ema. europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/11/WC500155491.pdf
3. Guidance for Industry Q9 Quality Risk Management. https://www.fda.gov/downloads/ Drugs/Guidances/ucm073511.pdf
4. ICH Harmonised Tripartite Guideline, Guideline For Good Clinical Practice E6(R1). https:// www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R1_ Guideline.pdf
5. Single Source of Truth: the Future of TMF Automation (webinar) http://www. lmkclinicalresearch.com/single-source-truth-future-of-tmf-recap/
6. Bioresearch Monitoring (BIMO) Metrics – FY’16 https://www.fda.gov/downloads/ ScienceResearch/SpecialTopics/RunningClinicalTrials/UCM561049.pdf
7. MHRA good clinical practice (GCP) metrics report for 2015 to 2016 https://www.gov.uk/ government/uploads/system/uploads/attachment_data/file/631254/GCP_INSPECTIONS_ METRICS_2015-2016__FINAL_21-07-17_.pdf