The COVID-19 pandemic is an unparalleled global event that caught all businesses off-guard. This is especially prevalent in the pharmaceutical industry as on-going clinical trials are disrupted due to lockdown measures and social distancing requirements. In many existing trials, subjects simply cannot get to site, visits are being missed, data are not being recorded and clinical monitoring is challenged as Clinical Research Associates (CRAs) experience travel restrictions.
Members of the statistical consultancy team from CRO Quanticate outline the three core principles to the conduct of clinical trials which can be considered to overcome these challenges.
Sample Size Calculations
Even under normal circumstances, getting the sample size right is the foundation that ensures a successful study while minimizing the number of patients undergoing potentially invasive study procedures and curtailing costs for the sponsor.
In studies where recruitment is ongoing, the pandemic has and will likely slow enrolment as patients’ movements and access to healthcare facilities continues to be limited. In an ideal world sponsors might think of continuing as planned until normality is restored and procedures restart as before. However, shareholders and investors might disagree: the costs of running a study that doesn’t deliver study results in the originally agreed timelines can be overwhelming for major pharma companies and simply a killer for smaller biotech companies. So, what can be done?
One potential solution is to investigate the impact on study power if recruitment was stopped and only patients currently recruited were allowed to continue and complete the study.
Such an exercise is similar to what is often done while planning the study. Sponsors and their partners evaluate the impact of assumptions (e.g. comparator arm response level, variability, etc), with the difference that new information obtained from external sources might allow to reassess study design features such as power and sample size in the light of this new knowledge. Visualize a new study where unblinded results had only been made available after the present study protocol was finalized. If the study suggested that the response in the comparator arm was in fact different from what was originally assumed, the current sample size might still allow sufficient power to detect a clinically relevant effect.
While this situation is not a very common one, re-evaluating study power in a large number of scenarios (either via closed formulas or simulations) will provide the study team with a better understanding of their next actions. However, if the only scenario to allow a sufficient power to be achieved requires a treatment effect twice as large as originally planned it might be worth considering if the study can continue with the same characteristics as before the pandemic.
This example brings up an important item - the extent to which study design itself can be altered to respond to the currently evolving scenario. For example, consider a study where it was planned to enroll 200 patients to demonstrate a difference in a continuous outcome between treatments ≥ 3. Assuming that the mean response was 10 in the treatment arm and 5 in the comparator arm with an 80% power (standard deviation = 5 and a one-sided test at a 2.5% level, dropout rate assumed to be 0% for simplicity). If no further information on the potential treatment effect has arisen from external sources, it is clear that if only 150 patients have currently been recruited it is not possible to halt recruitment now because this would leave only 68% power and increase the chances of the study being a failure.
In this situation, a viable option is to amend the study protocol to include an unblinded (and previously unplanned) interim analysis. The main purpose here is to estimate the current treatment effect and deriving measures of future study success (i.e. conditional power or predictive power, depending on whether you root for frequentist or Bayesian statistics) given the current data. Using this information (yet considering all available patient-level data) the Data Monitoring Committee can make a better decision as to whether the study is still likely to succeed or not.
The advantage of this approach is that only studies that are reasonably likely to deliver positive results, and for which no safety concerns arise, will continue, thus freeing up resources for other projects and minimizing unnecessary efforts on all sides. It is relevant to point out that going down this route has implications on study design: if an unblinded interim is added, preservation of type I error rate needs to be maintained via, e.g., alpha-spending functions that ultimately imply an increase in the overall sample size. While this might seem counter-productive, considering the difficulties in achieving the planned, and lower, sample size, this ensures that the efforts of recruiting additional patients are only done for promising compounds.
Estimands and Missing Data: Is COVID-19 an Intercurrent Event?
In normal circumstances, intercurrent events should be outlined in the protocol and the estimands defined to outline the approach to each anticipated intercurrent event. In these extraordinary times it is expected that protocol amendments will be written to document changes to the design and conduct of studies. This acknowledges adaptations to travel restrictions, limited access to sites, subjects and site staff suffering from COVID-19. It therefore seems reasonable to review and adapt study estimands as well.
Protocols may be adapted to allow for a pause in treatment, an alternative treatment, remote visits, larger visit windows and so on. Subjects may miss visits due to logistical reasons or having the virus. Each of these situations can be treated as an intercurrent event and, for each, the most appropriate strategy selected. The most suitable approach will depend on the details of the trial, the study treatment and the indication, and will need to be agreed by the whole study team. Consequently, adaptations may be required to the planned analyses to ensure consistency with the estimands defined in the protocol.
Depending on which strategies are used for COVID-19 related intercurrent events, there will potentially be an increase in missing data. Updates to the approach to dealing with missing data may be required to ensure it is appropriately dealt with and consistent with the estimands.
Subscribe to our e-Newsletters
Stay up to date with the latest news, articles, and events. Plus, get special offers
from Pharmaceutical Outsourcing – all delivered right to your inbox! Sign up now!
In the simplest scenario, where only minimal changes to the trial conduct have been, or need to be, implemented, it might be reasonable and acceptable not to treat events related to COVID-19 as intercurrent events. In this scenario, the practical impact would simply be a larger than previously anticipated amount of missing data, and this can be tackled by amending the missing data approach outlined in the Statistical Analysis Plan, or by justifying the reasons for no changes.
Trials are ideally designed with the intention of minimizing missing data. In this situation, adaptations will be required as a reaction to the pandemic, as the need to minimize the amount of missing data is fundamental. By, for example, making some changes at the procedural level such as making use of local labs or switching to standard of care or self-administration of the investigational product, data can be collected, where under the original protocol these data would have been missing. These procedural changes will of course have some impact on the data collected, and analyses may need to be adjusted to take this into account. It is therefore critical that case report forms are amended to capture changes to trial procedures at each data collection, as well as reasons for treatment and/or study withdrawal. This information can then be considered in the statistical analyses so that it does not confound the treatment effect.
There is a further consideration in the scenario where there is an unacceptably large amount of missing data when determining the number of responders for the primary efficacy endpoint at the point-in-time of interest. If the original analysis method was based on a Generalized Estimating Equation (GEE), a weighted GEE could be considered instead, which would make use of the data from the preceding visits when computing estimates.
Sensitivity analyses may also be included to assess the impact of intercurrent events using techniques such as multiple imputation.
Improving Data Quality Through Centralized Statistical Monitoring (CSM)
It is likely that access to sites will be restricted for many months regardless of the trial. This will necessitate the need for alternative mechanisms to provide monitoring and oversight activities. More than ever, there is a need for CSM - this is highlighted by recent guidance from the FDA on conduct of trials during the pandemic: ‘If planned on-site monitoring visits are no longer possible, sponsors should consider optimizing use of central and remote monitoring programs to maintain oversight of clinical sites.’
While CRAs are no longer travelling to site, CSM will be required to check for the usual data patterns and anomalies noted here:
- Identify missing data, inconsistent data, data outliers, unexpected lack of variability and protocol deviations
- Examine data trends such as the range, consistency, and variability of data within and across sites
- Evaluate systematic or significant errors in data collection and reporting at a site or across sites; or potential data manipulation or data integrity problems
- Analyze site characteristics and performance metrics
- Select sites and/or processes for targeted on-site monitoring.
However, sponsors must also examine areas not previously required, e.g. traditional visits versus non-traditional visits and protocol defined endpoint data collection versus updated endpoint data collection. When analyzing at site or regional levels, it may be more important to have some awareness or measure of how far the virus had progressed at the time of reporting in that region, what measures were being taken and how regional healthcare systems were coping. Patterns must be monitored as the trial progresses but with an awareness that full understanding of the situation, its impact and what results may or may not be acceptable is still unfolding. Communication between departments is therefore key; signals must be raised on anomalies and they must be opened up for multi-disciplined discussion across study teams.
One example of anomalous measures in sites which can be captured by CSM is if a temperature measurement at a particular site differs from the others due to, for example, incorrectly calibrated thermometers. Tools such as CluePoints, and SAS JMP can pick up these differences by analyzing means and variances of temperature readings and comparing across sites. Without site visits, the need for this kind of analysis is increased as such differences are less likely to be picked up. And during these times, analyses should not only be carried out across different sites but by also comparing, for example, traditional patient visits to those where they are carried out at a different time or place due to the pandemic.
Conclusion
The COVID-19 pandemic is the biggest challenge the world has faced in decades; for clinical trial sponsors it is impacting the integrity and feasibility of ongoing studies as they are threatened by the outbreak’s continuous global spread.
COVID-19 is highly disruptive to ongoing clinical trials. The impact over the coming months and years will be widespread and multifaceted and it is not possible yet to identify the full extent and severity of its impact. It is however possible to examine each study’s core components on an individual basis and thus identify the most affected areas. By putting mitigating strategies discussed here in place it may be possible to salvage some of, if not all of, a study’s potential.
All the above areas impacted by COVID-19 call for updates to existing study documentation. It is important to note that these updates can be made in different ways and should be thought through carefully. The rapidly evolving situation differs across regions and countries. As such, protocol amendments or updates to the Analysis Plans will have to account for this and factor in a level of uncertainty. Sponsors should also consider the need to document changes to Protocol Deviations Plans, Analysis of Safety Data and Data Management Plans.
These actions will need punctual documentation and constant reviewing to ensure they are up to date and aligned with the constantly evolving situation. This way it will be possible to demonstrate that the highest standards were pursued and that scientific integrity and patient safety were safeguarded. With support and cooperation across the industry, ultimately patients and trial participants can receive the best possible care.
Author Biography
Karen Ooms is responsible for overseeing the Statistics department at Quanticate. Karen is a Chartered Fellow of the Royal Statistical Society and has a background in biostatistics spanning over 25 years. Prior to joining Quanticate in 1999 (Statwood), Karen was a Senior Statistician at Unilever.