Evaluating New Data-Driven Approaches Informing Global Investigative Site Selection

Ample anecdotal stories and studies in the literature indicate that the global investigative site selection process is highly inefficient and ineffective. A 2012 study conducted by my colleagues at the Tufts Center for the Study of Drug Development (Tufts CSDD), for example, showed that site identification through activation timelines are long and highly variable. That same study indicated that across all therapeutic areas, clinical trial sponsors must double the planned enrollment period to recruit the target number of study volunteers. Other studies show that, even after doubling the planned enrollment period, a high percentage of sites under-enroll and fail to recruit the requisite number of patients.

Several studies have shown that one in ten sites in a given multi-center study will fail to enroll a single patient. This latter finding is particularly problematic given the high direct start-up cost for each investigative site estimated to range between $20,000 and $50,000.

For decades biopharmaceutical companies have implemented strategies and tactics aimed at addressing the site selection challenge. These approaches include the use of proprietary and commercial databases to assist with site identification; master service agreements to expedite the contracting and budgeting process; and enhanced feasibility assessment to improve the selection process.

Biopharmaceutical companies continue to pilot and adopt new approaches - many that rely on richer and aggregated datasets and more robust and predictive analytics. This article summarizes the approaches that biopharmaceutical companies and CROs are now utilizing and their experiences to date. Results and insights come primarily from three recent studies conducted by Tufts CSDD:

• The 2016 site-selection study: Tufts CSDD conducted a study assessing practices and inefficiencies with site selection, study start-up, and site activation.¹ They interviewed 26 individuals at the director level and above from 21 companies (13 biopharmaceutical companies and 7 CROs, all mid-sized and large) to assess current practices and inefficiencies.
• The 2016 chokeholds in patient engagement study: Tufts CSDD interviewed 20 stakeholders in drug development to assess chokeholds in the patient engagement process. The twenty respondents represent all stakeholders: sponsors, CROs, sites, patient recruitment experts, and patients via a patient advocacy group in order to capture a holistic view of patient engagement.
• The 2017 Study Start-up study: Tufts CSDD conducted a broad survey assessing current practices in clinical trial site selection and study start-up.² 591 respondents from sponsor organizations, CROs, and other groups provided feedback on new practices and performance metrics.

Summarizing Current Approaches

The 2016 site-selection study found that, on average, 70% of clinical sites on a given clinical trial are repeat or familiar to organizations, and 30% are completely new. Not surprising however, in disease indications that are new to the organization or in new countries with specific regulatory requirements, a greater percentage of sites are completely new. The study also found that the site selection process takes on average 3.2 months, ranging widely from two weeks to six months.¹

In doing due diligence, 10 of 21 companies noted using internal tools and metrics based on past performance, while 9 of 21 noted specifically using Informa’s Trialtrove. ¹ Only one of the 26 individuals interviewed noted using publicly available sources such as Clinicaltrials.gov or FDA’s Bioresearch Monitoring Information System (BMIS) database, even though:

• all organizations are required to submit any clinical trials occurring in the US or are looking for FDA approval, and
• the 2017 study start-up study found that 90% of respondents from medium and large-sized biopharmaceutical companies and 88% of respondents from small biopharmaceutical companies note submitting Form 1572s to the FDA (N=298 respondents).²

This suggests that while Clinicaltrials.gov contains up-to-date information on all US clinical trials, and BMIS contains up-to-date information on clinical investigators,² the inability to combine the two datasets makes it challenging for organizations to utilize these otherwise rich sources for site selection.

Respondents from the 2016 chokeholds in patient engagement study noted that while protocol design is a concern with site identification and selection, they rely on patient databases and electronic medical records (EMR) from sites during the site selection process. Other researchers have noted that predictive analytics and site forecasting tools are being used, as well as automating site feasibility to encourage completion, creating a site scoring algorithm, using shared investigator databases such as Transcelerate’s registry or those provided by patient advocacy groups, and capturing site-level performance metrics.^1,4-6

Organizations do not solely rely on data for site selection decisions but also on process improvements, resource management, and other strategies. Data from the 2017 Study Start-Up study showed that 56% of respondents have maintained the same level of investment in site selection and start-up technology solutions during the past three years. Results also showed that 81% of CROs with more than 1,000 employees (mid- and large-sized CROs) have a dedicated site selection functional group, compared with 65% of mid- and large-sized biopharmaceutical companies, 50% of small CROs, and 33% of small biopharmaceutical companies (pOf those companies with a dedicated site selection functional group, midand large-sized CROs and sponsors are looking to increase the number of resources, while the majority of respondents from small pharma are looking to improve technology around site selection. Table 1 lists the categories of solutions that biopharmaceutical companies and CROS are using to support the site selection. The majority of respondents rely on proprietary databases, personal networks, and recommendations from internal team members for site selection. Few respondents utilize trial brokers and/or site business development vendors.

Table 1. Proportion of Survey Respondents That Often or Always Use the Following Tools for Ranking and Selecting Global Investigative Sites (nmf = Not meaningful because sample size < 5 respondents). Sample sizes by company type and size are: 212 respondents from biopharmaceutical companies with ≤1,000 employees; 78 from biopharmaceutical companies with 1,000 employees.

Current Stakeholder Views

In the 2016 site-selection study, ten of 21 respondents noted that new practices companies have implemented for site selection include investments in data and data analytics tools. However, of the 10 organizations implementing new practices involving data-drive solutions, only six organizations have seen a positive impact on cycle times and overall costs. Three of the four were unsure of impact, and one noted no impact yet.¹

The majority of respondents from the 2016 site-selection study noted that challenges around data-driven strategies for site selection include increasing competition for sites that perform well, as well as not having context around metrics calculated from the data. One respondent noted that although new practices involve data-drive decisions, ultimately they “rely heavily on the judgement of the affiliate team.” Another respondent noted that while they too have new practices around software and datadrive decisions, that “no, software doesn’t solve problems [around site selection and activations].”

Respondents from the 2016 chokeholds in patient engagement study noted that while protocol design is a concern with site identification and selection, they rely on patient databases and electronic medical records (EMR) from sites during the site selection process. As one respondent from a site said, “we no longer have to roam the halls looking for nurses and patient charts. Now can go directly into an EMR.” However, respondents from both biopharmaceutical companies and sites noted that EMRs are not always up-to-date.

Respondents also indicated that although data solutions are available for site selection, not enough time is given for site selection. Five out of 18 respondents mentioned the importance of pre-planning, which starts three to six months prior to protocol development. During this time, intelligence on both sites and patients are gathered: intelligence on available sites and trial competition, on site performance, and on physician prescribing habits to determine the location of patients. Respondents from sponsor organizations warned that if that time is not provided, the proper intelligence gathering, even with all available tools, may not occur. One respondent for a CRO noted, “Often sponsors are in such a hurry to start trials they shortcut [the site selection step], so they take on sites that do not have a full understanding of the protocol. If sponsors do the detailed homework up front, they would have determined that. So they need to spend enough time during site identification/selection to do a good job with it. That involves not only emails/surveys but involves getting on the phone, talking with people to ensure they understand trial plans etc.”

Two respondents from CROs also mentioned that although big data are available for site selection, and sites may have an adequate number of patients fulfilling eligibility criteria, there are other issues affecting site performance. First, there is a lack of patient demand for clinical trials. One respondent from a CRO stated, “Expectations may be too high because [sponsors do] not plan accordingly. [The] majority of the world does the participate in a clinical trial.” Another respondent from a site stated, “Hard to enroll for certain indications, like C. Difficile. [The site] can enroll a weight study no problem, but others are more challenging.”

Second, there is limited incentive for physicians and nurses to tell patients about clinical trials. Another respondent stated, “Countries limit what we can compensate sites, therefore there is a lack of aggressiveness on physicians telling patients about clinical trials.” And last, as sites become differentiated as high-performing or not, competition at highperforming sites increases. If too many trials are at any given site, the study coordinator at the site may not be aware of the clinical trial.

Over half of all respondents in the 2017 study start-up study noted that even though companies are making data-driven decisions, the tools they are using are okay, and changes in cycle times have not been significant. The majority of respondents (88% of respondents from mid- to largesized CROs, 54% from small CROs, and 59% from biopharmaceutical companies) noted that current tools and technologies could be improved somewhat.

52% of all respondents reported that these tools have provided a small time savings; however, results from a model assessing site selection cycles times suggest differently. After adjusting for company type, size, and whether or not the company centralizes site selection, there were no statistically significant differences between companies that utilized proprietary databases or commercial databases always or often as compared to companies that do not. Cycle times were on average 5.2 weeks for repeat sites and 7.8 weeks for new sites.²

Respondents overall agreed that technology and lack of data on investigative site performance were not the main causes for an ineffective site selection process. 20% of all respondents think a lack of performance metrics on the site significantly contributes to an ineffective site selection process. However, over 69% of respondents thought that the delayed responsiveness of sites and investigators significantly contributes to an ineffective site selection process. Moreover, 68% of all respondents believe that better evidence of a site’s true enrollment potential before selection, regardless of whether this data is obtained through commercial databases or from the site directly, would enhance the site selection process a great deal.

The Path Forward: Solutions for Stronger Site Selection

Data-driven solutions around site selection is only a part of the solution for stronger site selection. The majority of interviewees from the 2016 chokeholds in patient engagement study noted that complex and infeasible protocol design, and/or conducting site selection prior to having a finalized protocol is a major hurdle in site selection and patient engagement. One respondent from a sponsor company summarized the problem, “if a trial is too restrictive or too much burden, will decrease recruitment and retention”.

Using claims databases and electronic health records (EHRs / EMRs) for clinical research may create more realistic protocols with less stringent eligibility criteria.⁷ In response to assessing the value of EHRs in cardiovascular clinical research found that using EHRs for patient recruitment and endpoint ascertainment is an emerging practice and an accepted practice for hypothesis generation and feasibility assesments.^7,8 One interviewee from the 2016 chokeholds in patient engagement study stated that they “would tap into EMR – invest in data…enroll enough sites so can start ASAP” in order to enhance site selection and patient engagement. A respondent from an investigative site stated, “We are becoming better – taking info and matching patients to the right trials.” Currently, networks combining global EHR data with clinical documentation are developing to provide biopharmaceutical companies and CROs real-time information on sites.

Another solution is to embed clinical research with medical care.⁹ Sacristán suggests increasing use of real-world and observational research into clinical research, given the role of observational data as a “first line of evidence”.⁸ Sacristán also notes that investigator-initiated studies are derived from observations in the clinic. In addition to data collected from EHRs, real-world evidence, observational studies and data for site and country selection can be gathered from social media channels, Google Health and BigQuery, Apple ResearchKit, Android’s ResearchDroid and patient advocacy groups.

Lastly, contract research organizations and ethical review companies are becoming leaders in patient and site intelligence. Results from the 2017 study start-up study indicated that study teams that often or always relied on their CRO partner for site recommendations for new sites completed site selection 1.6 weeks faster than those teams that rarely or never relied on their CRO partner for site recommendations. CROs have been collaborating or merging with intelligence companies. Since 2016, there have been ten notable transactions and alliances formed between CROs and intelligence companies. These transactions have positioned CROs as important future gatekeepers for investigative sites and patients (Table 2).

Table 2. Notable Recent Transactions / Alliances between Contract Research Organizations and Intelligence Companies Since 2016

References

1. Lamberti M.J., Chakravarthy, R.; Getz, K. Assessing Practices and Inefficiences with Site Selection, Study Start-Up and Site Activation. In. Applied Clinical Trials: UBM Plc; 2016.
2. Lamberti M.J., Harber, B.; Morgan, C.; Getz, K. Assessing Study Start-up Practices, Performance, and Perceptions Among Sponsors and Contract Research Organizations. Therapeutic Innovation & Regulatory Science. 2018:7.
3. Temkar P. Accelerating Study Start-Up: The Key to Avoiding Trial Delays. Clinical Researcher. 2017.
4. Sears CC, E. Using Public and Private Data for Clinical Operations. In. Applied Clinical Trials. Vol 252016.
5. Sullivan L. Defining “Quality that matters” in Clinical Trial Study Start Up Activities”. Monitor. 2011;December:4.
6. Cowie MR, Blomster JI, Curtis LH, et al. Electronic health records to facilitate clinical research. Clin Res Cardiol. 2017;106(1):1-9.
7. Sacristán JA. Clinical research and medical care: towards effective and complete integration. BMC medical research methodology. 2015;15:4.
8. Sacristán JA. Evidence based medicine and patient centered medicine: some thoughts on thier integration. Revista clinica espanola. 2013;213(9):460-464.

The author wishes to thank Ken Getz for his invaluable input and suggestions.