Qualification and Utilization of CRO Partners

Forming a Sensible CRO-Contracting Laboratory Partnership

As previously published in Part 1 of this series: “Collection, interpretation, and utilization of extractables or leachables (E&L) data are not trivial processes and the scientific and practical requirements for performing these activities may extend beyond the technical and/or resource capacities of many organizations that are required to possess such data. To bridge these knowledge and resource gaps, organizations turn to contract research organizations (CROs) to provide the needed expertise, the needed capacity, or both.” [1]

STEP 2: Identifying and Screening Potential CRO Partners (continued)

Screening for Intangibles

As important as technical issues might be, there are important nontechnical aspects that should be considered by a company when it screens for a CRO partner. The essential issue is compatibility. How compatible will the contracting organization and the CRO be? Aspects to consider include the organization’s quality systems, its financial models, its relative location, and its size and culture. It is relatively easy for incompatibilities to exist between a contracting organization and a CRO, and it is hard to do good science efficiently when incompatibilities exist. It may be that quality systems (and expectations) and financial models differ greatly from one organization to the next. Proximity of the contracting organization and the CRO may or may not be important, though generally, long distances are a barrier to collaboration and in some ways limit spontaneity. While the aspect of size versus capacity has already been considered, there are other aspects of size that may be relevant. While some contracting organizations may value a closer relationship with a smaller CRO, other organizations may be more comfortable working with a larger CRO that may have a greater depth or breadth.

Lastly, there is the aspect of culture. While divergent cultures in the contracting organization and the CRO can be managed and endured, such a relationship is not optimal and has potential for inefficiency. While it is beyond the ability of this author to provide detailed advice on how to assess and score an organization’s relationship with a CRO, it typically holds true that if something simply does not feel right, then it probably is not right.

Final Considerations for the Screening Stage

At the screening stage, the information that is collected is general, qualitative and high level as the purpose of screening is not to pick the final CRO but rather to narrow the field of potential CROs to a more manageable number of viable candidates. For example, consider the aspect of essential instrumentation. The question that is being answered at the screening stage is the general question into what and how many instruments and instrumental methods are available at the CRO. For instance, the purpose is to establish that the CRO has LC/MS instrumentation and obtain all of the essential capabilities of the CRO’s instrumentation, including relative capacity. At this stage, it is less useful to establish the specific instrument systems and the specific operating procedures. The process of gathering such detailed information is an investment for the contracting laboratory’s time which may prove to be time wasted if the CRO candidate is rejected.

The screening process must be focused on the type and level of support that the contracting organization is seeking. If a contracting organization requires only trace element or metals testing, it would be relatively meaningless that the CRO has a whole laboratory full of LC/MSn instruments operated by highly-qualified technicians. On the other hand, a contracting organization that is looking for a CRO to “do it all” would find this type of capability to be relevant and necessary.

STEP 3: Qualifying CRO Partners

As the pharmaceutical industry is highly regulated, vendors to this industry are audited against rigorous quality standards to exercise due diligence. As vendors of technical services to the pharmaceutical industry, E&L CROs are subject to the same level of scrutiny. CRO partners who have survived the screening process must be more rigorously audited than they were examined during screening. Additionally, it is prudent that the contracting organization re-examine the CRO more closely in terms of its financial systems and its technical capabilities. A series of actions that are performed as part of the qualification process are listed in Table 1.

Table 1. Qualifying CRO Partners (Step 3)

Paperwork Issues

The process of qualifying a CRO requires the examination and exchange of information that could be classified as confidential by either the CRO or the contracting organization. Therefore, documents that facilitate and control the exchange of such information are a prerequisite to the qualifying process. In some cases, the qualification process, particularly its technical aspects, may require that a considerable expense be incurred by one of the parties. The magnitude of such expenses and the responsibilities and obligations for those expenses must be established and documented. If laboratory studies are required as part of the technical audit, then the study design and the data reporting process must be established. Under certain conditions, additional documentation must be in place before the qualification can be executed.

Supplier Audit

As noted previously, pharmaceutical companies have established internal procedures for auditing and qualifying their vendors. Typically, the primary focus of a supplier audit is the CRO’s quality systems, although other operational aspects of the CRO may also be examined. As an organization’s supplier audit process may have been developed to deal with suppliers of materials instead of suppliers of services, it is possible that the process may need to be modified somewhat to deal with an E&L CRO. It is reasonable to expect that the spirit and intent of a contract organization’s supplier audit process will be exercised in its audit of an E&L CRO.

Financial Audit

Understandably, an E&L CRO will expect to be paid for its services. The contracting organization will desire to procure those services at what it considers to be a fair market price. The financial audit is an opportunity for both organizations to iron out the details of their financial relationship. While the quality audit seeks to find and address weakness or issues within the CRO’s quality system, the financial audit seeks to establish the rules for the financial engagement between the two organizations and serves to establish and document the financial expectations of both organizations.

An important aspect of the financial audit is to establish the cost basis for work performed for the contracting organization by the CRO. It is noted that there are a number of standard practices in terms of this aspect of the financial relationship. One standard practice is to quote individual jobs or projects. For example, the CRO might quote a fixed price for performing a certain extractables assessment. One presumes that the fixed price quotation would be based on a well-defined study design that enumerates the relevant aspects of the study including the numbers of test articles, numbers of replicates, the extraction process, its related analytical process, the testing completion criteria, and finally, the mechanism for reporting the study’s results. A fixed price quotation would address aspects of timing, such as project milestones.

The significant positive point of the fixed price quotation is that the exact financial commitment is clear and predictable. However, extractables can be notoriously unpredictable and it is questionable whether the cost of completing a study can reliably be predicted upfront. Organizations may be inclined to view contract modifications unfavorably. Managing the process of contract administration may consume time and resources that are better spent on the actual execution of the study.

Another financial approach is an FTE model, by which the contracting laboratory essentially contracts for a fixed portion of the CRO’s capacity. For example, the contracting laboratory may purchase a certain level of support over a defined time period (for example, 5 FTEs per month for a full calendar year). Such a model provides the contracting laboratory with a certain amount of flexibility in terms of the deployment of such purchased capacity to meet the specific needs of the contracting laboratory at any given time. For example, the activities in one month may be such that the contracting laboratory primarily needs chromatographic support, while the next month it might need technical writers to complete study reports. Presumably, such flexibility would allow the contracting laboratories to shift priorities between activities and between projects without the burden of contract adjustments. From the perspective of the contracting laboratory, the challenge of managing this type of FTE approach is the management of the work stream so that all the resources are utilized in slack times and no more than the available resources are used during peak times. From the perspective of the CRO, the challenge of managing this type of FTE approach is having the flexibility and capacity to deal with the contracting laboratory’s changing and shifting priorities, as well as dealing with the financial issue of not having all activities being billed at the same rate.

A third financial approach is the hiring of staff through a CRO and then placing the staff within the working laboratories of the contracting organization. While the staff is managed by the CRO and the work is performed consistent with the CRO’s standards and procedures, the work is performed with instrumentation and facilities provided by the contracting laboratory. Such an approach may be attractive to a contracting laboratory that has the facilities to support a larger staff in addition to a project load that requires a longer-term CRO relationship.

One could envision that a hybrid approach could provide both the CRO and the contracting laboratory with the financial framework that facilitates efficiency and effectiveness. For example, the two parties might develop a FTE-based relationship at a sustainable level and could choose to manage “excursions” (e.g., emergencies or short term bottlenecks) with fixed-priced, activity-based projects.

In addition to establishing the financial model that the CRO and contracting laboratory will work under, the financial audit is the proper time to deal with any inconsistencies between the partners’ financial systems (e.g., establishing payment terms).

Technical Audit

It is the scientific rigor and quality of an E&L study that dictates whether the efforts required to design, implement and report the study are well spent and produce useful, credible outcomes. Although a CRO’s technical capabilities might be established as being adequate during the candidate screening and selection stage, a more extensive and detail-driven technical audit should be conducted in order to qualify a CRO. As the primary tool used in an E&L study is analytical chemistry, this technical audit considers the analytical capabilities and capacity of the CRO, and specifically includes three activities: a more detailed review of the CRO’s technical resources, a very detailed review of the methods and procedures used by the CRO, and a thorough competency evaluation (Table 2).

Table 2. Technical Qualification of Partner CROs (Part D of Step 3)

The detailed and in-depth review of a CRO’s technical inventory takes into account the physical inventory noted in the CRO screening stage and evaluates each technical asset in terms of its capability to perform effectively in an E&L assessment. Individual instruments and instrument systems for generating extracts and testing extracts are evaluated in terms of matching their performance specifications with the requirements for E&L testing. For example, it is no longer sufficient at this stage to note that the CRO has LC/ MS capability; rather, it is necessary to take note of which specific LC/MS systems the CRO has as well as the system’s inherent capabilities. Consider TOC as a case in point. If a CRO has TOC capability because it frequently tests purified water, this TOC capability may not be adequate to test pH 9 extracts containing relatively large concentrations of multiple organic extractables. In terms of capabilities, both the manufacturer’s specifications and user input into system performance (for example, historical system suitability data) are examples of relevant considerations. An audit performed in this manner can be as extensive as to the due diligence that the contracting organization would have to exercise upon purchasing the equipment for its own laboratories.

In a similar vein, the actual technical qualifications of the CRO’s staff need to be examined more closely than the “number and degree” count that may have occurred while screening potential candidate CROs. At the qualification stage, the CRO’s staff must be assessed in terms of two important attributes: the breadth and depth of their experience in E&L assessment and their specific experiences with the contracting organization’s situation (dosage form, types of drug products, types of plastics used, etc.). An appropriately detailed staff qualification would require close and meaningful contact between the CRO’s key staff members (especially those doing the actual work) and the contracting organization’s qualification team. This close and meaningful contact could take a number of forms; for example, the contracting organization’s team could interview individual CRO staff members or the CRO staff could host a “symposium,” wherein the CRO’s staff members present case studies, theory and practice scenarios, methodological or technique details (such as the development and validation of a leachables method) or high level strategies to the contracting organization’s assessors. A personnel audit performed in this manner can rival the due diligence that the contracting organization might exercise if it was hiring the CRO’s staff members for their own organization.

In addition to reviewing the analytical tools and personnel, the CRO qualification process should include an assessment of the processes that are followed for performing the work, which may be documented in specifications, procedures and manuals. At the screening level, the activities of procedure and process reviews were focused on the more general aspects of business and laboratory procedures as part of the quality audit. For example, the initial screening investigations are focused on the “do you have?” level, manifested in questions such as, “do you have a procedure for reporting and addressing results that are out of specifications?” or, “do your instruments have maintenance logs?” At the qualification stage, the focus is on the contents of the documents and whether those contents are sufficiently rigorous to ensure the quality of the work that the documents are related to. Such an assessment could require a line-by-line examination of the CRO’s most critical Standard Operating Procedures, (SOPs), where the choice of SOPs to examine would be made by the contracting organization, not the CRO. For example, in one’s review of a procedure specification for a chromatographic method used to screen extracts for extractables, it might be necessary to look for required system suitability testing with the intent of establishing whether the system suitability test is meaningful (e.g., are the right method parameters considered and are the acceptance criteria sufficiently rigorous such that poor method implementation would actually cause a system suitability failure). If internal standard response factors are used to estimate the levels of extractables in extracts, then it could be a proper question for the contracting organization to ask the CRO, “where is the response factor database that was used to justify the choice of your internal standard?”. If the test method requires extensive sample processing prior to instrumental analysis, then the SOP for performing the sample processing should be examined from the perspective of scientific credibility, ease of implementation, and the presence of appropriate means of revealing and resolving cases of unacceptable performance. In the final analysis, the most rigorous assessment of a laboratory SOP may be for a member of the contracting organization’s qualification team to take the SOP and try to implement it as specified.

It is the nature of studies into extractables and leachables, especially at the extractables discovery and identification stages, that the path from initiation to completion is not always straight, and that the means of getting from point “A” to point “B” cannot always be captured in procedures and specifications. An experienced CRO will have encountered similar situations in its past efforts and will have implemented means of recognizing when such a situation is likely to happen, or whether it has already occurred. Based on this experience, the CRO will have established a means of documenting, addressing and resolving those situations. Additionally, it is the case that best demonstrated practices in E&L are not prescriptive and restrictive, but, rather, allow individual organizations to operate with the flexibility inherent in “good and prudent” science. In this situation, a CRO may have developed its own strategy and tactics around “good science” which may not be revealed in even the most rigorous SOP review. Therefore, it may be useful to include a “how do you handle this situation?” session as part of the CRO qualification. One would anticipate such a session focusing on issues that the contracting organization itself has experienced or heard about, and that the CRO’s responses would be judged in the context of how the well the CRO’s response agrees with the contracting laboratory’s own solution to the issue. Another important factor to consider is whether the CRO’s response meets the dual expectations of practicing good science and meeting regulatory compliance.

If one likens the process of qualifying a CRO to that of obtaining a driver’s license, one recognizes that the qualification process that has been described to this point is much like the written portion of a driver’s license test. In theory, the written test establishes that the prospective driver possesses the essential knowledge to handle a vehicle in a safe and proper manner. However, there still exists the considerable concern as to whether the prospective driver can translate and reconcile this knowledge with the practical challenges of actually operating a vehicle in an acceptable manner. Expanding this analogy to CRO qualification, the CRO qualification discussed up to this point has focused on evaluating the CRO’s ability to perform without witnessing the CRO’s actual performance. From the perspective of the contracting laboratory, it is useful that the qualification stage establishes the CRO’s actual ability to successfully perform the critical aspects of an E&L study. While one way to accomplish this objective would be to give the CRO a “real” study, this approach is not without an associated risk, and it could be considered akin to giving the prospective driver a license to operate a vehicle without taking a road test and then hoping for the best. The short comings ofthis approach could be mitigated if one considers the two essential aspects of an E&L study:

1. Can an E&L study be properly designed, and can the study design be effectively communicated in a document (such as a protocol)?
2. Can laboratory techniques be implemented so that extractables can be discovered, identified and quantified?

These two points suggest a means for assessing the CRO in the “heat of battle” without actually engaging the CRO in a test. Specifically, the aspect of study design and protocol generation can be addressed by providing the CRO with a case situation and having it generate a study design and protocol. Presumably this case situation would be one that the contracting laboratory has itself experienced previously. Therefore, the performance expectation in this case would be a safety design and protocol judged by the contracting organization as being as good as, if not better than, their own design and protocol in response to this situation. More objectively, the contracting organization can judge the actions of the CRO during the study design and protocol generation activity in the context of its ability to relate its experience to the contracting laboratory’s circumstances by considering key questions, such as:

1. How much time did the CRO spend understanding the product, the process, the circumstances and the ultimate objectives of the study?
2. Is the CRO offering their standard, “one size fits all” package or does the study design and protocol reflect a customized approach tailored to fit the study’s circumstances and needs?

Additional questions that the contracting organization should consider in evaluating the CRO’s study design and study protocol are listed in Table 3. For example, the CRO’s proposed extraction conditions and solvent(s) should effectively reflect the contracting organization’s study scenario and should contain appropriate and detailed justifications of the CRO’s choices. Important aspects to consider in terms of extract analysis include the CRO’s use and justification of qualification or reporting thresholds, the CRO’s strategy for identifying extractables and the CRO’s strategy or tactics designed to address the possibility that their screening method may leave important extractables undiscovered and unidentified.

Table 3. Technical Qualification of Partner CROs: Important Questions and Issues (Part D of Step 3)

The Competency Evaluation

By having a prospective CRO generate a mock protocol, the contracting organization has essentially established that the CRO can think and communicate in an appropriate and effective manner. While this is a significant point, it leaves a major question unanswered: can the CRO produce output (the E&L study results) that meets the contracting organization’s quality requirements? Points to consider in addressing this question include:

1. What is the technical quality of the analytical actions that the CRO has performed while conducting an E&L study?
2. How well has the CRO performed the technical functions required in an E&L study?
3. How good are the chromatograms that the CRO generates?
4. How accurate and credible are the identifications and concentration estimates provided by the CRO?
5. How well has the CRO correlated the disparate information it has created to provide a cohesive, consistent data package?
6. Has the CRO produced a set of data or has the CRO taken that data and converted it into useful and actionable information?

The answers to such questions can only be obtained if the CRO has actually performed the activities required in an E&L study. If these activities are not performed in the context of a real study, then the activities may be performed as part of an artificially-created competency evaluation. There are three possible formats for such an evaluation:

1. The CRO is provided a well-characterized reference material (that is, a material with a previously established extractables profile), and asked to determine this material’s extractables profile.
2. The CRO is provided with an “artificial” extract of known composition to evaluate (for example, a solution prepared in the laboratory containing known compounds at known levels).
3. The CRO is provided with an extract that the contracting laboratory has characterized for extractables using its own methods and laboratory.

Regardless of the format of the competency evaluation, the success of the CRO is judged primarily by the accuracy and completeness of the extractables profile it generates. Secondarily, the CRO is judged for the means it utilized to obtain the information contained in its evaluation report. For example, the means by which the CRO elucidated the identity or estimated the concentrations of extractables in a test sample may be as significant an indication of its technical savvy as its actually obtaining the correct answer.

It is difficult to establish objective acceptance criteria for the competency evaluation upfront, as the criteria depend somewhat on the nature and the complexity of the evaluation. If the evaluation consisted of analyzing a test sample containing a reasonable number of commonly encountered extractables at challenging (but not impossible) levels, then it may be appropriate to use the most stringent acceptance criteria which is that the CRO found all targets, correctly identified the targets, and properly estimated their concentrations. On the other hand, if the evaluation involves testing an actual material, it is unreasonable to expect that the CRO will produce the exact same profile that the contracting laboratory would generate. While one should expect reasonable inter-laboratory agreement for the major extractables, there may be significant differences in the results obtained for the more minor extractables. Rather than serving as reasons to eliminate a candidate CRO, such differences can lead to mutually-beneficial discussions between the CRO and the contracting organization.

STEP 4: Implementing a Qualified CRO as a Partner

After the selection and qualification of a CRO partner is complete, both the CRO and the contracting organization look forward to designing and implementing their first collaborative study. It will be gratifying to both parties to have reached this milestone in their relationship and there will be enthusiasm to begin meaningful, valuable, and complex work. However, before the CRO and contracting laboratory begin performing real studies, it is useful to debrief in terms of the whole screening and qualification process. Debriefing serves as an opportunity to review the issues that came up during these processes and to make sure that any actions taken to address potential issues were properly conceived and effectively implemented. If circumstances allow, it would be sensible to use a study of moderate importance and complexity as the first study conducted within the partnership. In doing so, both the CRO and the contracting organization are provided with the opportunity to build a solid foundation of success and likely avoids the trauma (and drama) of having to navigate the challenges inherent in the more complex E&L assessments.

References

1. Insights Gained into the Identification, Qualification and Utilization of CRO Laboratories in Extractables and Leachables Studies. D. Jenke. (Pharmaceutical Outsourcing, Vol. 14 Issue 2).
2. Compatibility of Pharmaceutical Solutions and Contact Materials: Safety Assessment of Extractables and Leachables for Pharmaceutical Products. D. Jenke. (Wiley, New York, 2009).
3. Leachables and Extractables Handbook: Safety Evaluation, Qualification, and Best Practices Applied to Inhalation Drug Products. D.J. Ball, D.L. Norwood, C.L.M. Stults, L.M. Nagao, Eds. (Wiley, New York, 2012).
4. Update on Undertaking Extractables and Leachables Testing. A. Feilden, (Smithers Rapra Technology, London, 2011).
5. Norwood, D.L. Considerations for outsourcing of leachables and extractables testing. Am Pharm Outsourcing, 9(2): 22 – 28 (2008).

Dr. Dennis Jenke is a Baxter Distinguished Scientist in the Technology Resources function at Baxter Healthcare Corporation. His responsibilities include assessing materialproduct compatibility, specifically with respect to leachables/ extractables and product ingredient binding. He has published extensively in the areas of analytical chemistry, environmental science and material-solution compatibility, is a member of professional and standard-setting organizations whose charter is to establish best demonstrated practices in the area of material-solution compatibility, is a frequently invited speaker on this subject, and is the author of “Compatibility of Pharmaceutical Solutions and Contact Materials; Safety Considerations Associated with Extractables and Leachables”.