The FDA Guidance on using EHR in clinical investigations, released in July, provides recommendations on the following: 

• “Deciding whether and how to use EHRs as a source of data in clinical investigations 
• Using EHR systems that are interoperable with electronic data capture (EDC) systems in clinical investigations 
• Ensuring the quality and integrity of EHR data collected and used as electronic source data in clinical investigations 
• Ensuring that the use of EHR data collected and used as electronic source data in clinical investigations meets FDA’s inspection, record-keeping, and record retention requirements”

Consider this required reading for everyone involved in clinical trials.  All trials should move to electronic source, based on the new ICHE6(R2) guidance on source data, which must meet the requirements for ALCOA (Attributable, Legible, Contemporaneous, Original, and Accurate). While the FDA will not require that EHRs meet 21 CFR Part 11, it does want to ensure that EHR data cannot be changed and the data are attributable to an individual with an audit trail.

There is additional information about selecting an Electronic Health Record.  The FDA encourages that EHR systems be certified through the Office of the National Coordinator for Health Information Technology at the Department of Health and Human Services (ONC) Health IT Certification process and that they use the ONC’s interoperability standards.  This is not a requirement and systems that do not have the certification should meet guidelines assuring confidentiality and integrity of data, have policies and processes for the use of the EHR system, limit access to authorized users, identify the data author and have audit trails available to track changes to data, and have records available for inspection. 

In addition, the guidance also recommends periodically checking a subset of the extracted data for accuracy, consistency, and completeness with the EHR source data.

eSource Principles
The FDA recommends documenting the primary source for data imported directly from the EHR system  be listed as the EHR system. In addition, data management documentation should include the manufacturer, model number, and version number of the EHR system and whether ONC certified the EHR system.

Sponsors should ensure monitors have adequate  access to all relevant subject information; this point was specifically mentioned in the Guidance.

Inspections:
EHR source data and the documentation associated with the EHR data and processes should be available to Inspectors.  They can be viewable within the EHR or as certified copies. 

RBM Impact:
This Guidance reaffirms the FDA’s expectation that data will be available for remote review as part of Risk Based Monitoring and quality oversight.  The FDA has provided an option that can eliminate transcription and provide data directly into the EDC.  There are many complexities to this process, particularly for multi-center trials with multiple EHRs, but we are moving in the right direction.
The link to the guidance is: https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM501068.pdf

Yesterday, Medrio released a press release announcing the addition of MANA RBM to its Partnership Program.   https://www.prweb.com/releases/2018/07/prweb15619842.htm
 
This exciting news means an efficient, cost effective, complete Risk Based Monitoring (RBM) solution and proven methods to meet the new ICHE6(R2) Good Clinical Practice Guidance are now available to companies of all sizes—within existing study budgets and timelines.
 
Whether you conduct the oversight yourself, or have MANA RBM conduct your data management and monitoring, MANA RBM’s complete RBM solution works to meet your needs..  
 
“Our RBM Solution focuses on the most important aspects of the trial and provides the most comprehensive review of those areas—far exceeding the review performed by Source Data Verification”, reported Penelope Manasco, M.D., CEO of MANA RBM.  “And we don’t stop with the systems—we provide our Sponsors with SOPs and Work Instructions to facilitate RBM implementation.”
 
“As Clinical Operations professionals, we believe trial oversight should include the same systematic, data-driven, evidence-based methods used in discovering new products. Clinical trials are the most important studies conducted in the lifecycle of a new treatment.  We owe patients, their caregivers, and physicians our best efforts to pursue scientific rigor.   Our complete RBM solution provides the tools and processes to achieve this goal”, said Dr. Manasco commenting on the Medrio press release.   

In March 2018, the FDA adopted the ICHE6(R2) Good Clinical Practice (GCP) Guidance.  A quality management system for clinical trials and the Sponsor’s requirement to properly oversee trial conduct are now a part of GCP.
 
Most clinical trial management approaches focus on whether the pages have been SDV’ed, number of open queries, and queries opened for a certain period of time.  These metrics, along with other metrics such as whether onsite visits occurred as contracted have been used to manage trials for many years.  They are easy to measure and easy to check performance against “goals”.  
 
But do those metrics really indicate quality?
 
If we closely review the guidance and think about the goals of a quality management system, the important metrics become more obvious as follows:  
 

Quality Goal	Evaluation Method Examples
Study conduct follows the protocol.	Identify and evaluate deviations to protocol conduct including patterns indicating issues to be addressed.
All data meet ALCOA requirements (Attributable, Legible, Contemporaneous, Original, Accurate)	Review of audit trail to confirm contemporaneous entry.
Human Subject Protection is maintained	Informed consents (IC) will be reviewed within 5 business days to assure all components of the IC process are followed correctly (in addition to confirming correct forms and dates)
Data quality supports confidence in primary endpoint(s)	Person assessing primary endpoint is identified in the data collector, is properly trained, and has been delegated the responsibility to perform the assessment.
Investigational Product (IP) chain of custody and administration is complete and correct	Data documenting site IP receipt, preparation, storage, dosing, return, and destruction are collected so errors can be quickly identified.

 
Moving to a quality management system or approach requires both processes and systems or technology to ensure that all aspects of trial conduct are evaluated (i.e., study processes, training, analysis data).   Simply checking that analysis data is present and within the expected range is insufficient quality oversight.
 
Traditional metrics used today determine whether certain activities occurred.  A quality management system focuses on determining what critical errors occurred and why.
For example, in a recent study, critical lab samples for a safety endpoint were received thawed from several sites.  A traditional project management approach would be to send an email to the sites telling the sites their samples were received thawed or providing instructions that may not be based on a root cause analysis of the reason for the thawed samples.  A quality approach would evaluatewhythe samples were received thawed (I.e., a root cause analysis).  This included evaluating how the instructions were provided, when and from where the thawed samples were shipped, the size of the shipping container, the amount of dry ice, the number of samples shipped, etc.  Remediation included revising instructions based on the root cause analysis to assure future samples were received frozen, and follow up to assure that the revised instructions corrected the issue.  This example shows how much more effective a quality management system is compared to SDV and traditional management systems used today.
 
In the mad dash to embrace quality metrics, ask first what SHOULD be measured not what CAN be measured.    

I recently read an interview with Oren Miron, a biomedical informatics research associate at Harvard and winner of the 2017 Next Einstein Competition researcher.  He repurposed a standard technology used to test an infant’s hearing for autism.  
 
Research from the 1970’s by Professor Hildesheimer showed that children with autism had a consistently delayed response to Auditory Brainstem Response (ABR) tests.  Autism was thought to be a disease of the frontal cortex, so this finding was “lost” in many ways until Miron found the published research.  
 
MRI had become the method of diagnosis.  A fine tool but prohibitively expensive to use as a screening tool.
 
Miron looked for a low-cost way to screen for autism and found it in a test now used routinely to test for hearing loss.   He screened thousands of infants for hearing loss and compared the results with the data on subsequent autism diagnosis and found a consistent response.  He is now searching for funding to conduct a prospective trial to validate his findings.
 
Finding children with autism early allows intervention to begin years earlier than 4 years old, the average age a child is diagnosed with autism.  This can significantly enhance a child’s life by what can be accomplished through early intervention.
 
This teaches us RBM lessons as follows:
 

1. The tools to conduct RBM are in place and the data are available, but it is HOW you look at the data that affects what you find.  The tools to collect and analyze the critical data (e.g., EDC/eSource, ePRO, CTMS, Protocol-Specific reporting) are available—just like the ABR responses unique to autism were there—it took an inquisitive, scientific mind to “see” the pattern and recognize its significance.

 

2. Scientific discipline is needed to evaluate the utility of new tools.  Miron used pre-existing data to conduct the first hypothesis testing, which he is following with prospective trials.  

We, in the clinical research arena, need to use the same scientific discipline to evaluate the optimal way to conduct oversight of trials.  We can’t keep using the methods, such as SDV, as our only means to evaluate trial quality. SDV has been proven to be ineffective; a position reiterated in Guidance by the Regulatory authorities.  We owe it to scientists everywhere to use the same scientific rigor in conducting clinical trials to prove the efficacy and safety of their important scientific discoveries and to help achieve the ultimate goal of improving patients’ lives.
 
We need to implement analytic approaches to determine whether the experiment (protocol) was performed correctly.  Did the right person, trained to conduct the experiment, conduct the experiment?  Were controls included for the primary endpoints?  In scientific experiments, we include positive and negative controls to be sure the experiment was performed correctly.  In clinical trials, placebo or active controls are often used for this purpose.  But research scientists review all aspects of the experiment (e.g., reagents, procedures, and analysis approaches) before expressing confidence in the results.  We, in clinical research, need to adopt and use the same approach.
 
MANA RBM is committed to advancing the scientific discipline of clinical research. We have, and will continue to validate and publish the findings and analysis of our proprietary methods for trial oversight. Links to the Journals and copies are/will be available on our MANARBM.com website.
 
We have previously published data on approaches superior to SDV and eCRF review, remote Informed Consent Review, Site Responses to Paperless trials, using electronic Investigator Site Files for Paperless trials, and monitor competencies for RBM.  Please contact us if you would like a copy of any of these papers or a link to it.
 
Stay tuned; we have many interesting papers that will be published in the near future, including a prospective comparison of SDV and a remote trial management approach.
 
Please join the mailing list on our website so we can notify you when these important papers are published and released.  Join us, also, as MANA RBM pioneers the field of Clinical Operations as a scientific discipline in lockstep with the scientists designing and developing our new treatments.
 

The FDA recently released Guidance on using image data for primary endpoints. (Clinical Trial Imaging Endpoint Process Standards. April 2018. https://www.fda.gov/downloads/drugs/guidances/ucm268555.pdf). This Guidance highlighted the important process of collecting and analyzing image data.
 
In the past, oversight of image analysis was limited to the interpretation data by the reviewer.  In this Guidance, the FDA clearly illustrates how many aspects of the process can affect the ultimate analysis.
 
The Guidance discusses many aspects of capturing and analyzing the images and how they can affect the final interpretation of the study results.  If images are not captured consistently, then the interpretation can be compromised.  If images are not collected at the correct timepoints, endpoints such as Progression Free Survival cannot be correctly identified.  If images are not read by a trained reader who is unaware of the treatment and the stage of treatment, then bias can be introduced, which can ultimately affect trial integrity.  Finally, if the correct aspects of the image are not identified consistently and correctly for analysis of endpoints, such as the RECIST score, then the results cannot be interpreted correctly.  
 
Guidance documents like this contain process improvements that should be incorporated into the Risk Assessment process.  For instance, after reading this guidance, processes may need to be modified to add a training aspect for the person interpreting the efficacy endpoint.  Process oversight should confirm that the correct person completed the analysis and had the proper training and authority to do so. Minimizing the possibility of bias is another area that can be addressed as part of Risk Assessment.  How will you minimize the risk, and how will you know if bias occurs?
 
Risk Based Monitoring was designed to help Sponsors, CROs, and Sites focus on the important aspects of trial conduct.  This Guidance document reaffirms the FDA’s position on RBM to focus on specific processes, usually identified in the Risk Assessment, rather than merely conduct SDV. Assuring that primary endpoints are correctly collected by evaluating the processes used to minimize bias and assure the highest quality data takes time and effort, but ultimately is a better use of scarce oversight/monitoring resources.  
 
For more information on MANA RBM’s Risk Assessment Service and how to integrate quality oversight and Risk Based Monitoring into your next trial, contact MANA RBM.  1-919-556-9456 or Dr. Manasco (pmanasco@manarbm.com).