Validation at the core
We have built Criterion AI purposely to comply with all relevant regulatory requirements and validation needs. That means that all of the features and workloads accessible in Criterion AI have been designed to make the process of staying compliant with regulators extremely easy.
We have even gone the extra mile and built even more advanced tools for evaluating model performance than what’s required by the authorities. One of these tools is called Facets Dive and is developed by Google’s People + AI Research Initiative. It is one of the absolute best tools out there to establish a visual overview of a dataset and evaluate the performance of a model on a given dataset. Take a look at a sample Facets Dive visualization below.
In our product demonstrations, we give in-depth introductions to our evaluation tools and how to use them to make validation of AI models as easy as pie. Book a demo right away. 👍
Tracking regulatory requirements
In order to make a platform like Criterion AI fully compliant, there are many regulatory requirements to track. First and foremost, we are compliant with the requirements laid out for IT systems in general. Specifically, we comply with 21 CFR Part 11 and other related documents laid other by other regulating bodies.
Moreover, there are a plethora of documents that govern the use of models based on multivariate statistics and related disciplines (such as deep learning). In the table below, you can see a list of the regulatory documents we track for validating the models we train in Criterion AI. We have implemented all of the requirements and recommendations laid out in the documents and, through our participation in forums such as the ISPE and the PDA, we stay up-to-date with new developments in the regulatory space that impact the use of AI in pharmaceutical settings.
|International Guidelines||ICH Quality Implementation Working Group on Q8, Q9, and Q10 Questions & Answers (R4)|
|ICH Quality Implementation Working Group Points to Consider (R2)|
|ICH Q11 Development and Manufacture of Drug Substances (Chemical and Biotechnological/Biological Entities)|
|ICH Q12 Product Lifecycle Management Step 2 document|
|National/Regional Guidelines||EMA Reflection Paper: Chemical, pharmaceutical and biological information to be included in dossiers when Process Analytical Technology (PAT) is employed|
|EMA Guideline on the use of near infrared spectroscopy by the pharmaceutical industry and the data requirements for new submissions and variations|
|FDA Guidance for Industry PAT—A Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Assurance|
|FDA draft Development and Submission of Near Infrared Analytical Procedures—Guidance for industry|
|Pharmacopeial||Ph.Eur. 5.21 Chemometric Methods Applied to Analytical Data|
|USP General Chapter <1039> Chemometrics|
|USP General Chapter <1005> Acoustic Emission|
|ASTM International Standards||E2891-13 Standard Guide for Multivariate Data Analysis in Pharmaceutical Development and Manufacturing Applications|
|E1655-05 Standard Practices for Infrared Multivariate Quantitative Analysis|
|E2617-10 Standard Practice for Validation of Empirically Derived Multivariate Calibrations|
|E2056-04 Standard Practice for Qualifying Spectrometers and Spectrophotometers for Use in Multivariate Analyses, Calibrated Using Surrogate Mixtures|
|E1790-04 e1 Standard Practice for Near Infrared Qualitative Analysis|
We thank Ana Patricia Ferreira, Jose C. Menezes, and Mike Tobyn suggesting this list in Multivariate Analysis in the Pharmaceutical Industry, published by Academic Press in 2018.
If there are one or more documents that you need to comply with but it isn’t listed in the table above, we would love to have a discussion with you about that. Please do not hesitate to reach out us to initiate the dialog. 💬