Published on 02/12/2025

Governance for AI in GxP: Roles, RACI, and Boards

The integration of Artificial Intelligence (AI) and Machine Learning (ML) into Good Practice (GxP) environments presents both opportunities and challenges. As organizations explore the adoption of AI/ML technologies, robust governance frameworks become crucial to ensure compliance with regulatory expectations and to manage related risks effectively. This tutorial serves as a comprehensive step-by-step guide for pharmaceutical professionals to understand the roles, responsibilities, and frameworks necessary for AI governance and risk management.

Understanding the Risks Associated with AI/ML in GxP

AI/ML technologies, while beneficial, come with inherent risks that professionals must understand to ensure compliance with industry standards. Firstly, the concept of intended use risk must be defined. This indicates how an AI/ML model will be utilized in the regulatory landscape and the implications it carries for patient safety and data integrity.

In pharmaceutical settings, AI/ML applications often handle sensitive health data. Thus, the risks can include:

• Data Accuracy: Ensuring that data inputted into the models are accurate and representative of real-world scenarios.
• Model Bias: Possibility of biases present in the data leading to skewed outcomes.
• Explainability: Ensuring that AI models provide understandable and clear decision pathways for end-users.

Incorporating thorough risk assessments during the preparatory phases of AI/ML model validation is essential. This includes ensuring data readiness curation, where the data quality is scrutinized, and data selection is justified based on the intended use of the AI/ML model. The regulatory frameworks specified by FDA, EMA, and MHRA provide guidelines that dictate the importance of mitigating such risks.

Establishing Governance Structures for AI/ML

The establishment of a governance structure is vital for overseeing AI/ML models operating within GxP. This governance framework has multiple components:

1. Defining Roles and Responsibilities

Creating clarity around roles is necessary for accountability in the AI/ML lifecycle. A crucial aspect of effective governance is the RACI matrix (Responsible, Accountable, Consulted, and Informed). The following roles should be detailed:

• Data Steward: Responsible for maintaining data integrity and quality throughout the data lifecycle.
• Data Scientist: Accountable for the development and validation of the AI/ML models. They must ensure compliance with GAMP 5 guidelines and applicable software validation standards.
• Quality Assurance (QA): Consulted during the development and deployment phases of the AI/ML models to ensure compliance with 21 CFR Part 11 and relevant regulations.
• Stakeholders: Informed parties who need to understand the implications and operations of the AI/ML models.

2. Constructing AI Governance Boards

Governance boards should be established to oversee the AI/ML model’s ethical and regulatory compliance. The composition of these boards may include:

• Executive Sponsors: Senior leaders who provide strategic oversight.
• Data Ethics Officers: Experts responsible for upholding ethical considerations.
• Compliance and Legal Advisors: Professionals knowledgeable in regulations, responsible for ensuring that all AI/ML implementations adhere to legal commitments.

Documentation and Audit Trails

Robust documentation is essential to maintain compliance and demonstrate accountability in AI/ML operations. This involves:

• Protocol Development: Creating written protocols for the validation process which include step-by-step methodologies for data validation and model testing.
• Change Control Procedures: Implementing changes in the AI/ML models must be documented, outlining the rationale and validation of changes.
• Audit Trails: AI systems must incorporate comprehensive audit trails that detail access, modifications, and outputs from AI systems, ensuring compliance with regulations such as Annex 11.

Utilizing software that supports seamless documentation processes will facilitate ease of audits and inspections by authorities including EMA and WHO.

Model Verification and Validation

Model verification and validation (V&V) are crucial phases that need to ensure the AI/ML outputs meet the predefined criteria. Steps in the V&V process include:

1. Model Verification

This involves checking whether the model’s development aligns with specifications and intended use. Activities may include:

• Code Reviews: Assessing the code used for model development to identify issues early in the process.
• Component Testing: Validating individual components of the AI system to confirm their proper functionality.

2. Model Validation

Validation entails a rigorous assessment of the model’s performance in real-world scenarios. Key aspects include:

• Bias and Fairness Testing: To ensure that the AI/ML model operates fairly across different populations.
• Explainability (XAI): Ensuring that outputs from the AI models can be interpreted and understood by stakeholders.

3. Drift Monitoring and Re-validation

Models may require ongoing assessment after deployment due to changes in the underlying data patterns or external factors. Monitoring for data drift is essential:

• Establishing Baselines: Using historical data to determine acceptable thresholds for model performance.
• Regular Audits: Conducting periodic evaluations of the model’s predictive accuracy and reliability.

Conclusion: Building a Comprehensive AI Governance Framework

A comprehensive AI governance framework is a multifaceted approach that aims to mitigate risks associated with AI/ML technologies within GxP. By clearly defining roles, establishing governance boards, and incorporating robust documentation processes, organizations can ensure compliance with regulatory requirements while safeguarding public health.

Incorporating continuous monitoring and evaluation processes, including bias and fairness testing, inherent to AI/ML model validation, is crucial. Maintaining alignment with GAMP 5 and evidencing compliance with 21 CFR Part 11 will support successful integration into existing pharmaceutical practices.

As organizations continue to explore the breadth of AI/ML in GxP, a proactive attitude towards governance can elucidate pathways for success and compliance in this rapidly evolving landscape.