Artificial intelligence (AI) is transforming a number of industries, including healthcare. A session at the AMCP 2025 Annual Meeting addressed the legal landscape and regulatory frameworks that will govern AI in managed care.
AI has the potential to affect managed care in a number of ways, including the utilization management process (eg, reviews and appeals), patient portals (eg, summarizing documents), and providing customer service. There are some concerns that must be considered with AI implementation such as patient dissatisfaction with a virtual assistant, inappropriate outcomes, and biased datasets.
Currently, regulation of AI is “the Wild West,” with multiple sources of authority, including Congress, the FDA, the Department of Health & Human Services, state governments, boards of pharmacy, and the courts. AMCP is tracking 60 bills in 23 states that would regulate the use of AI in managed care, many focused on AI in utilization management such as prior authorizations.
Another area of consideration for the use of AI is who may be liable, should AI result in an error: the organization deploying the AI tool, the software company that develops the tool, or both? In addition, there are questions surrounding the Health Insurance Portability and Accountability Act (HIPAA) protection impacts when using patient information in AI datasets.
The presentation included the following important components of AI compliance and ethics:
- Patients must trust AI-driven decisions related to their care.
- AI must be continuously monitored to ensure it serves the intended purpose.
- Oversight must be undertaken to understand how AI makes decisions.
- AI data should be free from sampling bias to avoid discrimination.
AI systems should be designed to complement, not replace, human judgment, and there should be mechanisms for human review and intervention when necessary. Patients should also be adequately informed of the use of AI in their care and be allowed to provide informed consent. There should be regular monitoring and assessment of AI systems to ensure fairness, accuracy, and freedom from bias. AI systems should be designed with transparency about how and why decisions are made.
The session concluded with some recommended best practices for responsibly integrating AI into the pharmacy practice:
- Adhere to the organization’s Ethics and Governance Committee.
- Ensure fairness and bias mitigation.
- Prioritize transparency and explainability.
- Strengthen data privacy and security.
- Embed continuous monitoring and accountability.
- Foster a responsible AI culture, including regular staff training.
Reference
Mesaros J, Wilson L. The regulatory landscape of AI in managed care pharmacy: compliance, ethics, and future trends. Session W1. Presented at: AMCP 2025; March 31-April 3, 2025; Houston, TX.
