Transforming Healthcare Documentation by Fine Tuning a Private LLM for MBCHC

A fine-tuned private LLM deployed on Azure AI Foundry, enhances healthcare documentation with automation, accuracy, and HIPAA compliance, leveraging Weights & Biases for optimization.

Comparison of ChatGPT-only and fine-tuned LLM outputs in a medical annotation task within the Agents Playground interface.

Kmeleon and MBCHC partnered to develop a fine-tuned private LLM for healthcare documentation, enhancing efficiency and compliance. Leveraging Weights & Biases for optimization and deploying on Azure AI Foundry, the AI model seamlessly integrates with EHR systems to automate clinical note-taking, reduce documentation time, and ensure HIPAA compliance. Designed to meet the demands of modern healthcare, this solution empowers physicians with accurate, real-time medical documentation while maintaining enterprise-grade security and scalability.

RESULTS

70% reduction in documentation time.

Full HIPAA compliance, with data security

Enterprise-grade deployment on Azure AI Foundry

Challenges:

For MBCHC, providing high-quality healthcare to underserved communities was always the top priority. However, outdated documentation workflows were creating a serious roadblock

Physicians were spending too much time manually writing or dictating notes—an inefficient, inconsistent process that slowed down patient interactions and increased administrative burden. These challenges led to: 

  • Excessive time spent on documentation, limiting patient interaction. 

  • Inconsistent medical records, leading to potential errors and compliance concerns. 

  • Physician burnout, with doctors overloaded by paperwork instead of focusing on patient care. 

  • Regulatory pressures, requiring strict compliance with HIPAA and data privacy regulations

MBCHC needed a modern, AI-powered solution that could reduce the documentation workload, maintain high medical accuracy, and integrate seamlessly with existing healthcare systems

Our Strategic Approach 

Recognizing the need for a domain-specific AI assistant, Kmeleon developed and fine-tuned a private LLM tailored to MBCHC’s unique requirements. Instead of relying on generic AI models, we created a solution that could understand, structure, and enhance medical annotations while maintaining strict compliance standards. 

Our approach included: 

  • Fine-Tuning with Weights & Biases (W&B) – Leveraging W&B’s experiment tracking and optimization tools, we refined the model using real doctor-patient interactions, ensuring high accuracy and medical specificity

  • A secure, HIPAA-compliant training environment – Implementing data encryption, anonymization, and strict access controls to protect patient records. 

  • An enterprise-ready deployment on Azure AI Foundry – The fine-tuned model was mounted onto Azure AI Foundry, ensuring enterprise-grade security, governance controls, and seamless cloud integration

  • Seamless EHR integration – Physicians could dictate, review, and edit AI-assisted notes in real time, allowing for faster and more structured documentation

The AI model was designed to not only generate structured and contextually relevant documentation but also adapt and improve over time, ensuring long-term efficiency. 

Implementation: From Development to Deployment 

To ensure the AI system was optimized for MBCHC’s needs, a rigorous fine-tuning and deployment strategy was implemented

  1. Data Preparation and ComplianceMedical records were anonymized, ensuring data privacy and HIPAA compliance before training. 

  1. Model Training and OptimizationW&B’s monitoring tools allowed for continuous hyperparameter tuning, ensuring optimal performance and accuracy

  1. Secure Enterprise Deployment – The fine-tuned model was deployed within Azure AI Foundry, allowing for scalable, compliant, and real-time medical documentation processing

  1. Continuous Learning and AdaptationReal-time feedback loops allowed the model to improve with ongoing physician interactions, ensuring long-term accuracy

This end-to-end implementation ensured the AI system was not only powerful and accurate but also fully compliant with industry standards

Graphs displaying training and validation performance metrics of different fine-tuning jobs, including mean token accuracy and loss over training steps.
Graphs displaying training and validation performance metrics of different fine-tuning jobs, including mean token accuracy and loss over training steps.
Graphs displaying training and validation performance metrics of different fine-tuning jobs, including mean token accuracy and loss over training steps.

Results and Impact

The introduction of AI-powered documentation at MBCHC significantly improved efficiency and compliance, providing measurable results that directly impacted physician workflows and patient care. With a streamlined documentation process, doctors could shift their focus from administrative tasks to patient interactions while maintaining high standards of accuracy and security. 

  • Seventy percent reduction in documentation time, giving physicians more time with patients. 

  • Enhanced accuracy and consistency in medical records, reducing errors. 

  • Full HIPAA compliance, ensuring data security and regulatory alignment. 

  • Enterprise-grade deployment on Azure AI Foundry, providing secure, scalable, and seamless AI integration

  • Effortless adoption, with a smooth transition into MBCHC’s daily workflows. 

Ready to Start Your First Gen AI Use Case?

Get a prototype up and running within a month to kickstart your Gen AI Evolution Journey.