On this page
- What is the difference between full fine-tuning and parameter-efficient fine-tuning?
- What is full fine-tuning?
- What is parameter-efficient fine-tuning (PEFT)?
- Full fine-tuning vs parameter-efficient fine-tuning: side-by-side comparison
- Why do clinical coding and documentation need a fine-tuned medical LLM?
- How do fine-tuned medical LLMs improve clinical coding?
- What compliance rules apply to fine-tuning a medical LLM?
- Where are fine-tuned medical LLMs headed?
- Summing up
- FAQ
Full fine-tuning vs. parameter-efficient fine-tuning is the first technical choice a healthcare team makes when it adapts a large language model to clinical coding and documentation. Full fine-tuning updates every weight in the model. Parameter-efficient fine-tuning (PEFT) freezes the model and trains a small add-on, such as a LoRA adapter. For most clinical coding projects in 2026, PEFT reaches accuracy close to full fine-tuning at a fraction of the cost.
This post explains how each method works, compares them side by side, and shows which one fits which clinical task, from ICD-10 code suggestion to specialty note drafting. It is written for founders, product managers, and CTOs planning a medical AI build.
What is the difference between full fine-tuning and parameter-efficient fine-tuning?
The difference is how much of the model changes during training. Full fine-tuning retrains all of the model’s parameters. Parameter-efficient fine-tuning trains a small set of new parameters, often less than 1% of the total, and leaves the original model frozen.
Both methods produce a fine-tuned medical LLM. A fine-tuned medical LLM is a general large language model (LLM) that has been trained further on clinical notes, code-description pairs, and coding guidelines, so it can read medical narratives and assign ICD-10-CM, CPT and HCPCS codes accurately. The two methods differ in cost, risk, flexibility, and how high the accuracy ceiling goes.
What is full fine-tuning?
Full fine-tuning is the process of continuing to train every weight of a pre-trained model on a new labeled dataset, so the whole model shifts toward the new domain.
How full fine-tuning works
Full fine-tuning runs the same training loop used to build the model, but on domain data such as de-identified discharge summaries paired with their final codes. Every training step stores the model weights, their gradients, and the optimizer states. A common estimate from Microsoft’s ZeRO research is that mixed-precision training with the Adam optimizer needs about 16 bytes per parameter before activations. By that estimate, a 7-billion-parameter model needs roughly 112 GB of GPU memory, which is more than a single 80 GB GPU holds.
When full fine-tuning is worth it
Full fine-tuning is worth the cost when the task needs the model’s core knowledge to change, not just its behavior.
- You have a large, high-quality body of clinical text, not just a few thousand labeled examples.
- You are building a medical foundation model. Meditron, from EPFL, was built by continued pretraining of Llama 2 (7B and 70B) on medical literature and guidelines.
- You need the highest possible accuracy on complex clinical reasoning and will run one model for one task.
Drawbacks of full fine-tuning
- Cost: multi-GPU clusters and long training runs.
- Storage: every task or specialty needs its own full copy of the model.
- Catastrophic forgetting: the model can lose general skills it had before training. Catastrophic forgetting is the loss of earlier knowledge when a model is retrained on new data.
- Overfitting: on small datasets, the model memorizes the training notes instead of learning the coding logic.
What is parameter-efficient fine-tuning (PEFT)?
Parameter-efficient fine-tuning (PEFT) is a family of methods that adapt a pre-trained model by training a small number of added or selected parameters while the original weights stay frozen. The open-source Hugging Face PEFT library implements the main methods below, and serving engines such as vLLM can run many adapters on one shared base model.
LoRA (Low-Rank Adaptation)
LoRA adds two small low-rank matrices next to the model’s frozen weight matrices and trains only those. In the original Microsoft paper (Hu et al., 2021), LoRA on GPT-3 175B cut trainable parameters by 10,000 times and GPU memory by 3 times, while matching or beating full fine-tuning on the benchmarks tested. A LoRA adapter is small enough to store and swap per task. Best for: most clinical coding and documentation tasks.
QLoRA (Quantized LoRA)
QLoRA trains LoRA adapters on top of a base model compressed to 4-bit precision. The QLoRA paper (Dettmers et al., 2023) fine-tuned a 65-billion-parameter model on a single 48 GB GPU. Best for: hospitals and clinics that must train on-premises with limited hardware because patient data cannot leave their network.
Adapters
Adapters are small bottleneck layers inserted inside each transformer block. In the original adapter paper (Houlsby et al., 2019), adapters came within 0.4% of full fine-tuning on the GLUE benchmark while adding about 3.6% parameters per task. Best for: teams that want a clean, modular add-on per task.
Structured Output and Workflow Integration
Fine-tuned models can be trained to produce:
- Structured ICD-10/CPT outputs
- Confidence scores
- Supporting documentation excerpts
- Rationale for code assignment
This makes them usable inside:
- EHR systems
- Revenue cycle management platforms
- Computer-Assisted Coding (CAC) tools
Instead of replacing human coders, they function as intelligent assistants, surfacing suggestions that speed up review cycles.
Full fine-tuning vs parameter-efficient fine-tuning: side-by-side comparison
PEFT wins on cost, speed, storage, and protecting what the base model already knows. Full fine-tuning wins only when the task needs big changes, and the team has the data and GPUs to support them.
| Factor | Full fine-tuning | PEFT (LoRA, QLoRA, adapters) |
|---|---|---|
| Parameters trained | All of them | Often less than 1% |
| GPU memory for a 7B model | About 112 GB or more, usually multi-GPU | A single GPU in most cases [VERIFY: typical VRAM for 7B LoRA and QLoRA runs] |
| Training time | Longer | Shorter |
| Storage per task | A full model copy | A small adapter file |
| Risk of forgetting general skills | Higher | Lower |
| Accuracy ceiling | Highest on complex, data-rich tasks | Close to full fine-tuning on most tasks |
| Data needed | Large, high-quality datasets | Works with smaller labeled sets |
| Serving many specialties | One model per specialty | One base model, many swappable adapters |
| Best for | Medical foundation models, deep domain shift | Clinical coding, specialty notes, fast updates |
Research supports the forgetting trade-off. A 2024 study, “LoRA Learns Less and Forgets Less” (Biderman et al.), found that LoRA usually trails full fine-tuning on hard new domains such as code and math but keeps more of the base model’s original abilities.
Why do clinical coding and documentation need a fine-tuned medical LLM?
Clinical coding needs a fine-tuned medical LLM because general models lack the code specificity, guideline awareness, and clinical context that billing and compliance demand. Every diagnosis, procedure, and patient interaction must be translated into standard codes such as ICD-10-CM, CPT, HCPCS, and SNOMED CT. These codes drive billing, reimbursement, compliance, quality reporting, and population health analytics.
The complexity of clinical coding
Clinical coding is complex because the code sets are large and change every year.
- ICD-10-CM contains more than 70,000 diagnosis codes.
- ICD-10-CM updates take effect each October 1, and the American Medical Association (AMA) releases CPT updates each January 1.
- Specialty documentation standards and payer policies add further rules.
A small documentation gap can cause undercoding, overcoding, claim denials, audits, or penalties. Coders must read notes full of abbreviations, incomplete sentences, and specialty jargon while tracking those changes.
The documentation burden on clinicians
Clinicians spend a large share of their day in the Electronic Health Record (EHR). A 2016 time-and-motion study in the Annals of Internal Medicine found that physicians spent nearly two hours on EHR and desk work for every hour of direct patient care.
- Physician burnout
- Less time and attention for patients
- Lower job satisfaction
- Copy-paste and templated notes that miss detail
How do fine-tuned medical LLMs improve clinical coding?
Fine-tuned medical LLMs improve clinical coding by reading the full clinical context of a note and suggesting the most specific code the documentation supports, with the evidence attached for a human coder to confirm.
From keyword matching to clinical understanding
Rule-based coding tools match keywords. If a note says “Type 2 diabetes,” the tool maps it to E11.9 (type 2 diabetes mellitus without complications). Real notes are rarely that clean:
- “Long-standing uncontrolled sugar levels”
- “DM2 with neuropathic complications”
- “Poor glycemic control, on insulin”
A general LLM recognizes diabetes. A fine-tuned medical LLM also recognizes the complications and the specificity ICD-10-CM requires, because it has learned from thousands of real notes paired with their final codes.
Structured output and workflow integration
A fine-tuned model can be trained to return structured ICD-10-CM and CPT codes, confidence scores, the supporting text from the note, and a short rationale. That output fits inside EHR systems, revenue cycle management platforms, and Computer-Assisted Coding (CAC) tools. Human coders stay in charge and review the suggestions faster.
Keeping up with annual code updates
Annual code updates are where PEFT has a clear practical edge. A team can retrain a small LoRA adapter for each new ICD-10-CM or CPT release instead of retraining the whole model. Many teams also pair the adapter with retrieval over the current code set, an approach compared in RAG vs fine-tuning.
Which should you choose for clinical coding in 2026?
For most clinical coding and documentation projects, choose PEFT, usually LoRA or QLoRA on an open base model. Choose full fine-tuning only when you are building a medical foundation model or when PEFT leaves a measured accuracy gap that your data and budget can close.
| Clinical task | Recommended method | Why |
|---|---|---|
| ICD-10-CM and CPT code suggestion | LoRA | High accuracy with modest labeled data |
| Specialty documentation (cardiology, radiology, orthopedics) | One LoRA adapter per specialty | One shared base model, small swappable adapters |
| On-premises training with limited GPUs | QLoRA | Patient data stays inside the network |
| Fast-changing payer rules | PEFT plus retrieval | Update the adapter or the rules source, not the model |
| New medical foundation model | Full fine-tuning or continued pretraining | Needs big changes to core knowledge |
Use these six steps to decide:
- Define the task and the metric. For example, code-level precision and recall, or first-pass claim acceptance.
- Audit your data. Check volume, label quality, and protected health information (PHI). Where real data is thin or sensitive, synthetic EHR data can fill gaps for validation.
- Set a baseline. Test a general model with good prompts and retrieval before you train anything.
- Run LoRA or QLoRA first. Compare the result against the baseline on held-out notes.
- Escalate only if needed. Move to full fine-tuning when a measured gap remains, and you have the data and compute to close it.
- Keep humans in the loop. Route every suggestion to a certified coder and monitor accuracy after each code update.
Budget follows the method you choose. The cost of implementing AI in healthcare breaks down into the wider build and integration costs. If you want an outside team, a custom AI development company such as Ailoitte is one option to scope the build.
What compliance rules apply to fine-tuning a medical LLM?
The Health Insurance Portability and Accountability Act (HIPAA) applies whenever protected health information is used to train, host, or run the model. In practice, that means four things:
- De-identify training data with the HIPAA Safe Harbor method (removing 18 identifiers) or the Expert Determination method.
- Sign a Business Associate Agreement (BAA) with any cloud or model vendor that touches PHI.
- Log access and outputs so every suggested code has an audit trail.
- Prefer explainable output. Coders and auditors need to see why a code was suggested, which is the core idea of explainable AI.
Software that only supports billing and administrative work is generally excluded from the FDA’s medical device definition under the 21st Century Cures Act. A tool that recommends diagnoses or treatment may fall under FDA clinical decision support guidance. The healthcare software development guide covers these rules across the full build.
Federated learning is the other privacy option. Federated learning trains a model across several hospitals without moving patient data to one place. Because LoRA adapters are small, they are cheap to share between sites, which makes PEFT a natural fit for federated machine learning.
Where are fine-tuned medical LLMs headed?
Fine-tuned medical LLMs are moving from after-the-fact coding help to real-time support inside clinical workflows, built on smaller specialty adapters and stronger audit trails.
- In-workflow coding: suggestions appear in the EHR while the clinician documents, so gaps are fixed before the claim is built.
- Multimodal models: models that read lab reports, imaging reports, and voice transcripts alongside the note, which cross-checks the codes.
- Coding copilots: tools that flag missing details and inconsistencies while humans make the final call.
- Denial prediction: systems that predict claim denials before submission.
- Clearer governance: standard audit trails and transparency expectations for AI-generated codes.
Teams that plan to build these products into a platform can work with an AI healthcare software development company that handles HIPAA, EHR integration, and model operations together.
Summing up
In the full fine-tuning vs parameter-efficient fine-tuning decision, PEFT is the right default for clinical coding and documentation in 2026. LoRA and QLoRA deliver accuracy close to full fine-tuning, cost far less, and make annual code updates and per-specialty models practical. Save full fine-tuning for foundation-level work, and keep a certified coder in the loop either way.
Sunil Kumar is CEO of Ailoitte, an AI-native engineering company building intelligent applications for startups and enterprises. He created the AI Velocity Pods model, delivering production-ready AI products 5× faster than traditional teams. Sunil writes about agentic AI, GenAI strategy, and outcome-based engineering.
About Ailoitte
