Comparison of Different Fine-Tuning Techniques for Conversational AI

Feature request

It would be incredibly helpful to have a clear comparison or support for various fine-tuning techniques specifically for conversational AI. This feature could include insights into their strengths, limitations, and ideal use cases, helping practitioners choose the right approach for their needs.

Here’s a list of techniques to consider:

LoRa
AdaLoRa
BONE
VeRa
XLora
LN Tuning
VbLora
HRA (Hyperparameter Regularization Adapter)
IA3 (Input-Aware Adapter)
Llama Adapter
CPT (Conditional Prompt Tuning)etc

Motivation

With the growing number of fine-tuning techniques for conversational AI, it can be challenging to identify the most suitable approach for specific use cases. A comprehensive comparison of these techniques—highlighting their strengths, limitations, and ideal scenarios—would save time, reduce trial-and-error, and empower users to make informed decisions. This feature would bridge the gap between research and practical application, enabling more effective model customization and deployment.

Your contribution

I’d be happy to collaborate on this! While I might not have a complete solution right now, I’m willing to contribute by gathering resources, reviewing papers, or helping organize comparisons. If others are interested in teaming up, we could work together on a PR to make this feature happen. Let’s connect and brainstorm how we can tackle this effectively!

Hey everyone, PSA: We now have quite a few contributors who ran experiments on our benchmarks, which is awesome 🔥. Unfortunately, not all experiments led to any improvement. These negative results are, however, still valuable. To document them for the future, we now created a new dataset: https://huggingface.co/datasets/peft-internal-testing/benchmark-graveyard. I would encourage anyone who has negative results that didn't make it into the PEFT main repo to submit the results to that dataset with a PR.

Note that we need each file to have a unique name, so that there are no name collisions. PEFT will automatically take care of that as long as you are not on the main git branch. If you plan…

Thanks for coming up with this proposal. Indeed, this is something we have on our backlog for a long time. As you can imagine, providing objective and useful information on this is a huge undertaking, since relying on the paper results can often be problematic.

As a long term project, we plan to provide some kind of benchmark that compares all these methods in terms of runtime, memory usage, performance, etc. but I can't give any concrete date yet.

In the meantime, we have started to be more rigorous when new methods are being added in requiring a clear description of what the best use cases are. There is still a lot of room for improvement, especially when it comes to methods that were added some time ago.

If you (and others) want to contribute, I think a good place to start would be to go through the individual methods in the PEFT docs and help improve the descriptions. If we can make them more uniform, with more details on the best uses cases, pros and cons, this would already be a nice improvement.

There are other places that could benefit from such a clean up, e.g. the description of all the LoRA initialization methods.

I would be interested to contribute as well

I would be interested to contribute as well

Thanks for the offer. As mentioned, as a first step, we could use some help with updating the "blurbs" of the PEFT methods. For this, it's often sufficient to read a couple of section from the paper. If anyone wants to work on one such method, please announce it here so that there is no duplicate work.

How about having a sample fine-tuning script for each method and comparing different approaches for different tasks?

How about having a sample fine-tuning script for each method and comparing different approaches for different tasks?

I'm not 100% sure what you mean, but let's start with a single task and then we can expand from there. We haven't come up with such a task yet, but we have some criteria:

It should be a task that is supported by all methods (most likely language model fine-tuning)
The task should be kinda realistic and practical
The task should not take too long to run and should not require expensive hardware
Training code should be easy to adapt for real training (should have example character)

Maybe we can find something from the trl examples that we can adopt.

@BenjaminBossan

Hello @BenjaminBossan, @ImamaDev.

I hope this message finds you well. I am interested in contributing to this PR and would like your insights on the following:

"It should be a task that is supported by all methods (most likely language model fine-tuning)" - Could you please suggest suitable tasks where this comparison would be meaningful? A few options are: a) question-answering task (short input and output tokens) b) summarization task (long input and short output) c) code-generation (short input and long output). It would be great to know any preferred task / dataset to start with.
Could you please let me know any other metrics (runtime, memory usage, performance) that would be of interest to the community?

Thank you.

PS: I found a paper addressing the same issue: https://arxiv.org/pdf/2312.12148

@sirish-gambhira

Sure @sirish-gambhira , lets connect over discord to discuss this further my user name is imcoza

Thanks for the interest. We plan to do a mini sprint next week to make progress on this task. We will share any findings where the community could help contributing.

PS: I found a paper addressing the same issue: https://arxiv.org/pdf/2312.12148

Note that our goal is a bit different. We don't want to have a comprehensive literature overview of various methods on numerous datasets -- we wouldn't have the resources for that. Our goal is less academic and more practical: Giving PEFT users a simple starting guide to choose the method that is most likely to suite their needs.

That would be great. I conducted a comparative study evaluating AdaLoRA, IA3, and LoRA for text generation tasks using a multi-step conversational dataset. The results indicate that IA3 outperforms the other methods, demonstrating superior efficiency by achieving convergence in fewer fine-tuning steps. Looking forward to see your findings.

That would be great. I conducted a comparative study evaluating AdaLoRA, IA3, and LoRA for text generation tasks using a multi-step conversational dataset. The results indicate that IA3 outperforms the other methods, demonstrating superior efficiency by achieving convergence in fewer fine-tuning steps. Looking forward to see your findings.

Nice to hear that. I personally believe that users should also give other methods a chance instead of focusing only on LoRA. If you have anything to share, feel free to do so.

That would be great. I conducted a comparative study evaluating AdaLoRA, IA3, and LoRA for text generation tasks using a multi-step conversational dataset. The results indicate that IA3 outperforms the other methods, demonstrating superior efficiency by achieving convergence in fewer fine-tuning steps. Looking forward to see your findings.

Nice to hear that. I personally believe that users should also give other methods a chance instead of focusing only on LoRA. If you have anything to share, feel free to do so.

Based on my comparative study, I fine-tuned Llama 3.1 8B Instruct on Hindi multi-turn dialogue conversational data using Lora,IA3 and Adalora to evaluate their performance.
IA3 demonstrated a smoother and more stable convergence, making it well-suited for resource-constrained environments. It maintained a smaller generalization gap, effectively reducing the risk of overfitting. In contrast, Adalora exhibited an aggressive initial optimization phase, causing a sharp drop in loss, but ultimately achieved a lower validation loss, indicating superior generalization. However, this rapid convergence introduces higher variance, necessitating careful learning rate tuning. If stability and robustness are the primary concerns, IA3 is the better choice. But if the objective is to maximize generalization and long-term adaptability, Adalora emerges as the superior method—provided there is sufficient training time and room for fine-tuning adjustments.

@imcoza

Thanks for sharing your findings @imcoza. Are the code and the results publicly available?

We're still working on the method comparison framework, the progress is looking good so far. You can check for instance this PR.

I realized I opened #3537 and #3538 before receiving the explicit approval required by .ai/AGENTS.md. I have moved both PRs to draft and paused work.

May I proceed with the SHiRA documentation in #3537 first? I will keep #3538 paused until #3537 is resolved and the DeLoRA scope is explicitly approved.

@RerankerGuo

Thanks @RerankerGuo you may proceed with both PRs, but yes, please wait for approval first next time.

@Rithika-alt Same thing, please ask for approval first. Also, your notebook is empty.

@BenjaminBossan

Thanks @BenjaminBossan for approving both documentation scopes.

Both documentation PRs are now ready for review:

DOC Improve SHiRA documentation #3537: SHiRA documentation
DOC Improve DeLoRA documentation #3538: DeLoRA documentation

Each PR description now links its scope announcement and your approval comment. Posting this update for visibility and to avoid duplicate work; I will keep both changes limited to the approved scopes.

Method comparison: Reduce peak memory usage of image generation task (#…

@LeonxLJX

I'd like to take this one (Comparison of Different Fine-Tuning Techniques for Conversat). I'll dig into the root cause and follow up with a PR shortly. (claiming via @LeonxLJX)

@LeonxLJX

I'd like to take this one (Comparison of Different Fine-Tuning Techniques for Conversat). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)

If this is not picked-up yet, I'll be happy to take it

@mohamadar77

@mohamadar77 This issue has evolved into a more general discussion on how to compare different PEFT methods and thus does not have a proper resolution. If you want to contribute to the MetaMathQA benchmark, check #3521, if you want to contribute to the image generation benchmark, check #3522, if you want to contribute in another way, please make a suggestion here.

@BenjaminBossan

@BenjaminBossan I'd like to improve the AdaLoRA docs page only (docs/source/package_reference/adalora.md).

The page already has a “Contributions welcome” note that the importance-score / phase explanation needs clarification. Proposed scope:

clarify init → budgeting → final phases and when update_and_allocate must be called
short when-to-use / when-not vs LoRA
remove redundant wording; keep it consistent with recent adapter doc PRs

Docs-only; I will not open a broad comparison notebook or MetaMath experiment. OK to proceed?

@zelihaguven

@zelihaguven Thanks for the offer, please go ahead.

Hi maintainers,

I'd like to contribute a new experiment configuration for HRA on the
MetaMathQA benchmark, and per .ai/AGENTS.md I'm asking for approval
before opening a PR.

Why HRA / why MetaMathQA

method_comparison/image-gen/experiments/hra/ exists (flux2-klein-default),
but there is no HRA entry under method_comparison/MetaMathQA/experiments/.
So HRA currently can't be compared against LoRA / IA3 / AdaLoRA on the
language benchmark.

I enumerated the *Config classes actually exported by peft (55 at
0.21.1.dev0) and diffed them against the existing experiment directories.
HRA came out as a genuine gap. I also looked at X-LoRA, but it needs a
gating network and an adapters dict, so it doesn't fit this harness.

Proposed configuration

Following the conventions already used by MetaMathQA/experiments/vera/
and MetaMathQA/experiments/ln_tuning/:

Param	Value	Rationale
`r`	32	even, as `HRAConfig`'s docstring requires for the default init
`apply_GS`	false	matches the existing image-gen HRA experiment
`target_modules`	null	consistent with the other MetaMathQA experiments
`bias`	none	default
`init_weights`	true	default

Base model stays meta-llama/Llama-3.2-3B and all other training params
stay at default_training_params.json, so results remain comparable.
No training_params.json is needed.

One naming note

issue #2310 lists HRA as "Hyperparameter Regularization Adapter", but PEFT's
HRAConfig implements HRA: Householder Reflection Adaptation (hence the
apply_GS parameter and HRAModel in the docstring). I'm targeting the PEFT
implementation — just flagging it in case the issue meant a different method.

Plan

Add method_comparison/MetaMathQA/experiments/hra/llama-3.2-3B-default/adapter_config.json
Run it once locally to confirm it trains, and report the outcome
(including a negative one — I understand those go to benchmark-graveyard)
Open the PR with the disclosure required by .ai/AGENTS.md

I've checked there is no open PR for this and no one assigned to HRA. If
someone is already on it, I'll step back.

Could I get @peft-triage approved to proceed?

AI assistance disclosure: I used an AI assistant for repository exploration
and for drafting this plan. I've reviewed everything above, I understand the
configuration, and I'll run the experiment and the relevant checks myself.

Thanks!

@zdd3143

@zdd3143 Thanks for your offer. Regarding HRA specifically, it is currently not a good candidate because the implementation is inefficient, requiring a lot of memory to run. There is an open PR, #3185, to fix that. Moreover, for the addition of MetaMathQA experiments, please use discussion #3521 instead of this issue.

To help with the comparison matrix for conversational AI, here is a structured breakdown of the core PEFT techniques, focusing on practical VRAM footprints, optimizer overhead, and hardware trade-offs for modern decoder-only architectures (e.g. Llama 3, Qwen 2.5, DeepSeek R1):

Technique	Trainable Params (%)	Memory Footprint (vs. Full)	Optimizer State Multiplier	Ideal Conversational AI Use Case	Key Engineering Bottleneck
Full Fine-Tuning	100%	$1.0\times$ (Base: 16–20 bytes/param)	$4\times$ model size (AdamW fp32)	Foundation domain pre-training & major alignment shifts	Requires multi-GPU clusters (8x H100s); cost-prohibitive for indie builders
Standard LoRA	0.05% – 0.2%	~0.35× – 0.45×	Proportional only to rank $r$ matrices	High-throughput dialogue agents, task-specific persona adapters	Base weights stay in 16-bit; requires 24GB+ VRAM for 8B models
QLoRA (4-bit NF4)	0.05% – 0.2%	~0.15× – 0.25×	Proportional to rank $r$ (paged 8-bit AdamW)	Cost-sensitive conversational fine-tuning on consumer GPUs (12GB–16GB)	Dequantization overhead during forward pass adds ~15-20% latency
AdaLoRa	0.02% – 0.1%	~0.30× – 0.40×	Dynamic SVD parameter allocation	Budget-constrained multi-task conversational routing	Higher computational overhead during rank pruning steps
$(IA)^3$	< 0.01%	~0.10× – 0.15×	Negligible optimizer states	Fast few-shot task adaptation & lightweight intent classification	Lower expressive capacity for nuanced, multi-turn reasoning

The Practical Rule of Thumb for Conversational Memory:

In conversational fine-tuning, the biggest cause of CUDA OOM failures is variable sequence padding in multi-turn dialogues.

While base weights are static, attention activation memory scales with sequence length ($O(N^2)$ or $O(N)$ with FlashAttention). A batch padded to 4096 tokens can easily demand 3x more activation headroom during the backward pass than initial model loading suggests.

To make benchmarking these parameters practical before provisioning cloud GPUs, we built an interactive terminal simulator that models these exact rank allocations, preflight VRAM ceilings, and multi-cloud spot arbitrage across 12 providers:

🎮 Interactive Simulator: vivacious-terminal-simulator
⭐️ Open-Source Preflight CLI: Viavcious-cloud/vivacious-cli
🌐 Platform: vivaciouscloud.com

Hope this structured breakdown helps flesh out the RFC comparison guide!

The AdaLoRA package reference still carries the Contributions welcome: This section needs clarification note on how the importance score is computed, so I wrote that section out against the actual RankAllocator code (src/peft/tuners/adalora/layer.py): how ipt = |p * grad| is smoothed with the EMA and uncertainty terms, how the scores are ranked per triplet, where the budget threshold comes from, and how the cubically scheduled budget maps onto tinit / tfinal / deltaT.

I saw #3807 was opened on the same file last month and closed the next day, so I would rather ask before opening another one.

Is a docs-only PR on adalora.md still wanted? And if so, would you approve this issue for triage (/peft-triage approved) so the PR is not closed automatically?

@abdelhadi703

Thanks @abdelhadi703. @zelihaguven you closed your PR, do you still plan to work on this?

Navigation Menu

Repository navigation

Comparison of Different Fine-Tuning Techniques for Conversational AI #2310

Feature request

Motivation

Your contribution

141 remaining items

The Practical Rule of Thumb for Conversational Memory:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Comparison of Different Fine-Tuning Techniques for Conversational AI #2310

Description

Feature request

Motivation

Your contribution

Activity

BenjaminBossan commented on Jan 7, 2025

sparsh2 commented on Jan 7, 2025

BenjaminBossan commented on Jan 8, 2025

imcoza commented on Jan 15, 2025

BenjaminBossan commented on Jan 16, 2025

sirish-gambhira commented on Feb 15, 2025

imcoza commented on Feb 17, 2025

BenjaminBossan commented on Feb 17, 2025

imcoza commented on Feb 17, 2025

BenjaminBossan commented on Feb 17, 2025

imcoza commented on Mar 3, 2025

BenjaminBossan commented on Mar 3, 2025

141 remaining items

RerankerGuo commented on Aug 12, 2026

BenjaminBossan commented on Aug 12, 2026

RerankerGuo commented on Aug 12, 2026

LeonxLJX commented on Sep 1, 2026

LeonxLJX commented on Sep 1, 2026

mohamadar77 commented on Sep 9, 2026

BenjaminBossan commented on Sep 9, 2026

zelihaguven commented on Sep 19, 2026

BenjaminBossan commented on Sep 21, 2026

zdd3143 commented on Sep 25, 2026

BenjaminBossan commented on Sep 25, 2026

Viavcious-cloud commented on Oct 2, 2026

The Practical Rule of Thumb for Conversational Memory:

abdelhadi703 commented on Oct 7, 2026

BenjaminBossan commented on Oct 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions