SkillAgentSearch skills...

fine-tuning-expert

Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, Op…

Install / Use

npx skills add Jeffallan/claude-skills --skill fine-tuning-expert

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

94/100

Supported Platforms

Universal

Tags

Our assessment of fine-tuning-expert

fine-tuning-expert scores 94/100 on our quality scale, 174th of 947 AI & Machine Learning skills we index (top 19%).

Its SKILL.md is 6.4 KB long, well organised into 14 sections with 3 code examples: a thorough specification that gives an agent plenty to work with.

With 12,363 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
29/30
Structure
18/20
Description
15/15
Adoption
17/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated 5 days ago, so fine-tuning-expert is actively maintained.
  • Our last check on 2026-11-16 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-28. Automated pattern scan on 2026-09-28. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

fine-tuning-expert compared with similar skills

All 4 of these similar skills score higher than fine-tuning-expert; compare them before choosing.

SkillScoreStarsUpdatedFormat
fine-tuning-expert (this skill)by Jeffallan9412.4k5d agoSKILL.md
claude-memby thedotmack100117.6ktodayCLAUDE.md
Understand-Anythingby Egonex-AI10092.8k1d agoCLAUDE.md
headroomby headroomlabs-ai10079.8ktodayCLAUDE.md
CowAgentby zhayujie10048.0ktodayCLAUDE.md

Frequently asked questions

How do I install fine-tuning-expert?
Run npx skills add Jeffallan/claude-skills --skill fine-tuning-expert. The install tabs above show the steps for each supported agent.
Which AI agents does fine-tuning-expert work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is fine-tuning-expert safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is fine-tuning-expert still maintained?
The repository was last updated 5 days ago, so fine-tuning-expert is actively maintained.

name: fine-tuning-expert description: "Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model." license: MIT metadata: author: https://github.com/Jeffallan version: "1.1.0" domain: data-ml triggers: fine-tuning, fine tuning, finetuning, LoRA, QLoRA, PEFT, adapter tuning, transfer learning, model training, custom model, LLM training, instruction tuning, RLHF, model optimization, quantization role: expert scope: implementation output-format: code related-skills: devops-engineer

Fine-Tuning Expert

Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.

Core Workflow

  1. Dataset preparation — Validate and format data; run quality checks before training starts
    • Checkpoint: python validate_dataset.py --input data.jsonl — fix all errors before proceeding
  2. Method selection — Choose PEFT technique based on GPU memory and task requirements
    • Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models
  3. Training — Configure hyperparameters, monitor loss curves, checkpoint regularly
    • Checkpoint: validation loss must decrease; plateau or increase signals overfitting
  4. Evaluation — Benchmark against the base model; test on held-out set and edge cases
    • Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers
  5. Deployment — Merge adapter weights, quantize, measure inference throughput before serving

Reference Guide

Load detailed guidance based on context:

| Topic | Reference | Load When | |-------|-----------|-----------| | LoRA/PEFT | references/lora-peft.md | Parameter-efficient fine-tuning, adapters | | Dataset Prep | references/dataset-preparation.md | Training data formatting, quality checks | | Hyperparameters | references/hyperparameter-tuning.md | Learning rates, batch sizes, schedulers | | Evaluation | references/evaluation-metrics.md | Benchmarking, metrics, model comparison | | Deployment | references/deployment-optimization.md | Model merging, quantization, serving |

Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
import torch

# 1. Load base model and tokenizer
model_id = "meta-llama/Llama-3-8B"
tokenizer = Auto…[redacted](model_id)
tokenizer.pad_token = toke…[redacted]

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# 2. Configure LoRA adapter
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,               # rank — increase for more capacity, decrease to save memory
    lora_alpha=32,      # scaling factor; typically 2× rank
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # verify: should be ~0.1–1% of total params

# 3. Load and format dataset (Alpaca-style JSONL)
dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"})

def format_prompt(example):
    return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}

dataset = dataset.map(format_prompt)

# 4. Training arguments
training_args = TrainingArguments(
    output_dir="./checkpoints",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,     # effective batch size = 16
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,                 # always use warmup
    fp16=False,
    bf16=True,
    logging_steps=10,
    eval_strategy="steps",
    eval_steps=100,
    save_steps=200,
    load_best_model_at_end=True,
)

# 5. Train
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    dataset_text_field="text",
    max_seq_length=2048,
)
trainer.train()

# 6. Save adapter weights only
model.save_pretrained("./lora-adapter")
tokenizer.save_pretrained("./lora-adapter")

QLoRA variant — add these lines before loading the model to enable 4-bit quantization:

from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")

Merge adapter into base model for deployment:

from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
merged = PeftModel.from_pretrained(base, "./lora-adapter").merge_and_unload()
merged.save_pretrained("./merged-model")

Constraints

MUST DO

  • Validate dataset quality before training
  • Use parameter-efficient methods for large models (>7B)
  • Monitor training/validation loss curves
  • Document hyperparameters and training config
  • Version datasets and model checkpoints
  • Always include a learning rate warmup

MUST NOT DO

  • Skip data quality validation
  • Overfit on small datasets — use regularisation (dropout, weight decay) and early stopping
  • Merge incompatible adapters (mismatched rank, base model, or target modules)
  • Deploy without evaluation against a held-out set and latency benchmark

Output Templates

When implementing fine-tuning, always provide:

  1. Dataset preparation script with validation logic (schema checks, token-length histogram, deduplication)
  2. Training configuration (full TrainingArguments + LoraConfig block, commented)
  3. Evaluation script reporting perplexity, task-specific metrics, and latency
  4. Brief design rationale — why this PEFT method, rank, and learning rate were chosen for this task

Documentation

Related Skills

View on GitHub
GitHub Stars12.4k
CategoryAI
Updated5d ago
Forks1.2k

Languages

JavaScript

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions