A fine-tuning SDK. Any open model — with any method, on any hardware, for any harness.
Open source · built by Lyzr Research Labs · maintained by Khush Patel · slm♥
pip install 'shadowlm[all]' # the full package — every dependency included
pip install shadowlm # core SDK only (zero dependencies)import shadowlm as slm
ds = slm.Dataset.from_jsonl("data.jsonl").as_chat() # datasets
model = slm.load("mlx-community/Qwen2.5-0.5B-Instruct-4bit", # load
accelerator="shadow")
run = model.finetune(ds, method="lora", max_steps=60) # finetune
print(run.loss, run.sparkline()) # live metrics
print(model.generate("What is the capital of France?")) # inference
model.save("out/", fmt="adapter") # ship itChange method="lora" to qlora, dora, full, dpo, grpo, more, bitfit,
prompt, ptuning, adapter, cpt — and nothing else changes. That's the idea.
Your agent runs on a rented frontier model — general, costly, someone else's. ShadowLM moves one task to a small model you own, without touching the agent: it keeps calling the same endpoint; only the model behind it changes.
What you end up with is a shadowLM — a small fine-tuned model that shadows the frontier model, runs in its shadow on real traffic until it does the job as well, then takes over. Lower cost, data stays inside, the weights are yours.
- Baseline — your agent runs on the frontier model.
- Capture & fine-tune —
slm.capture()records the real traffic; train a small open model on it. - Shadow mode — the shadowLM runs behind the same agent, answering in parallel so you can compare.
- Gradual switch — once it holds up, route traffic to the shadowLM. You own it.
This repo is the engine for that loop. The orchestration that wraps it into a one-click migration is ShadowLM Studio.
with slm.capture(model) as proxy: # 1. record your agent, unchanged
run_my_agent(base_url=proxy.base_url) # any OpenAI-client harness
group = slm.judge_group( # 2. score whole episodes (LLM judge)
slm.TrajectoryGroup(proxy.trajectories()), judge=judge)
run = model.finetune([group], method="grpo") # 3. train the shadowLM on themNo reward math, no rewriting the agent into an RL framework — the model API is the one boundary every agent already has, so ShadowLM trains from it.
The whole capture → judge → train → own a shadowLM loop runs on these:
| Block | What it does | API |
|---|---|---|
| Capture proxy | drop-in OpenAI endpoint that records your agent's traffic into trajectories — agent unchanged | slm.capture() |
| 12 methods | LoRA · QLoRA · DoRA · full · CPT · DPO · GRPO · MoRE · BitFit · prompt · p-tuning · adapter | method= |
| Judge → train | score episodes with an LLM judge, train with trajectory-GRPO or DPO | judge_group |
| MoRE | facts fused into attention — near-zero-hallucination recall | method="more" |
| Any hardware | CUDA · TPU · Trainium · Intel · Apple · CPU (whatever HF accelerate targets) | device= |
| Shadow accelerator | 4-bit, grad checkpointing, flash-attn, fused optimizer, optional Liger kernels — logged, never silent | accelerator="shadow" |
| Checkpoints | save every N steps, then load or A/B any version — step 200 vs final — in the playground |
save_steps= · run.checkpoint_at(step) |
| Remote + server | train on a GPU box or fleet over one JSON protocol; metrics stream back | backend="remote" · shadowlm serve |
| Studio | datasets → models → guided train → live runs (charts + console) → playground compare | shadowlm serve → / |
| CLI | finetune / runs / plot / chat / export / methods from the shell | shadowlm … |
| Own the weights | adapter/merged export, run records that survive restarts, nothing leaves your box | model.save() |
Each technique is a declarative spec under shadowlm/methods/; backends read the
spec (adapter kind, base requirements, data rendering), never the method name.
| method | what it does | base | default LR |
|---|---|---|---|
lora |
LoRA adapters | either | 2e-4 |
qlora |
LoRA on a 4-bit base, lowest memory | 4-bit | 2e-4 |
dora |
weight-decomposed LoRA, better at low rank | either | 2e-4 |
full |
update every transformer weight | unquantized | 2e-5 |
cpt |
continued pretraining on raw domain text | either | 5e-5 |
dpo |
preference optimization on {prompt, chosen, rejected} |
either | 5e-6 |
grpo |
RL from reward functions or scored TrajectoryGroups |
either | 5e-6 |
more |
mixture of retrieval experts — facts fused into attention | either | 1e-4 |
bitfit |
train only the bias terms (~0.1% of params) | unquantized | 5e-4 |
prompt/ptuning |
soft prompts / p-tuning — learned virtual tokens | either | 5e-3 |
adapter |
bottleneck adapter modules after each layer | either | 1e-4 |
Base requirements are enforced with clear errors (e.g. qlora on a 16-bit model
tells you to load a 4-bit one). Adding your own method is one file —
methods.register(TrainingMethod(...)).
torch (CUDA) is the production backend; mlx is the local-dev loop on Apple
Silicon; remote runs the same API against any ShadowLM server. auto picks
the right one. The torch path rides HuggingFace Trainer + accelerate, so it
trains on any accelerator HuggingFace supports — pick it with device=:
| ecosystem | how |
|---|---|
| NVIDIA CUDA | device="cuda" (+ 4-bit, flash-attn, fused optim) |
| AWS Trainium · Google TPU | device="xla" (Neuron / torch-xla) |
| Intel GPU | device="xpu" · Apple backend="mlx" · CPU device="cpu" |
On Microsoft Azure / any cloud you run on NVIDIA GPUs — the cuda path, nothing
to configure.
One command — installs the right backend for your machine and opens the studio:
curl -fsSL https://install.shadowlm.sh | shIt detects your hardware and installs the matching stack — Apple Silicon → mlx,
NVIDIA → torch + Liger fused kernels, otherwise torch CPU — into an isolated env
in ~/.shadowlm/venv, then launches shadowlm serve at http://127.0.0.1:8329.
Re-run any time to upgrade. Override with SHADOWLM_EXTRAS=cli (UI only),
SHADOWLM_PORT=…, or SHADOWLM_NO_SERVE=1 (install without launching).
Or with pip — pip install 'shadowlm[all]' gives you everything on a CUDA/CPU
box. Each extra is independent:
| extra | adds |
|---|---|
[torch] |
training on CUDA / CPU (transformers + trl + peft) |
[mlx] |
the Apple-Silicon dev backend |
[preference] |
dpo / grpo on the mlx backend |
[retrieval] |
the more method (fact index) |
[kernels] |
fused Triton kernels on NVIDIA (Liger, Apache-2.0) |
[cli] |
the shadowlm command (Typer + Rich) |
git clone https://github.com/open-gitagent/shadowLM && cd shadowLM
python3 -m venv .venv && source .venv/bin/activate && pip install -e '.[mlx]'
python examples/quickstart.py # datasets → finetune → inference, end to endNo hardware handy? examples/colab_quickstart.ipynb runs the same flow on a free
Colab GPU. Run output (mlx, a 0.5B model, ~3.5s):
[shadow] enabled: gradient checkpointing
[mlx:gpu] finetuning Qwen2.5-0.5B-Instruct-4bit · lora · 40 iters · lora r=16
[████████████████████████] step 40/40 loss 0.0718 lr 5.00e-05 1,048 tok/s
loss ▇▆█▇▆▇▇█▅▅▄▅▃▂▃▃▁▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ 4.2120 → 0.0718
♥ succeeded · 40 steps · 3.5s
shadowlm finetune data.jsonl --model Qwen/Qwen2.5-0.5B-Instruct --method lora
shadowlm finetune --config run.yaml --dry-run # reproducible runs, preview first
shadowlm chat out/adapter/ # talk to what you trained
shadowlm serve # studio UI + API on one portHeadline hyperparameters are typed flags; every other TrainConfig field is
reachable via --set field=value or a --config file (flags override config
override defaults). shadowlm serve opens the studio at http://127.0.0.1:8329
— Datasets (upload + HuggingFace) → Models → guided Train → live Runs (loss
charts + training console) → Playground (compare base ↔ finetuned). It's the
built React app, shipped in the wheel; the same JSON protocol powers
backend="remote".
accelerator="shadow" turns on the optimizations that are safe for your model
and hardware — gradient checkpointing, flash-attention-2, a fused 8-bit
optimizer, 4-bit QLoRA, and optional Liger
fused Triton kernels ([kernels] extra, NVIDIA). Modes: auto / shadow /
none. It logs exactly what it enabled and no-ops when something isn't
available — ShadowLM integrates proven optimizations rather than shipping its own
GPU kernels, so no magic multipliers, just the standard wins turned on safely.
The engine ships first; ShadowLM Studio (the hosted tier) wraps this exact API — nothing reimplemented — to turn the blocks into a one-click migration:
- Decision inbox — captured traces surfaced for human approve/correct into chosen-vs-rejected pairs (today: auto-scored by an LLM judge).
- Eval gates — advance only when quality holds and savings beat cost: task-level evals + cost-per-task on the run records.
- Shadow router — the capture proxy evolved: run the shadowLM in parallel behind the live agent, then shift traffic % frontier → owned.
- Fleet + teams — GPU job queue, shared run history, dataset/adapter registry.
[x] SDK — datasets → finetune → inference on mlx / torch / remote
[x] 12 methods incl. MoRE, trajectory GRPO, judge rewards
[x] Capture proxy · shadow accelerator · any-hardware
[x] Remote backend + reference server + the studio dashboard + CLI
[ ] Studio orchestration — decision inbox · eval gates · shadow router · switch
Adding a training method is one file; bug reports with a failing snippet are gold. Fork → branch → PR. ⭐ the repo if it trains something for you — it helps others find it.
MIT · slm♥