CODETIPS ACADEMY / COMPLETE RUNNABLE LESSON
All parts in this seriesPart 10 — Use a Pretrained Model and Publish Your Notebooks on GitHub
What you will build
The earlier lessons expose mechanisms without downloading pretrained weights. This final lesson supplies an entire optional Hugging Face LoRA script: imports, tokenizer setup, examples, label masking, training, generation comparison, and adapter saving.
Running the file without --run prints guidance and performs no model download. An explicit --run opts into a small external-model experiment. Five optimizer steps are a pipeline smoke test, not a claim that the model is ready for real users.
By the end: Inspect a complete optional pretrained-model recipe and publish reproducible notebook links.
- What you will build
- What you need before starting
- Understand the idea before the code
- A picture of this part's idea
- Create the lesson folder
- Complete code for this article
- Run it and compare the result
- Walk through what just happened
- Change one thing yourself
- If your result is different
- What you can now explain
What you need before starting
You need basic Python: variables, functions, dictionaries, loops, and imports. When this lesson introduces a class or a library-specific term, its role is explained below.
Use Python 3.12. These examples were executed on Windows with Python 3.12.4. Use the shown interpreter consistently when installing packages and running code. The default no-download command is tested. External pretrained-model training and GPU/QLoRA execution are optional and not claimed as verified runs.
You do not need to run an earlier article to make this part execute. The complete files and data used here are supplied below. The series order is for learning the ideas progressively.
Understand the idea before the code
Hugging Face hosts model repositories and develops several Python libraries. Transformers loads supported pretrained model architectures and tokenizers. PEFT provides implementations of parameter-efficient training methods such as LoRA. bitsandbytes provides specialized quantization/optimizer implementations for supported hardware. Installing one of these libraries does not download every model or make a compatible accelerator appear.
A model ID identifies a model repository, while a revision identifies a version such as a branch or immutable commit. The base model is the original pretrained parameter set. A saved LoRA adapter contains the added adaptation values/configuration and still needs its compatible base. A tokenizer converts text to token IDs. A chat template formats user/assistant messages using the markers expected by that model.
An attention mask tells the model which input positions are real rather than padding; it is different from the label mask, which chooses which target positions contribute to loss. The recipe uses the tokenizer's chat template and checks that the prompt token prefix matches the complete conversation before masking labels.
A GitHub repository is the versioned project folder online. A branch is a moving line of work; a commit identifies one saved version; a tag is a named reference often used for a release. An .ipynb notebook is a document containing Markdown explanations, code cells, and optionally outputs. GitHub displays a preview; it does not run those cells for readers.
A picture of this part's idea
This picture introduces the mechanism. The numbers and assertions in the program below are the actual experiment.
Create the lesson folder
Create this folder and keep the filenames exactly as shown. In particular, a local module is an ordinary file beside the program, not an extra package you must guess how to install.
lesson-folder/
lesson.py
requirements.txt
Save requirements.txt
These are the external Python packages for this lesson. The local files above are included in the article; they are not pip packages.
# Separate optional environment. No bitsandbytes in the default CPU recipe.
torch==2.2.2
numpy==1.26.4
transformers==4.47.0
peft==0.14.0
accelerate==1.2.1
safetensors==0.4.5
python -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt
On Linux/macOS, use python3 -m venv .venv and .venv/bin/python -m pip install -r requirements.txt. Installation requires internet access. The default examples do not call a paid API.
Complete code for this article
Save this complete file as lesson.py
This is the complete program, including imports, data, function definitions, and the entry point.
"""Optional pretrained-model smoke training. No network use unless --run is explicit.
Compatibility target: torch 2.2.2, transformers 4.47.0, peft 0.14.0.
External model training and QLoRA are NOT part of the executed CPU lesson report.
"""
import argparse
import json
from pathlib import Path
def arguments(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--run",
action="store_true",
help="Download the base and run training explicitly",
)
parser.add_argument("--steps", type=int, default=5)
parser.add_argument("--model", default="HuggingFaceTB/SmolLM2-135M-Instruct")
parser.add_argument(
"--revision",
default="main",
help="Prefer a verified immutable model commit for publishing",
)
parser.add_argument(
"--qlora",
action="store_true",
help="Optional CUDA/bitsandbytes NF4 path, not CPU-verified",
)
parser.add_argument("--output", type=Path, default=Path("hf-output"))
args = parser.parse_args(argv)
if args.steps < 1:
parser.error("--steps must be positive")
return args
def build_example(tokenizer, prompt, answer):
import torch
prefix = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}], tokenize=True, add_generation_prompt=True
)
full = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}, {"role": "assistant", "content": answer}],
tokenize=True,
add_generation_prompt=False,
)
if full[: len(prefix)] != prefix:
raise ValueError(
"Chat template prefix differs; inspect token boundaries before training."
)
if len(full) <= len(prefix):
raise ValueError("No supervised response tokens.")
if len(full) > 256:
raise ValueError("Example exceeds this smoke recipe sequence budget.")
ids = torch.tensor([full])
labels = ids.clone()
labels[:, : len(prefix)] = -100
return {"input_ids": ids, "attention_mask": torch.ones_like(ids), "labels": labels}
def main(argv=None):
args = arguments(argv)
if not args.run:
print(
"No model downloaded. Read the guide, install requirements.txt, then use --run."
)
return
import torch
import transformers, peft
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
torch.manual_seed(42)
if args.qlora and not torch.cuda.is_available():
raise RuntimeError(
"--qlora requires the supported CUDA/bitsandbytes setup described in the guide."
)
tokenizer = AutoTokenizer.from_pretrained(args.model, revision=args.revision)
options = {"revision": args.revision}
if args.qlora:
compute = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
options.update(
quantization_config=BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=compute,
),
device_map={"": 0},
)
else:
options["torch_dtype"] = torch.float32
base = AutoModelForCausalLM.from_pretrained(args.model, **options)
if not args.qlora:
base.to("cuda" if torch.cuda.is_available() else "cpu")
train = [
(
"Route this question: how do I append to a Python list?",
'{"category":"python"}',
),
("Route this question: explain a Python dictionary.", '{"category":"python"}'),
(
"Route this question: Linux file permissions are wrong.",
'{"category":"linux"}',
),
("Route this question: a Linux service stopped.", '{"category":"linux"}'),
("Route this question: ESP32 camera cannot connect.", '{"category":"iot"}'),
("Route this question: read a GPIO sensor.", '{"category":"iot"}'),
]
held_out = [
("Route this question: package import fails in Python.", "python"),
("Route this question: inspect a Linux process.", "linux"),
("Route this question: Raspberry Pi wifi setup.", "iot"),
]
def evaluate(model):
model.eval()
rows = []
for prompt, expected in held_out:
ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
output = model.generate(
ids,
attention_mask=torch.ones_like(ids),
max_new_tokens=48,
do_sample=False,
pad_token_id=tokenizer.eos_token_id,
)
response = tokenizer.decode(
output[0, ids.shape[1] :], skip_special_tokens=True
)
try:
correct = json.loads(response).get("category") == expected
except (ValueError, AttributeError):
correct = False
rows.append(
{
"prompt": prompt,
"expected": expected,
"response": response,
"exact_json_category": correct,
}
)
return rows
before = evaluate(base)
if args.qlora:
base = prepare_model_for_kbit_training(base)
model = get_peft_model(
base,
LoraConfig(
r=8,
lora_alpha=16,
lora_dropout=0.0,
bias="none",
target_modules="all-linear",
task_type="CAUSAL_LM",
),
)
model.print_trainable_parameters()
model.config.use_cache = False
opt = torch.optim.AdamW([p for p in model.parameters() if p.requires_grad], lr=2e-4)
model.train()
losses = []
for step in range(args.steps):
prompt, answer = train[step % len(train)]
batch = {
k: v.to(model.device)
for k, v in build_example(tokenizer, prompt, answer).items()
}
loss = model(**batch).loss
if not torch.isfinite(loss):
raise RuntimeError("Nonfinite loss; stop and inspect this configuration.")
opt.zero_grad()
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
opt.step()
losses.append(float(loss.detach()))
print("step", step + 1, "loss", losses[-1])
after = evaluate(model)
args.output.mkdir(parents=True, exist_ok=True)
model.save_pretrained(args.output)
tokenizer.save_pretrained(args.output)
report = {
"base": args.model,
"requested_revision": args.revision,
"resolved_revision": getattr(base.config, "_commit_hash", None),
"steps": args.steps,
"seed": 42,
"learning_rate": 2e-4,
"rank": 8,
"alpha": 16,
"qlora": args.qlora,
"torch": torch.__version__,
"transformers": transformers.__version__,
"peft": peft.__version__,
"loss": losses,
"base_outputs": before,
"adapted_outputs": after,
"limitation": "Tiny smoke experiment, not a deployment-quality evaluation.",
}
(args.output / "experiment.json").write_text(
json.dumps(report, indent=2), encoding="utf-8"
)
print(json.dumps(report, indent=2))
if __name__ == "__main__":
main()
Run it and compare the result
Run commands from the lesson folder. In the commands below, python means the environment you installed into. On Windows, substitute .venv\Scripts\python.exe; on Linux/macOS substitute .venv/bin/python. Do not create the environment with one interpreter and run the lesson with another.
Run 1: python lesson.py
python lesson.py
Captured standard output:
No model downloaded. Read the guide, install requirements.txt, then use --run.
The displayed output was captured from the supplied default program in a separate folder, with no project-level module on its import path. It is a reproducible demonstration on these examples, not a claim about arbitrary inputs. To deliberately try external training after installing requirements, use python lesson.py --run --steps 5. This downloads the selected public model and runs a small smoke experiment. The optional --qlora branch has additional supported-hardware and bitsandbytes requirements; neither external training branch is represented as executed in this edition.
Walk through what just happened
1. Identify the exact artifacts
A model ID identifies a repository, while a revision identifies a version. An adapter still depends on the compatible base and tokenizer; it is not a complete independent chatbot.
2. Check the chat boundary
build_example applies the tokenizer’s chat template to the prompt and full exchange. It verifies their shared token prefix before masking prompt labels, because a template mismatch can otherwise train the wrong positions.
3. Choose the memory path explicitly
The default recipe uses an ordinary float base. The optional --qlora branch requires a supported CUDA/bitsandbytes installation and is not part of the CPU training verification.
4. Publish the project, not only one notebook
Keep the notebook’s support assets and requirements with it. After publishing to your chosen repository, use the supplied link-generation helper and verify each normal GitHub blob URL in its matching article.
Change one thing yourself
Run the file without --run and inspect --help. Read the training and data code before deliberately enabling a model download.
If your result is different
Installing Transformers/PEFT does not install a model or guarantee compatible GPU support. The release report distinguishes the tested no-download path from unexecuted external training.
Check the complete imports and filenames first. Then compare your input and command with the shown run. Numeric results may differ slightly between platforms; the program's assertions check the intended relationships rather than relying on a decorative screenshot.
What you can now explain
Inspect a complete optional pretrained-model recipe and publish reproducible notebook links. You can now publish a reproducible starting point and choose a further experiment based on a specific task and held-out evidence.
Technical reference: PyTorch learning basics. This project and its wording are original CodeTips Academy material.