Part 3 — Full Fine-Tuning Versus a Frozen Encoder

Download the matching self-contained notebook · Read the notebook preview

What you will build

The next experiment needs an actual starting model. The program below explicitly trains a small source router first, then copies that same starting point for both adaptation methods. There is no downloaded pretrained LLM hidden in prepare.

The source corpus includes the topic keywords. Our held-out target examples test changed phrasing, not completely unseen subject knowledge. Both methods may already work well, which is a legitimate reason not to fine-tune further.

By the end: Compare changing the full tiny router with updating only its output head.

What you need before starting

You need basic Python: variables, functions, dictionaries, loops, and imports. When this lesson introduces a class or a library-specific term, its role is explained below.

Use Python 3.12. These examples were executed on Windows with Python 3.12.4. Use the shown interpreter consistently when installing packages and running code. This is an original small CPU teaching experiment, not a downloaded general-purpose LLM.

You do not need to run an earlier article to make this part execute. The complete files and data used here are supplied below. The series order is for learning the ideas progressively.

Understand the idea before the code

A tensor is a numeric array. Its shape gives the length of each dimension. A batch of 36 questions with 26 features is a two-dimensional tensor shaped [36, 26]: rows are examples and columns are features. A batch is the set of examples processed together in one calculation.

A layer is a calculation inside a model. A linear layer computes weighted sums plus biases. A bias is an additional learned offset. An activation, such as tanh, applies a nonlinear transformation so stacked layers can represent more than one linear map.

The encoder in our router turns input features into a hidden vector. “Hidden” means internal intermediate values; it is not a hidden file or human reasoning. The head is the final layer that returns one score per class. These raw scores are logits; they are not probabilities. Applying softmax converts logits into nonnegative values summing to one, but those values are not automatically well-calibrated confidence estimates.

Freezing a parameter means preventing its gradient-based update. The frozen layer still runs. A backbone is the main feature-producing part of a model; our tiny encoder plays that role. Head-only training changes the output layer while keeping that encoder fixed.

A picture of this part's idea

This picture introduces the mechanism. The numbers and assertions in the program below are the actual experiment.

Create the lesson folder

Create this folder and keep the filenames exactly as shown. In particular, a local module is an ordinary file beside the program, not an extra package you must guess how to install.

lesson-folder/
  lesson.py
  requirements.txt

Save requirements.txt

These are the external Python packages for this lesson. The local files above are included in the article; they are not pip packages.

torch==2.2.2
numpy==1.26.4
matplotlib==3.9.1

python -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt

On Linux/macOS, use python3 -m venv .venv and .venv/bin/python -m pip install -r requirements.txt. Installation requires internet access. The default examples do not call a paid API.

Complete code for this article

Save this complete file as lesson.py

This is the complete program, including imports, data, function definitions, and the entry point.

import copy
import re
import torch
from torch import nn
from torch.nn import functional as F

LABELS = ["python", "linux", "iot"]

GROUPS = [
    ("python", ["python", "list", "dictionary", "function", "script", "package"]),
    ("linux", ["linux", "shell", "permission", "service", "process", "terminal"]),
    ("iot", ["esp32", "camera", "sensor", "gpio", "wifi", "raspberry"]),
]


def seed(value=42):
    torch.manual_seed(value)
    torch.set_num_threads(1)


def records():
    """One group per topic keyword; all paraphrases stay in the same split."""
    rows = []
    for label, words in GROUPS:
        for i, word in enumerate(words):
            split = "train" if i < 4 else "validation" if i == 4 else "test"
            for template in ["help with {}", "please explain {}", "debug my {}"]:
                rows.append(
                    dict(
                        text=template.format(word),
                        label=label,
                        group=f"{label}-{word}",
                        split=split,
                    )
                )
    return rows


def tokens(text):
    return re.findall("[a-z0-9]+", text.lower())


def make_vocab(rows):
    return {
        word: i
        for i, word in enumerate(sorted({w for r in rows for w in tokens(r["text"])}))
    }


def encode(rows, vocab):
    x = torch.zeros(len(rows), len(vocab))
    for i, row in enumerate(rows):
        for word in tokens(row["text"]):
            if word in vocab:
                x[i, vocab[word]] += 1
    return (x, torch.tensor([LABELS.index(r["label"]) for r in rows]))


class Router(nn.Module):
    def __init__(self, width, hidden=16):
        super().__init__()
        self.encoder = nn.Linear(width, hidden)
        self.head = nn.Linear(hidden, 3)

    def forward(self, x):
        return self.head(torch.tanh(self.encoder(x)))


def train(model, x, y, steps=100, lr=0.03):
    params = [p for p in model.parameters() if p.requires_grad]
    optimizer = torch.optim.Adam(params, lr=lr)
    losses = []
    model.train()
    for _ in range(steps):
        optimizer.zero_grad(set_to_none=True)
        loss = F.cross_entropy(model(x), y)
        loss.backward()
        optimizer.step()
        losses.append(float(loss.detach()))
    return losses


def accuracy(model, x, y):
    model.eval()
    with torch.no_grad():
        return float((model(x).argmax(-1) == y).float().mean())


def prepare():
    """Pretrain on an explicitly separate, synthetic source corpus."""
    seed()
    target = records()
    source = []
    for label, words in GROUPS:
        for word in words:
            source.append(dict(text=f"general guide {word}", label=label))
    vocab = make_vocab(source + [r for r in target if r["split"] == "train"])
    sx, sy = encode(source, vocab)
    base = Router(len(vocab))
    train(base, sx, sy, steps=45, lr=0.02)
    return (base, vocab, target)


def method(base, name):
    model = copy.deepcopy(base)
    if name == "head":
        model.encoder.requires_grad_(False)
    return model


def counts(model):
    return {
        "trainable": sum((p.numel() for p in model.parameters() if p.requires_grad)),
        "total": sum((p.numel() for p in model.parameters())),
    }


def main():
    import torch

    base, vocab, rows = prepare()
    train_x, train_y = encode([r for r in rows if r["split"] == "train"], vocab)
    val_x, val_y = encode([r for r in rows if r["split"] == "validation"], vocab)
    results = {}
    for name in ["full", "head"]:
        model = method(base, name)
        before = {
            key: value.detach().clone() for key, value in model.named_parameters()
        }
        history = train(model, train_x, train_y, steps=80, lr=0.01)
        changed = [
            key
            for key, value in model.named_parameters()
            if not torch.equal(before[key], value.detach())
        ]
        results[name] = {
            "counts": counts(model),
            "changed": changed,
            "validation": accuracy(model, val_x, val_y),
            "history": history,
        }
        print(name, results[name]["counts"], changed, results[name]["validation"])
        if name == "head":
            assert all((key.startswith("head.") for key in changed))
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(figsize=(7, 3))
    for name, result in results.items():
        ax.plot(result["history"], label=name)
    ax.set(
        xlabel="Optimizer step",
        ylabel="Training cross-entropy",
        title="Training fit is not test quality",
    )
    ax.legend()
    plt.savefig("result-01.png", bbox_inches="tight")
    print("Saved result-01.png")
    plt.show()


if __name__ == "__main__":
    main()

Run it and compare the result

Run commands from the lesson folder. In the commands below, python means the environment you installed into. On Windows, substitute .venv\Scripts\python.exe; on Linux/macOS substitute .venv/bin/python. Do not create the environment with one interpreter and run the lesson with another.

Run 1: python lesson.py

python lesson.py

Captured standard output:

full {'trainable': 483, 'total': 483} ['encoder.weight', 'encoder.bias', 'head.weight', 'head.bias'] 1.0
head {'trainable': 51, 'total': 483} ['head.weight', 'head.bias'] 1.0
Saved result-01.png

The program saves result-01.png in the working folder. This is output from the shown code, not a separately invented chart.

The displayed output was captured from the supplied default program in a separate folder, with no project-level module on its import path. It is a reproducible demonstration on these examples, not a claim about arbitrary inputs.

Walk through what just happened

1. Build the encoder and head

Router converts word-count features into 16 hidden values, then into three class scores. All vocabulary and model definitions are included in the file.

2. Create the common starting point

prepare trains on the declared source examples. Each later method receives a deepcopy so the first experiment cannot alter the second starting model.

3. Freeze deliberately

Head-only training sets the encoder’s requires_grad flags to False. The encoder still participates in the forward calculation.

4. Inspect actual changes

The program snapshots named parameters before training and prints which changed. It also saves a training-loss plot; that curve is training fit, not a held-out quality claim.

Change one thing yourself

Print every parameter’s name and requires_grad value, then predict the changed-name list before running head-only training.

If your result is different

model.eval controls evaluation behavior such as dropout; it does not replace requires_grad=False for freezing parameters.

Check the complete imports and filenames first. Then compare your input and command with the shown run. Numeric results may differ slightly between platforms; the program's assertions check the intended relationships rather than relying on a decorative screenshot.

What you can now explain

Compare changing the full tiny router with updating only its output head. Next we will build on this result in Part 4.

Technical reference: PyTorch learning basics. This project and its wording are original CodeTips Academy material.