What happens when
you load a model?

Downloading and loading a model

When you download a model, you get files: learned weights, a configuration, and the tokenizer’s rules. They sit on disk. Downloading those files doesn’t start anything.

Your Python program loads the tokenizer and model into memory. Now you have a running process that can turn text into tokens, generate more tokens, and turn the result back into text.

DISK                  RUNNING PYTHON PROCESS
model files   ──────▶  tokenizer + model in memory
                               │
                               ▼
prompt → format → tokenize → generate → decode → text

Reusing a loaded model

In the first lab, model loading happens before the prompt loop. Each new prompt repeats the text-processing and generation steps. It reuses the model that is already loaded.

tokenizer, model = load_from_disk()

while True:
    prompt = read_prompt()
    if prompt == "quit":
        break

    tokens = tokenize(format_prompt(prompt))
    new_tokens = generate(model, tokens)
    print(decode(new_tokens))

This pseudocode shows the main steps; it won’t run as written. In the full lab, we’ll use a small open model and walk through the Python code for each step.

What happens when the process stops?

When the process exits, its memory is released. The files on disk remain. Next time, you load those files again—you don’t need to download them again.

Calling the model from another program

So far, we’ve entered prompts directly in the terminal. In the next lesson, we’ll wrap the same code in an HTTP service so another program can send prompts and receive answers.

GET /health

POST /generate
{"prompt": "Say hello."}