Inference/Foundation sample 4 of 4

Call your model from somewhere else.

Leave the server at home. Move the client to a cellular hotspot. Keep generating through a private network, then require a key before the model does any work.

Foundation samples · CPU inferencemacOS walkthroughDownload all four labs

This is part of the original four-sample CPU sequence, separate from the redesigned nine-week track. Sample numbers are not week numbers.

Two separate problems.

First, the client needs a network path to the server. Second, the application needs to decide whether to accept the request. A valid API key can’t repair a missing route. A working route doesn’t mean every request should run the model.

Request flowThe result we’re building
Client on a hotspot
    ↓ private Tailscale connection
Server at home :8000
    ↓ X-API-Key check
    ├─ missing or wrong → HTTP 401
    └─ correct → tokenize → generate → return text

This walkthrough uses two Macs, internet access on both, and a phone hotspot using cellular data. Finish the trusted-Wi-Fi experiment in lesson 3 first. Keep the server awake, plugged in, and connected at home. Only the client changes networks.

Prove the path before adding a key.

Start with lesson 2’s server listening on 0.0.0.0:8000, as in lesson 3. Install the native app using Tailscale’s macOS guide on both devices. Connect both to the same tailnet—the private network your account belongs to—and complete any required device approval.

Check that both devices are online and that the tailnet access policy allows the client to reach the server on port 8000. Identify the server by its device name and copy its Tailscale IPv4 address. Use the server’s address, not the client’s.

TerminalClient Mac · replace both example addresses
export HOME_WIFI_IP="192.168.1.25"
export TAILSCALE_IP="100.101.102.103"
export MODEL_SERVER_URL="http://$TAILSCALE_IP:8000"
curl --connect-timeout 5 --max-time 5 "$MODEL_SERVER_URL/health"

On home Wi-Fi, first confirm that the Tailscale address serves health, browser docs at http://SERVER_TAILSCALE_IP:8000/docs, and a generation request. Then move only the client to the cellular hotspot. Keep Tailscale connected on both Macs.

TerminalClient Mac · now on the hotspot
curl --connect-timeout 5 --max-time 5 "http://$HOME_WIFI_IP:8000/health"
curl --connect-timeout 5 --max-time 5 "$MODEL_SERVER_URL/health"
curl --connect-timeout 5 --max-time 60 \
  "$MODEL_SERVER_URL/generate" -H "Content-Type: application/json" \
  -d '{"prompt":"Hello from the hotspot.","max_new_tokens":30}'

Normally the old home address fails while the Tailscale address works. Confirm the matching request in your server’s logs: private address ranges can overlap, so an arbitrary response from the old address isn’t proof you reached your server. You’ve changed the network path; the model files and inference code haven’t moved.

The private connection works. Why add an application key?

Check your reasoning

Network access and permission to generate are different decisions. The application can reject a request even when its sender can reach the server. This shared-key exercise demonstrates that gate; it doesn’t create individual user accounts.

Now change who gets to generate.

Keep the client on the hotspot. On the server, stop lesson 2 with Control-C. Lesson 4 contains a separate server snapshot so you can compare the change without rewriting the earlier lab.

Create a fresh random key in a private server terminal. Save and transfer it using a private channel you control, such as your password manager. Keep it out of source files, screenshots, URLs, and messages.

TerminalServer Mac · private terminal
python -c 'import secrets; print(secrets.token_urlsafe(32))'

In the server’s serving terminal, enter that key through a hidden prompt and start the protected snapshot. These commands use the activated Python environment from the lab folder.

TerminalServer Mac · serving terminal
export MODEL_SERVER_API_KEY="$(python -c 'import getpass; print(getpass.getpass("Model server API key: "))')"
python -m uvicorn server:app --app-dir lessons/04-private-network --host 0.0.0.0 --port 8000

Enter the same key through the same hidden prompt in the client’s request terminal. Don’t enable shell tracing. The server reads its key at startup; changing the environment later requires restarting it. Missing configuration stops startup before the model loads.

Make three requests. Predict the status codes.

TerminalClient Mac · hotspot · missing key, wrong key, correct key
export MODEL_SERVER_API_KEY="$(python -c 'import getpass; print(getpass.getpass("Model server API key: "))')"
export MODEL_SERVER_URL="http://$TAILSCALE_IP:8000"

curl -w '\nHTTP %{http_code}\n' --max-time 60 \
  "$MODEL_SERVER_URL/generate" -H "Content-Type: application/json" \
  -d '{"prompt":"Say hello.","max_new_tokens":10}'

curl -w '\nHTTP %{http_code}\n' --max-time 60 \
  "$MODEL_SERVER_URL/generate" -H "Content-Type: application/json" \
  -H "X-API-Key: deliberately-wrong-demo-value" \
  -d '{"prompt":"Say hello.","max_new_tokens":10}'

curl -w '\nHTTP %{http_code}\n' --max-time 60 \
  "$MODEL_SERVER_URL/generate" -H "Content-Type: application/json" \
  -H "X-API-Key: $MODEL_SERVER_API_KEY" \
  -d '{"prompt":"Say hello.","max_new_tokens":10}'

What should those three requests do?

Check your reasoning

Expect 401, 401, then 200. The first two return the same “Invalid or missing API key” detail and never tokenize or generate. A 401 is evidence you reached the application. A timeout is a different problem.

Read require_api_key in the protected server.py. FastAPI checks the header through a dependency before the generation handler runs. The comparison uses secrets.compare_digest. After acceptance, the prompt-to-text flow is the same as before.

Use the clients. Check the evidence.

On the client, copy the lab source if needed. Its Python client requires Requests, not the model dependencies. In the terminal holding your URL and key, run:

TerminalClient Mac · project root
bash lessons/04-private-network/clients/curl_client.sh
python lessons/04-private-network/clients/python_client.py
node lessons/04-private-network/clients/javascript_client.mjs

These clients send the key in X-API-Key and refuse to run if it isn’t configured locally. Lesson 3’s clients don’t send that header, so they receive 401 from the protected server.

Compare /metrics before and after the three direct tests. With no other traffic, HTTP counts grow by two 401 responses and one 200, while generation attempts and successes grow by only one. The model should not spend compute on the rejected requests.

In /docs, use Authorize to supply the key, run a generation, then remove authorization and try again. Avoid screenshots of this page while authorized: generated request snippets can reveal the key.

What have we actually secured?

Tailscale encrypts the private device-to-device transport. The API key is an application gate; it doesn’t encrypt HTTP. Health, docs, OpenAPI, and metrics remain key-free to anyone with network access. Keep this demonstration private and don’t expose port 8000 publicly.

Everyone sharing this key has the same access. We haven’t built per-user identity, revocation, quotas, public HTTPS termination, or overload protection. When finished, stop the server, remove docs authorization, and run unset MODEL_SERVER_API_KEY in both terminals.

Now we can ask about capacity.

You’ve followed the whole path: model files, a running process, an API, separate clients, and a private remote request with an authorization check. The next question is how much useful work this CPU service can handle—and whether a GPU would help enough to justify its cost. That capacity work and later production-serving lessons are planned next.