From model weights to a working API
Follow tokens through generation. Reach private GPU inference in week 3, then build a stable, authenticated gateway with streaming.
Learn how models run, serve requests on a GPU, and deploy and monitor your service.
Dates coming soon.
I’ve taught thousands of engineers and engineering leaders at Interview Kickstart.
Your instructor’s background

THE PROJECT
Build the API around a model, control who can use it, and see what happens when requests arrive. Start locally, then take the same ideas to a GPU.
Docker brings up the engine, controller, gateway, Prometheus and Grafana. Inspect request rate, errors, in-flight requests and latency.
Recorded check · October 26, 2026
Synthetic engine; no GPU inference or performance claim.
Feedback from earlier classes and coaching sessions.
Explain inference in one sentence.
Send the example prompt to follow its path.
Interactive illustration, not a live model. Streaming is covered in the later labs.
Build and operate your own bounded inference platform, one problem at a time. Reuse vLLM as the inference engine and authorized compute, then own the gateway, deployment records, limits and recovery around them.
Follow tokens through generation. Reach private GPU inference in week 3, then build a stable, authenticated gateway with streaming.
Measure latency, throughput, and memory under load. Find the bottleneck before choosing a bigger GPU or a different engine.
Record a deployment specification and durable status. Deploy, stop, restart and roll back while explaining the control plane and request path.
Build a measured cost model from provisioned time and delivered tokens. Review eligibility for your own truthful OpenRouter application, with gaps made explicit.
The reference code is available now. Local fixture checks cover the platform and monitoring; real GPU and live-cluster validation are still pending.
Distinguish downloaded weights, a loaded model and inference. Then use FastAPI and Uvicorn to introduce the HTTP boundary; the web server and inference engine have different jobs.
Week 3 targets one compatible authorized host and actual generation through a private connection. Check environment, memory, readiness and spending before adding more infrastructure.
Keep the client API stable as the engine changes. Check missing, wrong and correct credentials; use authenticated HTTPS when sharing.
Measure delivered tokens and latency under bounded load. Add deploy/status/stop controls and durable metadata, separating deployment management from the request-serving path.
Use evidence to locate a failure, recover a worker and roll back deliberately. A one-GPU rollback may interrupt service. Finish with economics and a reviewed provider-readiness checklist.
Compare the public workflows of Baseten, Fireworks and Modal: packaging a model, deploying a revision, exposing an endpoint, observing demand and understanding cost. These are inspirations for our experiments; our architecture does not describe their private implementations.
Fine-tuning, extra engines, custom schedulers, Kubernetes and autoscaling are optional deep dives. Kubernetes manifests are implemented and schema-checked; live-cluster execution is pending. They are not prerequisites for the core platform.
For software engineers who know Python and want to understand model serving. We start with a small CPU model; you don’t need prior inference experience.
Weekly exercises and reference code are available. The track is still being rehearsed and validated: real GPU and live Kubernetes walkthroughs remain pending. The four CPU foundation samples below are a separate sequence, independent of week numbers.
Downloading weights did not create an answering system. Model weights, tokenizer, inference, CPU/GPU roles; no GPU purchase.
A local model runner and a request-path sketch; explain disk files, loaded model, prompt tokens and output. Observe what stops when the process exits.
Another program cannot type into our terminal; a basic handler does not solve efficient model serving. HTTP, client/server boundary, web server vs inference engine, prefill/decode and a small KV-cache/batching illustration.
Wrap the model in a small HTTP service, make a Python request, then compare that code with an engine API. Predict where multi-user serving gets difficult.
The laptop is constrained; a rented machine alone is not a running inference service. VRAM, weights vs request cache, driver/runtime, local/remote environments, load/warmup/readiness.
Inspect an authorized borrowed host or rent after reviewing costs; SSH in, verify a compatible environment, launch vLLM and get a private real-model response. Save a minimal deployment record and stop paid compute after the lab.
Users should not know a worker's changing address or SSH into it; exposing a raw engine is insufficient. Request path through gateway to engine; API compatibility, TLS, client localhost, separation of platform and engine responsibilities.
Add a small API gateway with a stable model alias, authentication and streaming; use a Python client through a private connection first and authenticated HTTPS for sharing. Check missing/wrong/correct keys.
A successful single request does not establish capacity, and unbounded demand can make everyone wait. Throughput vs latency, continuous batching, context/sequence limits, admission and backpressure; optional custom queue internals.
Run bounded repeated load experiments; measure actual tokens, first-text/total times and errors. Add explicit request/input/output/concurrency limits; change one engine setting and explain the tradeoff.
Shell commands and in-memory state do not give another operator a reproducible deployment. Control plane vs request-serving data plane, desired vs observed status, images vs containers, persistent model storage and configuration.
Turn the recorded configuration into a versioned deployment specification; package with a pinned engine image; add a small control API/CLI for deploy, status and stop with durable deployment metadata. Prove stop/start and a fresh generation.
Users report slowness or failures, but a live process does not explain them. Evidence-led diagnosis, request vs model timing, bounded retries, restart policy; use the existing engine's mechanisms rather than inventing a scheduler.
Add a small operational view of request counts, latency, errors and workload, plus useful logs and health/readiness. Reproduce one failure, locate its layer and recover under supervision.
A model/config update or worker failure can break a stable endpoint. Model version vs deployment revision, routing/health, reproducibility, rollback and capacity during change; a second real replica is optional if resources allow.
Record a new deployment revision, test readiness, change routing deliberately and roll back. Demonstrate recovery after stopping a worker. Add a minimal provider-catalog adapter based on the verified integration schema, and compare our platform's responsibilities with public provider workflows.
Working software is not yet a sustainable service or a substantiated provider offering. Provisioned vs busy time, input/output pricing, cost per million tokens, utilization, cold starts, storage/egress/platform overhead, margins and capacity commitments.
Present your accumulated platform, a deployment/evidence record and a measured cost model. Review actual OpenRouter criteria and prepare/submit your own truthful provider application when eligible. Retain application evidence without claiming acceptance.
A standalone CPU sequence: model runner, HTTP API, clients, then private access. These retain their original sample numbering and code; sample 3 is a clients exercise, separate from the new week 3 GPU lab.
Week 9 brings together your deployment evidence and measured economics: provisioned and idle GPU time, input and output tokens, storage, networking and other operating costs. Decide who the service is for and what capacity you can sustain.
Review current OpenRouter criteria and prepare or submit your own truthful provider application when eligible. Each learner owns their account, application, costs and post-course operating plan. If requirements cannot be met, record the gaps and leave submission incomplete. Eligibility and acceptance are not guaranteed; submission does not establish revenue or commercial readiness.
Find a bottleneck or a missing feature in our service.
Work through the code, then try the tool or technique that addresses it.
Measure whether it helped and work through your questions together.
Get candid feedback on your code, technical decisions, and career questions during the cohort.
Talk through technical questions and get feedback on how you explain your work.
Compare approaches, work through labs together, and stay connected after the course.
Where there’s a fit, I can help through my network. Referrals depend on the role and relationship and don’t guarantee an interview or job.
I’d like to bring in more instructors for mock interviews, feedback, and referrals. For now, I teach and support this cohort myself.

AI Inference & Post Training @ Baseten
I work on AI inference and post-training at Baseten. Before that, I worked at Databricks, Google, and Salesforce. I studied electrical engineering at Harvard.
Work, teaching & education

Over about three years at Interview Kickstart, I’ve taught thousands of engineers and engineering leaders, including classes on AI agents and voice systems.
An independent course by Samwel Emmanuel.
THE FIRST COHORT
A proposed nine-week path through your own inference platform, with live classes, guided labs, recordings and Samwel there for your questions. Reference code and weekly exercises are available; GPU walkthrough validation is pending.
$1,499 upfront or two payments of $749.50, 30 days apart. Enrollment dates coming soon.
Try a sample lessonDevelopers comfortable editing small Python programs, using a terminal and making an HTTP request. You don’t need prior CUDA, Docker or cloud-operations experience.
Week 1 starts on a laptop CPU. Week 3 targets private real-model generation with vLLM on an authorized, compatible borrowed GPU or rented host. Review the budget before renting; compute, storage and networking costs are separate from tuition. Without GPU access, you can use an authorized instructor service for client and measurement work and review server evidence; that observation path does not prove you provisioned a GPU.
Your own bounded inference platform: one model alias, one host and one active engine deployment by default. Build the gateway, deploy/status/stop controls, durable deployment records, request limits, measurements and rollback around vLLM. The engine runs the model; your platform manages access and deployment. Docker, Grafana and Prometheus examples are provided; extra replicas are optional.
Fine-tuning, extra serving engines, GPU kernels, Kubernetes and autoscaling are optional extensions. The nine-week core focuses on understanding and operating one model service with vLLM. It does not require a completed LoRA lab.
The nine-week track is a draft being authored and checked week by week. The four existing foundation samples are a separate CPU sequence, with their original numbering and source. They are not evidence that weeks 1–4 of the new track are validated. Weekly exercises, platform reference code and Docker/Grafana/Prometheus configuration are available. Local synthetic-engine checks have passed; real GPU and live Kubernetes validation remain pending. The final syllabus and weekly commitment will be confirmed before enrollment.
No. Week 9 ends with a measured cost model and an eligibility review for your own service and application. Inspect the actual current provider form and official requirements, then submit truthful information only when eligible. Public integration documentation has been reviewed, but live application fields and commercial screening remain unverified. If mandatory requirements cannot be met, record the gaps and leave submission incomplete. Submission does not guarantee listing, acceptance, revenue or commercial readiness.
Each learner owns their service, account, application information, capacity costs and consent to submission. Establish a budget and post-course operating plan before promising ongoing availability. The instructor’s demonstration service does not replace your operating responsibilities.
The existing foundation walkthroughs were tested on macOS. Windows and Linux setup commands for those samples are not yet verified. The planned GPU path uses a compatible Linux/NVIDIA host with a separately verified runtime; laptop CPU dependencies do not establish GPU compatibility.
The cohort will include recordings. I’ll share the schedule and how long you’ll have access to the recordings before enrollment opens.
I’m still setting the dates. Tuition for the founding cohort is $1,499. I’ll share the schedule and weekly time commitment before enrollment opens.
Interview preparation, feedback on your work, and introductions or referrals where there’s a fit. Referrals don’t guarantee interviews or jobs. A wider instructor community and dedicated mock interviews are future plans.
Yes. The organizations in my bio and the platforms discussed do not sponsor or endorse this course. Baseten, Fireworks and Modal are public workflow inspirations; our teaching architecture does not describe their private implementations.