Build your own AI inference platform.

Learn how models run, serve requests on a GPU, and deploy and monitor your service.

Dates coming soon.

I’ve taught thousands of engineers and engineering leaders at Interview Kickstart.

4.9/5 · IK teaching rating

Your instructor’s background

THE PROJECT

One service. From request to recovery.

Build the API around a model, control who can use it, and see what happens when requests arrive. Start locally, then take the same ideas to a GPU.

  1. ClientSend a prompt
  2. GatewayAuthenticate · limit · stream
  3. EngineFixture locally · vLLM on GPU

A local stack you can run

Docker brings up the engine, controller, gateway, Prometheus and Grafana. Inspect request rate, errors, in-flight requests and latency.

Recorded check · October 26, 2026
Synthetic engine; no GPU inference or performance claim.

Streamed requests completed
6
Prometheus scrape
up
Grafana panels
7
Grafana datasource
healthy
What is available and what still needs validation?
Available now
Four CPU foundation samples, nine weekly coding exercises and the platform reference implementation.
Tested locally
Docker stack, gateway streaming and authentication, deployment controls, Prometheus scraping and Grafana provisioning.
Validation pending
Real GPU engine execution, Brev/Runpod walkthroughs and a live Kubernetes cluster. Kubernetes manifests have passed schema checks only.
Code and exercise status · Exact verification evidence

What learners say

Feedback from earlier classes and coaching sessions.

4.9/5 · IK teaching rating

See how I teach. Start with a free lesson.

Run an open model

Follow a request.

Prompt

Explain inference in one sentence.

  1. ClientSend a prompt
  2. APIValidate the request
  3. ModelGenerate tokens
Response
Illustrated response appears after running the example.

Send the example prompt to follow its path.

Interactive illustration, not a live model. Streaming is covered in the later labs.

Understand the system. Make the engineering decisions.

Build and operate your own bounded inference platform, one problem at a time. Reuse vLLM as the inference engine and authorized compute, then own the gateway, deployment records, limits and recovery around them.

From model weights to a working API

Follow tokens through generation. Reach private GPU inference in week 3, then build a stable, authenticated gateway with streaming.

Know what limits your capacity

Measure latency, throughput, and memory under load. Find the bottleneck before choosing a bigger GPU or a different engine.

Deploy, diagnose, and recover

Record a deployment specification and durable status. Deploy, stop, restart and roll back while explaining the control plane and request path.

Decide whether the service should keep running

Build a measured cost model from provisioned time and delivered tokens. Review eligibility for your own truthful OpenRouter application, with gaps made explicit.

Tools follow the problem

The reference code is available now. Local fixture checks cover the platform and monitoring; real GPU and live-cluster validation are still pending.

Make a prompt produce an answer

Code & validation status
  • Python
  • PyTorch
  • Hugging Face Transformers

Distinguish downloaded weights, a loaded model and inference. Then use FastAPI and Uvicorn to introduce the HTTP boundary; the web server and inference engine have different jobs.

Reach a private GPU response

Code & validation status
  • SSH
  • Linux / NVIDIA GPU
  • vLLM
  • Brev / Runpod

Week 3 targets one compatible authorized host and actual generation through a private connection. Check environment, memory, readiness and spending before adding more infrastructure.

Give users a stable front door

Code & validation status
  • FastAPI gateway
  • Model alias
  • API keys
  • TLS
  • Streaming

Keep the client API stable as the engine changes. Check missing, wrong and correct credentials; use authenticated HTTPS when sharing.

Bound demand and reproduce deployments

Code & validation status
  • Docker / Compose
  • Load experiments
  • Request limits
  • Deployment records

Measure delivered tokens and latency under bounded load. Add deploy/status/stop controls and durable metadata, separating deployment management from the request-serving path.

Diagnose, recover and account for cost

Code & validation status
  • Grafana
  • Prometheus
  • Readiness
  • Rollback
  • Cost model

Use evidence to locate a failure, recover a worker and roll back deliberately. A one-GPU rollback may interrupt service. Finish with economics and a reviewed provider-readiness checklist.

Compare the public workflows of Baseten, Fireworks and Modal: packaging a model, deploying a revision, exposing an endpoint, observing demand and understanding cost. These are inspirations for our experiments; our architecture does not describe their private implementations.

Fine-tuning, extra engines, custom schedulers, Kubernetes and autoscaling are optional deep dives. Kubernetes manifests are implemented and schema-checked; live-cluster execution is pending. They are not prerequisites for the core platform.

Who this course is for

For software engineers who know Python and want to understand model serving. We start with a small CPU model; you don’t need prior inference experience.

A good starting point

  • You can read and change a small Python program.
  • You can work in a terminal.
  • You have a laptop for the labs.

Useful to know

  • Existing foundation samples use a macOS CPU walkthrough.
  • Week 3 needs an authorized, compatible Linux/NVIDIA host.
  • GPU rental, storage and networking are separate from tuition.
  • Approve a provider quote and shutdown plan before renting.

Nine weeks. One evolving platform.

Weekly exercises and reference code are available. The track is still being rehearsed and validated: real GPU and live Kubernetes walkthroughs remain pending. The four CPU foundation samples below are a separate sequence, independent of week numbers.

1Local model runnerWhat is actually running?Draft

Why don’t downloaded weights answer a prompt?

Downloading weights did not create an answering system. Model weights, tokenizer, inference, CPU/GPU roles; no GPU purchase.

Planned week outcome

A local model runner and a request-path sketch; explain disk files, loaded model, prompt tokens and output. Observe what stops when the process exits.

Python · PyTorch · Hugging Face Transformers

Open week 1 coding exercise →

2HTTP and engine boundariesFrom a model to a serviceDraft

How can another program use the model?

Another program cannot type into our terminal; a basic handler does not solve efficient model serving. HTTP, client/server boundary, web server vs inference engine, prefill/decode and a small KV-cache/batching illustration.

Planned week outcome

Wrap the model in a small HTTP service, make a Python request, then compare that code with an engine API. Predict where multi-user serving gets difficult.

FastAPI · Uvicorn · Python HTTP client

Open week 2 coding exercise →

3Private GPU generationAn engine on a borrowed GPUDraft

How do you turn a remote GPU into a running service?

The laptop is constrained; a rented machine alone is not a running inference service. VRAM, weights vs request cache, driver/runtime, local/remote environments, load/warmup/readiness.

Planned week outcome

Inspect an authorized borrowed host or rent after reviewing costs; SSH in, verify a compatible environment, launch vLLM and get a private real-model response. Save a minimal deployment record and stop paid compute after the lab.

SSH · NVIDIA GPU · vLLM

Open week 3 coding exercise →

4Gateway, authentication and streamingA usable front doorDraft

How can users reach one stable, protected endpoint?

Users should not know a worker's changing address or SSH into it; exposing a raw engine is insufficient. Request path through gateway to engine; API compatibility, TLS, client localhost, separation of platform and engine responsibilities.

Planned week outcome

Add a small API gateway with a stable model alias, authentication and streaming; use a Python client through a private connection first and authenticated HTTPS for sharing. Check missing/wrong/correct keys.

API gateway · Model alias · API keys · TLS

Open week 4 coding exercise →

5Capacity and request limitsWhen users arrive togetherDraft

What happens when demand exceeds capacity?

A successful single request does not establish capacity, and unbounded demand can make everyone wait. Throughput vs latency, continuous batching, context/sequence limits, admission and backpressure; optional custom queue internals.

Planned week outcome

Run bounded repeated load experiments; measure actual tokens, first-text/total times and errors. Add explicit request/input/output/concurrency limits; change one engine setting and explain the tradeoff.

Bounded load experiments · Token counts · vLLM limits

Open week 5 coding exercise →

6Deployment specification and lifecycleDeploy without memorizing a terminalDraft

Can you reproduce the deployment without your terminal history?

Shell commands and in-memory state do not give another operator a reproducible deployment. Control plane vs request-serving data plane, desired vs observed status, images vs containers, persistent model storage and configuration.

Planned week outcome

Turn the recorded configuration into a versioned deployment specification; package with a pinned engine image; add a small control API/CLI for deploy, status and stop with durable deployment metadata. Prove stop/start and a fresh generation.

Pinned engine image · Deployment records · Deploy / status / stop

Open week 6 coding exercise →

7Operational evidenceSee and diagnose the serviceDraft

Which layer failed, and what evidence shows it?

Users report slowness or failures, but a live process does not explain them. Evidence-led diagnosis, request vs model timing, bounded retries, restart policy; use the existing engine's mechanisms rather than inventing a scheduler.

Planned week outcome

Add a small operational view of request counts, latency, errors and workload, plus useful logs and health/readiness. Reproduce one failure, locate its layer and recover under supervision.

Metrics · Logs · Health and readiness

Open week 7 coding exercise →

8Revisions, routing and rollbackChange and recoverDraft

How do you update the service and recover when it breaks?

A model/config update or worker failure can break a stable endpoint. Model version vs deployment revision, routing/health, reproducibility, rollback and capacity during change; a second real replica is optional if resources allow.

Planned week outcome

Record a new deployment revision, test readiness, change routing deliberately and roll back. Demonstrate recovery after stopping a worker. Add a minimal provider-catalog adapter based on the verified integration schema, and compare our platform's responsibilities with public provider workflows.

Deployment revisions · Routing · Provider-catalog adapter

Open week 8 coding exercise →

9Cost model and individual applicationEconomics and provider submissionDraft

Can your service sustain its costs and substantiate a provider application?

Working software is not yet a sustainable service or a substantiated provider offering. Provisioned vs busy time, input/output pricing, cost per million tokens, utilization, cold starts, storage/egress/platform overhead, margins and capacity commitments.

Planned week outcome

Present your accumulated platform, a deployment/evidence record and a measured cost model. Review actual OpenRouter criteria and prepare/submit your own truthful provider application when eligible. Retain application evidence without claiming acceptance.

Measured costs · Capacity evidence · OpenRouter criteria

Open week 9 coding exercise →

Try the four foundation samples

A standalone CPU sequence: model runner, HTTP API, clients, then private access. These retain their original sample numbering and code; sample 3 is a clients exercise, separate from the new week 3 GPU lab.

Your platform. Your operating decision.

Week 9 brings together your deployment evidence and measured economics: provisioned and idle GPU time, input and output tokens, storage, networking and other operating costs. Decide who the service is for and what capacity you can sustain.

Review current OpenRouter criteria and prepare or submit your own truthful provider application when eligible. Each learner owns their account, application, costs and post-course operating plan. If requirements cannot be met, record the gaps and leave submission incomplete. Eligibility and acceptance are not guaranteed; submission does not establish revenue or commercial readiness.

How classes work

Start with a problem

Find a bottleneck or a missing feature in our service.

Make a change

Work through the code, then try the tool or technique that addresses it.

Compare the results

Measure whether it helped and work through your questions together.

Read a sample lesson

Support beyond the lessons

Direct access to me

Get candid feedback on your code, technical decisions, and career questions during the cohort.

Interview preparation

Talk through technical questions and get feedback on how you explain your work.

Meet other engineers

Compare approaches, work through labs together, and stay connected after the course.

Introductions and referrals

Where there’s a fit, I can help through my network. Referrals depend on the role and relationship and don’t guarantee an interview or job.

A longer-term plan

I’d like to bring in more instructors for mock interviews, feedback, and referrals. For now, I teach and support this cohort myself.

Samwel Emmanuel

Samwel Emmanuel

AI Inference & Post Training @ Baseten

I work on AI inference and post-training at Baseten. Before that, I worked at Databricks, Google, and Salesforce. I studied electrical engineering at Harvard.

Work, teaching & education

  • Education
  • Now
  • Previously
  • Previously
  • Teaching
  • Previously

Over about three years at Interview Kickstart, I’ve taught thousands of engineers and engineering leaders, including classes on AI agents and voice systems.

4.9/5 · IK teaching rating

Find me online

An independent course by Samwel Emmanuel.

THE FIRST COHORT

Learn live. Build together.

A proposed nine-week path through your own inference platform, with live classes, guided labs, recordings and Samwel there for your questions. Reference code and weekly exercises are available; GPU walkthrough validation is pending.

Course length
Nine weeks
Weekly commitment
To be confirmed before enrollment
Separate costs
GPU rental, storage and networking
Your work
Your code, deployment records and measurements
Schedule
Dates coming soon
  • Live instruction & labs
  • Session recordings
  • Working example code
  • Instructor access during the cohort
  • Interview prep guidance
  • Introductions and referrals when there’s a fit

Founding cohort tuition

$1,499USD / student

$1,499 upfront or two payments of $749.50, 30 days apart. Enrollment dates coming soon.

Try a sample lesson

Questions about the course

Who is this course for?

Developers comfortable editing small Python programs, using a terminal and making an HTTP request. You don’t need prior CUDA, Docker or cloud-operations experience.

Do I need a GPU or a cloud account?

Week 1 starts on a laptop CPU. Week 3 targets private real-model generation with vLLM on an authorized, compatible borrowed GPU or rented host. Review the budget before renting; compute, storage and networking costs are separate from tuition. Without GPU access, you can use an authorized instructor service for client and measurement work and review server evidence; that observation path does not prove you provisioned a GPU.

What will I own by the end?

Your own bounded inference platform: one model alias, one host and one active engine deployment by default. Build the gateway, deploy/status/stop controls, durable deployment records, request limits, measurements and rollback around vLLM. The engine runs the model; your platform manages access and deployment. Docker, Grafana and Prometheus examples are provided; extra replicas are optional.

Does the course include fine-tuning or other engines?

Fine-tuning, extra serving engines, GPU kernels, Kubernetes and autoscaling are optional extensions. The nine-week core focuses on understanding and operating one model service with vLLM. It does not require a completed LoRA lab.

Is all of the curriculum available now?

The nine-week track is a draft being authored and checked week by week. The four existing foundation samples are a separate CPU sequence, with their original numbering and source. They are not evidence that weeks 1–4 of the new track are validated. Weekly exercises, platform reference code and Docker/Grafana/Prometheus configuration are available. Local synthetic-engine checks have passed; real GPU and live Kubernetes validation remain pending. The final syllabus and weekly commitment will be confirmed before enrollment.

Does the capstone guarantee OpenRouter acceptance?

No. Week 9 ends with a measured cost model and an eligibility review for your own service and application. Inspect the actual current provider form and official requirements, then submit truthful information only when eligible. Public integration documentation has been reviewed, but live application fields and commercial screening remain unverified. If mandatory requirements cannot be met, record the gaps and leave submission incomplete. Submission does not guarantee listing, acceptance, revenue or commercial readiness.

Who operates the service after the course?

Each learner owns their service, account, application information, capacity costs and consent to submission. Establish a budget and post-course operating plan before promising ongoing availability. The instructor’s demonstration service does not replace your operating responsibilities.

Can I follow along on Windows or Linux?

The existing foundation walkthroughs were tested on macOS. Windows and Linux setup commands for those samples are not yet verified. The planned GPU path uses a compatible Linux/NVIDIA host with a separately verified runtime; laptop CPU dependencies do not establish GPU compatibility.

What if I miss a live session?

The cohort will include recordings. I’ll share the schedule and how long you’ll have access to the recordings before enrollment opens.

When does the first cohort start?

I’m still setting the dates. Tuition for the founding cohort is $1,499. I’ll share the schedule and weekly time commitment before enrollment opens.

What kind of career support is included?

Interview preparation, feedback on your work, and introductions or referrals where there’s a fit. Referrals don’t guarantee interviews or jobs. A wider instructor community and dedicated mock interviews are future plans.

Is this an independent course?

Yes. The organizations in my bio and the platforms discussed do not sponsor or endorse this course. Baseten, Fireworks and Modal are public workflow inspirations; our teaching architecture does not describe their private implementations.

Try a lesson before you decide.

Try the sample lesson