# py-10-fastapi-service — `askapi`: /ask, /stream, health vs readiness, a request id that sticks

The service you build here is the one that goes to Fargate on day 16 and becomes the RAG API
later: a FastAPI app with a JSON endpoint, an SSE streaming endpoint, the health/readiness split
every orchestrator expects, and a request id that follows one request through the whole stack —
the same idea as py-09's `request_scope`, now scoped to an asyncio task instead of a thread. The
brief is the same text as the page (`illustrated/7-mlops/exercise-fastapi-service.html`); the
scene it feeds is `illustrated/7-mlops/model-serving.html#request`.

## What to implement

`src/askapi/reqctx.py` — the request id's storage:

- **one value per asyncio task**, `"-"` outside a request;
- `request_scope(request_id)`: a context manager that sets it and restores the previous value on
  exit, even when the block raises; `current_request_id()`.

`src/askapi/schemas.py` — `AskRequest` (`prompt: str`, non-empty) and `AskResponse` (`answer: str`,
`request_id: str`), plain pydantic models.

`src/askapi/app.py`:

- `RequestIdMiddleware` — a **raw ASGI middleware**, the form Starlette's docs recommend over
  `@app.middleware("http")`. It is one coroutine around the whole exchange, so a single
  `with request_scope(...)` covers the route, the response headers and every streamed chunk.
  (The decorator would carry the id here too — it copies the ContextVar into the task that runs
  your route — but it re-wraps every response body in a stream of its own; the raw form is the
  one that shows the mechanism.) It reads `x-request-id` out of `scope["headers"]` (or mints a `uuid.uuid4()`), and
  patches the outgoing `X-Request-Id` header by wrapping `send`.
- `create_app(model: Model) -> FastAPI` — wires `POST /ask`, `GET /stream` (SSE: five
  `event: token` frames, then `event: done`, `media_type="text/event-stream"`), `GET /healthz`
  (always 200, never touches `model`), `GET /readyz` (503 until `model.loaded`, 200 after).

`src/askapi/model.py` (the `StubModel`) and `src/askapi/main.py` (the uvicorn entrypoint) are
**provided** — do not edit them.

## Run it

```
cd exercises/py-10-fastapi-service && uv sync && uv run pytest -q

# once the tests pass: talk to it for real
uv run uvicorn askapi.main:app --port 8000 &
curl -s -X POST localhost:8000/ask -H 'content-type: application/json' -d '{"prompt": "hi"}'
curl -sN localhost:8000/stream?prompt=hi
curl -si localhost:8000/readyz   # 503 for the first ~3 s — main.py load()s the stub on a timer, then 200

# the same bar the reference solution clears
uv run ruff check . && uv run mypy src
```

Done when `uv run pytest -q` prints **9 passed** — or **8 passed, 1 skipped** if you don't have a
Docker daemon running; the one skip is `test_dockerfile_builds_and_serves_healthz`, which is
marked `slow` and skips itself rather than failing when `docker info` can't reach a daemon. The
untouched starter fails all nine. The same line also says **3 deselected**: those are the live
checks in `integration/`, below — not part of the nine.

## `integration/` — the live check for cloud-07

Three tests (`/healthz` + `/readyz`, 20 `/ask` calls that must echo your `X-Request-Id`, and the
`/stream` first-chunk latency) that run against a deployed copy of this service, not an in-process
app. `pyproject.toml`'s addopts deselect them; cloud-07
(`illustrated/7-mlops/fargate-service.html#apply`) runs them against the ALB:

```
BASE_URL=http://<alb-dns> uv run pytest -q -m integration
```

and they print the `/ask` p50 and the first-chunk time at the end of the run. Against a local
`uvicorn` (`BASE_URL=http://127.0.0.1:8000`) they pass too, once `/readyz` has turned 200.

## The checks

- `test_ask_returns_200_with_answer_and_request_id` — a valid prompt gets a 200 with a non-empty
  `answer` and a `request_id`.
- `test_ask_missing_prompt_returns_422` — an empty JSON body 422s.
- `test_stream_sets_event_stream_content_type_and_five_tokens_then_done` — `content-type` starts
  `text/event-stream`; exactly five `event: token` frames, then `event: done`.
- `test_stream_delivers_chunks_incrementally_not_buffered` — the model's `emitted` counter is
  still under 5 the moment the app sends its first body chunk: the response is actually
  streaming, not buffered and sent all at once. (The test speaks ASGI to the app directly —
  httpx's `ASGITransport` collects the whole body before the client sees any of it.)
- `test_healthz_ok_and_readyz_503_until_loaded` — `/healthz` is 200 throughout; `/readyz` is 503
  with `{"ready": false}` before `model.load()`, 200 with `{"ready": true}` after.
- `test_incoming_request_id_is_echoed_in_response` — a caller-supplied `X-Request-Id` comes back
  unchanged, in both the response header and the JSON body.
- `test_generated_request_id_is_valid_uuid4` — with no incoming header, the minted id matches a
  uuid4 shape and the header matches the body.
- `test_log_line_carries_same_request_id_as_response` — `caplog` sees a log record whose
  `request_id` equals the one the response returned.
- `test_dockerfile_builds_and_serves_healthz` (slow, needs docker) — `docker build` on this
  folder succeeds and the running container answers `/healthz` with 200.

## Files

- [exercises/py-10-fastapi-service/README.md](../../exercises/py-10-fastapi-service/README.md) — this brief, offline
- [pyproject.toml](../../exercises/py-10-fastapi-service/pyproject.toml) — deps, ruff and mypy config
- [requirements.txt](../../exercises/py-10-fastapi-service/requirements.txt) — what the Dockerfile installs
- [Dockerfile](../../exercises/py-10-fastapi-service/Dockerfile) — the cloud-01 multi-stage recipe
- [src/askapi/app.py](../../exercises/py-10-fastapi-service/src/askapi/app.py) — the starter you edit first
- [src/askapi/reqctx.py](../../exercises/py-10-fastapi-service/src/askapi/reqctx.py) — the ContextVar
- [src/askapi/schemas.py](../../exercises/py-10-fastapi-service/src/askapi/schemas.py) — `AskRequest`/`AskResponse`
- [tests/test_app.py](../../exercises/py-10-fastapi-service/tests/test_app.py) — the checks

## If you get stuck

- **The header is right but `request_id` comes back as `"-"`** — the ContextVar was never set
  while the route ran: the `await self.app(...)` call sits outside the `with request_scope(rid):`
  block. Everything downstream — the route, its log line, every streamed chunk — runs inside
  that one `await`, so it goes inside the `with`.
- **`test_stream_delivers_chunks_incrementally_not_buffered` is red** — look for a list
  comprehension or `list(...)` around `model.stream(...)` before the `yield`s start; that
  exhausts the generator up front.
- **`/readyz` never 503s** — a route handler that wants to change the status code needs
  `response: Response` as a parameter and `response.status_code = 503`; returning a dict alone
  always answers 200.
- **`X-Request-Id` isn't echoed** — `scope["headers"]` is a list of `(bytes, bytes)` pairs with
  lower-cased keys; compare against `b"x-request-id"`, not the string.
- **Reading a red row** — `uv run pytest -q -x --tb=short` stops at the first failure and shows
  the assertion that tripped; the message names the observed value, not the fix.