> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hollerith.monarcha.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Latency and warm-up

> The four stages of a Hollerith call

A call spends its time in four places, and each is driven by something different. Find the
one that dominates before you change anything.

```python theme={null}
preds = h.predict(score, poll_interval=1.0, timeout=900.0)
```

* **Upload** — how many bytes your table gzips down to.
* **Queue and warm-up** — what is ahead of you, and whether the worker is up.
* **Forward pass** — how much table the model reads.
* **Result fetch** — how many rows come back, and whether they fit inline.

## Upload

The SDK serializes your payload to JSON, gzips it, hashes the compressed bytes, and PUTs
them to object storage. At or below 64 MiB compressed that is one PUT; above it, multipart.

* **The lever is bytes, not rows.** Drop columns you do not use, and stop carrying float64
  where you do not need it.
* **Nothing checks the ceiling locally.** A payload over 5,000,000,000 bytes is serialized and
  gzipped in full, then rejected at presign with `payload_too_large`. You pay the compression
  pass, not the transfer.

## Queue and warm-up

One worker drains the queue, oldest job first, one at a time. Your queue wait is the sum of
the jobs ahead of yours, so a single large `fit` delays every small `predict` behind it.

* **Whether the worker stays up is a deployment setting.** A persistent worker pays the model
  load on a rollout or restart; a scale-to-zero one pays it whenever the queue was empty.
* **An idle worker re-polls.** The interval is a deployment setting — 60 seconds in
  production — so an arrival into an empty queue can wait that long to be claimed.

```python theme={null}
h.predict(score, on_warming=lambda err: print("warming:", err.problem))
```

While a poll returns a retryable outage the SDK keeps waiting rather than raising, and calls
`on_warming` each time. The hook receives the `ServiceUnavailableError`, so a `lambda: ...`
taking no argument raises `TypeError` inside your own callback.

## Forward pass

This is the stage that scales with how much table the model reads.

* **`fit`** reads your whole training table.
* **Inline `predict`** reads the training rows and the scored rows in one pass.
* **Context-backed `predict`** reads only the rows you sent.

The relationship is worse than linear in rows: doubling the rows more than doubles the pass.
Columns behave the same way, which is what the cell budget is protecting.

No latency figure is published. The one measurement on file was taken on a GPU tier other than
the live fleet, and cannot be quoted as a guarantee.

Time your own table at the size you intend to run, and time it again when that size changes.

## Result fetch

A small result comes back inline. Above 512 KiB the worker gzips it to object storage
instead, and the API returns a presigned `resultUrl` and a content hash.

* **The large-result path costs a second round trip.** The SDK follows the URL, gunzips it,
  and verifies the hash for you.
* **Ask for probabilities and quantiles when you will use them.** Both multiply what has to
  come back.

## The fitted context is the main lever

Everything above is dominated by one choice: whether the training table is re-read on every
call. Fit once, and each later `predict` skips re-uploading it and skips re-reading it.

The worker also caches loaded contexts, so repeated predicts against the same one skip the
artifact download. The three ways you silently lose it are in
[The fitted context](/concepts/fitted-context).

## Bounding the wait

`poll_interval` defaults to 1.0 seconds. The loop sleeps before its first status check, so
every blocking call costs at least one interval.

```python theme={null}
handle = h.submit(score)     # keep the id, survive a timeout
handle.wait(timeout=60.0)
```

`timeout` defaults to 900.0 seconds and ends your wait, not the job. On expiry you get a
`TimeoutError` while the job keeps running, finishes, and bills.

## What this shape is for

One worker drains one queue, and there is no SLA on any of it. Nothing above is a guarantee
about when your call returns.

That makes Hollerith a fit for batch and near-line scoring. A synchronous request path that
blocks a person on `predict` is the case it does not fit — put your own queue in front of it.

## Next

* [The fitted context](/concepts/fitted-context) — fit once, predict many
* [Limits](/reference/limits) — rows, columns and the upload ceiling
* [Errors](/reference/errors) — which failures are retryable
* [Troubleshooting](/help/troubleshooting) — a call that never returns