> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hollerith.monarcha.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluating accuracy

> Compare with what you run

```python theme={null}
clf = Hollerith()
clf.fit(train, target="species", evaluate=True)
print(clf.evaluation_)
# Evaluation(accuracy=0.9733, method=kfold, folds=5, rows=150)
```

`evaluate=True` scores the table you fit on and stores the result. `clf.evaluate()` does the
same thing later, on demand.

## What you get back

`evaluation_` is an `EvaluationResult` frozen dataclass — not a dict, not a string. Read the
fields off it.

```python theme={null}
clf.evaluation_.metric    # "accuracy"
clf.evaluation_.value     # 0.9733
clf.evaluation_.method    # "kfold"
clf.evaluation_.folds     # 5 — None for a holdout
```

The attribute does not exist until an evaluation has run. After a plain `fit`,
`hasattr(clf, "evaluation_")` is `False`.

## Exactly one metric

Classification returns `accuracy`. Regression returns `RMSE`. Those are the only two metric
names the worker produces.

There is no AUC, no F1, no R² and no MAE. If you need one of those, hold rows back yourself
and score `predict` or `predict_proba` output against them.

## How the method is chosen

Row count decides it, server-side. You cannot override the choice.

* **10,000 rows or fewer** — k-fold, starting at 5 folds and trimmed toward 2 if the run
  would exceed the evaluation compute budget.
* **Above 10,000 rows** — a single holdout split. `folds` is `None`.
* **Fold membership is positional.** With 5 folds, the first fold is every fifth row rather
  than a contiguous block.

## The holdout split is unshuffled

The holdout is the last 20% of rows in file order. Training takes the first 80%, and nothing
shuffles.

The split is deterministic, so it reproduces exactly. It is also blind to how your file is
sorted.

A table sorted by its target produces a metric that means nothing. The model fits on the
classes that happen to come first and is scored on ones it never saw.

Shuffle once before you fit, with a seed you record.

```python theme={null}
train = train.sample(frac=1, random_state=0).reset_index(drop=True)
```

## What evaluate costs

`fit(..., evaluate=True)` is a second billed job. It re-uploads the whole frame,
label column included, after `fit` has already uploaded it once.

Two behaviours to know before you wire this into anything:

* **`fit(evaluate=True, wait=False)` drops the evaluation.** The early return skips it. No
  error, no warning, no `evaluation_`.
* **A client from `from_fitted_context()` cannot evaluate.** It raises
  `RuntimeError("call fit() before predict()")` — the labeled rows are not in that process.

## Comparing Hollerith with what you run today

This page carries no accuracy numbers of our own. A score on somebody else's table tells you
nothing about yours.

A tuned gradient boosting pipeline — XGBoost, LightGBM, CatBoost — may still win on a stable
table that someone already maintains. `evaluate` exists to settle that question, not to win
it.

## Hold the data fixed

Same rows, same features, same split, for every model in the comparison. Build the split
once and reuse that object — splitting separately per model hands each one different data.

Fit any preprocessor on the training split only. An encoder or scaler fitted on the full
frame leaks the test set into training and flatters whichever model consumed it.

Fix every seed, and report the seed next to the number.

## Measure more than one thing

A single metric does not decide anything on its own. Put fit time and inference time in the
same table as accuracy.

The costs sit in different places. A model you retrain nightly on your own hardware is a
different proposition from one you pay for per call — see [Latency](/guides/latency).

## Know your noise floor

A holdout metric is an estimate, and estimates have spread. Two models separated by less
than that spread are tied, not ranked.

Repeat the split under several seeds and look at how far the metric moves. On small tables
it moves a lot.

## Match the budget on both sides

If you tune the baseline, say so. Give both sides comparable effort, or state plainly which
one got more.

Hollerith has no hyperparameters, so the effort is not symmetric. Report the wall-clock and
engineering time each side consumed.

## Split the way you will use it

A random split assumes the rows you score look like the rows you fit on. If you will predict
next month from this month, use a time-based split.

If you will score customers the model has never seen, group the split by customer. A random
split on grouped data reports a number you will not see again in production.

## One dataset is an anecdote

Run at least three tables before making a general claim, including one about your own
stack.

A result that holds direction across tables is worth acting on. A single win is worth
running a second time.

## Checklist

Before you believe any comparison, yours or ours:

* [ ] Every model saw the same rows, the same features and the same split object.
* [ ] Seeds are fixed and recorded next to the result.
* [ ] Preprocessors were fit on the training split only.
* [ ] The test set was scored once, at the end.
* [ ] The metric matches what an error actually costs you.
* [ ] Fit time and inference time are in the table, not only accuracy.
* [ ] The gap between models is wider than the spread across seeds.
* [ ] Tuning budget was matched on both sides, and stated either way.
* [ ] The split is time-based or grouped where that is how the model will be used.
* [ ] More than one dataset stands behind any general claim.

## Next

* [Preparing your table](/guides/preparing-your-table) — dtypes, targets, and the traps
* [The model](/reference/model) — where it wins, and where it does not
* [Usage and billing](/account/usage-and-billing) — what each job kind bills
* [Python SDK](/reference/python-sdk) — full signatures for `fit` and `evaluate`