Skip to main content
Three questions decide where you start. Answer them before reading the rest of this page.
  • Is that number worse than always guessing the most common class? Start at rung 1.
  • Did it come from a table you never shuffled, or a target you never checked? Start at rung 2.
  • Is the table already clean and the number merely short? Start at rung 4.
The rungs are ordered by what they cost. Stop as soon as one of them explains the number. There are no tuning knobs on this page. Hollerith has no learning rate and no depth, so every rung is about the data or the measurement.

​
1. Check the floor

An accuracy of 0.8433 on a table that is 84.12% one class is not a model. Guessing the majority label every time scores almost the same. For regression the floor is the target’s standard deviation, because that is the RMSE you get by predicting the mean for every row. An RMSE above it means the model is losing to a constant, and no amount of work below will change that.
  • Cost: free. It ends the investigation more often than any other rung, and it is the only one that separates a weak model from no signal at all.

​
2. Check the measurement

The measurement fails before the model gets a chance. Three ways it happens:
  • The holdout is unshuffled. Above 10,000 rows evaluate takes a deterministic 80/20 split in file order. A file sorted by its target scores the model on classes it never saw.
  • The task was inferred, not chosen. A whole-number target with 20 or fewer distinct values becomes classification. Read clf.task_ after any fit on a numeric target.
  • read_csv returns every column as text. It does no dtype inference, so a numeric target arrives as strings and lands on classification, and numeric features are read as categories.
At 10,000 rows or fewer the folds are positional — every fifth row, with 5 folds — which survives a sorted file better than a single holdout does. That is why the same table can report one number at 9,000 rows and a much worse one at 11,000.
  • Cost: free. Shuffle once with a seed you record, cast the numeric columns, re-evaluate. Evaluating accuracy covers the split in full.

​
3. Fix the table

Leakage is the failure that looks like success. A column restating the outcome will be used, and the number stays excellent right up until production. Nothing in the model knows that a failure_code restates the thing you asked about. The rest of this rung is a page of its own — see Preparing your table for identifiers, rows with a blank target, and dates or free text arriving as categories.
  • Cost: an afternoon, plus one more fit and one more evaluate. Both are billed jobs.

​
4. Add signal the model cannot derive

Hollerith has no world knowledge. It cannot invent what is not in the columns, and it will not combine two of them in a way you have not written down.
  • Ratios and differences. A rate the model would have to infer from two columns is cheaper as one column.
  • Aggregates and joins. Per-customer history, per-region averages, anything sitting in another table.
  • A number where a date’s ordering matters. Dates are read as unordered categories, so the ordering is lost unless you encode it yourself.
These are the same features that earn their keep in a gradient boosting pipeline, for the same reason. Work you have already done for one transfers.
  • Cost: days, and the highest ceiling on this page. Domain knowledge beats every other rung here, and this is where it goes in.

​
5. Remove the columns carrying none

Wide tables degrade. 2,000 columns are accepted, but accuracy and latency both suffer as the informative-feature count grows. Past a few hundred informative features, selecting features first usually helps. Fewer, better columns is a result here rather than a tidiness argument, and it costs less on every call.
  • Constant and near-constant columns cost context and return nothing.
  • Near-duplicate columns are paid for on every pass, twice.
  • A text column is weighted four times a numeric one in the engine’s cost estimate.
  • Cost: one fit and one evaluate per candidate set. Drop the dead columns as a group rather than one at a time.

​
6. Send better rows, not more rows

Adding rows is the reflex, and it is the one that keeps charging you. The table is read on every forward pass, so its size is priced in seconds and in billed rows at each fit.
  • Recency beats volume where the world moved. Rows from before a pricing change describe something you are no longer predicting.
  • Duplicates buy nothing. They cost latency and billed rows and add no information.
  • Balance changes what the number means. The floor from rung 1 rises with the majority share, and evaluate reports only accuracy.
A compact informative table can beat a larger one. If the minority class is what you care about, hold rows back and score predict_proba output against them yourself, because evaluate will not tell you.
  • Cost: usually negative. This is the one rung that tends to lower the bill.

​
7. Accept the answer

Sometimes the signal is not in the table, and nothing above will put it there. A tuned gradient boosting pipeline, built by someone who knows the domain, may still win on a stable table. There is no rung eight. The ladder ends because the data is all there is, and a table with no signal in it is a finding rather than a failure.
  • Cost: one comparison, run properly. Which model wins on your table is measurable rather than arguable — Evaluating accuracy sets out how to make the answer mean something.

​
Next