prefit
Decision-driven EDA for tabular supervised learning. Run it before you fit: it turns what it measures into a short list of decisions, shows the evidence behind each one, tests the ones that depend on your model, and exports only what is safe, confirmed or verified.
pip install "prefit[verify]"What it does
Measures, then decides
Profiles your development data and turns each measurement into a decision with its evidence, 95% interval, rule version and threshold. Near-threshold results are marked uncertain.
Tests instead of assuming
Data- and model-dependent changes (log target, power transform, target encoding, class weights) are tested with bounded experiments on identical folds, with a practical effect threshold.
Asks what only you know
Which error is costlier, whether a column exists at prediction time, whether time order matters: these become questions, and nothing changes until you answer.
Exports only what holds
A scikit-learn model recipe with applied, confirmed or verified steps, plus an explicit list of what was left out and why.
Quickstart
import prefit as pf
from sklearn.ensemble import HistGradientBoostingRegressor
model = HistGradientBoostingRegressor(random_state=0)
dev_df, test_df = pf.split(df, target="price", test_size=0.2, strategy="random")
report = pf.analyze(dev_df, target="price", estimator=model)
print(report) # roadmap: one line of why / do per decision
report = report.answer({"metric_priority": "typical_error"})
report = report.verify(dev_df) # bounded experiments with your model
report.to_html("report.html")
built = report.export() # model recipe + CV + scoring
fitted = built.model.fit(dev_df[built.feature_columns], dev_df["price"])
pf.review(test_df["price"], fitted.predict(test_df[built.feature_columns]), report)Four kinds of decision
| class | what it is | enters the model recipe when |
|---|---|---|
| procedural | how to evaluate: fit preprocessing inside folds, stratify, keep split columns and the holdout out | always |
| conditional | may help or hurt depending on data and model | an experiment with your estimator verifies it |
| contextual | cannot be decided from data: the cost of errors, availability at prediction time | you answer the question |
| review | needs a human look: duplicates, informative missingness, infinite values, raw dates and text | never; it is a to-do |
Honest by construction
Leakage is only ever a hint for manual review. Proxy-model results are a sensitivity analysis, not a verdict on your model. Experiment results are development evidence; the generalization claim comes from data the analysis never read, which prefit checks by fingerprint. Reports are immutable, and every change is a new revision with a history entry.