prefit report · MedHouseVal

Rule

task_confirmation v1

Benchmark

rule detection pass rate on benchmark 2026.11-2: 1/1 checks (known-hazard fixtures; not a real-world accuracy claim)

Findings

f.task_inferred Task inferred as regression (continuous numeric target).

Columns

MedHouseVal

Full action

Analyze and evaluate as a regression problem.

Full reason

Inferred from the target: continuous numeric target.

Risk

A wrong task changes every metric and split recommendation.

Class

needs your context: Cannot be decided from data; applied only after you answer.

Rule

validation_split v2

Full action

Use KFold(n_splits=5, random_state=0, shuffle=True). Verification experiments and baselines use the same splitter repeated 2x (RepeatedKFold), for statistical power.

Full reason

Standard shuffled k-fold.

Risk

Without groups or a time column, rows that belong together (same patient, same customer, later in time) can still leak across folds.

Class

procedural: About how to evaluate; correct regardless of the data. Applied.

Rule

fit_in_fold v2

Full action

Put every preprocessing step in one sklearn Pipeline so it is re-fit on each training fold (defaults: 8 numeric/boolean: median imputation).

Full reason

Statistics computed on the full data (medians, category lists, encoders) leak test-fold information into the score.

Risk

The imputation and encoding defaults are generic; replace them where domain knowledge says otherwise. Feature scaling is estimator-dependent and is not part of this decision: export adds it with scale=True (default) and you can turn it off.

Class

procedural: About how to evaluate; correct regardless of the data. Applied.

CV: RepeatedKFold(n_splits=5, n_repeats=2) — the export splitter, repeated · 10 folds of repeated k-fold CV on identical folds; corrected repeated k-fold t-test (Nadeau & Bengio 2003; Bouckaert & Frank 2004): variance x (1/n + n_test/n_train), df = n-1; Holm across this run. Development evidence on the development rows only.

fig.col.AveBedrms · fig.col.AveOccup · fig.col.AveRooms · fig.col.Population

Rule

skewed_features v2 · thresholds: abs_skew = 2

Findings

f.skewed_features 4 skewed numeric feature(s) (|skew| >= 2; near-threshold = interval straddles 2).

Columns

AveBedrms, AveOccup, AveRooms, Population

Full action

Apply a Yeo-Johnson transform to ['AveRooms', 'AveBedrms', 'Population', 'AveOccup'].

Full reason

Skewed inputs can hurt linear and distance-based models; tree ensembles are usually far less sensitive. The effect is model-dependent, so it is tested, not assumed.

Risk

Transforms make coefficients harder to interpret and can amplify noise in the tails.

Check

Compare CV scores with and without the transform for your model family.

Class

conditional: May help or hurt depending on data and model; only an experiment can support it.

CV: RepeatedKFold(n_splits=5, n_repeats=2) — the export splitter, repeated · 10 folds of repeated k-fold CV on identical folds; corrected repeated k-fold t-test (Nadeau & Bengio 2003; Bouckaert & Frank 2004): variance x (1/n + n_test/n_train), df = n-1; Holm across this run. Development evidence on the development rows only.

fig.exp.log_target · fig.target

Rule

target_skew v1 · thresholds: abs_skew = 1; outlier_fraction = 0.05

Benchmark

rule detection pass rate on benchmark 2026.11-2: 2/2 checks (known-hazard fixtures; not a real-world accuracy claim)

Findings

f.target_skew Target is skewed (skew=0.98, 5.2% IQR-outliers): squared-error metrics will be driven by the largest values.

Columns

MedHouseVal

Full action

Wrap the estimator in TransformedTargetRegressor(func=log1p, inverse_func=expm1).

Full reason

On a skewed, non-negative target, fitting on the log scale often reduces absolute error; it also changes what the model optimizes (closer to relative error).

Risk

Predictions back-transformed from the log scale are biased low for the mean; RMSE can get worse while MAE improves.

Check

Compare MAE and RMSE with and without the transform.

Class

conditional: May help or hurt depending on data and model; only an experiment can support it.

Rule

metric_policy v1

Full action

Optimize and report: MAE; report alongside: median absolute error, RMSE, R².

Full reason

The data shows which metrics mislead here (see 'avoid'); which one to optimize depends on what an error costs, which only you know.

Risk

Optimizing a metric that does not match the real cost produces a model that looks good and fails where it matters.

Class

needs your context: Cannot be decided from data; applied only after you answer.

Rule

holdout_policy v2

Full action

Score the final model on the test set once.

Full reason

Prefit did not reserve an internal holdout; the analysis ran on the frame you passed, and the registered test set (4128 rows) was kept out of it.

Risk

Re-using it to choose between models turns it into a second dev set.

Class

procedural: About how to evaluate; correct regardless of the data. Applied.