Turn a business request into a measurable ML problem statement with a non-ML baseline, label plan and go/no-go conditions.
A fictional library receives 200 weekly support messages. A manager asks for AI to route them. A simple keyword rule correctly routes 150 in a synthetic review, leaves 30 for humans and misroutes 20. The team has not yet trained a model or agreed the cost of an error.
| Baseline outcome | Messages |
|---|---|
| Correct automatic route | 150 |
| Human review | 30 |
| Wrong automatic route | 20 |
Start with the work to improve and the cost of current failure. A business objective is not identical to a model metric. Identify who acts on a prediction and what happens when it is wrong. The manager’s request is a hypothesis about a solution, not proof that ML is appropriate.
Write an outcome statement and a decision flow for routing support messages.
Problem brief
Record your result and evidence here or in a separate file.
The brief names an action, an owner and an error consequence.
Replace use AI with an observable operational result.
Specify when a prediction is made and which information is available then. Avoid including future outcomes as inputs. A label needs a consistent definition and a process for ambiguous cases. Do not assume that historical routing is correct simply because it is stored in a system.
Create an input/label contract and an ambiguity-resolution procedure.
Data and label specification
Record your result and evidence here or in a separate file.
No future resolution field is used at prediction time.
Draw the prediction-time boundary before selecting features.
Evaluate the existing rule with both coverage and error measures. Automatic coverage is automatically routed cases divided by all cases; precision among automatic routes uses a different denominator. Human review is not a correct automatic prediction. Keeping these measures separate reveals the actual workflow burden.
Calculate coverage, correct routing among automatic routes and total error rate.
Baseline scorecard
Record your result and evidence here or in a separate file.
Coverage is 85%, automatic-route correctness is about 88.24%, and total misroute rate is 10%.
Include the 20 wrong routes in the automatic-routing denominator.
A success test should compare a candidate with the baseline under the same conditions. Include review workload, error cost, maintenance effort and a rollback condition. A higher offline score is insufficient if the system creates unacceptable operational work. Define what evidence would justify stopping the ML effort.
Write go/no-go criteria and a small offline evaluation protocol.
Evaluation and stop rules
Record your result and evidence here or in a separate file.
The rules include an acceptable human-review load and a failure condition.
Avoid a target with no baseline or no decision consequence.
A framing document should make unresolved questions visible. Record data access, privacy, label quality, integration and monitoring needs without pretending they are solved. The deliverable is a decision package for whether to proceed. It is not a trained model, approved deployment or evidence of improved support outcomes.
Write a one-page recommendation and a prioritized uncertainty register.
ML opportunity assessment
Record your result and evidence here or in a separate file.
The recommendation can legitimately be defer or keep the simple rule.
Remove claims of model performance that were never measured.
Deliver the outcome brief, prediction-time contract, label guide, baseline calculations, success criteria and recommendation. Include one reason to prefer a non-ML solution.
The proposed outcome, prediction task and business decision are distinct, and no untrained model is given invented results.
| Criterion | Points | Evidence |
|---|---|---|
| Evidence and correctness | 30 | Calculations, source use and results match the supplied case; limitations are honest. |
| Required artifacts | 30 | All five stage outputs and the capstone handoff are present and inspectable. |
| Reasoning and decisions | 20 | Choices, assumptions, alternatives and the case-specific critical condition are explained. |
| Reproducibility | 10 | A reviewer can follow the stated steps and distinguish expected from observed results. |
| Clear communication | 10 | Labels, units, structure and evidence references make the work easy to review. |
Score at least 80/100, supply every required artifact, and satisfy the path-specific critical condition. Rework missing evidence before marking complete.
The rule routes 170 of 200 messages automatically, so coverage is 85%. Of those 170, 150 are correct: 88.24%. The 20 misroutes are 10% of all messages. These figures describe a synthetic baseline and do not establish a target for a future model.
This is an ML feasibility brief, not model development, compliance approval or production deployment.