Machine Learning Problem Framing
Project workbook · Path 74

Turn a business request into a measurable ML problem statement with a non-ML baseline, label plan and go/no-go conditions.

The case

A fictional library receives 200 weekly support messages. A manager asks for AI to route them. A simple keyword rule correctly routes 150 in a synthetic review, leaves 30 for humans and misroutes 20. The team has not yet trained a model or agreed the cost of an error.

Baseline outcomeMessages
Correct automatic route150
Human review30
Wrong automatic route20
STAGE 1

Translate the request into an outcome

Start with the work to improve and the cost of current failure. A business objective is not identical to a model metric. Identify who acts on a prediction and what happens when it is wrong. The manager’s request is a hypothesis about a solution, not proof that ML is appropriate.

Your task

Write an outcome statement and a decision flow for routing support messages.

Required output

Problem brief

Record your result and evidence here or in a separate file.

Review gate

The brief names an action, an owner and an error consequence.

If the gate is not met

Replace use AI with an observable operational result.

STAGE 2

Define inputs and labels

Specify when a prediction is made and which information is available then. Avoid including future outcomes as inputs. A label needs a consistent definition and a process for ambiguous cases. Do not assume that historical routing is correct simply because it is stored in a system.

Your task

Create an input/label contract and an ambiguity-resolution procedure.

Required output

Data and label specification

Record your result and evidence here or in a separate file.

Review gate

No future resolution field is used at prediction time.

If the gate is not met

Draw the prediction-time boundary before selecting features.

STAGE 3

Measure the simple baseline

Evaluate the existing rule with both coverage and error measures. Automatic coverage is automatically routed cases divided by all cases; precision among automatic routes uses a different denominator. Human review is not a correct automatic prediction. Keeping these measures separate reveals the actual workflow burden.

Your task

Calculate coverage, correct routing among automatic routes and total error rate.

Required output

Baseline scorecard

Record your result and evidence here or in a separate file.

Review gate

Coverage is 85%, automatic-route correctness is about 88.24%, and total misroute rate is 10%.

If the gate is not met

Include the 20 wrong routes in the automatic-routing denominator.

STAGE 4

Set a proportionate success test

A success test should compare a candidate with the baseline under the same conditions. Include review workload, error cost, maintenance effort and a rollback condition. A higher offline score is insufficient if the system creates unacceptable operational work. Define what evidence would justify stopping the ML effort.

Your task

Write go/no-go criteria and a small offline evaluation protocol.

Required output

Evaluation and stop rules

Record your result and evidence here or in a separate file.

Review gate

The rules include an acceptable human-review load and a failure condition.

If the gate is not met

Avoid a target with no baseline or no decision consequence.

STAGE 5

Prepare a responsible handoff

A framing document should make unresolved questions visible. Record data access, privacy, label quality, integration and monitoring needs without pretending they are solved. The deliverable is a decision package for whether to proceed. It is not a trained model, approved deployment or evidence of improved support outcomes.

Your task

Write a one-page recommendation and a prioritized uncertainty register.

Required output

ML opportunity assessment

Record your result and evidence here or in a separate file.

Review gate

The recommendation can legitimately be defer or keep the simple rule.

If the gate is not met

Remove claims of model performance that were never measured.

Final project

Deliver the outcome brief, prediction-time contract, label guide, baseline calculations, success criteria and recommendation. Include one reason to prefer a non-ML solution.

Critical condition

The proposed outcome, prediction task and business decision are distinct, and no untrained model is given invented results.

CriterionPointsEvidence
Evidence and correctness30Calculations, source use and results match the supplied case; limitations are honest.
Required artifacts30All five stage outputs and the capstone handoff are present and inspectable.
Reasoning and decisions20Choices, assumptions, alternatives and the case-specific critical condition are explained.
Reproducibility10A reviewer can follow the stated steps and distinguish expected from observed results.
Clear communication10Labels, units, structure and evidence references make the work easy to review.

Score at least 80/100, supply every required artifact, and satisfy the path-specific critical condition. Rework missing evidence before marking complete.

Worked reference

The rule routes 170 of 200 messages automatically, so coverage is 85%. Of those 170, 150 are correct: 88.24%. The 20 misroutes are 10% of all messages. These figures describe a synthetic baseline and do not establish a target for a future model.

Boundaries

This is an ML feasibility brief, not model development, compliance approval or production deployment.