Recommendation Systems
Project workbook · Path 73

Build a small content-based recommender, exclude already-seen items and evaluate a documented reranking rule.

The case

A fictional learner prefers the topic vector [2,1,0] for data, project and design content. Four items have binary vectors: A=[1,0,0], B=[1,1,0], C=[0,0,1], D=[0,1,1]. Item A is already seen. Rank by dot product, break ties by item ID and return two unseen items.

ItemDataProjectDesignAlready seen
A100yes
B110no
C001no
D011no
STAGE 1

Separate the recommendation stages

Candidate generation, scoring and reranking solve different problems. A small catalog can score every eligible item, while a large one often narrows candidates first. State eligibility separately from preference. The case uses one fictional profile and does not infer a real person’s interests from browsing behavior.

Your task

Draw the eligible-items → score → rank → top-two pipeline.

Required output

Pipeline specification

Record your result and evidence here or in a separate file.

Review gate

Seen-item exclusion is a rule, not a negative preference score.

If the gate is not met

Move eligibility checks before the final selection.

STAGE 2

Implement a transparent score

A dot product rewards overlap weighted by the profile. It is easy to inspect but sensitive to representation and magnitude. Do not call it cosine similarity: that requires normalization. Keep the vector order fixed and explain the meaning of each position so the numbers remain reproducible.

Your task

Calculate all four scores and implement the scoring function.

Required output

Score table and function

Record your result and evidence here or in a separate file.

Review gate

Scores are A=2, B=3, C=0 and D=1.

If the gate is not met

Check the feature order and distinguish the chosen score from other similarities.

STAGE 3

Apply exclusions and stable ordering

A deterministic tie rule makes tests and explanations stable. Filter seen items before taking the top results; otherwise the list can be too short or contain repeats. Consider what happens when fewer eligible items remain than requested. Returning fewer is more honest than inserting an ineligible item.

Your task

Produce the top two unseen items and test a nearly exhausted catalog.

Required output

Ranking function and edge cases

Record your result and evidence here or in a separate file.

Review gate

The case returns B then D; no seen item is returned.

If the gate is not met

Apply the eligibility predicate before slicing the ranked list.

STAGE 4

Evaluate a reranking policy

A reranker can impose diversity, freshness or other constraints, but every rule has a trade-off. Explain what it changes and why. The case can test mechanics, not real satisfaction. A policy that promotes a low-scoring item should be visible rather than disguised as the original model score.

Your task

Propose a rule requiring a design-tagged item in the top two and show its effect.

Required output

Reranking note

Record your result and evidence here or in a separate file.

Review gate

D already satisfies the rule in the base result; no unnecessary change is made.

If the gate is not met

Check whether the constraint is already satisfied before altering the ranking.

STAGE 5

Document evaluation limits

Offline relevance checks need labeled outcomes or a defensible proxy. This case has neither real engagement nor satisfaction data, so do not invent accuracy or click-through improvements. Test correctness of exclusions and ordering, then outline how a voluntary future evaluation could examine usefulness and unwanted effects.

Your task

Write an evaluation plan and a model card for the toy scorer.

Required output

Recommendation handoff

Record your result and evidence here or in a separate file.

Review gate

No real-user performance metric is claimed.

If the gate is not met

Separate code correctness from recommendation quality.

Final project

Deliver the vectors, scorer, exclusion tests, reranking policy and evaluation plan. Add a tied candidate and demonstrate the documented ID ordering.

Critical condition

Eligibility is enforced before selection and the score is not mislabeled as observed satisfaction.

CriterionPointsEvidence
Evidence and correctness30Calculations, source use and results match the supplied case; limitations are honest.
Required artifacts30All five stage outputs and the capstone handoff are present and inspectable.
Reasoning and decisions20Choices, assumptions, alternatives and the case-specific critical condition are explained.
Reproducibility10A reviewer can follow the stated steps and distinguish expected from observed results.
Clear communication10Labels, units, structure and evidence references make the work easy to review.

Score at least 80/100, supply every required artifact, and satisfy the path-specific critical condition. Rework missing evidence before marking complete.

Worked reference

With profile [2,1,0], B scores 3 and D scores 1. A scores 2 but is excluded because it is seen; C scores 0. The final order is B, D. The design-diversity rule makes no change because D already carries that tag.

Boundaries

This is a content-based toy recommender, not a personalized production service or an engagement experiment.