Memra

The final's signature item

Given a paragraph-long scenario of five to seven steps, tag every step with its CRISP-DM phase, its generic task and the PSP stage it serves, resolving each borderline step with the purpose test: what is this step FOR? A histogram drawn to see the shape is data understanding; the same histogram on the dashboard is deployment; a merge re-run after a store was found missing is preparation again, not evaluation.

The item

The final ends with a paragraph: a project described in five to seven sentences, and for each sentence the question which CRISP-DM phase — and which task inside it — is this? Every module since Lesson 1.8 has tagged its own work; this lesson does the whole walk at once, on a case you have not seen, the way the paper will.

The method has not changed. For each step, ask what is this step for? — not which tool it uses, not which sheet it is on. str() in week one is data understanding; str() after a merge is preparation checking its own work. The tool is the same; the purpose is the phase.

Worked example — Crumb & Co.

Crumb & Co. runs four bakeries in Kitchener. The owner wants to know whether the weekday promotions she has been running lift Saturday sales. Seven steps, tagged:

  1. Agree the question and the criterion. "Do promo weeks have higher Saturday sales than non-promo weeks, by $500 or more per store?" → business understanding — determine business objectives and success criteria. PSP stage 1, assess the situation.
  2. Receive the POS export and write the data dictionary. Twelve weeks × four stores; week, store, promo (Y/N), sat_sales. → data understanding — describe data. PSP 2, identify and analyse the issues.
  3. Run str() and colSums(is.na()), and find two blank Saturdays. → data understanding — verify data quality. PSP 2.
  4. Derive promo_week with ifelse() and merge the store list for the store names. → data preparation — construct data and integrate data. PSP 2.
  5. aggregate() Saturday sales by promo flag and fit lm(sat_sales ~ promo). → modelling — build model. Recall Lesson 13.7: a group mean is a model. PSP 3, develop and analyse alternatives.
  6. Check the sign, compare the gap with $500, and note there are only eight promo weeks. → evaluation — evaluate results. PSP 4, decide and recommend.
  7. Build the dashboard and name who refreshes it. → deployment — produce final report and plan monitoring and maintenance. PSP 5, communicate.

The figure is this walk as a column. Read it once for the phases and once for the PSP stages: five stages, six phases, and the middle stage takes two phases.

The three borderline steps

The paper earns its reputation on steps like these; each resolves with the purpose test.

  • "Plot a histogram of Saturday sales to see the shape." A chart, so deployment? No — the purpose is to see the shape before any preparation: data understanding → explore data. Not preparation either: nothing in the data changes.
  • "The dashboard shows a histogram of Saturday sales." The same chart, drawn to communicate: deployment → produce final report. Same picture, opposite phase, because the purpose is opposite.
  • "After finding a store missing from the merge, re-run the merge with the corrected store list." Finding the missing store was preparation checking itself; the re-run is data preparation → integrate data again. It is iteration, the arrow back in Module 1's cycle — not evaluation, which judges the model's result against the criterion, and there is no model yet.

Two more that trip people: "decide the criterion is $500" is business understanding even if it happens in week three, because its purpose is to set the standard; and "write down that the June data were incomplete" is review process under evaluation if it is about this project, and review project under deployment if it is the lessons-learned note after the hand-over.

Tag a second case cold

The questions walk a second case — Pedal Loop, a Guelph bike-rental company asking whether rainy weekends cut rentals — with two of the borderline steps planted. One phase per step; the adjacent phases are the distractors, as they are on the paper.

CRISP-DM: all six phases, in one paragraph — the course, walked end to end.

agree the question and the $500 criterionbusiness understanding · PSP 1 assessPOS export + data dictionarydata understanding: describe · PSP 2str() · colSums(is.na())data understanding: verify quality · PSP 2ifelse() promo_week · merge() store listdata preparation: construct, integrate · PSP 2aggregate() by promo · lm(sat_sales ~ promo)modelling: build model · PSP 3sign · $500 gap · only eight promo weeksevaluation: evaluate results · PSP 4dashboard + refresh ownerdeployment: report, monitoring · PSP 5The purpose test: what is this step FOR? Thesame str() or histogram lands in differentphases for different purposes.
The phase walk. Five PSP stages, six CRISP-DM phases; stage 2 takes two phases. Each step's note is the answer the paper wants.
~/memra/learn/afm-112/the-end-to-end-phase-walk