Load → inspect → clean → derive → aggregate → write

One script, six steps, a comment each

Everything in Modules 10–12 was a line at the console. The crew project needs those lines in a file, in order, so that next month's orders.csv produces next month's dashboard by one source("prep.R") — reproducible, the word CRISP-DM uses for a deployment that can be repeated. Six steps, each tagged with the phase it belongs to:

# 1 collect initial data
orders <- read.csv("orders.csv")
# 2 verify data quality
str(orders)
colSums(is.na(orders))
# 3 clean and format
orders$prov <- trimws(orders$prov)
orders$order_date <- as.Date(orders$order_date)
# 4 construct data
orders$level <- ifelse(orders$amount >= 500, "High", "Low")
# 5 the model-ready summary
prov_summary <- aggregate(amount ~ prov, data = orders, FUN = sum)
# 6 deploy to the dashboard
write.csv(prov_summary, "prov_summary.csv", row.names = FALSE)

Run it and the last two steps produce the file the dashboard reads:

> prov_summary
  prov amount
1   AB   2300
2   BC   1360
3   ON   1995
4   QC    485
> cat(readLines("prov_summary.csv"), sep = "\n")
"prov","amount"
"AB",2300
"BC",1360
"ON",1995
"QC",485

Step 2 changes nothing and is never skipped — its output is what tells you whether steps 3 and 4 are right for this month's file. Step 6 has row.names = FALSE (Module 10) so the CSV has no unnamed first column. Every step is a lesson from the last three modules; the script is the order they run in.

Worked example — reading a pipeline you did not write

A colleague's script uses dplyr, a package whose verbs are named after what they do and chained with %>% ("then"):

orders %>%
  filter(amount > 500) %>%
  group_by(prov) %>%
  summarise(avg = mean(amount))

You will not write this in AFM 112 — it needs library(dplyr), which the exam machine does not have loaded — but you must be able to say what it does, because the quiz may show one. Read the verbs left to right and translate each to the base call it stands for:

• filter(amount > 500) → keep rows: subset(orders, amount > 500), or orders[orders$amount > 500, ]
• group_by(prov) + summarise(avg = mean(amount)) → one statistic per group: aggregate(amount ~ prov, data = …, FUN = mean)

So the pipeline is two base lines:

> big <- subset(orders, amount > 500)
> aggregate(amount ~ prov, data = big, FUN = mean)
  prov amount
1   AB   1150
2   BC    640
3   ON    635

Mean amount per province, for orders over $500—threeprovinces,becausenoQCorderclears500.Therestofthevocabularymapsthesameway:\ast \ast ‘select(orde{r}_{i}d,amount)‘\ast \ast \to ‘orders[,c("orde{r}_{i}d","amount")]‘;\ast \ast ‘mutate(share=amount/sum(amount))‘\ast \ast \to ‘orders$share <- …; **arrange(desc(amount))** → orders[order(-orders$amount), ]; **left_join(products, by = "sku")** → merge(…, by = "sku", all.x = TRUE)`. Six verbs, six base twins, and every one of them is a lesson you have already typed.

Type the skeleton and the pipeline, then map the verbs cold

The first snippet is the script with its six phase comments; the second is the pipeline, typed so that its shape is familiar when it appears on a paper.

CRISP-DM: the script is the deployment plan for data preparation — steps 1–2 are data understanding, 3–5 preparation, 6 deployment.