Memra

Look, then check

◈ 10 cards

hist() for one numeric, barplot(table()) for one categorical, boxplot(amount ~ channel, data = orders) for numeric by group, plot(x, y) for two numerics. Then three checks: colSums(is.na(orders)) for blanks, range(orders$amount) for impossible values, table(orders$channel) for stray spellings.

Four plots, matched to variable types

Module 8 chose charts by the question and the variable types. Base R has one function per case, and each takes the orders column directly:

  • hist(orders$amount) — one numeric variable: the distribution. Default bins are Sturges' rule; here 7 bins of width 200 with counts 2 3 3 2 1 0 1. breaks = seq(0, 1500, 300) gives Module 4's five bins.
  • barplot(table(orders$prov)) — one categorical variable: counts per category. barplot() wants the table, not the raw column; barplot(orders$prov) is an error.
  • boxplot(amount ~ channel, data = orders) — numeric by group. The ~ reads "amount by channel": one box per channel, side by side, and the outlier 1320 as a point above Online.
  • plot(ordersamount) — two numerics: a scatter, x first. Module 13 fits the line.

plot() on a factor draws a bar chart; on two numerics, a scatter — it looks at the types. Each call opens in the Plots pane; Export saves it.

Worked example — the three quality checks

A plot shows a problem; a check proves it. Three one-liners, phase-tagged verify data quality:

Blanks. is.na() marks each cell; colSums() counts per column:

> colSums(is.na(orders))
  order_id       prov    channel     amount      units order_date 
         0          0          0          0          0          0 

All zero — the file is complete. With one amount missing, the amount entry reads 1, and Module 12 decides what to do with the row.

Impossible values. range() returns the minimum and the maximum — two numbers:

> range(orders$amount)
[1]   90 1320

No negative order, nothing absurd. Plant a −999 sentinel — a value some systems write for "unknown" — and range() reports -999 1320 at once, while is.na() sees nothing, because −999 is a number. summary() shows the same minimum, and the mean sinks to 420.9.

Stray spellings. table() on every categorical column:

> table(orders$channel)

Online  Store 
     6      6 

Two levels, as the dictionary says. Plant an "Onatrio" in prov and table() prints five provinces — AB 2 BC 3 ON 3 Onatrio 1 QC 3 — which no range() could catch, because text has no range.

Three faults, three different catches: a blank is is.na(), a wrong number is range(), a wrong label is table(). Fixing them is Module 12; finding them is here.

Type the boxplot and the two checks, then match fault to check

CRISP-DM: the plots are data understanding → explore data; the checks are data understanding → verify data quality. Cleaning — replacing the −999, fixing "Onatrio" — is data preparation → clean data: the next module.

NORMAL ~/memra/learn/afm-112/base-plots-and-verifying-data-quality utf-8 LF