Audits & Reporting · Reference
How do you monitor catalog health with defect reporting?
Copy catalog data into a warehouse and write one query per defect, where a defect is one specific condition that should never be true of a product. Run them on a schedule and route the results to the people who can fix them. Building these checks inside the PIM degrades its performance and buries the logic; running them against a warehouse copy lets a small team keep a very large catalog healthy without slowing anyone's workflow.
What a defect is
A defect is one specific condition that should never be true of a product, written as a query.
Specificity is what makes the method work. “Products with poor data quality” cannot be assigned, counted or closed. “Active, shippable products where weight is null or zero” can be queried today, assigned to a named team, counted at 4,312, and driven to zero. A useful defect has four properties: it is unambiguous, it is queryable from data you already hold, it has an owner who can fix it, and fixing it changes something the business cares about. Drop any one and you get noise.
Why this belongs outside the PIM
The instinct is to build the checks where the data lives. Resist it.
Performance. Hundreds of catalog-wide scans across millions of records compete directly with the transactional work the PIM exists to do. Teams notice their save times degrading and blame the platform.
Scope. The most valuable defects are cross-system by nature. A PIM cannot tell you that a product is published and active but has no inventory record, that a category page is ranking with three products in it, or that a supplier’s entire feed silently stopped updating eleven days ago. Those questions need catalog data sitting next to commerce, inventory, and behavioural data.
Query power. A warehouse plus SQL lets you express conditions that no PIM rule builder can express: window functions, comparisons against category medians, joins across four systems, historical comparison against last week’s snapshot.
Blast radius. A defect query that goes wrong in a warehouse produces a wrong number. A validation rule that goes wrong in the PIM blocks people from working.
The architecture
It is unremarkable by design. The value sits in the queries, not the plumbing:
- Replicate. Catalog, commerce, inventory and asset metadata land in a warehouse on a schedule. Nightly is sufficient for most defects; near-real-time is worth it only for the handful that block revenue.
- Model. A modest transformation layer that resolves each entity to one row with consistent keys, so every query is not re-deriving what a product is.
- Query. One query per defect, version-controlled, each returning the identifiers of the offending records plus enough context to act.
- Surface. A dashboard showing counts by defect, by category, by brand, by supplier, by owner, with the ability to click through to the row-level list.
- Route. The row-level list reaches the person who fixes it, by export, by a queue in the PIM, or by a task list. Teams skip this step more often than any other, and then nobody fixes anything.
- Track. Store the daily count per defect. The trend line is the actual product of the program; a snapshot tells you nothing about whether you are winning.
Any warehouse and any dashboard tool will do this. Pick whatever your data team already runs, because the ongoing cost of this program is query maintenance, not licensing.
Twenty defects to start with
Most catalogs can write these against existing data in an afternoon:
Blocking. Stops the product working at all: published with no price; published with no primary image; active and shippable with zero or null weight or dimensions; no classification; classification pointing at a node that no longer exists; variant with no parent; parent with no variants.
Quality. The product works, but works badly: required category attribute empty; attribute value not in the controlled list; description shorter than a floor you set; description identical to another product’s; name not matching the naming convention; measurement stored without a unit; unit inconsistent with the category standard.
Cross-system. Only visible from outside the PIM: active in the PIM, absent from the storefront; published to a channel that rejected it; supplier feed with no updates in N days; in stock with no sales in ninety days while ranking on a category page; price below cost.
Nothing in that list is clever. All of it accumulates in a catalog nobody is querying.
Consolidation is what makes it survivable
A raw list of five hundred thousand defect rows helps nobody. Three moves make it workable for a small team:
Group by cause, not by row. Four thousand products missing the same attribute in one category is one problem with one owner and probably one bulk fix, not four thousand tasks. Report it that way.
Rank by impact, not by count. A defect on eleven high-velocity products usually outranks one on nine thousand discontinued items. Weight the count by revenue, traffic, or channel exposure before you sort.
Set severity and act on it differently. Blocking defects get a service-level target measured in days. Quality defects get worked down in batches. Structural defects get scheduled. Everything at the same priority means nothing has a priority.
Rule of thumb. The output of a defect program should be a short list of things to fix this week, ordered. If the output is a large number, the program is measuring rather than governing.
Feeding the rest of the system
Defect reporting is not isolated. Each new defect you find is evidence about a gate that was missing upstream: if you keep finding null weights, weight belongs in the required set at SKU creation or in the publishing gate for anything shippable. The point of the program is to run out of new defect types, not to run reports forever.
The counts also give you a defensible health metric over time, which is the number worth putting in front of an executive: not “data quality improved” but “blocking defects fell from 41,000 to 2,200 across two quarters and now sit under 500 on a rolling basis.”
Defect reporting answers whether anything is wrong. It does not tell you how complete a product is, which is a graded question rather than a binary one. That is completeness scoring, and the two programs are complementary.
Common questions
What counts as a defect?
One specific, queryable condition that should never be true of a product. "Bad data" is not a defect. "Weight is zero on a shippable item" is. The test is whether a person reading the row knows what to change without asking a follow-up question.
Why not build this inside the PIM?
Two reasons. Performance: hundreds of catalog-wide scans compete with the transactional work the PIM exists to do. Scope: a PIM only knows what a PIM knows, so it cannot see that a product is active with no inventory record, or ranked on the site with no sales in ninety days. The interesting defects live across systems.
How many defects should we track?
Start with fifteen to twenty covering your worst known problems, then grow. Mature programs run into the hundreds. The constraint is not query-writing capacity, it is whether anyone works the output. If nobody works the output, you are collecting metrics.
How often should defect reports run?
Daily for anything that blocks selling or publishing, weekly for enrichment quality and coverage, monthly for structural checks like taxonomy integrity. Set the cadence by how fast someone would actually respond. Query speed is not the constraint.
Who fixes them?
Whoever owns the field, which means ownership has to be assigned before the program is useful. A defect list with no owner column just circulates. Assign at the defect level, not the product level.
Does this replace validation at data entry?
No. Entry validation stops new bad data. Defect reporting finds the data that predates the rules, arrived through bulk load or integration, or became wrong when something else changed. Both are needed, and the defect reports are how you discover which entry rules you are missing.