Docs · Concepts

Rules

The thresholds that decide whether a comparison passed — every metric, every operator, and how a run is graded.

A Check without Rules reports differences without judging them. That is useful when a person reads every run. It is not useful at 03:00.

Rules are thresholds that decide whether a comparison passed. A scheduled run fails when an error-level Rule is breached, so a trigger can stay quiet while everything agrees.

Rules live on an aggregation, not on the Check, under the tab called Rules.

#The grammar

Every Rule is four fields and nothing more. It is deliberately not an expression language.

Field	Meaning
What is counted	The metric
Must be	at most · at least · exactly
Threshold	A number
On breach	Fail the run · Warn only

#What you can count

Metric	What it reads
Rows that differ	Rows where a measure disagrees beyond tolerance
Rows missing from a source	Rows present on one side only
Rows that differ, not yet known	The same, minus the ones somebody has marked known
Rows missing from a source, not yet known	Likewise
Rows that differ (%)	As a percentage of the total
Rows compared	The total — catches a source that returned nothing
Rows with no key	Rows excluded from the join for having a blank key

The two "not yet known" metrics are the ones a reconciliation ends up wanting after about a month, once a handful of permanent, understood differences would otherwise fail every run for ever.

Rows compared is the one people forget. A query that silently returns nothing produces zero differences and passes every other Rule. A Rule saying rows compared must be at least 1 turns that from a clean run into a failure.

#The default

Add a Rule and it arrives as: rows that differ, at most 0, fail the run.

That is the Rule almost everyone wants first, and the one that makes a nightly run meaningful — nothing should disagree.

#How a run is graded

Two gradings exist and they are not the same. This confuses people, so it is worth being precise.

Whether the run failed. Only error-level Rules can fail a run. A warning that failed a nightly run would be a warning in name only. Disabled Rules are not evaluated at all.

What the History tab shows, which has three states:

State	When
Failed	The run itself broke. It compared nothing.
Warning	The run finished and at least one Rule was breached — of either severity
Passed	The run finished and every Rule held

So a breached warn Rule shows as Warning in history while not failing the run. Warning is a real third state, not a cosmetic one.

#What a breach produces

Each Rule produces an outcome carrying what was expected and what was found, phrased for a human to read in a mail: 12 differing rows, expected at most 0.

Those outcomes are:

stored on the run, so history keeps them;
shown in the run log, which says "all 4 rules passed" or "1 of 4 rules failed" and badges each one PASS or FAIL;
sent in the webhook payload, in a field called assertions;
shown to published readers as the Rules they failed, in the words their author wrote;
used by a trigger set to notify only when something differs — with Rules defined, they decide what "interesting" means.

On the Overview, a Check whose last run breached a Rule reads Failed, outranked only by Stale.

#Writing Rules that hold

A few habits that survive contact with a real estate:

Start at zero and loosen with reason. A threshold of 3 that nobody can explain is worse than no Rule, because it makes the next person assume 3 is acceptable.

Prefer counts to percentages for small estates. Two differing rows out of eleven is 18%, which sounds alarming and is not.

Use warn for the Rule you are still calibrating, and promote it to error once you trust it. That is what the two severities are for.

Add a "rows compared" Rule to anything scheduled. It is the cheapest insurance against a silently empty source.