Reconciliation jobs: making a Shopify integration reliable

Every integration built purely on webhooks drifts. Not because the code is wrong — because webhook delivery is at-least-once and occasionally none-at-all, because networks fail, because a deploy happened during a burst of orders, because someone edited a record directly in the other system.

A reconciliation job is the periodic pass that compares both sides and fixes the difference. It's what separates an integration that's usually right from one that's reliably right, and it's the component most often left out of a quote.

What it does

At its simplest:

1. Fetch the current state of everything in scope from Shopify.
2. Fetch the same from the other system.
3. Compare.
4. Fix what can be fixed automatically.
5. Report what can't, to someone who will act on it.

Steps four and five are where design decisions live.

What to reconcile

Not everything, and not at the same frequency.

Orders are the one to never skip. An order that didn't reach finance or the warehouse is the failure with the highest cost and the lowest visibility.

Use bulk operations

Reconciliation means reading everything, which is exactly what bulk operations exist for. Paginating a 50,000-product catalogue nightly burns rate-limit budget and takes hours; a bulk query returns a JSONL file and is exempt from the normal cost limit.

This is what makes nightly reconciliation affordable rather than theoretical.

Deciding what to fix automatically

Not every difference should be auto-corrected — auto-correcting in the wrong direction propagates an error instead of fixing it.

• A field with a clear owner — inventory owned by the WMS, price owned by the ERP — can be corrected automatically toward the owner. This is why ownership per field has to be decided before any code.
• A missing record — an order in Shopify that never reached the other system — should be created, since the absence is unambiguous.
• A genuine conflict, where both sides changed since the last sync, needs a human. Report it; don't guess.

Report drift usefully

A reconciliation report that says "1,247 differences" is useless. A useful one says:

• How many differences, by type, and the trend against previous runs.
• The specific records, with both values and the last-modified timestamps.
• What was corrected automatically and in which direction.
• What needs a decision.

Send it to a person with a reason to care — operations or finance, not only the developer. And watch the trend: a stable non-zero count means something structural is wrong and will not fix itself.

The metrics that matter

Two numbers tell you whether the integration is healthy:

• Drift count per run. Should trend toward zero. A steady number means a systematic gap.
• Time to detection. How long between something going wrong and someone knowing. Nightly reconciliation makes that under 24 hours; without it, it's however long until a customer complains.

When to run it

Nightly, outside business hours, after the day's activity has settled. For high-volume stores, an additional lighter pass on inventory during the day.

Alert if the job doesn't run. A reconciliation job that silently stopped three weeks ago is worse than never having had one, because everyone believes they're covered — and "the sync didn't run" is invisible without an explicit check, which is why it's on the error handling checklist.

The argument for building it first

Reconciliation is usually scoped as a nice-to-have and cut when the budget tightens. It should be built with the first sync, for a practical reason: it's how you find out whether the sync works.

Without it, the integration appears to work from day one and quietly diverges. With it, you know within 24 hours.

Webhooks tell you what happened. Reconciliation tells you what's true. An integration with only the first is a guess that has been running for a while.