Zakat-Eligibility Triage: System Design

01

The pipeline and its trust boundary

ADR-0001ADR-0003ADR-0004ADR-0006ADR-0007ADR-0008

This is the whole pipeline and the line it is built around. A campaign comes in, one model call extracts a typed record of the facts, and a second maps the text against the eight recipient categories as supported, not supported, or insufficient evidence, with the exact span of story behind every mapping. Citations are the part I was strictest about: the model emits verbatim quotes and the server resolves them to offsets by exact search, so slicing the story at those offsets returns the quote by construction rather than by inspection, and a quote that cannot be located is a hard failure (ADR-0003). The refusal gate underneath is plain code over typed output with no model call, no clock and no network, so the same campaign refuses for the same reasons in the same order (ADR-0006). Four conditions fire it, and each carries the specific question a reviewer can answer, because a refusal that says only that a campaign needs review has moved the triage work back onto the person the triage was for.

The one decision behind this diagram is the red line. There is no code path in which an eligibility outcome is published without a recorded human decision, no schema field on the model side accepts a score or a confidence figure, and the only representation of an outcome anywhere in the schema is a row in the decisions table with a named reviewer and a written reason, held there by SQL CHECK constraints rather than by application code (ADR-0001, ADR-0008). Precedent retrieval sits behind a fence for the same reason: past adjudicated cases are useful to a person and dangerous in a prompt, so retrieval runs after generation, renders to the reviewer, and is kept out of the model's context by an import-graph test and a prompt-recording trace test rather than by a comment (ADR-0004). Everything in a citation position is either a byte-checked span of the campaign or human-authored corpus text retrieved by id, and the model's own prose is shape-guarded against quotation and chapter-and-verse references (ADR-0007).

02

The evaluation gate

ADR-0001ADR-0009

This is how I check the thing, and it is the part I would want a reviewer to press on. Eighteen hand-labelled synthetic campaigns run through two scorers that measure different things. The deterministic half counts what a written label can be right about: per-category status agreement, whether a citation slices its own quote back out of the story, exact-set agreement on which refusal conditions fired, and whether every category the label expects a question on got one. The judge half asks a different model, shown the campaign and the record and nothing else, for a pass or fail with a stated reason on four things no label can see. It is blind to the label because the deterministic half already scores agreement and scores it better, and blind to the precedent corpus because a judge holding past decisions would reward a record for resembling them (ADR-0009).

Every judge dimension is pass or fail and never a score, because a judge rating reasoning out of five would reintroduce through the test harness exactly the uncalibrated number ADR-0001 turned down. The timeline is the honest part. The gate went red four times before it went green, and each red caught something real, including one run where the harness charged its own parse failures to the system and reported twelve scholarly rulings that had never happened. A harness that cannot tell a failed measurement from a bad result reports the second when it means the first. One floor moved after all that, in a commit that argued from the report, and that is the only way a threshold is allowed to move here: lowering a bar to turn a red run green is the failure mode the rule exists to make visible.

03

Runtime and integrations

ADR-0002ADR-0005ADR-0008

The runtime is deliberately small. One Vercel project running Next.js App Router with TypeScript end to end, one Neon Postgres database with pgvector, and two model vendors: Anthropic for the extraction and mapping calls, and OpenAI for the embeddings behind precedent retrieval. Slack is a single outgoing webhook that fires only on an escalation and carries the refusal reasons, the questions and a link back to the reviewer page, because an alert nobody can act on from the notification is not worth sending. Models are injected everywhere, so the 441-test unit suite runs against mocks with no network, and PGlite boots the real shipped migrations so the tests exercise the schema production runs (ADR-0002, ADR-0005).

What I care about in this picture is which credential each thing holds. The unit job in CI holds none at all, and its build has to stay green with no database URL set, because an unset database is a state the reviewer page reports rather than crashes on. The eval job holds the model key and nothing else, no database URL and no webhook, and it fails when that key is missing rather than skipping, since a gate that passes without running is a green build for a run nobody performed. The eval job is also path-filtered, so a docs-only commit does not spend a corpus of model calls to measure nothing. The judge model exists only in CI and never runs in production, which is the whole reason it can afford to be a different model from the subject.

04

The operations layer this pattern comes from

context, drawn generically

This is where the habits came from, drawn without any of the specifics. I run a personal automation platform with more than thirty scheduled automations on it, and the parts that taught me the most are the boring safety parts. Email-triggered workers ingest mail and attachments written by someone else while holding the access that reads and sends the mailbox, so each one runs inside a bubblewrap sandbox with a tmpfs home and a curated environment, behind an egress allowlist that fails closed. That last word is the one that matters: an unknown destination is refused rather than logged, and if the sandbox cannot start then the worker does not run unconfined. Mail is read and sent through a Google Workspace service account with domain-wide delegation, and a note inside a message body is advisory context for a human rather than an instruction the pipeline follows.

The rest of the panel is the same idea in different clothes. Dead-letter sweepers re-drive jobs stranded by a worker that died mid-flight, up to a bounded number of attempts and then parked, because a job that dies quietly is worse than one that fails loudly. The local retrieval stack answers with citations in about four seconds and is measured against a hand-built 40-question gold set, since a retrieval system with no gold set has no way to tell a better index from a differently wrong one. The overnight build pipeline is eval-gated for the same reason the triage agent is. None of this is novel. It is the habit of writing the fence into the code rather than into the documentation, and that habit is what the zakat triage design is made of.

05

Orchestration: what moved onto a canvas, and what did not

ADR-0002ADR-0006ADR-0010

This system used to have one way to run. A reviewer opened a campaign and pressed a button, and the request stayed open for the length of two model calls, about seventy seconds on a real campaign, with nothing on screen to say so. A provider returning 529 ended the run as red text on that one reviewer's screen. A campaign nobody pressed the button on was never read at all. Those are operational questions, and ADR-0002 had not answered them because there was no operational layer to answer them with: it turned down an orchestration framework on the grounds that the interesting decisions here, refuse and escalate, are fixed rules over typed output that belong in auditable code rather than in framework config. That reasoning is about the decision path and it still holds, which is why what moved is the other half.

n8n owns when a run happens, how often, what retries, and who hears about a failure: a header-authenticated webhook that starts a run on demand, a sweep every hour over campaigns with no agent file, per-node retry with backoff, and an error workflow. It owns no rule. The three stages in the middle are HTTP calls into the same functions the reviewer's button calls, and that is the decision rather than a shortcut, because behind them sit the rules that make this tool worth anything and rebuilding those as Code nodes would put untyped JavaScript in a database blob, outside CI, where nobody could review it. Three properties keep the split honest. The workflow names a campaign and never supplies its text, so it cannot hand over a story edited to contain the quotes it wants verified. The escalation it branches on is recomputed server side before anything is stored, so what the canvas decided cannot become what the table records. The Slack message is composed here rather than there, because the escaping that stops a campaign title posting a channel-wide ping into the reviewers' room is a tested guarantee and not something a canvas should be trusted to remember. The button still works, and a run the workflow files is identical in shape to one a reviewer files by pressing it (ADR-0010).

06

The follow-up an organizer gets

ADR-0003ADR-0007ADR-0008ADR-0010

The second workflow is the one built out of n8n's own AI nodes, and its shape is an argument about where a model is allowed to write. When the agent cannot resolve a category it records the fact that is missing and the question that would obtain it, and that question was written by the mapping model with the campaign story in front of it, then de-duplicated, because asking an organizer the same thing twice in one message reads as carelessness. The obvious build hands those to a second model and asks for a friendly email. That throws the work away, and it is the thing ADR-0007 exists to refuse: model-authored prose standing in for what the pipeline recorded.

So the chain is asked for the envelope and never the contents. A subject, an opening and a closing, pinned to a schema by a structured output parser. The next node assembles the message with the questions inserted verbatim and then proves it: every question has to appear in the body character for character, which is the same check that resolves a citation back to its span of campaign story. It also rejects a covering note that reads on eligibility, or that asks a question of its own instead of leaving that to the list, and a draft failing either check goes to the engineers rather than to a reviewer. Approval is a person pressing a button in Slack, which holds the execution until they do. Approving posts the wording where a reviewer can use it and sends nothing at all, because writing to someone's organizer is a separate decision from approving the words, for the same reason the agent never publishes an outcome (ADR-0008).

07

What it looks like when it runs

ADR-0006ADR-0007ADR-0010

Three executions, all against this deployment. At the top a campaign submitted to the webhook is accepted in about ten milliseconds and finishes thirty-nine seconds later, the green path running the whole way through: read the campaign, extract, map, ask the rules whether it refuses, tell the reviewers, file. In the middle is a covering note waiting on a person. The execution is not polling for an answer and has not timed out; it is suspended at that node and resumes when somebody presses a button, which is what makes the approval a step in the workflow rather than a notification beside it.

The bottom panel is the one worth reading twice, because both messages in it are the system declining to do something. The first is a drafted note that was never shown to anyone: the model had written a subject line reading "A Few Details Needed: Zakat Eligible Campaign", and telling an organizer their campaign is zakat eligible is the determination this whole system exists not to make. The check that caught it is the same check the pipeline runs on citations, applied to a different artefact. The second is a deliberately broken run, included because a system that only demonstrates its successes has not demonstrated much: the campaign it names has no agent file, which makes it indistinguishable from one nobody has read yet, so it says so, and says that failing the same way twice would mean the cause is not the provider.

08

What the evals cannot prove

the limits, stated rather than hidden