Automation Worker and Trigger Specification

Status

Design update for the next implementation slice.

This supersedes the earlier polling-only trigger-watcher design. The worker is now conceptualized as an automation-worker that executes durable work the engine has already materialized.

The worker does not decide business semantics. The engine decides eligibility, writes durable work rows, and re-validates every fired event through process_event.

Goal

Support system-driven workflow progress without side doors:

• timed actions (schedule / cancel action fields),
• immediate automatic actions (auto: true),
• provenance for why automation happened,
• separate automation actor/key,
• realtime wake-up through PostgreSQL LISTEN/NOTIFY,
• safe recovery after worker restart.

Every automated action remains a real signed event in the process hash chain.

Automation lanes

The platform has four automation lanes.

Lane	Timing	Who signs?	Source of truth	Examples
Synchronous workflow effects	Same process_event	Original actor	Current action mutate	set fields, append note, spawn child, schedule timer
Automatic actions	ASAP after state/conditions become true	Automation actor	Engine-created automation_queue	start screening when onboarding reaches ready state; open case when possible match appears
Scheduled actions	At run_at	Automation actor	Engine-created scheduled_actions	SLA escalation, periodic review, timeout
External integration workers	Async external IO	Integration actor	Outbox/inbox tables	vendor screening request/response, notifications

This spec implements the middle two lanes: automatic actions and scheduled actions. External integration workers remain a separate future spec.

Core principle

Previous actions must not know downstream automation.

Bad model:

text submit_onboarding action says: after me, remember to start screening

Correct model:

text start_screening action says: I am automatic when state is ready_for_screening

So the logic lives in the action that will execute, not in every action that may lead to the state.

Automation actor

Use a dedicated automation actor/key instead of the broad system actor.

Recommended seed identity:

text actor id: automation role: AUTOMATION private key env: AUTOMATION_ACTOR_SK

Rules:

• private key never stored in DB,
• worker verifies env secret matches DB public key at startup,
• automated workflow actions must explicitly grant AUTOMATION role,
• generic system/bootstrap keys are not reused for routine automation.

This limits blast radius and makes audit provenance clearer.

Scheduled actions

Workflow syntax

In active Workflow DSL v2, scheduled work is declared on the action with a direct schedule field:

json { "actions": { "start_screening": { "type": "transition", "role": "analyst", "from": "intake_complete", "to": "screening_started", "schedule": [ { "id": "screening-deadline", "action": "escalate_review", "in": "48h", "data": { "reason": "screening-sla" } }, { "id": "periodic-review", "action": "start_review", "every": "1y", "data": {} } ] } } }

cancel cancels active schedules by stable id:

json { "actions": { "offboard": { "type": "transition", "role": "manager", "from": "active", "to": "offboarded", "cancel": ["periodic-review", "screening-deadline"] } } }

Schedule fields

Field	Required	Meaning
id	yes	Stable process-scoped schedule id.
action	yes	Workflow action name to fire.
data	yes, may be {}	Action data resolved at schedule creation time.
in	one of in/every/at	Relative delay such as 48h.
every	one of in/every/at	Recurrence interval.
at	one of in/every/at	Absolute RFC3339 timestamp.
until	no	Stop recurrence after timestamp.
count	no	Stop recurrence after N fires.

Duration grammar:

text <number><unit>

Units: m, h, d, w, y where y = 365d.

The engine resolves relative durations into absolute run_at; the worker does not parse workflow expressions to decide eligibility.

Automatic actions

Workflow syntax

An automatic action is a normal workflow action with auto enabled:

json { "actions": { "start_screening": { "role": "AUTOMATION", "auto": true, "from": "ready_for_screening", "to": "screening_started", "when": "state = 'ready_for_screening'", "mutate": { "$set": { "state": "screening_started", "screening_status": "started" } } } } }

Optional data form:

json { "auto": { "data": { "reason": "state-entry", "source_event": "event.hash" } } }

V1 rules:

• auto: true means queue with {} canonical action input plus provenance.
• auto: { input } may provide static/template input resolved at queue time.
• automatic actions must be executable by the automation actor role.
• engine evaluates eligibility after every successful event.
• worker fires queued actions ASAP.
• engine re-validates role, from, when, and all workflow rules at fire time.

Why a queue still exists

The queue is an implementation detail, not workflow logic.

process_event should not recursively call itself because that creates unclear failure semantics and can cause unbounded cascades. Instead:

text process_event(A) -> state changes -> engine detects auto action now available -> engine inserts automation_queue row with provenance -> NOTIFY automation_queue_changed -> automation-worker signs event B -> process_event(B)

Every automated action remains its own signed event.

Schema

scheduled_actions

sql CREATE TABLE scheduled_actions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), schedule_id TEXT NOT NULL, process_id TEXT NOT NULL REFERENCES processes(id), workflow_path TEXT NOT NULL, action_name TEXT NOT NULL, action_data JSONB NOT NULL, run_at TIMESTAMPTZ NOT NULL, interval TEXT, until_ts TIMESTAMPTZ, max_count INTEGER, fire_count INTEGER NOT NULL DEFAULT 0, status TEXT NOT NULL DEFAULT 'active', created_event_hash TEXT NOT NULL REFERENCES events(event_hash), created_by_action TEXT NOT NULL, created_by_actor TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), last_attempt_at TIMESTAMPTZ, last_attempt_result TEXT, last_attempt_error TEXT, attempts INTEGER NOT NULL DEFAULT 0, last_fired_at TIMESTAMPTZ, last_fired_event_hash TEXT, UNIQUE (process_id, schedule_id, status) );

Status values:

text active | claimed | cancelled | exhausted | failed

automation_queue

sql CREATE TABLE automation_queue ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), process_id TEXT NOT NULL REFERENCES processes(id), workflow_path TEXT NOT NULL, action_name TEXT NOT NULL, action_data JSONB NOT NULL DEFAULT '{}'::jsonb, status TEXT NOT NULL DEFAULT 'pending', reason TEXT NOT NULL, available_state TEXT, queued_by_event_hash TEXT NOT NULL REFERENCES events(event_hash), queued_by_action TEXT NOT NULL, queued_by_actor TEXT NOT NULL, queued_at TIMESTAMPTZ NOT NULL DEFAULT now(), claimed_at TIMESTAMPTZ, completed_at TIMESTAMPTZ, fired_event_hash TEXT, attempts INTEGER NOT NULL DEFAULT 0, last_error TEXT, UNIQUE (process_id, action_name, queued_by_event_hash) );

Status values:

text pending | claimed | completed | skipped | failed

Privileges

• tables are append/update-only by engine/security-definer functions,
• app roles cannot insert/update/delete directly,
• worker uses narrow RPC/security-definer functions to claim/complete work,
• worker never mutates processes or events directly.

Notifications

The engine emits PostgreSQL notifications after writing durable work rows. Notifications are delivered only after transaction commit.

Channels:

text scheduled_actions_changed automation_queue_changed

Payload example:

json { "id": "...", "process_id": "customer-123", "workflow_path": "customer/retail/retail-customer", "action_name": "escalate_review", "run_at": "2026-06-20T10:00:00Z", "kind": "scheduled_action" }

Notifications are wake-up hints. The durable tables remain source of truth.

Automation-worker

trigger-watcher/ should become automation-worker/ or be renamed in place.

Responsibilities:

1. verify AUTOMATION_ACTOR_SK against the seeded automation actor public key,
2. open direct PostgreSQL connection for LISTEN,
3. scan active scheduled actions and pending automation queue on startup,
4. schedule timers for future scheduled_actions.run_at,
5. fire overdue schedules immediately,
6. fire automation_queue rows ASAP,
7. sign all fired events as automation actor,
8. submit via process_event,
9. mark schedule/queue rows completed, exhausted, skipped, or failed through narrow RPCs,
10. reconnect and rescan after DB connection loss,
11. run a low-frequency safety sweep.

Realtime scheduled flow

text process_event(A) -> inserts/updates scheduled_actions row S -> NOTIFY scheduled_actions_changed -> automation-worker receives notification -> fetches/claims S -> setTimeout until run_at -> signs event B as automation actor -> process_event(B) -> mark S exhausted or reschedule recurrence

Realtime auto-action flow

text process_event(A) -> evaluates auto:true actions against new process state -> inserts automation_queue row Q -> NOTIFY automation_queue_changed -> automation-worker receives notification -> claims Q -> signs event B as automation actor -> process_event(B) -> marks Q completed or skipped/failed

Recovery

On startup/reconnect:

• active schedules with run_at <= now() fire immediately,
• active schedules with future run_at get timers,
• pending auto queue rows fire ASAP,
• claimed rows older than a lease timeout return to active/pending.

A safety sweep every 5–10 minutes catches missed notifications.

Timer limits

Node timers have a maximum delay. For far-future schedules, the worker should schedule a maximum-size timer chunk and re-check when it fires.

Fired event provenance

The worker must copy provenance from durable rows into the fired event. It must not invent provenance. Provenance is included only in events that the engine accepts and appends.

Recommended injected canonical input field:

json { "meta": { "automation": { "kind": "scheduled_action", "schedule_id": "screening-deadline", "work_id": "...", "scheduled_by_event_hash": "...", "scheduled_by_action": "submit_onboarding", "scheduled_by_actor": "<public-key>", "scheduled_at": "2026-06-18T10:00:00Z", "run_at": "2026-06-20T10:00:00Z", "fired_by_actor": "automation" } } }

For auto actions:

json { "meta": { "automation": { "kind": "auto_action", "work_id": "...", "queued_by_event_hash": "...", "queued_by_action": "submit_onboarding", "queued_by_actor": "<public-key>", "queued_at": "2026-06-18T10:00:00Z", "available_state": "ready_for_screening", "reason": "auto action became available", "fired_by_actor": "automation" } } }

If action input already contains meta.automation, the worker must reject or overwrite according to a strict engine-defined rule. V1 recommendation: reserve input.meta.automation and reject user-provided schedule/auto input that sets it.

Worker result feedback and diagnostics

process_event must not update scheduled_actions or automation_queue with execution results. Core event execution should not know whether a caller is a human, CLI, integration worker, scheduled action, or auto action.

The automation-worker owns durable work-row bookkeeping through narrow security-definer functions.

Result categories

Result	Meaning	Work-row outcome
accepted	process_event appended the event	completed, exhausted, or rescheduled
business skip	workflow/state/when/role no longer allows the action	skipped
transient failure	DB/PostgREST/network unavailable or timeout	release/retry later
permanent worker failure	malformed row, missing process, repeated internal error	failed after retry policy
signing/key failure	automation key invalid	worker fatal; no work-row mutation except operator intervention

A scheduled action means “attempt this action at run_at,” not “force this action.” If state changed meanwhile, the worker still submits the signed event; the engine rejects it; the worker records a diagnostic skip.

Auto actions follow the same rule. Queueing means “this action appeared eligible after event X.” If another event changes state before the worker fires it, the engine may reject it and the queue row becomes skipped.

Bookkeeping RPCs

V1 should expose narrow functions rather than direct table writes:

```sql claim_scheduled_action(work_id, worker_id) complete_scheduled_action(work_id, fired_event_hash) skip_scheduled_action(work_id, reason) fail_scheduled_action(work_id, reason) reschedule_scheduled_action(work_id, next_run_at)

claim_automation_work(work_id, worker_id) complete_automation_work(work_id, fired_event_hash) skip_automation_work(work_id, reason) fail_automation_work(work_id, reason) ```

These functions may update only work-row status/attempt fields. They must not mutate processes, events, registry data, or workflow definitions.

Diagnostic fields

Work rows should capture enough detail for operations and audit triage:

text last_attempt_at last_attempt_result # accepted | skipped | failed | transient_error last_attempt_error # concise engine/worker error text attempts fired_event_hash # only when accepted

The system_log should also capture worker attempts where practical, especially skips and failures. This is diagnostic feedback; it is not process state.

Recurring schedules

For recurring schedules:

• accepted fire: increment fire_count, reschedule unless bounds are exhausted,
• business skip: record skip and schedule the next recurrence unless bounds say stop,
• transient failure: retry the same due occurrence,
• repeated failures: mark failed according to retry policy.

For one-shot schedules:

• accepted fire: exhausted or completed,
• business skip: skipped,
• transient failure: retry.

Engine integration

Mutate effects

apply_mutate returns all side effects for the engine to persist atomically:

rust pub struct MutateEffects { pub new_data: JsonValue, pub spawns: Vec<ResolvedSpawn>, pub schedules: Vec<ResolvedSchedule>, pub unschedules: Vec<String>, }

The engine persists schedules/unschedules inside the same transaction as the causing event.

Auto-action evaluation

After a successful event mutation and before transaction commit, the engine:

1. records the event,
2. computes current state,
3. inspects actions in the current workflow,
4. selects actions with auto enabled,
5. checks automation role eligibility,
6. evaluates from/state and when against the new process state,
7. inserts deduplicated automation_queue rows,
8. emits NOTIFY hints.

The eventual fired event is still fully checked by process_event. Queueing is an optimization/materialization of availability, not a bypass.

Guard rule compliance

Rule	Compliance
No delete/edit/reorder	Work rows are statused, not deleted. Fired events are appended.
Chain never breaks	Worker fetches latest hash and submits normal signed events.
Actor = public key	Automation actor has its own Ed25519 key.
No dev bypass	No "0000"; all automated events are signed.
Keys/tokens not co-located	Automation private key lives in worker env, not DB.
No action without workflow	Engine queues only defined actions and revalidates at fire time.
Rules at decision time	Fire-time process_event validates against current workflow rules.
State is authority	Auto eligibility is evaluated against committed process state.
Engine is chokepoint	Worker submits only through process_event.
Every event verifiable	Automated events include public key, signature, hash chain, provenance.
System log append-only	Attempts and fired events log like other actions.

Implementation plan

Task 1 — Actor and schema foundations

• Add automation actor seed/public key.
• Add AUTOMATION role.
• Add scheduled_actions provenance columns if missing.
• Add automation_queue table.
• Add statuses, indexes, lease fields, and privileges.
• Add source invariants for no automation private key in DB/source.

Acceptance:

• reset/reseed creates automation actor with public key only,
• table privileges block raw app writes,
• zero "0000" signatures remain.

Task 2 — Scheduled actions provenance and NOTIFY

• Persist schedule provenance from causing event:
• event hash,
• action name,
• actor public key,
• created timestamp.
• Emit pg_notify('scheduled_actions_changed', payload) on schedule/unschedule.
• Reserve input.meta.automation in scheduled action input.

Acceptance:

• scheduling event writes provenance,
• notification payload identifies the changed work row,
• cancellation emits notification,
• existing schedule tests pass.

Task 3 — Auto-action DSL validation

• Add auto field to workflow action model.
• Validate auto is boolean or object with optional canonical input.
• Require auto actions to grant AUTOMATION role.
• Reject user-provided input.meta.automation in auto action input.
• Ensure editor/WASM validation reports auto-action issues.

Acceptance:

• valid auto: true action passes,
• auto action without automation role fails,
• invalid auto shape fails,
• TypeScript remains render-only.

Task 4 — Engine auto-action queueing

• After successful event processing, evaluate auto actions in the active workflow against new state.
• Insert deduplicated automation_queue rows with provenance.
• Emit automation_queue_changed notifications.
• Avoid recursive process_event.
• Add loop/dedupe protections.

Acceptance:

• reaching a state queues exactly one eligible auto action,
• multiple incoming actions reaching the same state do not require duplicated logic,
• auto action is not queued when when/state fails,
• queue rows link to the causing event.

Task 5 — Automation-worker realtime scheduler

• Rename/refactor watcher to automation-worker.
• Use direct pg connection for LISTEN.
• Keep signing/canonicalization via governed-events WASM.
• Maintain timer queue for scheduled actions.
• Process automation_queue ASAP.
• Add startup scan, reconnect scan, and safety sweep.
• Add claim/complete/fail RPCs or safe SQL functions.

Acceptance:

• no steady-state polling for normal work,
• schedule fires close to run_at,
• auto queue fires immediately,
• restart catches overdue/pending work,
• multiple workers do not double-fire claimed rows.

Task 6 — Fired event provenance

• Worker injects _automation provenance into fired event data.
• Engine rejects forged/reserved _automation from non-worker origins if needed.
• Fired event includes caused-by links from scheduled/auto queue rows.

Acceptance:

• auditor can trace fired event to scheduling/queueing event,
• worker does not invent provenance,
• provenance survives in event JSON.

Task 7 — Tests, docs, reset/reseed

• Rust tests for schedule provenance, notify path, auto validation, auto queueing.
• Worker tests for timer/queue/reconnect/result-classification behavior where practical.
• Bookkeeping RPC tests proving status updates cannot mutate process/event data.
• Integration tests for:
• scheduled action fired by automation actor,
• auto action fired after state reached,
• provenance links resolve,
• stale scheduled/auto actions are marked skipped with diagnostic reason,
• no raw process mutation.
• Update docs and guard invariants.
• Reset/reseed source DB if schema changes land.

Acceptance:

• all existing gates pass,
• get_workflows() unchanged except workflows that intentionally use auto,
• zero "0000" signatures,
• schedule and auto fired events are signed by automation actor,
• provenance verification query succeeds.

Non-goals for this slice

• external vendor integration worker,
• UI for automation queue administration,
• dynamic namespace creation,
• workflow versioning/history,
• deleting work rows,
• recursive same-transaction automated event execution.