Draft Proposal: Workflow-Governed Process Data and FEC UI Overhaul

Status: Draft for product/architecture review
Owner: LeadArchitect
Inputs: Product Owner feedback, DISC-02 team design discussion
Scope: Proposal only; no implementation commitment

1. Executive Summary

The eFEC engine is converging on the right architecture: all business mutations flow through process_event, actions are workflow-governed, event history is verifiable, and the universal action dialog can construct signed events from workflow/server output. The current UX, however, exposes engine pieces directly rather than presenting a coherent platform to analysts, auditors, and admins. Pages such as Workbench, Customers, Cases, and Screening overlap, repeat data, and do not clearly map to user jobs.

The root cause is not only frontend layout. The platform does not yet have a formal, enforced process data model. Customer data currently feels arbitrary because process data is effectively a loose JSON object, while workflows contain only partial/de-facto schema metadata.

This proposal has two parts:

1. FEC-independent process data structure definition: formalize workflow-owned process data schema, using the existing $fields and $display conventions as the starting point, and enforce it inside the engine.
2. FEC-specific UI overhaul: redesign navigation and presentation around user personas and schema-governed process instances rather than hardcoded customer/case/screening pages.

Key recommendation: extend and enforce the existing workflow $fields model instead of inventing a parallel data_schema syntax. $fields already appears in most seeded workflows and already functions as de-facto process schema/display metadata. Formalizing it reduces migration risk and keeps schema attached to immutable workflow versions.

2. Goals and Non-Goals

Goals

• Make process data structure explicit, workflow-governed, versioned, and enforceable.
• Keep the model generic: applicable to any workflow namespace, not specific to customers or FEC.
• Preserve eFEC guardrails:
• all mutations through process_event;
• canonical input envelope;
• no frontend workflow evaluation;
• no raw writes;
• immutable event history;
• Rust/WASM parity.
• Enable schema-driven process views, forms, tables, and detail pages.
• Simplify FEC UX around personas and primary jobs.
• Separate analyst operational UX from auditor/admin diagnostic UX.

Non-Goals

• Do not design a full low-code UI builder.
• Do not introduce frontend-only validation as a substitute for engine validation.
• Do not make customer data a separate application model outside process data.
• Do not change canonical event hash formula.
• Do not require per-state process schemas in the first iteration.
• Do not implement automatic child-process spawning as part of this proposal; model it as a related but separate workflow/process graph capability.

3. Part A: FEC-Independent Process Data Structure Definition

3.1 Current State

Current workflows already contain schema-like metadata:

• $fields describes fields using attributes such as type, label, section, required, system, and options.
• $display describes presentation metadata such as label, icon, list_columns, and default_sort.
• Action set rules commonly map from input.fields.* into process data.

This is already a de-facto process data schema. The missing pieces are:

• formal syntax and validation semantics;
• deploy-time validation of workflow schema;
• runtime enforcement during process_event;
• clear distinction between consensus validation metadata and UI display metadata;
• schema-driven frontend rendering.

3.2 Recommendation: Formalize $fields

Use workflow-level $fields as the authoritative process data schema for a workflow version.

Example draft shape:

json { "$schema_enforce": true, "$display": { "label": "Corporate Customer", "icon": "building", "list_columns": ["legal_name", "jurisdiction", "state", "risk_score"], "default_sort": [{ "field": "updated_at", "direction": "desc" }] }, "$fields": { "legal_name": { "type": "string", "label": "Legal Name", "section": "identity", "required": true, "readonly": false, "audit_sensitivity": "pii" }, "jurisdiction": { "type": "string", "label": "Jurisdiction", "section": "identity", "required": true }, "risk_score": { "type": "number", "label": "Risk Score", "section": "risk", "readonly": true, "computed": true, "audit_sensitivity": "internal" }, "kyc_case": { "type": "reference", "label": "KYC Case", "section": "related", "reference": { "namespace": "case/review", "workflow": "kyc-review" }, "readonly": true }, "raw_screening_payload": { "type": "json", "label": "Screening Payload", "section": "screening", "audit_sensitivity": "restricted" } }, "$sections": { "identity": { "label": "Identity", "order": 10 }, "risk": { "label": "Risk", "order": 20 }, "related": { "label": "Related Processes", "order": 30 }, "screening": { "label": "Screening", "order": 40 } } }

Notes:

• $fields is consensus-relevant schema metadata because it controls process mutation validity.
• $display and $sections are presentation metadata. They may live in the immutable workflow snapshot, but the engine should only depend on validation-relevant parts.
• $schema_enforce allows opt-in rollout per workflow version.

3.3 Field Types

Start with a small deterministic type set:

Type	Meaning	Validation
string	UTF-8 JSON string	value must be JSON string
number	JSON number	value must be finite JSON number
boolean	JSON boolean	value must be boolean
date	ISO date string	strict YYYY-MM-DD string
datetime	ISO/RFC3339 timestamp string	strict timestamp string
enum	one of configured options	value must match option value
reference	process reference	process id string initially; later structured reference object
json	opaque object/array/scalar	accepted as JSON, optionally constrained later
object	declared nested object	only if a nested field model is explicitly needed

Avoid expanding the type system too early. money, country, identifier, etc. should initially be display/widget hints over canonical primitive types unless they require consensus validation.

3.4 Field Attributes

Initial attributes:

Attribute	Owner	Purpose
type	engine	deterministic field validation
label	frontend	human-readable display
section	frontend	grouping/detail layout
required	engine	required presence constraint
default	engine	create/init default, applied deterministically
options	engine/frontend	enum values and labels
readonly	engine	rejects analyst/action writes unless system-derived path is allowed
computed	engine/frontend	indicates derived field, not submitted by users
system	engine/frontend	backwards-compatible alias/flag for engine-owned fields
audit_sensitivity	frontend/policy	redaction and visibility hints
reference	engine/frontend	allowed target namespace/workflow for process references
description	frontend	help text
widget	frontend	rendering hint only, not validation logic

3.5 Engine Responsibilities

The engine owns deterministic, workflow-versioned validation:

1. Deploy-time validation during workflow registration/update:
2. parse $fields;
3. reject invalid types/options/attributes;
4. reject set paths targeting undeclared fields when $schema_enforce is true;
5. type-check set source references such as input.fields.doc_count against target $fields types, not only literal values;
6. reject self-contradictory input schemas that ask users to provide readonly/computed fields;
7. warn or reject unsupported schema/display combinations;

8. preserve schema in immutable workflow version snapshot.

9. Runtime validation during process_event:

10. resolve workflow/action/state/guard as today;
11. validate action input if/when action input schema exists;
12. construct the process data delta from set/merge/append rules;
13. validate the resulting delta/final data against $fields;
14. reject unknown fields, invalid types, missing required fields, and forbidden user-input writes to readonly/computed fields;
15. allow workflow-authored deterministic set rules to write derived readonly/computed fields when those rules pass deploy-time validation;

16. only after validation succeeds, proceed to canonical event persistence.

17. Canonical errors:

18. return stable error codes and field paths;
19. avoid frontend-specific phrasing in engine semantics.

3.6 Frontend Responsibilities

The frontend owns presentation only:

• render list columns from $display.list_columns;
• render detail sections/cards from $fields.section and $sections;
• select controls/widgets from type, options, and widget hints;
• show required/readonly/computed/sensitivity indicators;
• render allowed actions from server/Rust-WASM flow output;
• submit canonical input envelopes through the shared event builder/dialog.

The frontend must not:

• decide action availability;
• bypass engine validation;
• infer hidden workflow semantics;
• write directly to process data;
• treat customer/case/screening data as separate models outside workflow process data.

3.7 Validation Ordering and Hash Integrity

Validation must not alter canonical event hash semantics.

Recommended order:

1. Receive canonical action envelope with input.
2. Verify actor/signature/roles/workflow/state/guard.
3. Build data delta from workflow action rules.
4. Validate delta/final process data against $fields.
5. If invalid: reject; no event is appended.
6. If valid: append the original signed event using the existing canonical hash formula.

Important principle: schema validation is a pre-commit gate. It rejects invalid mutations; it does not rewrite the event payload.

3.8 Requiredness and Defaults

Open design choice: requiredness can mean either:

• lifecycle required: field must exist after create/init; or
• eventually required: field must exist before particular transitions.

Recommendation for first iteration:

• Process-wide required means required after create/init when $schema_enforce is true.
• Transition-specific requiredness remains in action guards/input requirements.
• Defaults apply only during create/init or deterministic schema initialization.
• Defaults never mutate historical events retroactively.
• A default added in workflow version N+1 must not backfill process data created under workflow version N during replay.

3.9 Readonly, Computed, and System Fields

System-derived data should be explicit.

Examples:

• state: system/readonly, engine-owned.
• risk_score: computed/readonly if produced by rules/automation.
• kyc_result: readonly if written only by a linked child process result event or automation actor.

Initial rule:

• readonly means "not user-submitted" rather than "never changed";
• analyst/action input must not directly provide readonly or computed fields;
• workflow-authored deterministic set rules may write readonly/computed fields, because that is how derived/system fields are produced;
• deploy-time validation must reject action input schemas that request readonly/computed fields from users;
• deploy-time validation must still type-check workflow set rules that write readonly/computed fields;
• automation writes require explicit actor/workflow semantics before broadening this rule.

This precedence avoids a contradiction where a field such as risk_score is correctly marked readonly for analysts but still needs to be written by approved workflow logic.

3.9.1 set, merge, append, and Nested Data

The first schema-enforced version should avoid a full nested schema language.

Recommended v1 rule:

• scalar set targets are validated against declared $fields type and attributes;
• set values that reference input.fields.* must be type-compatible with their target fields at deploy time and runtime;
• merge targets must be declared as an opaque json field unless/until a nested schema extension is approved;
• append targets must be declared as an opaque json or array-compatible json field in v1;
• v1 does not validate array element schemas or nested object member schemas;
• future nested schemas can be added after the flat field contract is proven.

This keeps consensus validation deterministic and small while still blocking accidental writes to undeclared top-level process data.

3.10 References and Process Graphs

KYC, screening, investigations, accounts, and cases should be modeled as process instances, not duplicated JSON fields on the customer.

First iteration reference model:

json { "kyc_case": { "type": "reference", "reference": { "namespace": "case/review", "workflow": "kyc-review" }, "readonly": true } }

A reference field initially stores a process id. Later it may become a structured object:

json { "process_id": "...", "namespace": "case/review", "workflow": "kyc-review", "relationship": "kyc_case" }

Related but separate future work:

• explicit child process creation action semantics;
• process graph query API;
• relationship events between processes;
• UI graph of parent/customer and child KYC/screening/case branches.

3.11 Migration and Compatibility

Recommendation:

• Introduce $schema_enforce: true as opt-in per workflow version.
• Existing workflows/processes continue without enforcement until migrated.
• Convert current $fields metadata into formal schema in seeded workflows.
• Add missing $fields to workflows that currently lack them.
• Run reconcile/dry-run tooling against existing process data before enabling enforcement.
• A clean dev reset is acceptable if Product Owner approves, but the architecture should not depend on resets long term.

3.12 Consensus and Parity Risks

High-risk areas:

• schema validation becomes consensus logic;
• Rust/WASM parity must be exact;
• canonical error semantics should be stable;
• validation must not mutate submitted events or alter hash formula;
• display metadata must not become hidden consensus logic accidentally.

Mitigations:

• implement schema parser/validator in shared governed-events core;
• expose the same logic to WASM;
• test Rust and WASM parity with shared fixtures;
• add a dedicated schema_validation_tests fixture set covering valid schemas, invalid schemas, every supported field type, required/default/readonly/computed attributes, input-ref-to-target type compatibility, and invalid-submission rejection before hashing/persistence;
• keep initial type system small;
• keep validation deterministic and independent of database lookups except where explicitly designed, such as reference existence validation.

4. Part B: FEC-Specific UI Overhaul

4.1 Current UX Problem

The current UI exposes implementation pieces rather than a coherent product:

• Workbench, Customers, Cases, Screening, Processes, Events, and diagnostics overlap.
• Users see repeated process data without a clear task model.
• Customer detail mixes arbitrary JSON, workflow actions, related processes, screening/case concepts, and audit timeline.
• Some pages are unclear: are they operational surfaces, diagnostics, or admin tools?
• Child-process concepts such as KYC are not represented clearly.

The UI should be rebuilt around personas and jobs-to-be-done, with process schema as the rendering source.

4.2 Personas

Analyst

Primary user for day-to-day compliance work.

Needs:

• know what requires attention now;
• open an entity/case;
• see current state and key data;
• perform allowed workflow actions;
• review related KYC/screening/case processes;
• avoid audit/admin noise unless needed.

Investigator / Compliance Manager

Needs:

• escalations and approvals;
• queue oversight;
• exception/risk views;
• ability to inspect linked cases and decision rationale.

Auditor

Needs:

• immutable event history;
• hash chain/signature verification;
• exact submitted event payloads;
• workflow version/snapshot at decision time;
• read-only access, minimal operational UI.

Admin / Workflow Owner

Needs:

• workflow deployment/versioning;
• actor/role management;
• automation/scheduler configuration;
• diagnostics and health checks;
• schema/display review.

4.3 Proposed Navigation

Recommended top-level navigation:

1. Inbox
2. Processes
3. Audit
4. Admin

Optional shortcuts can appear under Processes, but should not become separate data models.

Inbox

Primary analyst landing page.

Purpose:

• show available work across process types;
• group by priority, SLA, state, workflow, assignment, or risk;
• drive users to the next allowed action.

Data source:

• server/Rust flow projection and process metadata;
• not hardcoded action lists.

Rows could show:

• process type/workflow label;
• process display name from $display/schema fields;
• current state;
• available action count/action labels;
• risk/SLA indicators;
• assigned actor/team where available.

Processes

Unified process browser.

Purpose:

• replace fragmented Customers/Cases/Screening pages with one schema-driven browser;
• provide saved filters for common domains.

Saved filters/shortcuts:

• Customers
• KYC Reviews
• Screening Alerts
• Investigations
• Accounts

These are filters over process namespace/workflow/state, not separate frontend models.

List rendering:

• columns from $display.list_columns;
• labels from workflow $display;
• field formatting from $fields;
• no hardcoded customer table columns except temporary migration fallback.

Process Workspace

Detail page for any process instance.

Purpose:

• single operational workspace for an entity/case/screening process.

Recommended layout:

1. Header
2. workflow label/icon;
3. process id truncated with copy/full view;
4. state/status;

5. key badges from schema display metadata.

6. Summary

7. key fields from $display.summary_fields or first important $fields.

8. Allowed Actions

9. rendered from server/Rust flow output;

10. submit through universal ActionDialog.

11. Data Sections

12. generated from $fields and $sections;
13. readonly/computed/sensitive indicators;

14. schema-driven formatting.

15. Related Processes

16. references/process graph: KYC case, screening alert, investigation, account, etc.;

17. no duplicated child data unless shown as a summary card.

18. Timeline

19. process-local event timeline from create event onward;
20. action, date, process id, event hash;

21. click opens audit event detail.

22. Evidence/Notes

23. only when represented by workflow-governed process data or linked processes/events;
24. avoid side tables that bypass process_event.

Audit

Read-only audit explorer.

Purpose:

• event detail;
• hash chain verification;
• signature verification;
• workflow snapshot/version at event time;
• replay/lineage views;
• process graph and event graph diagnostics.

Audit should not be the analyst default experience. It is for proof and investigation.

Admin

Purpose:

• workflows and schema review;
• actors/roles;
• automation and schedules;
• namespace registries;
• diagnostics/health;
• seed/bootstrap state.

Admin is where raw tables/diagnostics belong, clearly separated from operational UX.

4.4 Pages to Remove, Merge, or Reframe

Current Concept	Proposed Treatment
Workbench	Rename/reframe as Inbox if task/action-driven; otherwise remove as redundant.
Customers	Saved filter under Processes + schema-driven customer workspace.
Cases	Saved filter under Processes, not separate model.
Screening	Saved filter or queue only if it has distinct alert workflow; otherwise related process tab.
Processes	Keep as unified process browser.
Events	Move under Audit.
Raw diagnostics	Move under Admin/Diagnostics.

4.5 Customer Data Model in FEC

A customer should be a process instance governed by its workflow. There should be no separate customer data model that competes with process data.

Corporate customer example:

• namespace: customer/corporate
• workflow: corporate-customer
• data governed by $fields
• related KYC/screening/investigation/account processes linked via reference fields or process graph relationships

Example sections:

• Identity
• Registration
• Ownership
• Risk
• KYC
• Screening
• Relationships
• Audit Summary

System fields:

• state
• risk_score
• kyc_result
• screening_status
• last_reviewed_at
• references to child process ids

Analyst-facing customer view should show curated schema sections and allowed actions. Auditor-facing view should show event lineage and exact submitted events.

4.6 KYC, Screening, and Cases

The current spawn_kyc behavior demonstrates the need for clearer modeling:

• an action named spawn_kyc recorded intent but did not create a KYC process;
• users then had no way to find the expected KYC process.

Recommendation:

• model KYC review as a real child/related process using case/review/kyc-review;
• create/link child processes through explicit signed events;
• store the child process id as a reference field or relationship event;
• show child processes in the parent Process Workspace under Related Processes;
• avoid duplicating child status except as read-only summary fields derived from linked process state.

The exact child-process creation mechanism should be specified separately, but the UI model should assume linked process graph support.

5. Implementation Sequence

Phase 0: Product/UX Inventory

No engine changes.

Deliverables:

• map current pages to personas and jobs;
• identify duplicated data and unclear surfaces;
• decide final top-level navigation labels;
• define minimum analyst, auditor, and admin workflows.

Phase 1: $fields Schema Specification

No implementation until approved.

Deliverables:

• formal $fields grammar;
• type semantics;
• required/default/readonly/computed rules;
• set source/target type-checking rules, including input.fields.* indirection;
• merge/append v1 rule for opaque json fields;
• $schema_enforce semantics;
• validation order in process_event;
• canonical error model;
• Rust/WASM parity fixtures;
• relationship/reference semantics v1.

Decision required:

• confirm extending $fields instead of adding parallel data_schema;
• confirm no per-state schemas in v1;
• confirm opt-in enforcement.

Phase 2: Schema-Driven UI Prototype

Frontend prototype only; may use static/sample schemas.

Deliverables:

• schema-driven process list;
• schema-driven process detail sections;
• schema-driven action input rendering using existing ActionDialog;
• prototype navigation: Inbox / Processes / Audit / Admin;
• no frontend workflow evaluation.

Purpose:

• validate UX before making schema enforcement consensus-critical.

Phase 3: Engine Schema Parser and Deploy-Time Validation

Deliverables:

• parse $fields in shared Rust core;
• validate workflow schema during deployment/registration;
• validate set target paths and input.fields.* source type compatibility;
• validate readonly/computed input-vs-workflow ownership rules;
• expose parsed schema through WASM/API;
• reject invalid schemas;
• parity tests.

No runtime process mutation enforcement yet unless behind a test flag.

Phase 4: Runtime Enforcement Behind $schema_enforce

Deliverables:

• validate action input, resulting data deltas, and final process data before commit;
• reject unknown fields and invalid types;
• enforce readonly/computed/required/default semantics;
• validate opaque-json behavior for merge/append v1;
• ensure invalid submissions append no event;
• preserve canonical event hash formula;
• add Rust/WASM/SQL integration tests, including a reject-before-hash/persistence fixture.

Phase 5: Seed Workflow Migration

Deliverables:

• update all seeded workflows to formal $fields;
• add missing schemas for KYC, screening, accounts, and other workflows;
• reconcile/dry-run existing process data;
• reset dev seed data if approved.

Phase 6: UI Consolidation

Deliverables:

• Inbox replaces current scattered Workbench behavior;
• Processes browser replaces standalone Customers/Cases/Screening as primary model;
• process workspace becomes universal detail shell;
• Audit Explorer owns events/hash/signature views;
• Admin owns workflow/actor/diagnostic views;
• old redundant pages removed or converted to saved filters.

6. Open Decisions

1. Schema syntax: extend $fields as recommended, or introduce a new data_schema block?
2. Enforcement rollout: opt-in $schema_enforce per workflow version, or mandatory after migration?
3. Per-state schemas: defer as recommended, or support state-specific field requirements early?
4. References: store process id string initially, or structured reference object from v1?
5. Child process creation: specify now, or handle in a separate process-graph proposal?
6. Navigation labels: should analyst home be called Inbox, Workbench, or Queue?
7. Role-specific navigation: separate analyst/auditor/admin navigation roots, or same nav with role-gated pages?
8. Dev reset: is a clean seed reset acceptable when schema enforcement lands?

7. Proposed Immediate Next Steps

1. Product Owner reviews this draft and confirms direction on open decisions.
2. LeadArchitect converts this into two approved specs:
3. $fields process data schema enforcement spec;
4. FEC UX information architecture spec.
5. Team prototypes schema-driven process list/detail using current $fields before engine enforcement.
6. Team writes shared schema validation fixtures before modifying process_event.

8. Team Input Incorporated

Developer input:

• proposed workflow-level process schema separate from action input schemas;
• emphasized engine validation vs frontend rendering separation;
• recommended queue/entity/case/audit/admin UX organization;
• warned against overloading schema as both validation and layout.

SeniorDeveloper input:

• identified $fields as existing de-facto process schema;
• recommended extending/enforcing $fields instead of inventing parallel syntax;
• proposed $schema_enforce opt-in migration;
• emphasized validate-before-hash ordering and Rust/WASM parity;
• recommended collapsing Customers/Cases/Screening into unified Processes browser with saved filters;
• recommended Inbox-first analyst UX and separate Audit/Admin surfaces.