gigafibre-fsm/docs/FLOW_EDITOR_ARCHITECTURE.md

Flow Editor — Architecture & Integration Guide

Authoritative reference for the Flow Editor subsystem: visual builder in the Ops PWA, JSON flow-definition language, runtime engine in targo-hub, and the Frappe scheduler that wakes delayed steps. Companion document to ARCHITECTURE.md and DATA_AND_FLOWS.md.

---

1. Goals & Principles

The Flow Editor lets operators compose automations — onboarding, follow-ups, SLA escalations, billing side-effects — without writing code or leaving the ERPNext/Ops perimeter. It behaves like n8n or Shopify Flow, but is natively embedded in the Ops PWA and talks directly to ERPNext via the targo-hub.

Design tenets:

1. One UI, many domains. A single FlowEditor.vue tree is parameterised by a kind catalog (PROJECT_KINDS, AGENT_KINDS, …). Adding a new catalog adds a new editor, no UI fork.
2. Inline-editable from anywhere (Odoo pattern). FlowQuickButton + useFlowEditor() let any page pop the editor with preset filters (category/applies-to/trigger) — the user never has to route to Settings to create a flow.
3. Pluggable runtime via a kind dispatcher. KIND_HANDLERS in flow-runtime.js is a plain map — no if/else chain. Adding a step kind = write one async function.
4. No code-as-config. Predicates go through a pure evaluator (evalCondition) — never eval() / Function(). Templates are rendered with a regex ({{a.b.c}}), no Mustache lib.
5. Persist decisions, not heartbeats. Delayed steps become Flow Step Pending rows with a trigger_at. The scheduler never re-executes logic itself — it just nudges the Hub.
6. Seeded templates are protected (is_system=1). Operators can duplicate them but not delete or rename.

---

2. File Inventory

Front-end — apps/ops/src/

Path	Responsibility
components/flow-editor/FlowEditor.vue	Vertical-tree view + drag-to-reorder, consumes a kind catalog
components/flow-editor/FlowNode.vue	Single step card (icon, label, menu, branch connectors)
components/flow-editor/StepEditorModal.vue	Modal that edits one step's payload (trigger + kind fields)
components/flow-editor/FieldInput.vue	Polymorphic field widget (text/number/select/datetime/…)
components/flow-editor/FlowEditorDialog.vue	Full-screen dialog for template-level edits (meta + tree)
components/flow-editor/FlowTemplatesSection.vue	Settings list: filter, duplicate, delete, toggle active
components/flow-editor/FlowQuickButton.vue	Drop-in "New flow / Edit flow" button
components/flow-editor/kind-catalogs.js	PROJECT_KINDS, AGENT_KINDS, TRIGGER_TYPES, buildEmptyStep
components/flow-editor/index.js	Barrel export
composables/useFlowEditor.js	Module-level singleton reactive store
api/flow-templates.js	REST client for /flow/templates/*

Back-end — services/targo-hub/lib/

Path	Responsibility
flow-templates.js	REST API: CRUD + duplicate + validation
flow-runtime.js	Execution engine: kind handlers, wave loop, template render
flow-api.js	REST API: /flow/start, /flow/advance, /flow/complete, /flow/event, /flow/runs
contracts.js	Fires on_contract_signed on JWT confirm
payments.js	Fires on_payment_received on Stripe webhook
acceptance.js	Fires on_quotation_accepted on accept/confirm

ERPNext — erpnext/

Path	Responsibility
flow_scheduler.py	Frappe cron hook (every minute) — claims due pending steps
seed_flow_templates.py	Seeds the 4 project templates + residential_onboarding + quotation_follow_up
setup_flow_templates.py	DocType creation (Flow Template / Flow Run / Flow Step Pending)

---

3. Data Model (ERPNext doctypes)

Three custom doctypes live under ERPNext. All named by series (FT-NNNNN, FR-NNNNN, FSP-NNNNN).

3.1 Flow Template

Authoritative definition of an automation. Edited via the Flow Editor.

Field	Type	Notes
template_name	Data	Human label (unique within category)
category	Select	residential / commercial / incident / billing / other
applies_to	Link	Target DocType (Service Contract, Quotation, Issue, …)
icon	Data	Material icon name
description	Small Text	One-liner
is_active	Check	Only active templates dispatch on events
is_system	Check	Seeded by code — cannot be deleted; duplicating is allowed
version	Int	Bumped on every save (used for audit trail)
trigger_event	Select	on_contract_signed, on_payment_received, on_subscription_active, on_quotation_created, on_quotation_accepted, on_issue_opened, on_customer_created, on_dispatch_completed, manual
trigger_condition	Code	Optional JSONLogic-ish gate — today only == is wired
flow_definition	Long Text (JSON)	The whole graph — see §4
step_count	Int	Computed on save for list display
tags, notes	Data / Text	Free-form metadata

3.2 Flow Run

One execution of a template against a specific trigger doc.

Field	Type	Notes
flow_template	Link	The FT we're executing
template_version	Int	Snapshot at start — template edits don't retro-affect
status	Select	running / completed / failed / cancelled
trigger_event	Data	What caused this run (for audit)
context_doctype	Data	Trigger doc's doctype
context_docname	Data	Trigger doc's name
customer	Link Customer	Resolved customer for template rendering
variables	Long Text (JSON)	Runtime bag passed by dispatchEvent (amount, signed_at, …)
step_state	Long Text (JSON)	{ [stepId]: { status, started_at, completed_at, result, error } }
started_at, completed_at	Datetime	Timestamps

3.3 Flow Step Pending

One row per delayed step. The scheduler processes these.

Field	Type	Notes
flow_run	Link	Parent run
step_id	Data	The step within flow_definition.steps
trigger_at	Datetime	When to fire (indexed — scheduler queries this)
status	Select	pending → running → completed or failed
retry_count	Int	Increments on scheduler failures (caps at 5)
last_error	Small Text	Truncated error from last attempt
context_snapshot	Long Text (JSON)	Step payload captured at schedule time (optional)
executed_at	Datetime	Set when scheduler successfully nudges the Hub

---

4. Flow Definition JSON Schema

Stored as the flow_definition field of Flow Template and as {step} in Flow Step Pending.context_snapshot.

{
  "version": 1,
  "trigger": { "event": "on_contract_signed", "condition": "" },
  "variables": { "default_group": "Tech Targo" },
  "steps": [ /* Step[] */ ]
}

4.1 Step shape

{
  "id": "step_abc123",
  "kind": "dispatch_job",
  "label": "Installation fibre",
  "parent_id": null,
  "branch": null,
  "depends_on": [],
  "trigger": { "type": "on_flow_start" },
  "payload": {
    "subject": "Installation {{customer.customer_name}}",
    "job_type": "Installation",
    "priority": "medium",
    "duration_h": 2,
    "assigned_group": "Tech Targo"
  }
}

Field	Type	Notes
id	string	Stable UID, generated client-side (step_xxxxxx)
kind	string	Must exist in KIND_HANDLERS
label	string	Free text shown in the editor + timeline
parent_id	string|null	If set, this step is inside the parent's branch
branch	string|null	Matches parent.result.branch (e.g. "yes" / "no")
depends_on	string[]	Must all be done before this step becomes ready
trigger	object	When to fire — see §4.2
payload	object	Kind-specific fields (see §4.3)

4.2 Trigger types

trigger.type	Extra fields	Semantics
on_flow_start	—	Fires in the very first wave of advanceFlow
on_prev_complete	—	Fires when all depends_on are done
after_delay	delay_hours, delay_days	Scheduled — creates a Flow Step Pending row
on_date	at (ISO)	Scheduled — creates a Flow Step Pending row
on_webhook	—	Stays pending, external POST to /flow/complete fires it
manual	—	Stays pending, user clicks a button to fire

4.3 Payloads per kind

The authoritative list lives in apps/ops/src/components/flow-editor/kind-catalogs.js (PROJECT_KINDS). The runtime map in flow-runtime.js must be kept in sync.

Kind	Payload
dispatch_job	subject, job_type, priority, duration_h, assigned_group, on_open_webhook, on_close_webhook, merge_key
issue	subject, description, priority, issue_type
notify	channel (sms/email), to, template_id, subject, body
webhook	url, method, body_template (JSON string)
erp_update	doctype, docname_ref, fields_json (JSON string)
wait	— (delay controlled by trigger.type=after_delay)
condition	field, op, value → exposes result.branch = "yes"|"no"
subscription_activate	subscription_ref

4.4 Template interpolation

Strings in payload support {{a.b.c}} dotted paths. The context is:

{
  doc,          // the trigger doc (contract, quotation, issue)
  customer,     // resolved Customer doc
  run,          // the Flow Run (name, variables, step_state)
  template,     // the Flow Template (name, version)
  variables,    // run-level variable bag
  now           // current ISO datetime
}

Unknown paths render as empty string — rendering never throws.

---

5. Front-end Architecture

5.1 Component hierarchy

MainLayout.vue
  └─ FlowEditorDialog.vue          (mounted once, globally)
       └─ FlowEditor.vue
            └─ FlowNode.vue (xN)
                 └─ StepEditorModal.vue
                      └─ FieldInput.vue (xN)

Any page
  └─ FlowQuickButton.vue           (opens the global dialog)

SettingsPage.vue
  └─ FlowTemplatesSection.vue      (list/filter/toggle/duplicate)

5.2 useFlowEditor() — singleton reactive store

Module-level refs shared across every call to useFlowEditor() — no provide/inject, no Pinia store. Any component can call openTemplate() and the globally-mounted <FlowEditorDialog> reacts automatically.

Read-only exposed refs: isOpen, loading, saving, error, dirty, mode, templateName. Mutable: template (the draft body).

Actions:

Method	Behaviour
openTemplate(name, opts)	Fetches FT, populates template, isOpen = true, mode=edit
openNew(opts)	Blank draft with category/applies_to/trigger_event presets
close(force?)	Confirms unsaved changes unless force=true
save()	Create or patch — bumps version, invokes onSaved callbacks
duplicate(newName)	Clones current, loads the clone
remove()	Deletes current (forbidden on is_system=1)
markDirty()	Called on every field mutation — O(1), avoids deep-watch cost

5.3 Pluggable kind catalogs

FlowEditor takes a kindCatalog prop. Two are shipped:

• PROJECT_KINDS — 8 kinds for service-delivery automations
• AGENT_KINDS — 6 kinds for conversational agent flows (preserves the old AgentFlowsPage semantics for a later drop-in refactor)

Each catalog entry defines: label, icon, color, optional hasBranches/branchLabels, and a fields[] array of field descriptors. Descriptor shape is documented at the top of kind-catalogs.js. Adding a field descriptor = the modal renders the right widget automatically.

5.4 FlowQuickButton — inline entry points

Drop-in button with two modes:

<!-- New flow, pre-filtered -->
<FlowQuickButton category="residential" applies-to="Service Contract" />

<!-- Edit an existing template -->
<FlowQuickButton template-name="FT-00005" label="Modifier le flow" />

Used in: ProjectWizard.vue, TicketsPage.vue, IssueDetail.vue, ClientDetailPage.vue (Contrats de service header). Add it wherever a contextual "spin up an automation from here" shortcut makes sense.

---

6. Runtime (flow-runtime.js)

6.1 Wave loop — advanceFlow(runName)

1. Fetch run + template + build context (_buildContext).
2. Loop up to 50 waves:
   1. For each step, isStepReady(step, state, def):
      • all depends_on done?
      • parent done + branch matches parent's result?
   2. If ready:
      • after_delay/on_date → schedule Flow Step Pending, state=scheduled
      • on_webhook/manual → leave pending, caller resumes later
      • else → inline execute via KIND_HANDLERS[step.kind], mutate state
   3. If no step progressed, break.
3. Compute run-level status: completed if all done, failed if any failed.
4. Persist step_state + patch once (single PUT).

The wave loop means a single advanceFlow call collapses an entire chain of instant steps into one write — the scheduler's job is just to kick the first domino after a delay.

6.2 Kind dispatcher

const KIND_HANDLERS = {
  dispatch_job, issue, notify, webhook, erp_update,
  subscription_activate, wait, condition,
}

Every handler has the same signature:

async function handleXxx(step, ctx) { … return { status, result?, error? } }

status is one of STATUS.DONE, STATUS.FAILED, or STATUS.SCHEDULED.

6.3 Condition evaluator

evalCondition({ field, op, value }, ctx)  // → boolean

Supported ops: == != < > <= >= in not_in empty not_empty contains starts_with ends_with. No eval, no Function, no prototype access. condition steps return { branch: ok ? 'yes' : 'no' } so child steps can filter on branch.

6.4 dispatchEvent(eventName, opts)

The public event hook used by contracts.js, payments.js, acceptance.js, etc. It:

1. Queries Flow Template with trigger_event = eventName AND is_active = 1.
2. For each, evaluates trigger_condition (simple == JSON gate today).
3. Calls startFlow for matching templates.

Callers use it non-blocking (.catch(()=>{})) so automation failures never break the primary flow (contract signature, payment, etc.).

6.5 Scheduler (flow_scheduler.py)

Runs every minute via Frappe hooks.py:

scheduler_events = {
    "cron": { "* * * * *": ["flow_scheduler.tick"] }
}

Algorithm (_claim_due_rows → _fire_row):

1. Select Flow Step Pending where status='pending' AND trigger_at <= now() (limited to 50 per tick).
2. Atomically flip them to status='running' in one SQL UPDATE.
3. For each, POST {run, step_id} to HUB_URL/flow/complete.
4. On success: status='completed', executed_at=now().
5. On failure: increment retry_count, set status='pending' (or 'failed' after 5 retries), store truncated error in last_error.

The claim-then-fire split guarantees at-most-once execution even under concurrent ticks. retry_count gives durable back-off without polling logic.

---

7. HTTP API

7.1 /flow/templates (mounted in flow-templates.js)

Verb	Path	Body / Query	Response
GET	/flow/templates	?category=&applies_to=&is_active=&trigger_event=	{templates: FT[]}
GET	/flow/templates/:name	—	{template: FT}
POST	/flow/templates	{template_name, …}	{template}
PUT	/flow/templates/:name	partial patch	{template}
DELETE	/flow/templates/:name	—	{ok: true} (blocks is_system)
POST	/flow/templates/:name/duplicate	{new_name?}	{template}

Validation runs on every write: unique step IDs, known kinds, referential integrity of depends_on/parent_id.

7.2 /flow/* (mounted in flow-api.js)

Verb	Path	Body	Response
POST	/flow/start	{template, doctype, docname, customer, variables}	{run, executed, scheduled}
POST	/flow/advance	{run}	{run, executed, scheduled}
POST	/flow/complete	{run, step_id, result?}	{run, …}
POST	/flow/event	{event, doctype, docname, customer, variables}	{results: [...]}
GET	/flow/runs	?status=&template=&customer=	{runs: FR[]}
GET	/flow/runs/:name	—	{run: FR}

All endpoints support optional Authorization: Bearer $HUB_INTERNAL_TOKEN (controlled via INTERNAL_TOKEN env). Used by the Frappe scheduler.

---

8. Trigger Wiring

Event emitters are intentionally thin — two lines each, failure-tolerant.

8.1 on_contract_signed (in contracts.js)

Fired by the JWT accept flow (residential) on POST /contract/confirm:

_fireFlowTrigger('on_contract_signed', {
  doctype: 'Service Contract',
  docname: contractName,
  customer: payload.sub,
  variables: { contract_type, signed_at },
})

_fireFlowTrigger lazy-requires ./flow-runtime so the cost is paid only when a trigger actually fires.

8.2 on_payment_received (in payments.js)

Fired from the Stripe checkout.session.completed webhook handler, after the Sales Invoice is marked paid:

require('./flow-runtime').dispatchEvent('on_payment_received', {
  doctype: 'Sales Invoice',
  docname: invoiceName,
  customer,
  variables: { amount, payment_intent: piId },
})

Wrapped in try/catch so a flow bug never blocks a payment.

8.3 on_quotation_accepted (in acceptance.js)

Fired by the commercial accept flow (DocuSeal/JWT) on POST /accept/confirm:

require('./flow-runtime').dispatchEvent('on_quotation_accepted', {
  doctype: 'Quotation',
  docname: payload.doc,
  customer: payload.sub,
  variables: { accepted_at },
})

8.4 Future wiring

To hook another event (e.g. on_ticket_resolved):

1. Pick the source of truth (ERPNext Issue webhook, Ops inline save, etc.).
2. From that handler, call require('./flow-runtime').dispatchEvent(...).
3. Add the event name to TRIGGER_EVENT_OPTIONS in FlowEditorDialog.vue so operators can pick it from the UI.
4. (Optional) add a pre-seeded template in seed_flow_templates.py.

---

9. How-To Recipes

9.1 Add a new step kind

1. Front-end — in kind-catalogs.js, extend PROJECT_KINDS with a new entry:

   send_portal_invite: {
     label: 'Invitation portail',
     icon: 'mail_lock',
     color: '#0ea5e9',
     fields: [
       { name: 'to',   type: 'text', label: 'Email', required: true },
       { name: 'role', type: 'select', label: 'Rôle', options: ['customer', 'tech'] },
     ],
   }
   

2. Back-end — in flow-runtime.js, implement the handler:

   async function handlePortalInvite (step, ctx) {
     const p = renderDeep(step.payload || {}, ctx)
     // … call the existing portal-invite endpoint …
     return { status: STATUS.DONE, result: { … } }
   }
   

3. Register it in KIND_HANDLERS:

   const KIND_HANDLERS = { …, send_portal_invite: handlePortalInvite }
   

4. Add the kind name to flow-templates.js → validateFlowDefinition.validKinds.
5. Rebuild + deploy Hub; rebuild + deploy Ops SPA.

No UI changes: the new kind renders its fields automatically.

9.2 Add a new trigger event

1. Emit it: require('./flow-runtime').dispatchEvent('on_foo', {…}) from wherever the event occurs.
2. Add the label in FlowEditorDialog.vue TRIGGER_EVENT_OPTIONS.
3. (Optional) pre-seed a template in seed_flow_templates.py.

9.3 Add a new kind catalog (e.g. for billing flows)

1. Export a new const from kind-catalogs.js: export const BILLING_KINDS = {...}.
2. Pass it into any <FlowEditor :kind-catalog="BILLING_KINDS" /> instance.
3. Reuse the same FlowEditorDialog if the kinds live side by side; fork it only if you need dramatically different metadata panels.

9.4 Add a trigger condition

Today trigger_condition supports a simple { "==": [{"var": "path"}, "value"] } JSON gate. To extend to full JSONLogic:

1. Replace _matchCondition in flow-runtime.js with the json-logic-js library (already a stable dep elsewhere in Hub — check first).
2. Update validateFlowDefinition to reject unknown operators.
3. Document the new operators in this file.

---

10. Debugging Guide

10.1 Flow didn't run when expected

1. Check the event was emitted. docker logs targo-hub | grep '\[flow\]'. You should see [flow] started FR-XXXXX from FT-YYYYY.
2. Check the template is active + matches. In Ops → Settings → Flow Templates, confirm is_active, trigger_event, and trigger_condition (if set) match the event.
3. Inspect the Flow Run. GET /flow/runs/:name returns full state. step_state[stepId].status = 'failed' + .error pinpoints the cause.

10.2 Delayed step never fires

1. Check the scheduler is ticking. ERPNext backend logs: docker logs erpnext-backend-1 | grep flow_scheduler. If silent, confirm the cron entry in hooks.py and the Frappe scheduler worker is up (bench doctor).
2. Check the pending row. ERPNext → Flow Step Pending list: status, trigger_at, retry_count, last_error.
3. Manually fire it: POST /flow/complete with {run, step_id} from any host with access to the Hub.

10.3 Template renders empty strings

{{foo.bar}} returned empty means the path is absent from the context. Add a console.log(ctx) in the handler, or use a condition step upstream to verify the data is populated before the branch that uses it. Remember that customer is only loaded if run.customer is set or the trigger doc has a customer field.

10.4 "Step has no handler" error

The kind name in the template no longer matches KIND_HANDLERS. Either a deploy mismatch (front-end ahead of Hub) or a kind was removed. Revert the Hub or update the Flow Template.

10.5 Retry loop stuck

Flow Step Pending.retry_count maxes at 5 then flips to failed. To retry manually: set status='pending', retry_count=0 on the row, wait a minute. Fix root cause before retrying — the scheduler won't back off past 5.

---

11. Seeded Templates

Living in erpnext/seed_flow_templates.py. All created with is_system=1.

FT Name	Category	Trigger event	Purpose
fiber_install	residential	manual	4-step install (dispatch install + notify + activate sub + 24h survey)
phone_service	residential	manual	Phone-only activation
move_service	residential	manual	Move existing service to new address
repair_service	incident	on_issue_opened	Triage + dispatch repair
residential_onboarding	residential	on_contract_signed	Install + welcome SMS + sub activate + 24h survey + 11-month renewal reminder
quotation_follow_up	commercial	on_quotation_accepted	3-touch email sequence (D+3, D+7, D+14)

Run the seeder once after deploying the scheduler:

docker exec -u frappe erpnext-backend-1 bash -c \
  'cd /home/frappe/frappe-bench/sites && \
   /home/frappe/frappe-bench/env/bin/python -c \
   "import frappe; frappe.init(site=\"erp.gigafibre.ca\"); frappe.connect(); \
    from seed_flow_templates import seed_all; seed_all()"'

---

12. Security & Governance

• Seeded templates (is_system=1) cannot be deleted — only duplicated. This preserves the baseline automations even after operator mistakes.
• Internal token: all /flow/* endpoints honour Authorization: Bearer $HUB_INTERNAL_TOKEN. Set it in both Hub and ERPNext environment to lock down the API surface between containers.
• No eval anywhere: conditions run through a fixed predicate table, templates through a regex. Adding arbitrary code to a flow is not a supported feature — if a use case needs it, the answer is "add a new kind".
• Validation is mandatory on every write: duplicate IDs, unknown kinds, dangling depends_on, orphan parent_id all reject at the API.
• Audit: every Flow Run carries template_version so historical runs stay traceable even after the template is edited.

---

13. Deployment Topology

ERPNext backend ──cron every 60s──► flow_scheduler.tick()
                                        │
                                        │ (INTERNAL_TOKEN)
                                        ▼
Ops PWA  ───REST───►  targo-hub  ─PUT/POST─►  ERPNext (Flow Template, Flow Run, Flow Step Pending)
 FlowEditorDialog    flow-api.js              (+ Dispatch Job, Issue, Customer, Subscription, …)
                     flow-runtime.js
                     flow-templates.js
   ▲                       │
   └── dispatchEvent ◄──────┘
      (contracts.js, payments.js, acceptance.js)

Deploy order (important):

1. ERPNext: create doctypes via setup_flow_templates.py, wire flow_scheduler.tick in hooks.py, restart scheduler.
2. targo-hub: ship flow-runtime.js, flow-api.js, flow-templates.js + the trigger emitters in contracts.js / payments.js / acceptance.js. Restart container.
3. Ops SPA: build + deploy. The editor is usable instantly; it just can't dispatch until the Hub is up.
4. Seed the templates (seed_flow_templates.py).

---

14. Glossary

• Flow Template (FT) — the definition, reusable, edited by operators.
• Flow Run (FR) — one execution against one trigger doc.
• Flow Step Pending (FSP) — one row per delayed step, the scheduler's inbox.
• Kind — a step type (dispatch_job, notify, …). Runtime-dispatched via KIND_HANDLERS.
• Catalog — a set of kinds exposed in one editor instance (PROJECT_KINDS, AGENT_KINDS).
• Wave — one iteration of advanceFlow's inner loop. A flow typically needs 1–3 waves; the 50 cap is a safety net against cycles.
• Trigger event — the incoming signal that kicks off flows (on_contract_signed, …).
• Trigger type — the per-step firing rule inside a flow (on_prev_complete, after_delay, …).

---

Last updated: 2026-04-21. Owners: Targo Platform team (louis@targo.ca). See ROADMAP.md for upcoming Flow Editor v2 items (JSONLogic conditions, sub-flows, parallel branches, retry policies).