Gigafibre Field Service Management - ERPNext doctypes, dispatch app, architecture docs
d897bcedb4
The Map CSV migrated from the legacy ERP carries names with two common
defects: missing French accents (Stephane, Andre, Frederic), and
compound first names that were typed without a separator (Marcandre,
Mariejosee, Jeanphilippe). Sending an email "Bonjour stephane," instead
of "Bonjour Stéphane," reads as sloppy automation. Fix both at parse
time so the user sees the corrected names in Step 2 and can override
inline if the auto-cleaner got it wrong.
Backend (lib/campaigns.js):
- FR_NAME_FIXES — 100+ entry dictionary mapping lowercase no-accent
Québec first names to their canonical accented form (André, Stéphane,
Frédéric, Geneviève, Hélène, Joséée, etc.). Sourced from MIQ baby
names + older-generation curation.
- COMPOUND_PARTS — list of common name parts (jean, marie, anne, marc,
philippe, françois, etc.) that combine into QC compound first names.
When two parts appear concatenated with no separator, the cleaner
splits and hyphenates them. Example: "Marcandre" → ["marc","andre"]
→ "Marc-André" (dictionary then applies accent).
- titleCaseToken — proper Title Case respecting apostrophes (O'Brien,
L'Heureux) and hyphens (Marie-Ève). Uses \p{L} Unicode class so it
works on accented chars correctly.
- cleanName(raw) — full pipeline: trim → Title Case → dictionary
lookup per word → compound split fallback. Applied to firstname AND
lastname in parseMapCsv.
- nameWarning(name) — heuristic flag for cases the cleaner couldn't
confidently handle: digit in name, single letter, abnormally long
without separator (likely two stuck names not in COMPOUND_PARTS).
Returns a short FR description for the UI tooltip.
- parseMapCsv now returns firstname/lastname (cleaned) + firstname_raw/
lastname_raw (original from CSV) + cleaned_changed bool + name_warnings
per recipient. UI uses these to show before/after + flags.
UI (CampaignNewPage Step 2):
- New counter card "Noms à vérifier" — count of recipients with at least
one nameWarning. Only renders if > 0.
- Info banner above the recipients table:
"X nom(s) auto-corrigés (...) Y nom(s) suspects (...)"
- Per-row icons in the firstname + lastname columns:
• ⚠ amber WARNING — cleaner flagged this name as suspicious
(tooltip shows the reason: "deux prénoms collés", "contient un
chiffre", etc.)
• ✨ green AUTO_FIX_HIGH — auto-cleaner changed something at parse
time (tooltip shows the original raw value)
Both icons are tooltip-only — no action required.
- Click any name cell → q-popup-edit opens an inline input. Type the
correction, Enter saves. ESC cancels. This is the manual override
path for any name the auto-cleaner mishandled.
Tests (manual via end-to-end smoke against prod):
STEPHANE TREMBLAY → Stéphane Tremblay ✓ accent + Title Case
marie tremblay → Marie Tremblay ✓ Title Case only
Marcandre Boileau → Marc-André Boileau ✓ compound + accent
Jean Francois Lebrun → Jean François Lebrun ✓ accent only
Mariejosee Lapierre → Marie-Josée Lapierre ✓ compound + double accent
Andre LAPRISE → André Laprise ✓ both fixed
Helene St-Pierre → Hélène St-Pierre ✓ accent, hyphen preserved
Frederic O'Brien → Frédéric O'Brien ✓ accent, apostrophe preserved
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|---|---|---|
| apps | ||
| docs | ||
| erpnext | ||
| patches | ||
| scripts | ||
| services | ||
| .gitignore | ||
| README.md | ||
Gigafibre FSM
Gigafibre FSM is the operations platform for Gigafibre (consumer brand of TARGO Internet), a fiber ISP in Quebec. It replaces a legacy PHP/MariaDB billing system with ERPNext v16 + Vue 3/Quasar apps for ops, dispatch, field service, and customer self-service.
Repository Structure
gigafibre-fsm/
apps/
ops/ Targo Ops -- main operations PWA (Vue 3 / Quasar v2)
field/ Targo Field -- mobile app for technicians
client/ Gigafibre Portal -- customer self-service
website/ www.gigafibre.ca -- marketing site (React / Vite / Tailwind)
portal/ Customer portal deploy configs
services/
targo-hub/ Node.js API gateway (ERPNext, GenieACS, Twilio, Traccar)
modem-bridge/ SNMP/TR-069 bridge for CPE diagnostics
legacy-db/ Legacy MariaDB read-only access
docuseal/ Document signing service
erpnext/ Custom doctype setup scripts (setup_fsm_doctypes.py)
scripts/
migration/ 51 Python scripts for legacy-to-ERPNext data migration
bulk_submit.py, fix_ple_*.py/sh -- PostgreSQL patches, bulk ops
docs/ Architecture, infrastructure, migration, strategy docs
docker/ Docker compose fragments
patches/ ERPNext patches
Architecture
Internet
|
96.125.196.67 (Proxmox VM, Ubuntu 24.04, Docker)
|
Traefik v2.11 (TLS via Let's Encrypt)
|
+----------+----------+----------+----------+----------+
| | | | | |
ERPNext Ops PWA Authentik n8n Website DocuSeal
erp. erp. auth. n8n. www. docs.
gigafibre gigafibre targo.ca giga giga gigafibre
.ca .ca/ops/ id.giga fibre fibre .ca
fibre.ca .ca .ca
|
targo-hub (API gateway, msg.gigafibre.ca)
|
+----------+----------+----------+----------+
| | | | |
GenieACS Twilio Mailjet Stripe Traccar
(TR-069) (SMS) (email) ($) (GPS)
|
modem-bridge (SNMP / TR-069 deep-dive for TP-Link)
Services & Dependencies
| Service | URL | Stack | Purpose |
|---|---|---|---|
| ERPNext | erp.gigafibre.ca | Frappe v16, PostgreSQL | ERP / billing / ticketing — Source of Truth |
| Ops SPA | erp.gigafibre.ca/ops/ | Vue 3, Quasar v2, Pinia | Internal operations app (dispatch, clients, settings) |
| targo-hub | msg.gigafibre.ca | Node.js 20, Express | API gateway: SMS, SSE, AI, OAuth admin, Stripe webhooks, Traccar proxy |
| modem-bridge | internal | Node.js, Playwright | TR-181 deep-dive for TP-Link XX230v / Deco mesh |
| Authentik (staff) | auth.targo.ca | Python, PostgreSQL | SSO for ops + n8n + Gitea + ERPNext OAuth provider |
| Authentik (clients) | id.gigafibre.ca | Python, PostgreSQL | SSO for customer-facing portals |
| n8n | n8n.gigafibre.ca | Node.js | Workflow automation (campaigns, SMS, email) |
| Traefik | — | Go | Reverse proxy, TLS, forwardAuth middleware |
| Website | www.gigafibre.ca | React, Vite, Tailwind | Marketing site + address API |
| DocuSeal | sign.gigafibre.ca | Ruby on Rails | Contract e-signature |
| GenieACS | (external) | Node.js, MongoDB | TR-069 ACS for ONT/router fleet |
| Traccar | (external) | Java | GPS tracking for techs |
ERPNext Custom Doctypes
| Doctype | ID Pattern | Purpose |
|---|---|---|
| Service Location | LOC-##### | Customer premises (address, GPS, OLT port, network config) |
| Service Equipment | EQP-##### | Deployed hardware (ONT, router, TV box -- serial, MAC, IP) |
| Service Subscription | SUB-##### | Active service plans (speed, price, billing, RADIUS) |
| Dispatch Job | DJ-##### | Work orders with equipment, materials, checklist, photos, signature |
| Dispatch Technician | DT-##### | Tech profiles with GPS (Traccar), skills, color coding |
| Dispatch Tag | -- | Skill/service/region tags with levels (Fibre, TV, Telephonie, etc.) |
Key Custom Fields
| Doctype | Custom Fields |
|---|---|
| Customer | legacy_account_id, legacy_customer_id, ppa_enabled, stripe_id |
| Item | legacy_product_id, download_speed, upload_speed, olt_profile |
| Subscription | radius_user, radius_pwd, legacy_service_id |
| Issue | legacy_ticket_id, assigned_staff, issue_type, is_important, service_location |
Tech Stack
Frontend: Vue 3, Quasar v2, Pinia, Vite, Mapbox GL JS Backend: ERPNext v16 / Frappe (Python), PostgreSQL, Node.js (targo-hub) Infra: Docker, Traefik v2.11, Authentik SSO, Proxmox Integrations: Twilio (SMS), Mailjet (email), Stripe (payments), Traccar (GPS), GenieACS (TR-069), Gemini 2.5 Flash via targo-hub (vision/OCR — see docs/features/vision-ocr.md)
Data Volumes (migrated from legacy)
| Entity | Volume |
|---|---|
| Customers | 6,667 (active + terminated) |
| Subscriptions | 21,876 (with RADIUS credentials) |
| Sales Invoices | 115,000+ |
| Payments | 99,000+ (with invoice references) |
| Tickets (Issues) | 242,000+ (parent/child hierarchy) |
| Ticket Messages | 784,000+ |
| Devices | 7,600+ (ONT, router, TV box) |
| Service Locations | ~17,000 |
Development
# Ops app
cd apps/ops && npm install && npx quasar dev
# Website
cd apps/website && npm install && npm run dev
# targo-hub
cd services/targo-hub && npm install && npm run dev
# Deploy ops to production
cd apps/ops && bash deploy.sh
Auth Pattern
Two parallel Authentik instances — not a migration in progress:
auth.targo.ca— staff-facing. Protects/ops/, n8n, Gitea, the hub admin endpoints. Also acts as the OAuth provider for ERPNext sign-in. Lives on the prod box at/opt/authentik(managed by Targo's IT team).id.gigafibre.ca— customer-facing. Protects the client portal. Lives at/opt/authentik-clienton the prod box.
Staff apps go through Traefik forwardAuth → Authentik outpost → cookie. The ops SPA reads X-Authentik-Email from the proxied header to identify the user, then maps Authentik groups to the in-app capability set (useUserGroups.js). All ERPNext API calls from targo-hub use Authorization: token <ERP_SERVICE_TOKEN> (Bearer from .env).
New users are added via ops Settings → Utilisateurs → "Inviter" which hits POST /auth/users on the hub. The hub creates the Authentik user, sets a temporary password, emails it via Mailjet, and creates the matching ERPNext System User in one round-trip. See docs/SETUP.md §6.
Documentation
Start at docs/README.md — it indexes every doc with a "I want to…" intent table. Quick map:
| Area | Entry point |
|---|---|
| Plan & live module URLs | docs/roadmap.md |
| System architecture (services, Docker, SSO) | docs/architecture/overview.md |
| ERPNext data model + customer flows | docs/architecture/data-model.md |
| Frontend patterns (Vue/Quasar/Pinia) | docs/architecture/app-design.md |
| Billing, Stripe, invoices | docs/features/billing-payments.md |
| CPE / modems / ONTs / TR-069 | docs/features/cpe-management.md |
| Scanner / OCR / Gemini pipeline | docs/features/vision-ocr.md |
| Agent flows (Flow Template) | docs/features/flow-editor.md |
| Wizard SKU vs legacy audit | docs/reference/erpnext-item-diff.md |
| Historical snapshots & migration logs | docs/archive/ |