louis/gigafibre-fsm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
231bb6fbcc
gigafibre-fsm/README.md
louispaulb 101faa21f1 feat: inline editing, search, notifications + full repo cleanup 
- InlineField component + useInlineEdit composable for Odoo-style dblclick editing
- Client search by name, account ID, and legacy_customer_id (or_filters)
- SMS/Email notification panel on ContactCard via n8n webhooks
- Ticket reply thread via Communication docs
- All migration scripts (51 files) now tracked
- Client portal and field tech app added to monorepo
- README rewritten with full feature list, migration summary, architecture
- CHANGELOG updated with all recent work
- ROADMAP updated with current completion status
- Removed hardcoded tokens from docs (use $ERP_SERVICE_TOKEN)
- .gitignore updated (docker/, .claude/, exports/, .quasar/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:34:41 -04:00

199 lines
9.1 KiB
Markdown
Raw Blame History

# Gigafibre FSM
Complete operations platform for **Gigafibre** (consumer brand of TARGO Internet), a fiber ISP in Quebec. Replaces a legacy PHP/MariaDB billing system with ERPNext v16 + custom Vue.js (Quasar) apps.
## What This Repo Contains
| Directory | Description |
|-----------|-------------|
| `apps/ops/` | **Targo Ops** — main operations PWA (Vue 3 / Quasar v2) |
| `apps/field/` | **Targo Field** — mobile app for technicians (barcode, diagnostics, offline) |
| `apps/client/` | **Gigafibre Portal** — customer self-service portal |
| `apps/website/` | **www.gigafibre.ca** — marketing site (React / Vite / Tailwind) |
| `apps/dispatch/` | Legacy dispatch app (replaced by `apps/ops/` dispatch module) |
| `apps/portal/` | Customer portal deploy configs |
| `erpnext/` | ERPNext custom doctype setup scripts |
| `scripts/migration/` | 51 Python scripts for legacy-to-ERPNext data migration |
| `scripts/` | Utility scripts (bulk submit, PostgreSQL fixes) |
| `docs/` | Architecture, infrastructure, migration plan, changelog |
## Architecture
```
96.125.196.67 (Proxmox VM, Ubuntu 24.04)
|
Traefik v2.11 (TLS via Let's Encrypt)
|
+-- erp.gigafibre.ca ERPNext v16.10.1 (PostgreSQL, 9 containers)
+-- erp.gigafibre.ca/ops/ Ops PWA (Quasar/Vue3, Authentik SSO)
+-- id.gigafibre.ca Authentik SSO (customer-facing)
+-- auth.targo.ca Authentik SSO (staff, federated to id.gigafibre.ca)
+-- n8n.gigafibre.ca n8n workflow automation
+-- git.targo.ca Gitea
+-- www.gigafibre.ca Marketing site + address API
+-- oss.gigafibre.ca Oktopus CE (TR-069 CPE management)
+-- tracker.targointernet.com Traccar GPS tracking
```
## Features
### Ops App (`apps/ops/`)
- **Client Management** — customer list with search by name, account ID, legacy ID. Inline editing (double-click any field to edit, saves to ERPNext in background)
- **Client Detail** — full customer view with contact, billing KPIs, service locations, subscriptions, equipment, tickets, invoices, payments, notes
- **Inline Editing** — Odoo-style double-click-to-edit on all fields (locations, equipment, tickets, invoices). Uses `InlineField` component + `useInlineEdit` composable
- **Dispatch Timeline** — drag-drop job scheduling with Mapbox map, GPS tracking (Traccar), technician management, tag/skill system
- **Ticket Management** — list with inline status/priority editing, detail modal with reply thread
- **Equipment Tracking** — serial/MAC, status lifecycle, OLT info, per-location grouping
- **SMS/Email Notifications** — send from contact card via n8n webhooks (Twilio SMS, Mailjet email)
- **Invoice OCR** — scan paper bills using Ollama Vision (llama3.2-vision)
- **PWA** — installable, offline-capable with Workbox
### Legacy Migration (completed)
Migrated from a 15-year-old PHP/MariaDB billing system:
| Data | Volume | Status |
|------|--------|--------|
| Customers | 6,667 (active + terminated) | Migrated |
| Contacts + Addresses | ~6,600 each | Migrated |
| Service Locations | ~17,000 | Migrated |
| Subscriptions | 21,876 (with RADIUS credentials) | Migrated |
| Items (products/plans) | 833 | Migrated |
| Sales Invoices | 115,000+ | Migrated |
| Payments | 99,000+ | Migrated with invoice references |
| Tickets (Issues) | 242,000+ (parent/child hierarchy) | Migrated |
| Ticket Messages (Communications) | 784,000+ | Migrated |
| Customer Memos (Comments) | 29,000+ | Migrated with real dates |
| Employees | 45 ERPNext Users from legacy staff | Migrated |
### Bugs Fixed From Legacy System
- **Date corruption** — Unix timestamps stored as strings; fixed during import with timezone-aware conversion
- **Invoice outstanding amounts** — payment_item references broken in legacy; reconciled during migration
- **Customer links** — orphan records (invoices/payments without valid customer); rebuilt relationships
- **Subscription details** — missing service_location links, incorrect billing frequencies; corrected via analysis scripts
- **Reversal transactions** — credit notes improperly recorded; mapped to ERPNext return invoices
- **Annual billing dates** — yearly subscriptions had wrong period boundaries; recalculated
- **Duplicate customer IDs** — legacy allowed duplicate customer_id; resolved with rename script
- **Staff/ticket ownership** — legacy used numeric IDs for assignment; mapped to ERPNext User emails
### ERPNext Adjustments for Import
- **Direct PostgreSQL inserts** — Frappe ORM too slow for 100K+ records; used psycopg2 with proper `tabXxx` schema
- **Scheduler paused** — disabled auto-invoicing during import to prevent 21K subscriptions from generating invoices
- **Custom fields on Customer** — `legacy_account_id`, `legacy_customer_id`, `ppa_enabled`, `stripe_id`, date fields
- **Custom fields on Item** — `legacy_product_id`, download/upload speeds, quotas, OLT profiles
- **Custom fields on Subscription** — `radius_user`, `radius_pwd`, `legacy_service_id`
- **Custom fields on Issue** — `legacy_ticket_id`, `assigned_staff`, `opened_by_staff`, `issue_type`, `is_important`, `service_location`
- **PostgreSQL GROUP BY patch** — ERPNext v16 generates MySQL-style queries; patched `number_card.py` and PLE reports
- **Subscription API unlock** — removed `read_only` and `set_only_once` restrictions on Subscription fields for REST API updates
- **Portal auth bridge** — server script for legacy MD5 password migration to PBKDF2
## ERPNext Custom Doctypes
### Field Service Management
| Doctype | ID Pattern | Purpose |
|---------|-----------|---------|
| Service Location | LOC-##### | Customer premises (address, GPS, OLT port, network config) |
| Service Equipment | EQP-##### | Deployed hardware (ONT, router, TV box — serial, MAC, IP) |
| Service Subscription | SUB-##### | Active service plans (speed, price, billing cycle, RADIUS) |
### Dispatch
| Doctype | Purpose |
|---------|---------|
| Dispatch Job | Work orders with equipment, materials, checklist, photos, signature |
| Dispatch Technician | Tech profiles with GPS link (Traccar), skills, color coding |
| Dispatch Tag | Categorization with skill levels (Fibre, TV, Telephonie, etc.) |
| Dispatch Tag Link | Child table linking tags to jobs/techs with level + required flag |
### Child Tables
Equipment Move Log, Job Equipment Item, Job Material Used, Job Checklist Item, Job Photo, Dispatch Job Assistant, Checklist Template + Items
## Quick Start
### Development
```bash
# Ops app
cd apps/ops
npm install
npx quasar dev
# Website
cd apps/website
npm install
npm run dev
```
### Deploy Ops to Production
```bash
cd apps/ops
bash deploy.sh # builds PWA + deploys to server via SSH
```
### Run Migration Scripts
```bash
# Scripts run inside the ERPNext backend container
docker cp scripts/migration/migrate_all.py frappe_docker-backend-1:/tmp/
docker exec frappe_docker-backend-1 bench --site erp.gigafibre.ca execute /tmp/migrate_all.main
```
### Setup FSM Doctypes
```bash
docker cp erpnext/setup_fsm_doctypes.py frappe_docker-backend-1:/home/frappe/frappe-bench/apps/frappe/frappe/
docker exec frappe_docker-backend-1 bench --site erp.gigafibre.ca execute frappe.setup_fsm_doctypes.create_all
```
## Documentation
| Document | Content |
|----------|---------|
| [ARCHITECTURE.md](docs/ARCHITECTURE.md) | Data model, tech stack, authentication flow, doctype reference |
| [INFRASTRUCTURE.md](docs/INFRASTRUCTURE.md) | Server, DNS, Traefik, Authentik, Docker, n8n, gotchas |
| [MIGRATION-PLAN.md](docs/MIGRATION-PLAN.md) | Legacy system portrait, mapping, phases, risks |
| [CHANGELOG.md](docs/CHANGELOG.md) | Detailed migration log with volumes and methods |
| [ROADMAP.md](docs/ROADMAP.md) | 5-phase implementation plan |
| [MIGRATION_MAP.md](scripts/migration/MIGRATION_MAP.md) | Field-level mapping legacy tables to ERPNext |
| [COMPETITIVE-ANALYSIS.md](docs/COMPETITIVE-ANALYSIS.md) | Comparison with Gaiia, Odoo, Zuper, Salesforce, ServiceTitan |
## Infrastructure
| Service | URL | Technology |
|---------|-----|------------|
| ERP | erp.gigafibre.ca | ERPNext v16.10.1 (Frappe, PostgreSQL) |
| Ops | erp.gigafibre.ca/ops/ | Quasar v2 PWA |
| SSO (staff) | auth.targo.ca | Authentik |
| SSO (customers) | id.gigafibre.ca | Authentik (federated from auth.targo.ca) |
| Workflows | n8n.gigafibre.ca | n8n |
| Git | git.targo.ca | Gitea |
| GPS | tracker.targointernet.com | Traccar |
| Website | www.gigafibre.ca | React / Vite / Tailwind |
| CPE Mgmt | oss.gigafibre.ca | Oktopus CE (TR-069) |
| DNS | Cloudflare | gigafibre.ca (DNS-only, Traefik handles TLS) |
| Email | Mailjet | noreply@targo.ca |
| SMS | Twilio | +1 438 231-3838 |
## Tech Stack
| Layer | Technology |
|-------|-----------|
| Backend | ERPNext v16 / Frappe (Python) on PostgreSQL |
| Frontend (ops) | Vue 3, Quasar v2, Pinia, Vite |
| Frontend (website) | React, Vite, Tailwind, shadcn/ui |
| Maps | Mapbox GL JS + Directions API |
| GPS | Traccar (REST + WebSocket) |
| Auth | Authentik SSO (forwardAuth via Traefik) |
| Proxy | Traefik v2.11 (Let's Encrypt) |
| Automation | n8n webhooks |
| SMS | Twilio (via n8n) |
| Email | Mailjet (via n8n) |
| OCR | Ollama (llama3.2-vision) |
| Hosting | Proxmox VM, Ubuntu 24.04, Docker |
Reference in New Issue View Git Blame Copy Permalink