gigafibre-fsm/docs/MIGRATION.md

Legacy to ERPNext Migration

1. Legacy System

Source database: gestionclient (MariaDB on 10.100.80.100). Monthly recurring revenue ~$742K across fibre, wireless, TV, and telephony.

Table	Records	Description
account	15,303	Customers (6,510 active, 8,594 terminated)
delivery	17,114	Installation addresses with GPS
service	66,879	Subscriptions (39,624 active)
product / product_cat	833 / 34	Catalogue items and categories
invoice / invoice_item	630K / 1.86M	Invoices and line items
payment / payment_item	541K / 685K	Payments and allocations
compta_comptes / ecritures	48 / 1.2M	Chart of accounts and journal entries
ticket / ticket_msg	243K / 784K	Support tickets and messages (8K with base64 images)
device / fibre	10,377 / 16,057	Network equipment and fibre connections
staff / bon_travail	155 / 14,472	Employees and work orders

2. Legacy to ERPNext Mapping

Legacy Table	ERPNext DocType	Records	Status
account	Customer + Contact + Address	6,667 + 6,600 + 6,700	Migrated
delivery + fibre	Service Location	17,114	Migrated
product	Item (type=Service)	833	Migrated
product_cat	Item Group	34	Migrated
service (active)	Subscription	21,876	Migrated (scheduler paused)
invoice + invoice_item	Sales Invoice	630K	Opening balance only
payment + payment_item	Payment Entry	99K imported	Migrated (recent)
compta_comptes	Account (Chart)	48	Mapped to ERPNext CoA
compta_journal_ecriture	Journal Entry	1.2M	Opening balance only
ticket + ticket_msg	Issue + Communication	243K + 784K	Migrated
device	Service Equipment	10,377	Migrated
staff	Employee / User	45 users	Migrated
bon_travail	Dispatch Job	14,472	Pending
tax	Sales Taxes Template	4	Migrated (TPS 5% + TVQ 9.975%)

3. Key Architecture Differences

• Naming: Legacy uses auto-increment IDs; ERPNext uses naming series (CUST-.YYYY.-, LOC-.#####)
• Customer/Address: Legacy stores everything in account; ERPNext separates Customer, Address, and Contact as linked doctypes
• Subscriptions: Legacy service is a static row; ERPNext Subscription auto-generates Sales Invoices each billing cycle
• Custom Pricing: Legacy hijack_price override on service; ERPNext uses additional_discount on Subscription or Pricing Rules
• Taxes: Legacy calculates TPS/TVQ in PHP code; ERPNext applies Sales Taxes Templates automatically
• Accounting: Legacy writes journal entries via PHP; ERPNext auto-creates GL Entries from submitted Sales Invoices
• Permissions: Legacy uses staff.rights (PHP serialized); ERPNext uses Role-based DocType permissions
• Files: Legacy stores images as base64 in mediumtext columns; ERPNext uses File doctype with /files/ directory
• Dates: Legacy uses Unix timestamps (bigint); ERPNext uses ISO dates (YYYY-MM-DD)
• RADIUS: Legacy stores credentials on service; ERPNext uses custom fields on Subscription

4. Migration Phases

1. Foundation -- Done (2026-03-28)

   • 34 Item Groups, 833 Items with ISP custom fields (speeds, quotas, fibre profiles)
   • Tax Templates: TPS 5% + TVQ 9.975%
   • 15 custom fields created across Item, Customer, and Subscription doctypes

2. Customers & Addresses -- Done (2026-03-28)

   • 6,667 Customers, ~6,600 Contacts, ~6,700 Addresses via direct PostgreSQL (~30s)
   • Customer-Contact-Address links established with GPS coordinates

3. Subscriptions & Items -- Done (2026-03-28)

   • 92 Subscription Plans created from active products
   • 21,876 Subscriptions imported with RADIUS data
   • Scheduler PAUSED to prevent automatic invoice generation

4. Invoices & Payments -- Done (2026-03-29)

   • Outstanding invoice analysis and AR reconciliation
   • 99,000 payments imported with invoice references and mode mapping

5. Opening Balances -- Done (2026-03-29)

   • Customer balance reconciliation (invoices minus payments)
   • Opening balance journal entries created

6. Tickets -- Done (2026-03-29, 242K)

   • 242,618 tickets migrated as Issues with parent/child hierarchy
   • 784,290 ticket messages imported as Communications
   • Issue types mapped from 21 legacy departments
   • Staff assignment mapped from legacy IDs to ERPNext User emails

7. Staff & Memos -- Done (2026-03-29)

   • 45 ERPNext Users created from legacy staff table
   • 29,000 customer memos imported as Comments with original creation dates

5. Field Gap Analysis

~73 custom fields needed across 5 doctypes to reach zero data loss.

DocType	Gaps	Key Missing Areas
Customer	~23	PPA banking (8 fields), Stripe (2), termination reason/date (4), VIP/flags (3), portal login (2), other (4)
Subscription	~10	Custom pricing flag/desc (3), suspension/active-until dates (3), quotas (2), static IP (1), other (1)
Service Equipment	~17	Management access (6: URL, port, protocol for GUI+CLI), provisioning from GenieACS (8: WiFi, VoIP, GPON/CWMP serials), ACS integration (4)
Service Location	~18	Fibre infrastructure (10: ONT serial, OLT IP/name, ONT ID, distance, spans), individual VLANs (4), legacy service IDs (4)
Dispatch Job	~5	Second technician + arrival/departure timestamps (5)

Already migrated fields per doctype: Customer ~18, Subscription ~14, Equipment ~10, Location ~9.

6. Execution Log

• 2026-03-28: Frappe Assistant Core v2.3.3 installed, legacy MariaDB access granted. 34 Item Groups + 833 Items imported. 15 custom fields created (Item 7, Customer 4, Subscription 3). 6,667 Customers + Contacts + Addresses via direct PG in ~30s. 92 Subscription Plans + 21,876 Subscriptions with RADIUS data. Scheduler paused.
• 2026-03-29: Opening balance + AR analysis. 242K tickets + 784K messages migrated as Issues/Communications. 45 Users created. 29K customer memos imported. 99K payments imported.
• 2026-03-30: Ops App V2 with ClientDetailPage, Dispatch module, Equipment page, OCR. Field Tech App (PWA with barcode scanner, offline support).
• 2026-03-31: Inline editing (InlineField component), client search with legacy ID + or_filters, SMS/Email notifications via n8n/Twilio/Mailjet, ticket reply via Communication docs, Authentik federation (auth.targo.ca to id.gigafibre.ca).

7. Data NOT Migrated

• Journal entries (1.2M) -- opening balance only, no line-by-line import
• Invoices > 2 years -- covered by opening balance
• Closed tickets (241K) -- available in legacy read-only archive
• RADIUS consumption (conso_*) -- operational data, not financial
• Phone call logs (phonecall_log_*) -- historical records
• Temp/backup tables (tmp*, *_bk, *_archive)
• Camping tables (camping_*) -- confirm if still in use
• Password manager (passwords_manager*) -- never migrate third-party credentials
• Work orders (bon_travail, 14K) -- historical, Dispatch Job doctype ready for future import

8. Risks & Decisions

Risks:

• Incorrect balances after migration -- mitigated by before/after reconciliation reports
• PPA payment disruption during cutover -- mitigated by parallel operation (legacy bills, ERPNext takes over gradually)
• Missing subscriptions -- validation script compares active legacy services against ERPNext Subscriptions
• Lost base64 images -- batch extraction scheduled before cutover
• ERPNext performance with 630K invoices -- only 2 years imported, rest as opening balance
• Scheduler is PAUSED -- reactivation would trigger billing for 21,876 subscriptions

Open Decisions:

• Invoice history depth: currently 2 years, rest as opening balance
• PPA (AccesD Desjardins): ERPNext has no native AccesD support, middleware needed
• Legacy ID retention: legacy_* fields on all doctypes for traceability
• Customer portal: 659 portal profiles use unsalted MD5 passwords, force reset via email/SMS OTP required
• RADIUS integration: webhook from Subscription to FreeRADIUS via n8n
• Camping module: confirm if camping_* tables are still active