gigafibre-fsm/scripts
louispaulb 10afd696ae fix(migration): clean address_line + postal_code + connection_type at import
Three legacy data-quality issues that were leaking into ERPNext on every
import run. Caught while auditing C-LPB4's mis-pinned dispatch job.

1. **Postal code embedded in address_line.** Legacy `gestionclient` had
   rows like `2200-3 chemin de la riviere de la guerre  J0S1B0` with
   the postal code concatenated at the end (and the same code repeated
   in the dedicated zip column). Caused 48-char address_line on what
   should have been a 39-char address. Now stripped at import: a regex
   matches `\\s+<FSA><LDU>\\s*$` (with or without space) and removes
   it; the dedicated postal_code field carries the canonical form.

2. **Abbreviations + Cobol-style capitalization.** Legacy stored
   `2066 Ch De La 1Re-Concession` instead of the canonical
   `2066 Chemin de la 1re-Concession`. ABBREV_MAP expands `Ch` →
   `Chemin`, `Av` → `Avenue`, `Bd`/`Boul` → `Boulevard`, `Rte` →
   `Route`, `St-` → `Saint-`, `Ste-` → `Sainte-`, `Mtl` → `Montréal`.
   Title-casing rule preserves French articles lowercase (`de`, `du`,
   `des`, `la`, `le`, `les`, `au`, `aux`, `à`, `et`, `sur`, `en`)
   and ordinal markers (`1re`, `2e`, `3e`). 96 SLs in production had
   the `1Re-Concession` style; they'll be re-normalized on next
   migration run.

3. **`connection_type` left empty even when ONT/CPE devices existed.**
   Pre-loads device→delivery mapping at import start; if the legacy
   delivery has any device whose category/name/model contains "ont",
   "onu", "cpe", "fibre", "gpon", or "ftth", we set
   connection_type='Fibre FTTH'. Without devices on file, the field
   stays empty (rep fills it later) — we don't guess.

4. **`postal_code` normalized too** — `j0s1b0` → `J0S 1B0` (uppercase
   + canonical space). Was being inserted in lowercase no-space form.

Self-tested on 8 representative cases including the actual broken
records found in production (LOC-15903, LOC-6227, LOC-4 / C-LPB4).

These changes affect only re-imports of locations. Existing data
needs a separate backfill script — a follow-up will cover that
either as a one-shot migration or by running the existing
`reimport_subscriptions.py` after this script.
2026-05-08 15:38:19 -04:00
..
migration fix(migration): clean address_line + postal_code + connection_type at import 2026-05-08 15:38:19 -04:00
test refactor: major cleanup — remove dead dispatch app, commit all backend code, extract client composables 2026-04-08 17:38:38 -04:00
bulk_submit.py feat: inline editing, search, notifications + full repo cleanup 2026-03-31 07:34:41 -04:00
fix_ple_groupby.py feat: inline editing, search, notifications + full repo cleanup 2026-03-31 07:34:41 -04:00
fix_ple_postgres.sh feat: inline editing, search, notifications + full repo cleanup 2026-03-31 07:34:41 -04:00
import-legacy-accounts.js refactor: reduce token count, DRY code, consolidate docs 2026-04-13 08:39:58 -04:00
server_bulk_submit.py feat: telephony UI, performance indexes, Twilio softphone, lazy-load invoices 2026-04-02 13:59:59 -04:00