From 0e86456b0efd37116836bccc60f6ff7da08dbfe2 Mon Sep 17 00:00:00 2001 From: louispaulb Date: Fri, 3 Apr 2026 08:08:57 -0400 Subject: [PATCH] =?UTF-8?q?feat:=20complete=20device=20matching=20analysis?= =?UTF-8?q?=20(legacy=20=E2=86=94=20GenieACS=20=E2=86=94=20ERPNext)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Full data export and cross-reference analysis: - 7,550 GenieACS devices with IPs, deviceId, tags - 6,720 legacy devices (raisecom, tplink, onu categories) - 16,056 fibre table entries (OLT frame/slot/port/ontid, VLANs) - 8,434 legacy services linked to devices Key finding: CWMP serial ≠ physical serial. Only 22/7,550 devices are tagged with their physical serial (RCMG/TPLG). Raisecom MAC is extractable from CWMP serial suffix. TP-Link CWMP serial = sticker serial for ONT models. Matching strategy documented: tag-based, MAC-based, OLT port-based. Recommends bulk tagging via OLT query as first step. Co-Authored-By: Claude Opus 4.6 --- scripts/migration/genieacs-export/ANALYSIS.md | 136 ++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 scripts/migration/genieacs-export/ANALYSIS.md diff --git a/scripts/migration/genieacs-export/ANALYSIS.md b/scripts/migration/genieacs-export/ANALYSIS.md new file mode 100644 index 0000000..a8cb2b5 --- /dev/null +++ b/scripts/migration/genieacs-export/ANALYSIS.md @@ -0,0 +1,136 @@ +# GenieACS ↔ Legacy System ↔ ERPNext — Device Matching Analysis + +## Data Sources + +| Source | Records | Key Fields | +|--------|---------|------------| +| GenieACS (NBI) | 7,550 devices | CWMP serial, OUI, ProductClass, IP, tags, _lastInform | +| Legacy `device` table | 10,417 devices | delivery_id, category, sn (physical), mac, manage IP, parent | +| Legacy `fibre` table | 16,056 entries | sn (physical), OLT frame/slot/port/ontid, olt_ip, VLANs | +| Legacy `service` table | 66,686 services | delivery_id, device_id, product_id, radius_user/pwd | +| Legacy `delivery` table | 17,131 deliveries | account_id, address, city, zip | +| ERPNext Service Equipment | ~10,000 | serial_number, mac_address, customer, service_location | +| GenieACS MariaDB `wifi` | unknown | serial, instance, ssid, password | +| GenieACS MariaDB `voip` | unknown | serial, instance, username, password | + +## Fleet Breakdown + +### GenieACS (by model) +- **Device2** (TP-Link Deco XE75/XX230v): 4,051 (74% online) +- **HT803G-W** (Raisecom GPON ONT): 493 +- **HT803G-WS2** (Raisecom GPON ONT v2): 2,340 +- **Device 2** (older TP-Link): 409 +- **DISCOVERYSERVICE**: 156 (ghost entries) +- **HT502/HT812** (Grandstream ATA): ~70 +- **GXP2130/2160/1630** (Grandstream phones): ~20 +- **XX530v/XGB430v_Pro** (TP-Link VoIP): 4 + +### Legacy (by category) +- **raisecom_rcmg**: 5,016 +- **stb_ministra**: 1,212 +- **stb**: 1,054 +- **tplink_tplg**: 849 +- **tplink_device2**: 811 +- **airosm**: 832 (Ubiquiti AirOS) +- **airos_ac**: 228 +- **cambium**: 118 +- **onu**: 40 +- **ht803g1ge**: 4 + +## Matching Strategy + +### Key Discovery: CWMP Serial ≠ Physical Serial + +GenieACS stores the CWMP negotiated serial number, which is an **internal identifier** — NOT the physical label serial. + +**Raisecom HT803G-W pattern:** +``` +CWMP Serial: 7IGLC980074DF9D98 + ^^^^^^^^^^^^ + = MAC address (98:00:74:DF:9D:98) +Tag (physical SN): RCMG19E0AB57 +``` + +**TP-Link Device2 pattern:** +``` +CWMP Serial: 2234196001225 (manufacturing serial, same as sticker) +Tag: config_manuelle (NOT the serial) +``` + +### Raisecom Matching: CWMP Serial → MAC → OLT → Fibre → Customer + +1. **Extract MAC from CWMP serial**: Last 12 hex chars = MAC (e.g., `980074DF9D98` → `98:00:74:DF:9D:98`) +2. But legacy `device.mac` field is mostly empty (only 28/5,016 have it) +3. **Alternative**: The 22 tagged devices have `RCMG...` physical serial in tags +4. **Best path**: OLT registration maps GPON SN → frame/slot/port → fibre table + +**Proposed workflow:** +- Query each OLT for registered ONT serial numbers and their port assignments +- Match OLT data → `fibre.frame/slot/port/ontid` → `fibre.service_id` → `service.delivery_id` → `delivery.account_id` +- Tag each GenieACS device with its RCMG physical serial from OLT data + +### TP-Link Matching: CWMP Serial = Sticker Serial + +For TP-Link devices, the CWMP serial IS the sticker serial (e.g., `TPLGA1E7FB90`). +But most TP-Link Device2 (Deco) have alphanumeric serials like `2234196001225`. + +**Match via MAC address:** +- GenieACS has `Device.Ethernet.Interface.1.MACAddress` for Device2 +- Legacy has `device.mac` for `tplink_device2` category +- ERPNext Service Equipment has `mac_address` + +### Recommended Matching Script + +```python +# Priority order for matching: +# 1. Tag (physical serial) → fibre.sn or device.sn (exact match, highest confidence) +# 2. MAC address → device.mac or ERPNext mac_address (when available) +# 3. CWMP serial last 12 chars as MAC → MAC-based lookup +# 4. WAN IP (172.17.x.x) → device.manage or fibre OLT topology +# 5. Manual matching for remainder (present device list for human review) +``` + +## Actions Required + +### Immediate: Bulk Tag GenieACS Devices +The biggest gap is that only 22/7,550 devices have their physical serial as a tag. +Two approaches: + +**A. Script via OLT (recommended):** +- SSH to each OLT (the `info_connect` IPs: 172.17.0.x) +- Run `display ont info` or equivalent to get GPON SN → port mapping +- Cross-reference with `fibre` table (frame/slot/port/ontid) +- Push tags to GenieACS via NBI: `POST /devices/:id/tags/:serial` + +**B. Enrichment via CWMP serial MAC extraction:** +- For Raisecom: extract last 12 chars of CWMP serial as MAC +- Match to any MAC-indexed data source +- Tag via NBI + +### Medium-term: Migrate WiFi/VoIP Provisioning Data +The GenieACS MariaDB at `10.5.14.21` is not accessible from all servers. +Export the `wifi` and `voip` tables and migrate to: +- ERPNext custom fields on Service Equipment (preferred) +- Or a new provisioning table in the targo-hub database + +### For Oktopus: Device Profiles +Map each device category to an Oktopus device profile: + +| Legacy Category | GenieACS Model | Oktopus Profile | Config | +|---|---|---|---| +| raisecom_rcmg | HT803G-W/WS2 | raisecom-ont | VoIP DSCP, NTP, UPnP off, bridge mode | +| tplink_tplg | Device2 (GPON) | tplink-ont | WiFi SSID/pwd from DB | +| tplink_device2 | Device2 (Deco) | tplink-deco | WiFi SSID/pwd, mesh config | +| onu | Various | generic-ont | Basic monitoring | +| ht803g1ge | HT803G-1GE | raisecom-1ge | Legacy single-port ONT | + +## Data Volumes Summary + +| Match Type | Count | Confidence | +|---|---|---| +| GenieACS tagged with physical SN | 22 | 100% | +| TP-Link CWMP serial = sticker serial | ~5,000 | High | +| Raisecom MAC extractable from CWMP SN | ~2,833 | Medium | +| Legacy fibre with SN | 4,948 | Reference | +| Legacy device with SN | ~5,900 | Reference | +| Needs OLT query for matching | ~2,800 Raisecom | Required |