feat: complete device matching analysis (legacy ↔ GenieACS ↔ ERPNext)
Full data export and cross-reference analysis: - 7,550 GenieACS devices with IPs, deviceId, tags - 6,720 legacy devices (raisecom, tplink, onu categories) - 16,056 fibre table entries (OLT frame/slot/port/ontid, VLANs) - 8,434 legacy services linked to devices Key finding: CWMP serial ≠ physical serial. Only 22/7,550 devices are tagged with their physical serial (RCMG/TPLG). Raisecom MAC is extractable from CWMP serial suffix. TP-Link CWMP serial = sticker serial for ONT models. Matching strategy documented: tag-based, MAC-based, OLT port-based. Recommends bulk tagging via OLT query as first step. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
8ba73251f3
commit
231bb6fbcc
136
scripts/migration/genieacs-export/ANALYSIS.md
Normal file
136
scripts/migration/genieacs-export/ANALYSIS.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# GenieACS ↔ Legacy System ↔ ERPNext — Device Matching Analysis
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Source | Records | Key Fields |
|
||||
|--------|---------|------------|
|
||||
| GenieACS (NBI) | 7,550 devices | CWMP serial, OUI, ProductClass, IP, tags, _lastInform |
|
||||
| Legacy `device` table | 10,417 devices | delivery_id, category, sn (physical), mac, manage IP, parent |
|
||||
| Legacy `fibre` table | 16,056 entries | sn (physical), OLT frame/slot/port/ontid, olt_ip, VLANs |
|
||||
| Legacy `service` table | 66,686 services | delivery_id, device_id, product_id, radius_user/pwd |
|
||||
| Legacy `delivery` table | 17,131 deliveries | account_id, address, city, zip |
|
||||
| ERPNext Service Equipment | ~10,000 | serial_number, mac_address, customer, service_location |
|
||||
| GenieACS MariaDB `wifi` | unknown | serial, instance, ssid, password |
|
||||
| GenieACS MariaDB `voip` | unknown | serial, instance, username, password |
|
||||
|
||||
## Fleet Breakdown
|
||||
|
||||
### GenieACS (by model)
|
||||
- **Device2** (TP-Link Deco XE75/XX230v): 4,051 (74% online)
|
||||
- **HT803G-W** (Raisecom GPON ONT): 493
|
||||
- **HT803G-WS2** (Raisecom GPON ONT v2): 2,340
|
||||
- **Device 2** (older TP-Link): 409
|
||||
- **DISCOVERYSERVICE**: 156 (ghost entries)
|
||||
- **HT502/HT812** (Grandstream ATA): ~70
|
||||
- **GXP2130/2160/1630** (Grandstream phones): ~20
|
||||
- **XX530v/XGB430v_Pro** (TP-Link VoIP): 4
|
||||
|
||||
### Legacy (by category)
|
||||
- **raisecom_rcmg**: 5,016
|
||||
- **stb_ministra**: 1,212
|
||||
- **stb**: 1,054
|
||||
- **tplink_tplg**: 849
|
||||
- **tplink_device2**: 811
|
||||
- **airosm**: 832 (Ubiquiti AirOS)
|
||||
- **airos_ac**: 228
|
||||
- **cambium**: 118
|
||||
- **onu**: 40
|
||||
- **ht803g1ge**: 4
|
||||
|
||||
## Matching Strategy
|
||||
|
||||
### Key Discovery: CWMP Serial ≠ Physical Serial
|
||||
|
||||
GenieACS stores the CWMP negotiated serial number, which is an **internal identifier** — NOT the physical label serial.
|
||||
|
||||
**Raisecom HT803G-W pattern:**
|
||||
```
|
||||
CWMP Serial: 7IGLC980074DF9D98
|
||||
^^^^^^^^^^^^
|
||||
= MAC address (98:00:74:DF:9D:98)
|
||||
Tag (physical SN): RCMG19E0AB57
|
||||
```
|
||||
|
||||
**TP-Link Device2 pattern:**
|
||||
```
|
||||
CWMP Serial: 2234196001225 (manufacturing serial, same as sticker)
|
||||
Tag: config_manuelle (NOT the serial)
|
||||
```
|
||||
|
||||
### Raisecom Matching: CWMP Serial → MAC → OLT → Fibre → Customer
|
||||
|
||||
1. **Extract MAC from CWMP serial**: Last 12 hex chars = MAC (e.g., `980074DF9D98` → `98:00:74:DF:9D:98`)
|
||||
2. But legacy `device.mac` field is mostly empty (only 28/5,016 have it)
|
||||
3. **Alternative**: The 22 tagged devices have `RCMG...` physical serial in tags
|
||||
4. **Best path**: OLT registration maps GPON SN → frame/slot/port → fibre table
|
||||
|
||||
**Proposed workflow:**
|
||||
- Query each OLT for registered ONT serial numbers and their port assignments
|
||||
- Match OLT data → `fibre.frame/slot/port/ontid` → `fibre.service_id` → `service.delivery_id` → `delivery.account_id`
|
||||
- Tag each GenieACS device with its RCMG physical serial from OLT data
|
||||
|
||||
### TP-Link Matching: CWMP Serial = Sticker Serial
|
||||
|
||||
For TP-Link devices, the CWMP serial IS the sticker serial (e.g., `TPLGA1E7FB90`).
|
||||
But most TP-Link Device2 (Deco) have alphanumeric serials like `2234196001225`.
|
||||
|
||||
**Match via MAC address:**
|
||||
- GenieACS has `Device.Ethernet.Interface.1.MACAddress` for Device2
|
||||
- Legacy has `device.mac` for `tplink_device2` category
|
||||
- ERPNext Service Equipment has `mac_address`
|
||||
|
||||
### Recommended Matching Script
|
||||
|
||||
```python
|
||||
# Priority order for matching:
|
||||
# 1. Tag (physical serial) → fibre.sn or device.sn (exact match, highest confidence)
|
||||
# 2. MAC address → device.mac or ERPNext mac_address (when available)
|
||||
# 3. CWMP serial last 12 chars as MAC → MAC-based lookup
|
||||
# 4. WAN IP (172.17.x.x) → device.manage or fibre OLT topology
|
||||
# 5. Manual matching for remainder (present device list for human review)
|
||||
```
|
||||
|
||||
## Actions Required
|
||||
|
||||
### Immediate: Bulk Tag GenieACS Devices
|
||||
The biggest gap is that only 22/7,550 devices have their physical serial as a tag.
|
||||
Two approaches:
|
||||
|
||||
**A. Script via OLT (recommended):**
|
||||
- SSH to each OLT (the `info_connect` IPs: 172.17.0.x)
|
||||
- Run `display ont info` or equivalent to get GPON SN → port mapping
|
||||
- Cross-reference with `fibre` table (frame/slot/port/ontid)
|
||||
- Push tags to GenieACS via NBI: `POST /devices/:id/tags/:serial`
|
||||
|
||||
**B. Enrichment via CWMP serial MAC extraction:**
|
||||
- For Raisecom: extract last 12 chars of CWMP serial as MAC
|
||||
- Match to any MAC-indexed data source
|
||||
- Tag via NBI
|
||||
|
||||
### Medium-term: Migrate WiFi/VoIP Provisioning Data
|
||||
The GenieACS MariaDB at `10.5.14.21` is not accessible from all servers.
|
||||
Export the `wifi` and `voip` tables and migrate to:
|
||||
- ERPNext custom fields on Service Equipment (preferred)
|
||||
- Or a new provisioning table in the targo-hub database
|
||||
|
||||
### For Oktopus: Device Profiles
|
||||
Map each device category to an Oktopus device profile:
|
||||
|
||||
| Legacy Category | GenieACS Model | Oktopus Profile | Config |
|
||||
|---|---|---|---|
|
||||
| raisecom_rcmg | HT803G-W/WS2 | raisecom-ont | VoIP DSCP, NTP, UPnP off, bridge mode |
|
||||
| tplink_tplg | Device2 (GPON) | tplink-ont | WiFi SSID/pwd from DB |
|
||||
| tplink_device2 | Device2 (Deco) | tplink-deco | WiFi SSID/pwd, mesh config |
|
||||
| onu | Various | generic-ont | Basic monitoring |
|
||||
| ht803g1ge | HT803G-1GE | raisecom-1ge | Legacy single-port ONT |
|
||||
|
||||
## Data Volumes Summary
|
||||
|
||||
| Match Type | Count | Confidence |
|
||||
|---|---|---|
|
||||
| GenieACS tagged with physical SN | 22 | 100% |
|
||||
| TP-Link CWMP serial = sticker serial | ~5,000 | High |
|
||||
| Raisecom MAC extractable from CWMP SN | ~2,833 | Medium |
|
||||
| Legacy fibre with SN | 4,948 | Reference |
|
||||
| Legacy device with SN | ~5,900 | Reference |
|
||||
| Needs OLT query for matching | ~2,800 Raisecom | Required |
|
||||
7552
scripts/migration/genieacs-export/devices-full.json
Normal file
7552
scripts/migration/genieacs-export/devices-full.json
Normal file
File diff suppressed because it is too large
Load Diff
7552
scripts/migration/genieacs-export/devices-with-ip.json
Normal file
7552
scripts/migration/genieacs-export/devices-with-ip.json
Normal file
File diff suppressed because it is too large
Load Diff
6721
scripts/migration/genieacs-export/legacy-devices.tsv
Normal file
6721
scripts/migration/genieacs-export/legacy-devices.tsv
Normal file
File diff suppressed because it is too large
Load Diff
16057
scripts/migration/genieacs-export/legacy-fibre.tsv
Normal file
16057
scripts/migration/genieacs-export/legacy-fibre.tsv
Normal file
File diff suppressed because it is too large
Load Diff
8393
scripts/migration/genieacs-export/legacy-services.tsv
Normal file
8393
scripts/migration/genieacs-export/legacy-services.tsv
Normal file
File diff suppressed because it is too large
Load Diff
0
scripts/migration/genieacs-export/legacy-wifi.tsv
Normal file
0
scripts/migration/genieacs-export/legacy-wifi.tsv
Normal file
|
|
Loading…
Reference in New Issue
Block a user