feat: complete device matching analysis (legacy ↔ GenieACS ↔ ERPNext)

Full data export and cross-reference analysis:
- 7,550 GenieACS devices with IPs, deviceId, tags
- 6,720 legacy devices (raisecom, tplink, onu categories)
- 16,056 fibre table entries (OLT frame/slot/port/ontid, VLANs)
- 8,434 legacy services linked to devices

Key finding: CWMP serial ≠ physical serial. Only 22/7,550 devices
are tagged with their physical serial (RCMG/TPLG). Raisecom MAC
is extractable from CWMP serial suffix. TP-Link CWMP serial = sticker
serial for ONT models.

Matching strategy documented: tag-based, MAC-based, OLT port-based.
Recommends bulk tagging via OLT query as first step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
louispaulb 2026-04-03 08:08:57 -04:00
parent 8ba73251f3
commit 231bb6fbcc
7 changed files with 46411 additions and 0 deletions

View File

@ -0,0 +1,136 @@
# GenieACS ↔ Legacy System ↔ ERPNext — Device Matching Analysis
## Data Sources
| Source | Records | Key Fields |
|--------|---------|------------|
| GenieACS (NBI) | 7,550 devices | CWMP serial, OUI, ProductClass, IP, tags, _lastInform |
| Legacy `device` table | 10,417 devices | delivery_id, category, sn (physical), mac, manage IP, parent |
| Legacy `fibre` table | 16,056 entries | sn (physical), OLT frame/slot/port/ontid, olt_ip, VLANs |
| Legacy `service` table | 66,686 services | delivery_id, device_id, product_id, radius_user/pwd |
| Legacy `delivery` table | 17,131 deliveries | account_id, address, city, zip |
| ERPNext Service Equipment | ~10,000 | serial_number, mac_address, customer, service_location |
| GenieACS MariaDB `wifi` | unknown | serial, instance, ssid, password |
| GenieACS MariaDB `voip` | unknown | serial, instance, username, password |
## Fleet Breakdown
### GenieACS (by model)
- **Device2** (TP-Link Deco XE75/XX230v): 4,051 (74% online)
- **HT803G-W** (Raisecom GPON ONT): 493
- **HT803G-WS2** (Raisecom GPON ONT v2): 2,340
- **Device 2** (older TP-Link): 409
- **DISCOVERYSERVICE**: 156 (ghost entries)
- **HT502/HT812** (Grandstream ATA): ~70
- **GXP2130/2160/1630** (Grandstream phones): ~20
- **XX530v/XGB430v_Pro** (TP-Link VoIP): 4
### Legacy (by category)
- **raisecom_rcmg**: 5,016
- **stb_ministra**: 1,212
- **stb**: 1,054
- **tplink_tplg**: 849
- **tplink_device2**: 811
- **airosm**: 832 (Ubiquiti AirOS)
- **airos_ac**: 228
- **cambium**: 118
- **onu**: 40
- **ht803g1ge**: 4
## Matching Strategy
### Key Discovery: CWMP Serial ≠ Physical Serial
GenieACS stores the CWMP negotiated serial number, which is an **internal identifier** — NOT the physical label serial.
**Raisecom HT803G-W pattern:**
```
CWMP Serial: 7IGLC980074DF9D98
^^^^^^^^^^^^
= MAC address (98:00:74:DF:9D:98)
Tag (physical SN): RCMG19E0AB57
```
**TP-Link Device2 pattern:**
```
CWMP Serial: 2234196001225 (manufacturing serial, same as sticker)
Tag: config_manuelle (NOT the serial)
```
### Raisecom Matching: CWMP Serial → MAC → OLT → Fibre → Customer
1. **Extract MAC from CWMP serial**: Last 12 hex chars = MAC (e.g., `980074DF9D98``98:00:74:DF:9D:98`)
2. But legacy `device.mac` field is mostly empty (only 28/5,016 have it)
3. **Alternative**: The 22 tagged devices have `RCMG...` physical serial in tags
4. **Best path**: OLT registration maps GPON SN → frame/slot/port → fibre table
**Proposed workflow:**
- Query each OLT for registered ONT serial numbers and their port assignments
- Match OLT data → `fibre.frame/slot/port/ontid``fibre.service_id``service.delivery_id``delivery.account_id`
- Tag each GenieACS device with its RCMG physical serial from OLT data
### TP-Link Matching: CWMP Serial = Sticker Serial
For TP-Link devices, the CWMP serial IS the sticker serial (e.g., `TPLGA1E7FB90`).
But most TP-Link Device2 (Deco) have alphanumeric serials like `2234196001225`.
**Match via MAC address:**
- GenieACS has `Device.Ethernet.Interface.1.MACAddress` for Device2
- Legacy has `device.mac` for `tplink_device2` category
- ERPNext Service Equipment has `mac_address`
### Recommended Matching Script
```python
# Priority order for matching:
# 1. Tag (physical serial) → fibre.sn or device.sn (exact match, highest confidence)
# 2. MAC address → device.mac or ERPNext mac_address (when available)
# 3. CWMP serial last 12 chars as MAC → MAC-based lookup
# 4. WAN IP (172.17.x.x) → device.manage or fibre OLT topology
# 5. Manual matching for remainder (present device list for human review)
```
## Actions Required
### Immediate: Bulk Tag GenieACS Devices
The biggest gap is that only 22/7,550 devices have their physical serial as a tag.
Two approaches:
**A. Script via OLT (recommended):**
- SSH to each OLT (the `info_connect` IPs: 172.17.0.x)
- Run `display ont info` or equivalent to get GPON SN → port mapping
- Cross-reference with `fibre` table (frame/slot/port/ontid)
- Push tags to GenieACS via NBI: `POST /devices/:id/tags/:serial`
**B. Enrichment via CWMP serial MAC extraction:**
- For Raisecom: extract last 12 chars of CWMP serial as MAC
- Match to any MAC-indexed data source
- Tag via NBI
### Medium-term: Migrate WiFi/VoIP Provisioning Data
The GenieACS MariaDB at `10.5.14.21` is not accessible from all servers.
Export the `wifi` and `voip` tables and migrate to:
- ERPNext custom fields on Service Equipment (preferred)
- Or a new provisioning table in the targo-hub database
### For Oktopus: Device Profiles
Map each device category to an Oktopus device profile:
| Legacy Category | GenieACS Model | Oktopus Profile | Config |
|---|---|---|---|
| raisecom_rcmg | HT803G-W/WS2 | raisecom-ont | VoIP DSCP, NTP, UPnP off, bridge mode |
| tplink_tplg | Device2 (GPON) | tplink-ont | WiFi SSID/pwd from DB |
| tplink_device2 | Device2 (Deco) | tplink-deco | WiFi SSID/pwd, mesh config |
| onu | Various | generic-ont | Basic monitoring |
| ht803g1ge | HT803G-1GE | raisecom-1ge | Legacy single-port ONT |
## Data Volumes Summary
| Match Type | Count | Confidence |
|---|---|---|
| GenieACS tagged with physical SN | 22 | 100% |
| TP-Link CWMP serial = sticker serial | ~5,000 | High |
| Raisecom MAC extractable from CWMP SN | ~2,833 | Medium |
| Legacy fibre with SN | 4,948 | Reference |
| Legacy device with SN | ~5,900 | Reference |
| Needs OLT query for matching | ~2,800 Raisecom | Required |

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff