feat: GenieACS config export + TR-069 to TR-369 migration plan
- Add /acs/export endpoint: dumps all provisions, presets, virtual params, files metadata in one call (insurance policy for migration) - Add /acs/provisions, /acs/presets, /acs/virtual-parameters, /acs/files - Shell script export_genieacs.sh for offline full backup - TR069-TO-TR369-MIGRATION.md: phased migration plan from GenieACS to Oktopus with parallel run, provision mapping, CPE batching Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
louispaulb
parent
ea71eec194
commit
56ad97bc71
144
docs/TR069-TO-TR369-MIGRATION.md
Normal file
144
docs/TR069-TO-TR369-MIGRATION.md
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
# TR-069 (GenieACS) → TR-369 (Oktopus) Migration Plan
|
||||
|
||||
## Why Migrate
|
||||
|
||||
1. **GenieACS maintenance risk** — single maintainer, slow release cadence, Node.js/MongoDB stack aging
|
||||
2. **TR-069 is polling-based** — CPE connects every X minutes, no real-time push. Reboot command waits for next inform.
|
||||
3. **TR-369 (USP) is the ITU/BBF successor** — MQTT/WebSocket transport, bidirectional, real-time, certificate-based auth
|
||||
4. **Oktopus is actively developed** — Go-based, NATS messaging, supports both TR-069 and TR-369
|
||||
|
||||
## Current GenieACS Architecture
|
||||
|
||||
```
|
||||
CPE fleet (TR-069)
|
||||
│ HTTP/SOAP (CWMP)
|
||||
▼
|
||||
96.125.192.25 — GenieACS Core (CWMP, NBI, FS)
|
||||
│
|
||||
▼
|
||||
10.5.2.125 — MongoDB (devices, tasks, faults, provisions)
|
||||
│
|
||||
▼
|
||||
10.5.2.124 — GenieACS GUI (admin portal)
|
||||
│
|
||||
▼
|
||||
targo-hub /devices/* — Ops UI integration
|
||||
```
|
||||
|
||||
## Target Oktopus Architecture
|
||||
|
||||
```
|
||||
CPE fleet (TR-369 USP + legacy TR-069)
|
||||
│
|
||||
├── MQTT/WebSocket (USP) ──▶ Oktopus Controller
|
||||
│ │
|
||||
├── HTTP/SOAP (CWMP) ──────▶ Oktopus TR-069 Adapter
|
||||
│ │
|
||||
│ ▼
|
||||
│ NATS (internal messaging)
|
||||
│ │
|
||||
│ ▼
|
||||
│ PostgreSQL / TimescaleDB
|
||||
│
|
||||
▼
|
||||
targo-hub /devices/* — Ops UI integration (same endpoints, new backend)
|
||||
```
|
||||
|
||||
## Migration Phases
|
||||
|
||||
### Phase 0: Export & Document (NOW)
|
||||
- Run `scripts/migration/export_genieacs.sh` on GenieACS server
|
||||
- Catalog all provisions, presets, virtual parameters
|
||||
- Document each provision's purpose and parameter paths
|
||||
- Download all firmware files from GridFS
|
||||
|
||||
### Phase 1: Deploy Oktopus with TR-069 Adapter (Parallel Run)
|
||||
- Deploy Oktopus CE alongside GenieACS (different port/IP)
|
||||
- Enable Oktopus TR-069 adapter to accept CWMP connections
|
||||
- Point a small test group of CPEs (5-10 units) to Oktopus
|
||||
- Verify: inform, reboot, parameter read/write, firmware upgrade
|
||||
- Keep GenieACS running for the rest of the fleet
|
||||
|
||||
### Phase 2: Reproduce Provisions in Oktopus
|
||||
- Map each GenieACS provision to Oktopus equivalent:
|
||||
|
||||
| GenieACS Concept | Oktopus Equivalent |
|
||||
|---|---|
|
||||
| Provision script (JS) | USP Set/Get/Operate messages + webhooks |
|
||||
| `declare(path, ...)` | USP Get/Set on TR-181 path (same paths!) |
|
||||
| `ext('script', ...)` | Oktopus webhook to targo-hub |
|
||||
| Preset (trigger rule) | Device group + event subscription |
|
||||
| Preset tag filter | Oktopus device group membership |
|
||||
| Preset event (inform/boot) | MQTT topic subscription |
|
||||
| Virtual parameter | Oktopus computed metric or targo-hub enrichment |
|
||||
| File (firmware) | Oktopus firmware repository |
|
||||
|
||||
### Phase 3: Gradual CPE Migration
|
||||
- Update CPE ACS URL via GenieACS setParameterValues:
|
||||
```
|
||||
Device.ManagementServer.URL = "https://acs-new.gigafibre.ca"
|
||||
```
|
||||
- Migrate in batches: 50 → 500 → all
|
||||
- Monitor for faults, missed informs, failed tasks
|
||||
- Keep GenieACS read-only as fallback for 30 days
|
||||
|
||||
### Phase 4: TR-369 Firmware Upgrade
|
||||
- For CPEs that support USP (newer models):
|
||||
- Push firmware with TR-369 agent enabled
|
||||
- Configure USP controller URL and MQTT broker
|
||||
- Migrate from TR-069 adapter to native TR-369
|
||||
- For legacy CPEs (TR-069 only):
|
||||
- Keep on Oktopus TR-069 adapter permanently
|
||||
- Replace hardware on failure with TR-369 capable units
|
||||
|
||||
### Phase 5: Decommission GenieACS
|
||||
- Verify 100% of fleet reports to Oktopus
|
||||
- Archive MongoDB data (export to JSON/PostgreSQL)
|
||||
- Shut down GenieACS servers
|
||||
- Update targo-hub to point to Oktopus API instead of GenieACS NBI
|
||||
|
||||
## targo-hub Integration Changes
|
||||
|
||||
The `/devices/*` endpoints in targo-hub currently proxy GenieACS NBI. For Oktopus:
|
||||
|
||||
```
|
||||
# Current (GenieACS NBI)
|
||||
GENIEACS_NBI_URL=http://10.5.2.115:7557
|
||||
|
||||
# Future (Oktopus API)
|
||||
OKTOPUS_API_URL=https://oss.gigafibre.ca/api
|
||||
```
|
||||
|
||||
The `summarizeDevice()` function already handles both TR-098 and TR-181 parameter paths, so the Ops UI won't need changes — only the backend proxy target.
|
||||
|
||||
## CPE Compatibility Check
|
||||
|
||||
Before migration, verify each CPE model's protocol support:
|
||||
|
||||
| Model | TR-069 | TR-369 | Notes |
|
||||
|---|---|---|---|
|
||||
| ZTE F670L | ✅ | ❌ | Legacy, keep on TR-069 adapter |
|
||||
| ZTE F680 | ✅ | ✅ | Firmware update enables USP |
|
||||
| Huawei HG8245H | ✅ | ❌ | Legacy |
|
||||
| Nokia G-010G-Q | ✅ | ✅ | USP via firmware 3.x+ |
|
||||
|
||||
(Fill in with actual fleet models after running device summary export)
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
1. **GenieACS stays running** during entire migration — no big bang cutover
|
||||
2. **Same TR-181 data model** — parameter paths don't change
|
||||
3. **targo-hub abstraction layer** — Ops UI doesn't care which ACS backend serves `/devices/*`
|
||||
4. **Export script** preserves all business logic — can recreate GenieACS from scratch if needed
|
||||
5. **Oktopus TR-069 adapter** means even legacy CPEs work without firmware changes
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
| Phase | Duration | Prerequisite |
|
||||
|---|---|---|
|
||||
| Phase 0: Export | 1 day | SSH access to GenieACS |
|
||||
| Phase 1: Parallel deploy | 1 week | Oktopus server provisioned |
|
||||
| Phase 2: Reproduce provisions | 1-2 weeks | Understanding of all scripts |
|
||||
| Phase 3: CPE migration | 2-4 weeks | Phase 2 validated |
|
||||
| Phase 4: TR-369 firmware | Ongoing | CPE vendor firmware availability |
|
||||
| Phase 5: Decommission | 1 day | 30 days after Phase 3 |
|
||||
120
scripts/migration/export_genieacs.sh
Executable file
120
scripts/migration/export_genieacs.sh
Executable file
|
|
@ -0,0 +1,120 @@
|
|||
#!/bin/bash
|
||||
# ============================================================
|
||||
# GenieACS Full Config Export
|
||||
# ============================================================
|
||||
# Exports all provisions, presets, virtual parameters, and file
|
||||
# metadata from GenieACS NBI API to local JSON files.
|
||||
#
|
||||
# This is the "insurance policy" — captures all ACS logic before
|
||||
# migrating to Oktopus TR-369 or if GenieACS becomes unmaintained.
|
||||
#
|
||||
# Usage:
|
||||
# ./export_genieacs.sh [NBI_URL]
|
||||
# ./export_genieacs.sh http://10.5.2.115:7557
|
||||
#
|
||||
# Output: ./genieacs-export-YYYY-MM-DD/ directory with:
|
||||
# provisions.json — all provision scripts (the business logic)
|
||||
# presets.json — trigger rules (when provisions fire)
|
||||
# virtual-parameters.json — computed/virtual parameters
|
||||
# files.json — firmware/config file metadata
|
||||
# faults.json — current active faults
|
||||
# devices-summary.json — device fleet summary
|
||||
# full-export.json — everything combined
|
||||
# ============================================================
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
NBI="${1:-http://10.5.2.115:7557}"
|
||||
DATE=$(date +%Y-%m-%d)
|
||||
DIR="genieacs-export-${DATE}"
|
||||
|
||||
echo "=== GenieACS Config Export ==="
|
||||
echo "NBI URL: ${NBI}"
|
||||
echo "Output: ${DIR}/"
|
||||
echo ""
|
||||
|
||||
mkdir -p "${DIR}"
|
||||
|
||||
echo "[1/7] Exporting provisions (scripts)..."
|
||||
curl -sf "${NBI}/provisions/" | python3 -m json.tool > "${DIR}/provisions.json" 2>/dev/null || echo "[]" > "${DIR}/provisions.json"
|
||||
COUNT=$(python3 -c "import json; print(len(json.load(open('${DIR}/provisions.json'))))" 2>/dev/null || echo "?")
|
||||
echo " ${COUNT} provisions exported"
|
||||
|
||||
echo "[2/7] Exporting presets (trigger rules)..."
|
||||
curl -sf "${NBI}/presets/" | python3 -m json.tool > "${DIR}/presets.json" 2>/dev/null || echo "[]" > "${DIR}/presets.json"
|
||||
COUNT=$(python3 -c "import json; print(len(json.load(open('${DIR}/presets.json'))))" 2>/dev/null || echo "?")
|
||||
echo " ${COUNT} presets exported"
|
||||
|
||||
echo "[3/7] Exporting virtual parameters..."
|
||||
curl -sf "${NBI}/virtual-parameters/" | python3 -m json.tool > "${DIR}/virtual-parameters.json" 2>/dev/null || echo "[]" > "${DIR}/virtual-parameters.json"
|
||||
COUNT=$(python3 -c "import json; print(len(json.load(open('${DIR}/virtual-parameters.json'))))" 2>/dev/null || echo "?")
|
||||
echo " ${COUNT} virtual parameters exported"
|
||||
|
||||
echo "[4/7] Exporting files metadata..."
|
||||
curl -sf "${NBI}/files/" | python3 -m json.tool > "${DIR}/files.json" 2>/dev/null || echo "[]" > "${DIR}/files.json"
|
||||
COUNT=$(python3 -c "import json; print(len(json.load(open('${DIR}/files.json'))))" 2>/dev/null || echo "?")
|
||||
echo " ${COUNT} files catalogued"
|
||||
|
||||
echo "[5/7] Exporting active faults..."
|
||||
curl -sf "${NBI}/faults/" | python3 -m json.tool > "${DIR}/faults.json" 2>/dev/null || echo "[]" > "${DIR}/faults.json"
|
||||
COUNT=$(python3 -c "import json; print(len(json.load(open('${DIR}/faults.json'))))" 2>/dev/null || echo "?")
|
||||
echo " ${COUNT} active faults"
|
||||
|
||||
echo "[6/7] Exporting device summary..."
|
||||
curl -sf "${NBI}/devices/?projection=DeviceID,_lastInform,_tags&limit=10000" | python3 -c "
|
||||
import json, sys
|
||||
from datetime import datetime, timezone
|
||||
devices = json.load(sys.stdin)
|
||||
now = datetime.now(timezone.utc)
|
||||
stats = {'total': len(devices), 'online': 0, 'offline': 0, 'models': {}, 'manufacturers': {}}
|
||||
for d in devices:
|
||||
did = d.get('DeviceID', {})
|
||||
model = did.get('ProductClass', {}).get('_value', 'Unknown') if isinstance(did.get('ProductClass'), dict) else 'Unknown'
|
||||
mfr = did.get('Manufacturer', {}).get('_value', 'Unknown') if isinstance(did.get('Manufacturer'), dict) else 'Unknown'
|
||||
li = d.get('_lastInform')
|
||||
online = False
|
||||
if li:
|
||||
try:
|
||||
age = (now - datetime.fromisoformat(li.replace('Z', '+00:00'))).total_seconds()
|
||||
online = age < 300
|
||||
except: pass
|
||||
if online: stats['online'] += 1
|
||||
else: stats['offline'] += 1
|
||||
stats['models'][model] = stats['models'].get(model, 0) + 1
|
||||
stats['manufacturers'][mfr] = stats['manufacturers'].get(mfr, 0) + 1
|
||||
json.dump(stats, sys.stdout, indent=2)
|
||||
" > "${DIR}/devices-summary.json" 2>/dev/null || echo "{}" > "${DIR}/devices-summary.json"
|
||||
echo " Done"
|
||||
|
||||
echo "[7/7] Creating combined export..."
|
||||
python3 -c "
|
||||
import json
|
||||
data = {
|
||||
'exportedAt': '$(date -u +%Y-%m-%dT%H:%M:%SZ)',
|
||||
'source': '${NBI}',
|
||||
}
|
||||
for name in ['provisions', 'presets', 'virtual-parameters', 'files', 'faults', 'devices-summary']:
|
||||
try:
|
||||
with open('${DIR}/' + name + '.json') as f:
|
||||
data[name.replace('-', '_')] = json.load(f)
|
||||
except:
|
||||
data[name.replace('-', '_')] = []
|
||||
json.dump(data, open('${DIR}/full-export.json', 'w'), indent=2)
|
||||
" 2>/dev/null
|
||||
echo " ${DIR}/full-export.json"
|
||||
|
||||
echo ""
|
||||
echo "=== Export Complete ==="
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Review provisions.json — these are your business logic scripts"
|
||||
echo " 2. Review presets.json — these define WHEN each provision runs"
|
||||
echo " 3. Map each provision to an Oktopus USP equivalent"
|
||||
echo " 4. Download firmware files: curl -o firmware.bin '${NBI}/files/<file_id>'"
|
||||
echo ""
|
||||
echo "Provision → Oktopus mapping guide:"
|
||||
echo " GenieACS declare() → USP Set message"
|
||||
echo " GenieACS ext() → Oktopus webhook/script"
|
||||
echo " GenieACS log() → Oktopus event logging"
|
||||
echo " Preset tags → Oktopus device groups"
|
||||
echo " Preset events → Oktopus MQTT subscriptions"
|
||||
|
|
@ -683,6 +683,11 @@ const server = http.createServer(async (req, res) => {
|
|||
return handleGenieACS(req, res, method, path, url)
|
||||
}
|
||||
|
||||
// ─── GenieACS Config Export (provisions, presets, virtual params, files) ───
|
||||
if (path.startsWith('/acs/')) {
|
||||
return handleACSConfig(req, res, method, path, url)
|
||||
}
|
||||
|
||||
// ─── 404 ───
|
||||
json(res, 404, { error: 'Not found' })
|
||||
|
||||
|
|
@ -1238,6 +1243,95 @@ async function handleGenieACS (req, res, method, path, url) {
|
|||
}
|
||||
}
|
||||
|
||||
// ── GenieACS Config Export ──
|
||||
// Extract provisions, presets, virtual parameters, files metadata from GenieACS NBI
|
||||
// Used for migration to Oktopus TR-369 and as backup of all ACS logic
|
||||
|
||||
async function handleACSConfig (req, res, method, path, url) {
|
||||
if (!GENIEACS_NBI_URL) return json(res, 503, { error: 'GenieACS NBI not configured' })
|
||||
|
||||
try {
|
||||
const parts = path.replace('/acs/', '').split('/').filter(Boolean)
|
||||
const resource = parts[0]
|
||||
|
||||
// GET /acs/provisions — list all provision scripts
|
||||
if (resource === 'provisions' && method === 'GET') {
|
||||
const r = await genieRequest('GET', '/provisions/')
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/provisions/:id — get single provision script
|
||||
if (resource === 'provisions' && parts[1] && method === 'GET') {
|
||||
const id = decodeURIComponent(parts[1])
|
||||
const r = await genieRequest('GET', '/provisions/' + encodeURIComponent(id))
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/presets — list all presets (trigger rules)
|
||||
if (resource === 'presets' && method === 'GET') {
|
||||
const r = await genieRequest('GET', '/presets/')
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/virtual-parameters — list all virtual parameters
|
||||
if (resource === 'virtual-parameters' && method === 'GET') {
|
||||
const r = await genieRequest('GET', '/virtual-parameters/')
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/files — list all files (firmware, configs)
|
||||
if (resource === 'files' && method === 'GET') {
|
||||
const r = await genieRequest('GET', '/files/')
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/faults — list all active faults
|
||||
if (resource === 'faults' && method === 'GET') {
|
||||
const r = await genieRequest('GET', '/faults/')
|
||||
return json(res, r.status, r.data)
|
||||
}
|
||||
|
||||
// GET /acs/export — full export of all config (provisions + presets + virtual params + files metadata)
|
||||
if (resource === 'export' && method === 'GET') {
|
||||
const [provisions, presets, virtualParams, files, faults] = await Promise.all([
|
||||
genieRequest('GET', '/provisions/').then(r => r.data).catch(() => []),
|
||||
genieRequest('GET', '/presets/').then(r => r.data).catch(() => []),
|
||||
genieRequest('GET', '/virtual-parameters/').then(r => r.data).catch(() => []),
|
||||
genieRequest('GET', '/files/').then(r => r.data).catch(() => []),
|
||||
genieRequest('GET', '/faults/').then(r => r.data).catch(() => []),
|
||||
])
|
||||
|
||||
const exportData = {
|
||||
exportedAt: new Date().toISOString(),
|
||||
source: GENIEACS_NBI_URL,
|
||||
provisions: Array.isArray(provisions) ? provisions : [],
|
||||
presets: Array.isArray(presets) ? presets : [],
|
||||
virtualParameters: Array.isArray(virtualParams) ? virtualParams : [],
|
||||
files: Array.isArray(files) ? files.map(f => ({
|
||||
_id: f._id,
|
||||
metadata: f.metadata || {},
|
||||
filename: f.filename,
|
||||
length: f.length,
|
||||
uploadDate: f.uploadDate,
|
||||
})) : [],
|
||||
faultCount: Array.isArray(faults) ? faults.length : 0,
|
||||
summary: {
|
||||
provisionCount: Array.isArray(provisions) ? provisions.length : 0,
|
||||
presetCount: Array.isArray(presets) ? presets.length : 0,
|
||||
virtualParamCount: Array.isArray(virtualParams) ? virtualParams.length : 0,
|
||||
fileCount: Array.isArray(files) ? files.length : 0,
|
||||
},
|
||||
}
|
||||
return json(res, 200, exportData)
|
||||
}
|
||||
|
||||
return json(res, 404, { error: 'Unknown ACS config endpoint' })
|
||||
} catch (e) {
|
||||
log('ACS config error:', e.message)
|
||||
return json(res, 502, { error: 'GenieACS config error: ' + e.message })
|
||||
}
|
||||
}
|
||||
|
||||
// ── GenieACS NBI Proxy ──
|
||||
// Proxies requests to the GenieACS NBI API for CPE/ONT device management
|
||||
// Endpoints:
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user