gigafibre-fsm/scripts/migration/import_memos.py
louispaulb 607ea54b5c refactor: reduce token count, DRY code, consolidate docs
Backend services:
- targo-hub: extract deepGetValue to helpers.js, DRY disconnect reasons
  lookup map, compact CAPABILITIES, consolidate vision.js prompts/schemas,
  extract dispatch scoring weights, trim section dividers across 9 files
- modem-bridge: extract getSession() helper (6 occurrences), resetIdleTimer(),
  consolidate DM query factory, fix duplicate username fill bug, trim headers
  (server.js -36%, tplink-session.js -47%, docker-compose.yml -57%)

Frontend:
- useWifiDiagnostic: extract THRESHOLDS const, split processDiagnostic into
  6 focused helpers (processOnlineStatus, processWanIPs, processRadios,
  processMeshNodes, processClients, checkRadioIssues)
- EquipmentDetail: merge duplicate ROLE_LABELS, remove verbose comments

Documentation (17 → 13 files, -1,400 lines):
- New consolidated README.md (architecture, services, dependencies, auth)
- Merge ECOSYSTEM-OVERVIEW into ARCHITECTURE.md
- Merge MIGRATION-PLAN + ARCHITECTURE-COMPARE + FIELD-GAP + CHANGELOG → MIGRATION.md
- Merge COMPETITIVE-ANALYSIS into PLATFORM-STRATEGY.md
- Update ROADMAP.md with current phase status
- Delete CONTEXT.md (absorbed into README)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 08:39:58 -04:00

101 lines
3.1 KiB
Python

#!/usr/bin/env python3
"""Import account_memo as Comments on Customer. Fix timestamps."""
import pymysql
import psycopg2
import uuid
from datetime import datetime, timezone
from html import unescape
LEGACY = {"host": "legacy-db", "user": "facturation", "password": "VD67owoj",
"database": "gestionclient", "connect_timeout": 30, "read_timeout": 300}
PG = {"host": "db", "port": 5432, "user": "postgres", "password": "123",
"dbname": "_eb65bdc0c4b1b2d6"}
ADMIN = "Administrator"
def uid(p=""):
return p + uuid.uuid4().hex[:10]
def ts_to_dt(t):
if not t or t <= 0: return None
try: return datetime.fromtimestamp(int(t), tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
except: return None
def clean(v):
if not v: return ""
return unescape(str(v)).strip()
def log(msg):
print(msg, flush=True)
def main():
log("=== Import Account Memos as Comments ===")
mc = pymysql.connect(**LEGACY)
cur = mc.cursor(pymysql.cursors.DictCursor)
cur.execute("SELECT * FROM account_memo ORDER BY id")
memos = cur.fetchall()
cur.execute("SELECT id, email FROM staff WHERE email IS NOT NULL AND email != ''")
staff_email = {r["id"]: r["email"] for r in cur.fetchall()}
mc.close()
log(" {} memos loaded".format(len(memos)))
pg = psycopg2.connect(**PG)
pgc = pg.cursor()
pgc.execute('SELECT legacy_account_id, name FROM "tabCustomer" WHERE legacy_account_id > 0')
cust_map = {r[0]: r[1] for r in pgc.fetchall()}
pgc.execute('SELECT name FROM "tabUser" WHERE enabled = 1')
valid_users = set(r[0] for r in pgc.fetchall())
ok = err = 0
for m in memos:
cust_name = cust_map.get(m["account_id"])
if not cust_name:
err += 1
continue
content = clean(m.get("memo")) or "(empty memo)"
created = ts_to_dt(m.get("date_orig"))
if not created:
created = ts_to_dt(m.get("last_updated")) or datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
owner = staff_email.get(m.get("staff_id"), ADMIN)
if owner not in valid_users:
owner = ADMIN
try:
pgc.execute("""
INSERT INTO "tabComment" (
name, creation, modified, modified_by, owner, docstatus, idx,
comment_type, comment_email, content,
reference_doctype, reference_name, published
) VALUES (
%s, %s, %s, %s, %s, 0, 0,
'Comment', %s, %s,
'Customer', %s, 0
)
""", (uid("MEMO-"), created, created, owner, owner,
owner, content, cust_name))
ok += 1
except Exception as e:
err += 1
pg.rollback()
if err <= 5:
log(" ERR memo#{} -> {}".format(m["id"], str(e)[:80]))
if ok % 2000 == 0 and ok > 0:
pg.commit()
log(" {} memos imported".format(ok))
pg.commit()
pg.close()
log("")
log("=" * 50)
log("Memos: {} imported, {} errors".format(ok, err))
log("=" * 50)
if __name__ == "__main__":
main()