From c31a9e029e4bc9076c6c62a238377c6ab6b370ff Mon Sep 17 00:00:00 2001 From: louispaulb Date: Thu, 21 May 2026 14:40:36 -0400 Subject: [PATCH] docs: recommend frappe_pg community app for ERPNext PostgreSQL compat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ERPNext was built for MariaDB; we run it on PostgreSQL because that's what fit the legacy migration. Frappe's SQL generator is loose on MariaDB (missing GROUP BY columns OK, double-quoted strings OK, HAVING without GROUP BY OK) but strict on Postgres, so we end up hand-patching files in `patches/fix_pg_groupby.py` after every ERPNext upgrade. The community has packaged a comprehensive fix as a Frappe app — `frappe_pg` — that covers the same bugs in one place. The cleaner path long-term is to install that app instead of growing our own patch set. Two doc updates: - docs/architecture/overview.md §6 item 8 — full background: the 3 SQL patterns that break (GROUP BY, HAVING, double-quoted string literals), the 12 hotspots we've already patched, the 4 known remaining (bank_clearance, bank_reconciliation_tool, accounts/utils L1660, gross_profit), and the install recommendation with trade-offs (pin a commit, validate on staging, keep our patches as backup for 4-6 weeks). - docs/SETUP.md §7 — quick-start install commands for whoever decides to flip the switch, plus the warning about pinning rather than tracking main. Also notes that custom Server Scripts with raw SQL (like `customer_balance`) need the same single-quote vs double-quote vigilance even after installing frappe_pg, and the export-fixtures hint to version-control them. --- docs/SETUP.md | 35 +++++++++++++++++++++++++++++++++++ docs/architecture/overview.md | 15 +++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/docs/SETUP.md b/docs/SETUP.md index edeab5d..456b9db 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -96,3 +96,38 @@ docs/ This file + future runbooks matching ERPNext System User. - The Authentik recovery email flow isn't configured (no `flow_recovery` on the brand) — the hub sends the credentials itself instead. + +## 7. ERPNext on PostgreSQL — known incompatibilities + +ERPNext was built for MariaDB. We run on PostgreSQL because the legacy +migration data was easier to handle there. Frappe & ERPNext generate +SQL that's lenient under MariaDB but strict under Postgres — symptoms +on the UI are "column X does not exist" errors or empty/blank reports +on certain accounting screens (Bank Clearance, Payment Reconciliation, +Gross Profit, etc.). + +**Strongly suggested**: install the +[`frappe_pg`](https://github.com/the-commit-company/frappe_pg) +community app, which bundles a comprehensive set of PostgreSQL +compatibility patches as a Frappe app. It's the cleaner alternative +to maintaining our own per-file patches in `patches/fix_pg_groupby.py`, +which we have to re-apply after every ERPNext upgrade. + +```bash +# On the prod box, inside the erpnext-backend container: +docker exec -it erpnext-backend-1 bench get-app frappe_pg +docker exec -it erpnext-backend-1 bench --site erp.gigafibre.ca install-app frappe_pg +docker exec -it erpnext-backend-1 bench restart +``` + +Before installing on prod: pin a known-good commit in `apps.txt` +rather than tracking `main` — the app is community-maintained and can +lag behind ERPNext releases. Validate on staging first by running the +smoke test on the 4 known-broken accounting UIs. + +Our own custom Server Scripts with raw SQL (e.g. `customer_balance`) +need the same vigilance regardless: PostgreSQL treats `"Customer"` as +a column identifier; use `'Customer'` (single quotes) for string +literals. Add a `bench export-fixtures` step to version-control any +Server Script we tweak so the fix isn't lost on re-deployment. See +`docs/architecture/overview.md` §6 item 8 for the full background. diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 1f2a1d8..3033cf9 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -173,3 +173,18 @@ When a CSR clicks "Diagnostiquer" in the Ops app: 5. **Weekly prune** runs via `/etc/cron.d/docker-prune` Sunday 03:00 ET — clears anything not used in 30 days. Don't add a stack you only run monthly without `restart: always` or it'll get pruned out. 6. **PostgreSQL transaction-aborted errors** in the backend log — usually benign (one bad query in the Frappe scheduler) but if persistent, it's the connection pool needing a recycle. `docker restart erpnext-backend-1` resolves. 7. **Authentik recovery flow** isn't configured on the brand. Don't use `recovery_email/` from the API — use the hub invite flow described in §4 instead. +8. **ERPNext was built for MariaDB; we run it on PostgreSQL.** Frappe and ERPNext generate SQL that's tolerant under MariaDB but strict under Postgres — three recurring incompat patterns: + - `GROUP BY` clauses missing non-aggregated columns (Postgres rejects, MariaDB doesn't) + - `HAVING` without a `GROUP BY` (same) + - `"Customer"` interpreted as a column reference under Postgres (it's a string literal under MariaDB) + + We've hand-patched 12 hotspots (see `feedback_erpnext_postgres.md` in the working memory + `patches/fix_pg_groupby.py`), but 4 known issues remain in `accounts/utils.py` ~L1660, `bank_clearance.py`, `bank_reconciliation_tool.py`, and `gross_profit.py`. Symptom on the UI: a report or doctype list returns *"column "X" does not exist"* or stays blank. + + **Recommendation — install the [`frappe_pg`](https://github.com/the-commit-company/frappe_pg) community app.** It bundles a comprehensive set of PostgreSQL compatibility patches for Frappe + ERPNext as an external app — one `bench install-app frappe_pg` instead of patching files one by one on every ERPNext upgrade. Trade-off: a third-party app can lag behind ERPNext releases and may introduce its own issues, so: + - Evaluate it first on staging (re-run the smoke test on the 4 known-broken UIs to confirm coverage) + - Pin a known-good `frappe_pg` commit in `apps.txt` rather than tracking `main` + - Keep our `patches/fix_pg_groupby.py` as a backup; remove it only after frappe_pg has been stable for 4-6 weeks + + When we apply our own patches, they go in `patches/` (Python files that run during `bench update`) so they survive ERPNext upgrades. Never edit ERPNext source files in-place inside the container — the next `bench update` clobbers it. + + Custom Server Scripts with raw SQL (e.g. our `customer_balance` endpoint) need the same vigilance: use `'Customer'` not `"Customer"` for string literals. Add a `bench export-fixtures` step to version-control any Server Script we tweak so the fix isn't lost if ERPNext is re-deployed elsewhere.