The infrastructure for the Totum low-code platform was migrated to a high-availability architecture using two primary nodes and a third lightweight witness server. This setup ensures system continuity without requiring a third full-scale server for quorum.
The architecture utilizes Patroni for PostgreSQL failover management and etcd for distributed configuration. The witness node acts as the third voting member in the etcd cluster, allowing the system to maintain a quorum if one of the two main nodes fails.
Application-level high availability is achieved through a role-check mechanism that determines which node is active. L7 load balancers route traffic based on health checks, while lsyncd handles the synchronization of application files and directories between nodes.
The entire infrastructure is managed via Ansible, with specific playbooks used to avoid unintended code deployments. The migration process involved setting up the new environment in parallel, verifying all components, and performing failover tests before switching traffic.
Post-migration performance issues were addressed by analyzing PHP-FPM slow logs. Identifying bottlenecks in external HTTP requests and long polling allowed for the optimization of the PHP-FPM worker pool, increasing max_children to 40 to handle concurrent requests.
