The Ænix company is developing a turnkey public cloud platform that includes billing, project management, self-service capabilities, and external system integration. The entire infrastructure relies on Kubernetes, utilizing the Kubernetes API Aggregation Layer and controller-runtime for its backend operations.
The platform stores custom resources in etcd clusters managed by a proprietary etcd-operator. For relational data, the system uses Postgres managed by the CloudNativePG operator. The architecture avoids external storage replication tools like Ceph or DRBD, relying instead on the native replication mechanisms of etcd and Postgres.
The design leverages Kubernetes features such as declarative state management, leader election, and health probes to ensure high availability across multiple data centers. Each etcd member and Postgres replica operates on local storage volumes to minimize latency and avoid complex cross-site disk synchronization.
Despite the theoretical resilience of Kubernetes, the Ænix team encountered practical limitations during implementation. To achieve true fault tolerance across data centers, the developers had to build additional custom tools and extensions to address gaps in the standard Kubernetes behavior.
