Tutu, a travel industry company, maintains a complex infrastructure to ensure 24/7 service availability. The system includes 3,000 microservices across multiple Kubernetes clusters, a monolith, and hundreds of database, queue, and cache clusters. The company aims to keep the site operational even if any major infrastructure component fails.
The company transitioned from managing physical servers to a multi-cloud architecture. This approach utilizes both private clouds and third-party providers. Private clouds handle constant workloads, reducing TCO, while public clouds are used for specific requirements like GPU access, rapid scaling, or compliance with certifications like PCI DSS.
Infrastructure is distributed across three logical sites, each using an independent cloud vendor to mitigate technical and legal risks. This multi-cloud strategy allows for a balanced resource portfolio and provides leverage in vendor negotiations. The company employs cloud-agnostic tools like Terraform for infrastructure management and Ansible for configuration.
For stateless services, Tutu uses Anycast and LVS (Layer 4 load balancer) to manage traffic across data centers. This setup ensures load balancing and allows for the graceful removal of backends from service without disrupting client requests. The company prioritizes network-level connectivity, often using dark fiber to connect to cloud providers.
Stateful services, such as databases, are deployed on virtual machines rather than Kubernetes to ensure data integrity. These systems include custom load balancers and management elements to handle replication switching. The company maintains strict separation between databases and application servers to prevent performance interference.
