Apollo Hospitals at Google Cloud microservices zero-downtime

zero downtime migration of a complex micro services application to Google Cloud for Apollo Hospitals

Achieving zero downtime migration of a complex micro services application to Google Cloud for Apollo Hospitals, a healthcare company based in Chennai, India. One of India’s largest multidisciplinary healthcare systems, it offers online doctor consultations, home diagnostic tests, and online pharmacy to make healthcare affordable and accessible.

This blog describes how it migrated a critical 24×7 national application with 97 services and 40+ SQL databases to Google Cloud with zero downtime.

Growing faster with better performance, security, and reliability

COVID-19 presented new growth challenges for Apollo Hospitals, which was founded before the pandemic.Google needed rapid development in functional and non-functional areas to meet demand. Apollo 24|7 deploys VMs and containers. The application integrates SQL and NoSQL databases, Redis, VMs, Kubernetes, load balancers, WAF, and third-party public endpoints.

After the pandemic, google engineering leadership saw an opportunity to improve platform performance, security, and reliability to continue growth. Modern automation and Infrastructure as Code (IaC) interventions would improve robustness and scale. To provide affordable healthcare to the masses, cost efficiency was prioritized. her app has millions of active users, so needed to avoid downtime to maintain user experience.

They collaborated with award-winning Google Cloud partner Searce to implement to needs smoothly. Apollo Hospitals, Searce, and Google Cloud teams collaborated on development, testing, deployment, and monitoring, which helped the migration project succeed.

Search supported google teams throughout the project with expertise and support.

Full-stack migration without downtime

The goal was to migrate the entire app to Google Cloud without downtime. The QA, pre-production, and production environments were included.

Google process is outlined below:

• DORA survey, then CAST assessment.
• Developed migration waves, dependency graphs, and vulnerable library fixes
• Created a Google Cloud enterprise landing zone and enabled posture management security controls.
• Virtual machines, GKE cluster, Cloud SQL, and Redis instances
• Built CI/CD pipelines for all services
• Deployed Google Cloud services and dependencies
• The Google Cloud Marketplace allows ISV services.
• Connect source DBs to Cloud SQL for change data capture (CDC).
• Split traffic between services to Google Cloud before the final cutover.
• CI/CD pipelines, Terraform scripts, and other resources were used to deploy the QA environment on Google Cloud.

Google next environment was a pre-production replica of the production environment with lower capacity due to low traffic. Simulating a production migration helped us write the right SOPs for each application service.Then also defined service dependencies, split traffic between the old locations and Google Cloud, and tested the functionality end-to-end by sending 1-10% of service traffic to Google Cloud. They installed CDC for databases between source and target on Google Cloud and migrated all traffic to Google Cloud to complete the cutover migration.

They used many pre-production lessons during production migration. On migration day, Identity and Access Management (IAM) issues in a few services integrated with third-party services like payments caused some issues. During partial deployment on GKE, service pods communicated with source location DBs via internet and/or VPN tunnel. Therefore, bandwidth monitoring and NAT gateway port allocation had to be closely monitored to avoid packet drops. Traffic splitting allowed us to divert all traffic from Google Cloud if issues arose. This enabled a zero-downtime application migration.

This migration’s main achievements:

• Migration of 97 services across three GKE clusters with gateway and ingress controllers
• Used 22 Redis instances, Cloud Functions, Pub/Sub, Kafka.
• Took static content to Cloud CDN
• Cloud Armor WAF/DDoS, Security Command Center Premium
• Select Marketplace services like MongoDB and Aiven Managed Kafka
• Split production traffic between the old location and Google Cloud for zero downtime.
• Migration of 40+ MySQL and PostgreSQL databases to CloudSQL using CDC
• Transferred 2+ TB to Google Cloud.
• Reduced operational costs and increased agility and security

Apollo 24|7 successfully migrated through applications to Google Cloud, improving GKE and Cloud SQL performance and latency. Moving away from monolithic code improved architectural tenets and reduced costs by using committed use discounts (CUD) pricing and Google Cloud’s per-second billing, on-demand resources, and custom-sized VMs.

They improved agility and performance by using Terraform to deploy services efficiently with fewer errors using Infrastructure as Code (IaC). The deployment found IAM policy gaps, improving security. Google’s security principles guided thorough cleanup with this project.

Path forward

Apollo Hospitals is also expanding its AI-powered solution partnership beyond the Clinical Decision Support System (CDSS) built with Google Cloud Consulting AI teams. They are introducing Med-PaLM 2 generative AI models for Ask Apollo and beyond.

Google and Apollo Hospitals will continue to transform Indian healthcare. Google Cloud has proven valuable in Apollo 24|7’s core applications, and they want to use it in marketing, supply chain, and customer service.