Moloco: Using TPUs on Google Kubernetes Engine to train models ten times faster.
Businesses of all sizes must optimize their marketing expenses in the crowded digital world of today. With so many messages competing for the attention of potential consumers, they need to figure out how to stand out. Additionally, they struggle with declining retention rates and growing client acquisition expenses, which hinder their profitability.
The wealth of customer data, which companies frequently find difficult to efficiently use to target the appropriate audience, further complicates matters. Businesses want data-driven advertising tactics to solve these challenges and stay relevant and profitable.
AI-powered Moloco advertising solutions enable user acquisition, retention, and revenue creation. By leveraging its clients’ exclusive first-party data, Moloco Ads’ demand-side platform (DSP) helps businesses target and attract high-value consumers based on real-time consumer behavior, eventually increasing conversion rates and return on investment.
It uses predictions from a dozen deep neural networks to satisfy this demand, and it is constantly creating and testing new models. The platform handles bid requests daily at a maximum pace of 10.5 million queries per second (QPS) and consumes 10 petabytes of data.
Over the past three years, Moloco‘s company has grown by more than eight times, and many of its customers have spent more than $50 million a year. A cost-effective infrastructure that could manage large data processing and real-time machine learning predictions was necessary due to Moloco’s fast expansion. Training times increased as Moloco’s models became more complicated, which hampered creativity and production. The Moloco team independently recognized that in order to grow low-latency ad experiences for users worldwide, they also needed to enhance serving efficiency.
Training complex ML models with GKE
Because of Google Cloud’s scalability, versatility, and strong partner ecosystem, to chose it after assessing many cloud providers and their offerings. Google Cloud’s infrastructure met Moloco’s needs for managing its quickly expanding data and machine learning workloads, which are crucial for maximizing the effectiveness of its clients’ advertising.
It chose Google Cloud over other cloud providers mostly because of Google Kubernetes Engine (GKE). GKE is more than just a container orchestration tool, as Moloco found; it’s a doorway to use AI and ML to their fullest. GKE supports a large number of frameworks and offers scalability and performance optimization tools to accommodate a variety of machine learning workloads, enabling Moloco to tailor the platform to their own requirements.
GKE provides a solid environment for the data processing and distributed computing that support Moloco‘s intricate AI and ML operations by interacting with other Google Cloud services and acting as the cornerstone of a unified AI/ML platform. For workloads that include a lot of reading, GKE’s ML data layer provides the high-throughput storage options that are essential. Efficient resource allocation is ensured by features like cluster autoscaler, node-auto provisioner, and pod autoscalers.
To started utilizing GKE for model training shortly after moving to Google Cloud. But Moloco soon discovered that, both in terms of cost and speed, employing conventional CPUs was not competitive at its size. Moloco’s success was largely due to GKE’s capacity to autoscale on multi-host Tensor Processing Units (TPUs), which are Google’s specialized processing units for machine learning workloads. This enabled Moloco to utilize TPUs at scale, leading to notable improvements in training efficiency and speed.
It also made use of GKE’s AI and ML capabilities to manage its computing resources more efficiently, reducing downtime and saving money while enhancing performance. Notably, it was able to extend its machine learning infrastructure without putting undue burden on its technical team thanks to GKE. Because of this, Moloco’s developers were able to focus on creating AI and ML software rather than maintaining infrastructure.
It deploys ML models into production using GPUs on GKE in addition to training models on TPUs. This improves speed and supports more complicated models by enabling the Moloco platform to efficiently manage real-time inference requests and take use of GKE’s scalability and operational stability.
Throughout the implementation process, to worked closely with the Google Cloud team, taking advantage of their knowledge and direction. Moloco received assistance from the Google Cloud team in putting solutions into place that guaranteed a seamless transition and little interruption to business operations.
In order to maximize resource use and cost effectiveness, it specifically collaborated with the Google Cloud team to move its machine learning workloads to GKE by utilizing the platform’s pod prioritization and autoscaling features. Furthermore, it incorporated Cloud TPUs into its training pipeline, which led to a notable decrease in training durations for intricate machine learning models. Additionally, Moloco used GPUs to enhance its serving infrastructure, guaranteeing its clients low-latency ad experiences.
A powerful foundation for ML training and inference
Moloco’s ability to innovate was significantly altered by their partnership with Google Cloud.
Moloco‘s engineers were able to create with previously unheard-of speed and efficiency as a result of the rapid model iteration and experimentation that this enabled. Furthermore, it was able to develop and deploy state-of-the-art machine learning solutions by managing ever-more complex models and large datasets thanks to the scalability and performance of Google Cloud’s infrastructure. Notably, Moloco’s GPU-supported low-latency advertising experiences promoted improved client retention and satisfaction.
The success of it shows how Google Cloud‘s solutions may help organizations reach their greatest potential. Moloco was able to grow its infrastructure, speed up its ML training, and provide its clients with outstanding ad experiences by utilizing GKE, Cloud TPU, and GPUs. Google Cloud will continue to be a vital component of Moloco’s success as it expands and innovates.
In the meantime, GKE is revolutionizing the field of AI and ML by providing a combination of performance, cost-effectiveness, scalability, and adaptability. Additionally, Google Cloud keeps funding GKE to enable it to manage even the most taxing workloads related to AI training. For instance, GKE now offers unparalleled scale for training or inference by supporting clusters with up to 65,000 nodes.
