Challenge
JustPremium, the rich media advertising company, was heavily reliant on a service developed by a third-party company to collect, process, and aggregate data from their fleet of ad servers. This dependency created significant challenges, as the third-party provider continually increased pricing, leveraging JustPremium’s reliance on their service. Faced with escalating costs and lack of control over their data processing infrastructure, they approached us with a clear objective: to build their own data lake. This would eliminate the hard dependency on the third-party service, enabling JustPremium to regain control, reduce costs, and achieve greater operational efficiency.
Solution
Our team designed and implemented a scalable data lake that empowered JustPremium to store, process, and aggregate their data efficiently at a fraction of the previous cost. We began by conducting a thorough analysis of JustPremium’s underlying data, ensuring the new system could seamlessly replicate and enhance the functionality provided by the third-party service. A robust backend was developed to support their reporting system, fully replacing the existing solution. To ensure cost efficiency, the system was designed with autoscaling capabilities to dynamically adjust resources based on traffic, reducing unnecessary expenditure during periods of low activity. We implemented advanced configurations for Kafka, focusing on data retention policies to handle high volumes of streaming data effectively. Additionally, performance was significantly enhanced by adding indexes to Athena, which accelerated data retrieval for reporting and analytics.
Results
The new data lake solution significantly reduced costs across data collection, storage, processing, and aggregation, offering substantial savings compared to the previous third-party provider. While the former provider proposed to charge $500,000 per month, the new solution cut costs to just $10,000 per month—achieving a 98% reduction. This shift was even more impactful considering that 3,000GB (3TB) of data was being processed by the data lake daily, efficiently handling massive volumes of information at a fraction of the previous cost. By eliminating reliance on the third-party provider, JustPremium regained control over their data infrastructure, removing pricing pressures and constraints. This shift not only delivered immediate financial benefits but also laid the foundation for sustainable, long-term success with a flexible, scalable infrastructure tailored to their specific needs.
JustPremium was acquired by GumGum, which folded the rich media formats into its contextual advertising platform: read the acquisition announcement.
