Centralized RTB ML Platform

Factored built a centralized ML platform for RTB that cut costs by 30%, improved data quality, and standardized CI/CD workflows.

Key Takeaways:

Identifying Strategic Opportunities.

The platform team faced three critical challenges:

• Data Quality Issues. The need to detect missing or corrupted data before it was used in ML models.
• High Snowflake Costs. Inefficient SQL queries and unused data led to excessive cloud storage and computation costs.
• CI/CD Standardization. Multiple CI/CD tools required migration and standardization to streamline workflows.

Building a Platform to Elevate AI Function.

• Data Quality Framework. Built a monitoring system using Victoria Metrics to track data integrity, with PagerDuty for real-time notifications.
• Snowflake Cost Optimization. Implemented query profiling and performance tuning.
• CI/CD Migration to GitHub Actions. Migrated and standardized CI/CD pipelines using GitHub Actions, ensuring a more consistent and automated deployment process.

Lowering Cost by 30%.

• Improving overall efficiency.
  • Preventing data errors.
  • Optimizing query performance.
  • Optimized data usage.