# Real-Time Ad Data Streaming

Factored built a real-time data streaming platform that scaled to millions of requests, cut latency, and boosted ad performance.

## Key Takeaways:

#### Improving Performance in Real-Time.

The advertisement platform needed to transition from batch processing to a real-time, multi-tenant data streaming platform to support real-time machine learning (ML) and analytics. This new system had to handle massive data volumes, scale efficiently, reduce latency, and ensure high availability while providing self-service capabilities for different teams.

#### A Multi-Stage Solution.

We implemented a Kafka-based data streaming platform, scaling to 1M requests/sec and 100TB daily.

- Trino, Flink, Vertica, and ClickHouse enabled real-time analytics, while self-managed Kubernetes Airflow improved orchestration across 40 teams.
- A self-service platform allowed teams to deploy pipelines with 24/7 monitoring. <10ms latency for Kafka-to-Aerospike transfers ensured real-time bidding.
- Custom monitoring, metadata tracking, and open-source enhancements improved governance, visibility, and platform stability.

#### Improved Data Streaming Capacity 10x.

- Reducing operational overhead.
- Achieved ultra-low latency (sub-10ms) for real-time bidding and streaming analytics.
- Enhanced metadata and lineage tracking.

### Continue Reading

### [Governed Metrics for AI](/content/case-studies/semantic-layer-enterprise-metrics/index.html)

Single logic across every interface

### [Compliance Reporting](/content/case-studies/compliance-reporting/index.html)

Reporting automated

### [Audience Segmentation](/content/case-studies/audience-segmentation/index.html)

Targeting efficiency improved