Smart Connected Fleet: Ingesting Telemetry from 500,000 Vehicles
High-scale IoT ingestion pipeline handling 500k connected vehicles with AWS IoT Core MQTT message broker, Amazon Timestream, and real-time anomaly alerting.
1. Business Problem & Context
An electric vehicle manufacturer maintains a fleet of 500,000 connected cars worldwide. Every vehicle transmits 50 telemetry points (battery voltage, motor temperature, tire pressure, GPS coordinates) every 10 seconds over cellular networks. The legacy polling architecture suffered from dropped packets in tunnels and collapsed under 1.2 Billion daily telemetry messages.
2. Requirements & Constraints
- Lightweight Cellular Protocol: Use standard MQTT over TLS with minimal bandwidth consumption.
- Massive Scalability: Ingest up to 60,000 telemetry messages/second during rush hour peaks.
- Time-Series Compression: Store 3 years of historical vehicle telemetry while keeping storage costs manageable.
- Real-Time Collision / Critical Alerting: Trigger emergency dispatch in < 3 seconds upon crash sensor trigger.
3. Architecture Overview & Data Flow
Interactive Architecture Diagram (Use controls to zoom & pan)
- Mutual TLS Authentication: Vehicles authenticate to AWS IoT Core using dedicated device X.509 certificates provisioned in factory HSM chips.
- IoT SQL Rules Engine: Evaluates incoming JSON payloads and splits normal telemetry from urgent crash alerts.
- Kinesis Buffer Stream: Buffers massive telemetry surges and batches writes to Amazon Timestream.
- Tiered Time-Series Storage: Amazon Timestream stores the most recent 24 hours of data in ultra-fast in-memory storage, automatically transitioning older data to magnetic storage at 90% lower cost.
4. AWS Services Used & Rationales
AWS Services Architecture Rationale
Concrete reasons why these specific services were chosen over alternatives
| Service | Category | Architectural Rationale ("Why this service?") |
|---|---|---|
| AWS IoT Core | IoT | Maintains persistent, low-overhead MQTT connections to 500,000 vehicles with X.509 certificate authentication. |
| Amazon Timestream | Database | Serverless time-series database that scales storage and queries independently with automated data tiering. |
| Amazon Kinesis Data Streams | Analytics | Buffers high-frequency vehicle telemetry bursts and enables multiple downstream consumer applications. |
5. Key Design Trade-offs
Architecture Decision & Trade-Off Matrix
Evaluating alternative approaches under real-world constraints
Custom EC2 Mosquitto Broker + Self-Hosted InfluxDB
- + Open source
- − High maintenance overhead
- − Complex clustering and disk resizing
- − Vulnerable to cellular reconnect storms
AWS IoT Core + Amazon Timestream (Chosen)
✓ Chosen Design- + Fully managed serverless MQTT broker
- + Automated in-memory to magnetic storage tiering
- + Built-in time-series SQL functions
- − Requires IoT Rules Engine configuration
6. Implementation Highlights
IoT Rule SQL AWS IoT Core SQL Topic Rule
-- IoT Rule to filter urgent battery overheating alerts
SELECT
topic(2) as vehicle_id,
battery_temp_celsius,
speed_kmh,
gps_lat,
gps_lon,
timestamp() as received_at
FROM 'vehicles/+/telemetry'
WHERE battery_temp_celsius > 55.07. Results & Key Metrics
- Daily Ingestion: Successfully processes over 1.2 Billion messages per day.
- Predictive Maintenance: Reduced roadside breakdown incidents by 34% via early ML detection.
8. Key Architectural Takeaways
IoT Architecture Law: For connected devices, always decouple the device connection broker (AWS IoT Core) from the analytics storage engine (Amazon Timestream) using streaming buffers (Amazon Kinesis) to withstand cellular reconnect storms.
