Logging Infrastructure

Requirements:

• Highly Available
• Minimal Data loss
• Eventually consistent
• Scalable
• Low latency
• Minimum operation overhead

Collecting Logs and putting in storage

App Server

Message Transportation — Kafka

Data Transportation — Out of Kafka Store in Object Store

Disadvantage

• Data Loss — Broker Failure, high back pressure
• Operability — Broker go down, Rolling restart so restart all related service

Solution — Replication at Kafka

Features

Decouple System — Logging Agent failure to Application Failure

Minimize impact of message queue failure

Dynamic Configuration Detection — Adding new pipelines, removign existing pipelines

Adjustable Logging latency

Auditing

Heartbeat