Universal Containers (UC)uses Apache Kafka on Heroku to stream shipment inventory data in real time throughout the world. A Kafka topic is used to send messages with updates on the shipping container GPS coordinates as they are in transit. UC is using a Heroku Kafka basic-0 plan.The topic was provisioned with 8 partitions, 1 week of retention, and no compaction. The keys for the events are being assigned by Heroku Kafka, which means that they will be randomly distributed between the partitions.
UC has a single-dyno consumer application that persists the data to their Enterprise Data Warehouse (EDW). Recently, they've been noticing data loss in the EDW.
What should an Architect with Kafka experience recommend?
Amina
4 days agoTamie
4 days agoNorah
5 days agoCassi
6 days agoSharika
8 days agoRodrigo
13 days agoEllsworth
16 days ago