Live Order Sync (Webhooks, Kafka/EKS)

Orders take hours to sync and times vary. Major things this impacts: 1) Inventory - it's impossible not to oversell if orders are done in batches instead of immediately when the order is place. Orders might not sync for 2,3,4 hours at a time. This means that even if you sync inventory when it hits 0 ("zero-blast inventory" is apparently in progress), it may be too late because they may have had 3 units in stock on their platform, but by the time ShipEdge syncs orders, they've gotten 10 orders. So now, it does the inventory sync since it's <=0, but there are -7 units available so it has oversold anyways. 2) SLAs - every warehouse NEEDS SLAs - it is non-negotiable with clients. Unfortunately, orders don't sync when they come in and there is no set time that we can guarantee they'll sync by. So, there is no way to truthfully implement and hold to an SLA where we "ship orders placed by 2pm", because ShipEdge might not sync orders for hours. 3) Express Shipping - related to #2, if orders don't sync on time, then Next Day/2 Day orders aren't guaranteed to get there in 1/2 days because they might not sync until after pickups have come for the day. This is obviously very bad because if someone is using overnight shipping, they clearly need it at a certain time and urgently. ... and many more We've been told live order sync has been due to Shopify/Amazon/etc rate limits, but I don't see that applying here as long as you have throttling logic in place. Rate limit for webhooks is I believe 2 events/sec which should be plenty to do a live sync. Then you can just have a consumer through Kafka (I presume over EKS) listening for those events (AKA orders) and writing those events to ShipEdge immediately, executing the query only when events are available to save resources, while keeping it a real-time sync.