Warehouse syncs not starting
Incident
Report for Segment
Resolved
We have confirmed that we have fully recovered and expect no further issues. Warehouse and source syncs will now start and complete correctly for all customers. No data was dropped during the extended outage and subsequent sync delays. If you see any problems, please don't hesitate to contact us at http://segment.com/help/contact
This extended outage was caused by a series of cascading failures in our upstream infrastructure provider that was initially triggered by a scaling event on our end. We're actively working closely with them on diagnosing and resolving the underlying performance issue on their end as well as improving our ability to recover more quickly in situations similar to this.
This extended outage was caused by a series of cascading failures in our upstream infrastructure provider that was initially triggered by a scaling event on our end. We're actively working closely with them on diagnosing and resolving the underlying performance issue on their end as well as improving our ability to recover more quickly in situations similar to this.
Monitoring
We have resolved the underlying issue with our infrastructure provider. Warehouse and Source syncs are now recovering. No data was dropped, and warehouses will catch up during their next sync. We will post an update when we're satisfied that this incident is resolved.
Update
We're still working with our infrastructure provider to resolve the underlying issue as quickly as we can.
Update
We've identified the root cause with our infrastructure provider and will be restoring service shortly.
Update
The underlying cause of the outage is an issue with our infrastructure service provider and we are actively working with them to resolve the issue. We'll update this status incident as soon as we know more.
Update
This issue also affects Cloud Source syncs from 4:00 PM PST onwards. As soon as the underlying issue is resolved, Cloud Sources will recover with no data loss.
Identified
Warehouse syncs scheduled to start on or after 4:00 PM PST are not starting due to an issue in our underlying infrastructure. Engineers are working on the issue now.
No data is being lost and all warehouses will automatically catch up during their next scheduled sync if they missed any sync windows.
No data is being lost and all warehouses will automatically catch up during their next scheduled sync if they missed any sync windows.
This incident affected: Warehouses (Warehouses (US)).