Delayed Data Flow for Server-Side Integrations

Incident Report for Segment

Resolved

We've finished replaying all the data, and the event processing has returned to normal. No data was lost during the delay.
Posted Jun 25, 2016 - 02:10 PDT

Update

We've begun replaying the data that was delayed. It's currently streaming through to our integrations and should finish over the course of the next few hours.
Posted Jun 25, 2016 - 00:32 PDT

Update

Today at 3:55 PM PDT, we deployed a new version of our integrations service which caused issues to outbound data flow for our server-side integrations. We were able to restore proper data flow by 6:11 PM PDT. We are currently working on replaying data for that time period. All integrations offering historical data processing will have data replayed with the proper timestamps. The other integrations may be time shifted reflecting the time of the replay.

No data was lost, and the tracking API, data backup service, and warehouses were not affected.

We will update the status of this page as we make progress.
Posted Jun 24, 2016 - 18:58 PDT

Monitoring

We've resolved the data flow issues and are now replaying the data sent during the outage.
Posted Jun 24, 2016 - 18:24 PDT

Identified

We've noted a major outage for all data flowing outbound through server-side integrations as of 4pm today. We've identified the problem on our side, and we're working to find a fix. No data has been been lost, and we will be doing a backfill and replay from our raw logs as soon as possible.
Posted Jun 24, 2016 - 18:07 PDT