First of all I would like to personally apologize for the outage this evening. I understand how important it is to maintain a highly available service for your businesses.

We made a recent change to the orders dashboard designed to make it easier to navigate. When we deployed the change we realized there was a bug in the code. We executed a standard rollback. However, during the rollback the navigation caches did not get primed properly. This caused the database servers to become overloaded. Once this condition occurred it became difficult to get out from under to restore service.

We have had incidents like this before. They have been infrequent. We have come up with a way to mitigate this situation faster if it should occur again.

Sincerely,

William Carr