Near-term maintenance

2 days and 7 hours

Damn! I jinxed it. Now we are really done.

Almost done. All services run on patched instances now. The average downtime was about 7 minutes.

All affected HTTP nodes are migrated. This happened seamlessly without any interruption of web delivery. Next: SSH & Storage clusters

Scheduled maintenance is currently in progress. We will provide updates as necessary.

As you might have heard: Yesterday AWS announced a near-time maintenance of all their data-centers around the globe on the weekend.

About 20% of our instances are affected. Of course we do have automated fail-over and recovery mechanisms in place, but on an event of this magnitude, we rather like to make triple sure.

This is why we start our maintenance on Friday evening. Since AWS officially finishes on Monday night, we will, too. At this moment it's not clear if additional instances might be affected as well.

Here is what we'll do: All affected instances will be gracefully failed over. Storage clusters will be re-build.

What you can expect: Since all storage clusters are afflicted, there will be downtimes. If everything proceeds as planned: We expect those to be in the range of 30-60 Minutes for non-HA Apps and 5-10 Minutes for HA Apps. In addition Git and SSH/SFTP will be down for about 15 minutes.

We'll keep you updated here.

Began at: