Partial deployment issue

Resolved
{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}
After 3 hours and 33 minutes

We've experienced a higher CPU than usual load on the Git node starting this morning around 4am (UTC). The load itself was not a dramatic issue, since this is caused (and expected) by extensive post-deploy scripts which run in parallel. However, the availability of the node continued to decrease till it was around 9am (UTC) completely unavailable.

At that time, we replaced the node with a new one, which is since available. Note: The IP address of the node has changed, so you might be informed (by your Git/SSH client) about a possible man-in-the-middle attack. This can be safely ignored (in this case). Follow the instructions of your client and remove the old fingerprint from your known-hosts file.

{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}
After 2 hours and 43 minutes

This incident has been resolved.

{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}

One git node (git8) is not responsive at the moment. We are currently looking into the issue.

Began at: