One Cloud Infrastructure Requires Immediate Maintenance
Currently, several servers in one cloud structure are not responding due to a bug in Windows Server 2012 related to Hyper-V. This is a known issue. Information on this issue is available on the Microsoft website here: http://support.microsoft.com/kb/2823643
Currently, this impacts a small subset of our customers in the Omaha 1 datacenter. Starting immediately, PowerDNN will be performing maintenance on the affected cluster. Applying the Microsoft hot-fix and rebooting the servers will take approximately 30 minutes.
We anticipate that this will solve the issue and that no further interruptions will be necessary.
If you notice any issue inconsistent with this notice, or if you want to know whether your server or website will be affected by this maintenance, contact the support team for immediate assistance.
Update - 1:00 PM CST
Two servers within the affected cluster have failed to restart once the patch was applied. The infrastructure team is addressing this as both a concomitant issue and as a potentially unrelated issue. The rest of the servers on that cluster have returned to normal operating status. We will provide additional information as soon as it is available.
Update - 1:30 PM CST
As of 1:30 PM CST, all sites and servers have returned to fully accessible, normal operations. Since the originating cause or today's issue is the Microsoft Windows Server 2012 bug, PowerDNN is exploring whether the hot-fix provided by Microsoft conflicted with other software or technologies once deployed or whether some other issue, as yet unidentified, prevented certain servers from effectively and promptly restarting. Again, this issue is under investigation.
We understand how even a limited interruption such as this can be disruptive to our customers. We sincerely appreciate the patience of those who were affected. As additional information becomes available we will post it here, in this news item.
If you are continuing to experience any issues at this time, such are likely not related to this notice. Contact the support team for immediate assistance.
The PowerDNN Infrastructure Team
Published: April 11, 2014 at 10:30 AM