Service affecting issue 23/06/2018
Please note that timings are approximate
The following is a summary of the service outage that occurred on Saturday morning, the 23rd June, 2018. We understand the serious impact this has had on our customers, and apologise wholeheartedly to those that this incident affected. We appreciate the patience of all our customers through both the initial outage, and the disruption to service that followed.
At 07:44 AM, a power failure occurred within one of our data-centres which resulted in three servers being restarted. One of these servers being a nameserver used to facilitate DNS requests. While the nameserver in question became operational, our internal monitoring mistakingly identified it as being in good health.
Between the hours of 8 AM and 10AM, the scope of customers affected by this issue grew. At 10 AM, our on-call enginers, who were already investigating the prior power failure, identified that the impact was greater than previously thought and found the root cause to be with the problematic name server.
Finally, at 10:15 AM, the problematic host was recovered and affected customers began to recover full functionality.
This continued until the all clear was given at 10:24 AM
For further information about this, including a full RFO,please contact support on 03333 443 443 or support@soho66.co.uk.
