B Power Feeds to Racks Fault
Resolved
Feb 28 at 01:32pm GMT
Update 4 - RESOLVED: Following the delivery and installation of replacement components, all remedial work has now been completed. The UPS battery infrastructure has been fully restored to operational health, with all systems confirmed stable and performing within expected parameters.
All battery strings have been impedance tested using a calibrated Hioki BT3554-50 battery tester, with every cell returning healthy readings. Bus voltage has been restored to nominal float levels and the UPS is operating normally.
This incident is now closed. We will continue to monitor UPS infrastructure closely and our updated monthly visual inspection procedure remains in place going forward.
We apologise for any concern this incident may have caused and thank you for your patience throughout.
Kind regards,
William Allison | CTO @ EUGameHost
Affected services
Updated
Feb 26 at 04:28pm GMT
Update 3: This incident will be fully resolved on Saturday.
The delay is due to awaiting delivery of replacement batteries. Services remain stable and under constant monitoring.
Going forward, we have also updated our procedures to include monthly UPS battery visual inspections instead of quaterly to help prevent a recurrence of this. We will provide a final confirmation update once remedial work is completed.
Kind regards,
William Allison | CTO @ EUGameHost
Affected services
Updated
Feb 24 at 06:39pm GMT
Update 2: The issue has been fully identified as a faulty battery in one of our battery strings. Upon investigation, a single cell was found to have failed catastrophically. It is worth noting that these batteries were only 6 months old, making this an unexpected and very rare failure according to our UPS Techs.
As a precautionary measure and to ensure the continued reliability of our UPS infrastructure, we will be replacing all batteries in the cabinet.
The system will remain on mains power until the replacement is complete.
We apologise for any inconvenience and will provide a further update once the work has been completed.
Kind regards,
William Allison | CTO @ EUGameHost
Affected services
Updated
Feb 23 at 10:25pm GMT
Update: The affected UPS battery cabinet has been isolated due to an identified battery anomaly. The UPS remains operational on mains, however battery runtime is currently unavailable for the B power feeds to Rack 5 and Tower Rack 2.
There is no active fault affecting power delivery at this time. The risk is limited to loss of battery ride-through in the event of a mains interruption.
Our UPS engineers have been scheduled to attend, investigate and remediate.
We will provide a further update once more information becomes available to us.
For users unsure of if they are affected by this, please reach out to us.
Kind regards,
Will Allison - CTO @ EUGameHost
Affected services
Created
Feb 23 at 07:11pm GMT
We have noticed an issue within the B power feeds to Rack 5 and Tower Rack 2. There is a UPS issue and the load is currently unprotected from mains failure so, as a warning, we are putting this notification out in the event that the power has an issue while in bypass.
Affected services