The generator has been switched off with all systems behaving normally. This incident is now closed.
Posted May 17, 2016 - 13:50 PDT
Update
We have successfully transferred from generator to utility power. The UPS operated normally, holding the load during the transition. Once the generator is shut down, this incident will be concluded.
Posted May 17, 2016 - 13:45 PDT
Update
The replacement UPS has been installed and tested and is functioning properly. The load is protected by the UPS, and we will be transitioning from generator power to utility shortly. No outage is expected. We will update once this issue is closed, or if the situation changes.
Posted May 17, 2016 - 13:32 PDT
Update
We are still waiting for the replacement UPS to arrive at the facility. Updates will follow when the UPS is on-site, installed, and when the situation is normal. Mains power is still being supplied by a generator with sufficient fuel.
Posted May 12, 2016 - 22:52 PDT
Update
Due to issues with the currently installed replacement UPS, the install of another unit will be occurring later this week. It is slated to arrive tomorrow. Updates will follow when the UPS is on-site, installed, and when the situation is normal. Mains power is still being supplied by a generator with sufficient fuel.
Posted May 11, 2016 - 11:54 PDT
Update
Due to the issues related to turn-up, a separate UPS has been sourced to ensure restoration of power protection. The factory technicians will continue work on the current system and complete turn-up if they're able. Otherwise, the other replacement UPS will be on-site for installation early next week. Updates will follow at that time, or if the situation changes significantly. Main power is still supplied by generator with sufficient fuel.
Posted May 06, 2016 - 14:55 PDT
Update
Although turn-up was attempted, we were unsuccessful in bringing the UPS online. Work will continue tomorrow. In the interim, main power is still supplied by generator.
Posted May 05, 2016 - 23:49 PDT
Update
UPS turn-up has been delayed pending a shipment of parts to arrive tomorrow. Power is still being supplied from the generator. The generator has sufficient fuel and will be refueled when needed. No further outages are expected, and we expect the UPS to be online no later than Thursday.
Posted May 03, 2016 - 15:23 PDT
Update
The replacement UPS is currently scheduled for turn-up tomorrow. Prior to turn-up, a component was found to be suspect and is to be replaced before turn-up. An update will follow if the situation changes, or when the replacement UPS is online and protecting the critical load from utility outages. Currently, the building is running on generator power and has sufficient fuel for several days.
Posted May 02, 2016 - 12:55 PDT
Update
At about 12:25PM local time utility power had a failure. Machines are powering back on now.
We are awaiting the window when the UPS will be brought online from the facility.
UPDATED: The time of the outage was incorrectly entered.
Posted May 02, 2016 - 09:28 PDT
Update
At about 5AM local time utility power had a failure.and this caused equipment to reboot.
Posted May 02, 2016 - 04:04 PDT
Update
The new UPS is installed. Turn-up is scheduled for tomorrow after routine diagnostics are run prior to switching the load to the UPS.
Posted May 01, 2016 - 19:42 PDT
Update
The old UPS has been removed and the new UPS is in-place. The new UPS will now be cabled.
Posted Apr 30, 2016 - 01:26 PDT
Update
The replacement UPS is on-site. The process of removing the old UPS is about to begin.
Posted Apr 29, 2016 - 22:16 PDT
Update
UPS diagnostics have been completed. Although the capacitors in the unit have been maintained recently, they have failed. In order to feel confident in UPS performance, the unit is being completely replaced; no outage is expected. We estimate this project to be complete this weekend. Updates will follow if there are any issues, and at the following milestones:
1. The replacement UPS is on-site.
2. The old UPS is removed.
3. The replacement UPS is installed.
4. The replacement UPS is in normal mode.
Posted Apr 29, 2016 - 10:07 PDT
Update
Power restoration to all cabinets has been confirmed. Remaining individual issues are isolated and will be dealt with on a case-by-case basis. UPS diagnostics and repair is scheduled for tomorrow morning to bring the UPS back into normal operation.
Posted Apr 28, 2016 - 21:15 PDT
Monitoring
Power has been restored to the facility and we are in the process of restoring connectivity to all servers.
Posted Apr 28, 2016 - 20:23 PDT
Update
A technician is currently investigating the cause at the UPS and is working to bypass the fault.
Posted Apr 28, 2016 - 20:09 PDT
Investigating
A power outage has been experienced on a UPS in the Miami facility. Redundant power has not been impacted, but this is affecting all customers on "A" power only. We will reply with updates as the issue is resolved.