Loona - NY Datacenter Fire – Incident details

This incident has been resolved.

The DC has restored power, we are just waiting for them to restore Network Access.

We have received the following disappointing response from the New York datacenter:

The EWR Secaucus data center remains powered down at this time per the fire marshal.
We have just finished the meeting with the fire marshal, electrical inspectors, and our onsite management. We have made great progress cleaning and after reviewing it with the fire marshal, they have asked us to clean additional spaces and they have also asked us to replace some components of the fire system. They have set a time to come back and review these requests at 9am EDT Wednesday. We are working to comply completely with these new requests with these vendors and are bringing in additional cleaning personnel onsite to make the fire marshal's deadline.
In preparation for being able to allow clients onsite, the fire marshal has stated that we need to perform a full test of the fire/life safety systems which will be done after utility power has been restored and fire system components replaced. We have these vendors standing by for this work tomorrow.
Assuming that all goes as planned, the earliest that clients will be allowed back into the site to power up their servers would be late in the day Wednesday.

We are working to see what alternatives we have, if any.

We have also set our Game Nodes to "degraded performance", due to the Panel being offline.

The latest statement is as follows:

The fire marshal is requiring extensive cleaning of the affected systems and areas before the site can be re-energized. That work is in progress. We were informed that Evocative will need to get an OK from the Mayor, the fire marshal, and the local utility to re-energize the site

The DC Ops team hope to have everything cleaned by mid-day, at which point they will request certification from the fire marshal.
Once approved, the re-energizing process can take up to 5 hours.

Our New York Server Provider has experienced a fire involving their electrical systems. No user data has been lost and this fire has only affected the Datacenters power

The current ETA for power recovery is NEXT TUESDAY.

Updates will be posted as we learn more.