Friday, March 22, 2019

Mid-Day Outage

Our outage today--approximately 90 minutes between 11:00am and 12:30pm lasted longer than it should have, because we initially mis-diagnosed it.

Because it presented exactly like the upstream fiber outage earlier this month, we wasted time having FirstLight confirm the condition of their handoff to us. Only after they were able to verify it did we look inside the network to discover we had no power to our principal server cabinet in Ripton. It appears that we suffered a premature failure of an AC breaker there, which was easily corrected once we discovered the problem. We are addressing methods of prevention and faster diagnosis of any future occurrence. This will include some combination of redundant AC circuitry, more robust UPS capacity, and networked signalling of power events.

No comments:

Post a Comment