Internet Outage Update

[Glen <[e-mail]>]

All -

I've received the following two items from our ISP.  Note that there
are (at least) two cuts in the fiber.

--snip--
IP NOC reports that at the first fiber cut location, the anticipated
slack in the fiber span located in the secondary manhole for the south
end of the span is unable to be used. As such, crews are making
preparations to pull in approximately 3,000 feet of new fiber into a
viable fiber duct. Preparations to the fiber span at the north end
successfully remain ongoing at this time. Field Operations have
provided a tentative estimated time to restore (ETTR) of 0200 GMT on
April 22, 2020.
At the secondary fiber cut location, to expedite service restoration
fiber crews will be performing a single splice point and splicing a
temporary hand hold into service. Splicing efforts are anticipated to
commence at 19:30 GMT.
--snip--

and this just arrived right as I was preparing to send the above:

--snip--
The network is engineered to withstand a single fault in the backbone.
Today, there were multiple fiber cuts that created this unique
situation resulting in abnormal loss and latency inside the backbone
of the network.  The network rerouted traffic as designed around the
cuts and additional traffic engineering has taken place to mitigate
latency observed on IP services. Preparations are underway and
splicing activities are expected to begin in the next 30-minutes on
one of the damaged cables. As these fiber cuts are repaired continued
improvements will be seen as traffic shifts to the most optimal path
per design.  These fiber cuts have been escalated to the highest
levels and all essential repair teams are engaged and focused on the
most expeditious means of repair.
--snip--

Thank you for your patience.

Glen
--
Glen Barney
IT Director
AMS (IETF Secretariat)