Yesterday’s global meta outage seemingly took out the company’s entire network, with users unable to access Facebook, Messenger, Instagram, Threads, and Quest headsets.
The outage lasted between one and two hours for most users, and while everything now appears back to normal, questions are naturally being asked about what went wrong …
The company has been tight-lipped, simply acknowledging the outage, and apologising for the inconvenience.
Earlier today, a technical issue caused people to have difficulty accessing some of our services. We resolved the issue as quickly as possible for everyone who was impacted, and we apologize for any inconvenience.
Almost certainly not a broader problem
There was some suggestion that the issue might have been a wider one, with Meta not the only company affected.
Cybersecurity expert Matthew Green said the outage appeared to go beyond Meta. “There are a number of services having trouble with at least parts of their systems, particularly the ability to log into websites,” said Green, an associate professor of computer science and member of the Johns Hopkins University Information Security Institute. “This may indicate a common cause, like a failure at a major cloud services provider.”
However, given the fact that the Meta outage appears to have been total, and no other platforms experienced problems on anything like the same scale, this seems unlikely. Far more limited issues reported with other services may be linked to damaged submarine cables in the Red Sea.
Damage to submarine cables in the Red Sea is disrupting telecommunications networks and forcing providers to reroute as much as a quarter of traffic between Asia, Europe and the Middle East, including internet traffic.
That re-routing would have stressed other parts of the Internet.
Cyber attack possible, but unlikely
There was also inevitable speculation that Meta may have been experiencing a cyber attack, especially as it happened on Super Tuesday, when the greatest number of US states hold their primary presidential elections.
However, while security experts say that this cannot be ruled out, the US Cybersecurity and Infrastructure Security Agency says it considers this unlikely.
At a press briefing yesterday on election security, CISA said it was “aware of the incident and the global scope of it,” adding that malicious cyberactivity was likely not involved.
“We are aware of the incident and at this time we are not aware of any specific election nexus or any specific malicious cyber activity nexus to the outage,” the US agency said.
Some groups did claim to be responsible for an attack, but it’s usual for unsupported claims to be made by organisations hoping to generate publicity for their cause.
Most likely a repeat of 2021
The most likely explanation appears to be that it was effectively a repeat of what happened back in 2021: a mistake made by someone at Meta.
On that occasion, it was an error which occurred, somewhat ironically, when engineers were checking the availability of network capacity.
During one of these routine maintenance jobs, a command was issued with the intention to assess the availability of global backbone capacity, which unintentionally took down all the connections in our backbone network, effectively disconnecting Facebook data centers globally. Our systems are designed to audit commands like these to prevent mistakes like this, but a bug in that audit tool didn’t properly stop the command.
This change caused a complete disconnection of our server connections between our data centers and the internet.
While the exact cause may be different this time (one would certainly hope that particular mistake wasn’t allowed to happen again!), all the signs to point to another internal error of this general nature.
Photo by Kelly Sikkema on Unsplash
FTC: We use income earning auto affiliate links. More.
