“The expiration of a key digital encryption service on Thursday sent major tech companies nationwide scrambling to deal with internet outages that affected millions of online users,” reports the Washington Examiner.
The expiring certificate was issued by Let’s Encrypt — though ZDNet notes there’s been lots of warnings about its pending expiration:
Digital Shadows senior cyber threat analyst Sean Nikkel told ZDNet that Let’s Encrypt put everyone on notice back in May about the expiration of the Root CA Thursday and offered alternatives and workarounds to ensure that devices would not be affected during the changeover. They have also kept a running forum thread open on this issue with fairly quick responses, Nikkel added.
Thursday night the Washington Examiner describes what happened when the big day arrived:
Tech giants — such as Amazon, Google, Microsoft, and Cisco, as well as many smaller tech companies — were still battling with an endless array of issues by the end of the night… At least 2 million people have seen an error message on their phones, computers, or smart gadgets in the past 24 hours detailing some internet connectivity problems due to the certificate issue, according to Scott Helme, an internet security researcher and well-known cybersecurity expert. “So many people have been affected, even if it’s only the inconvenience of not being able to visit certain websites or some of their apps not working,” Helme said.
“This issue has been going on for many hours, and some companies are only just getting around to fixing it, even big companies with a lot of resources. It’s clearly not going smoothly,” he added.
There was an expectation before the certificate expired, Helme said, that the problem would be limited to gadgets and devices bought before 2017 that use the Let’s Encrypt digital certificate and haven’t updated their software. However, many users faced issues on Thursday despite having the most cutting-edge devices and software on hand. Dozens of major tech products and services have been significantly affected by the certificate expiration, such as cloud computing services for Amazon, Google, and Microsoft; IT and cloud security services for Cisco; sellers unable to log in on Shopify; games on RocketLeague; and workflows on Monday.com.
Security researcher Scott Helme also told ZDNet he’d also confirmed issues at many other companies, including Guardian Firewall, Auth0, QuickBooks, and Heroku — but there might be many more beyond that:
“For the affected companies, it’s not like everything is down, but they’re certainly having service issues and have incidents open with staff working to resolve. In many ways, I’ve been talking about this for over a year since it last happened, but it’s a difficult problem to identify. it’s like looking for something that could cause a fire: it’s really obvious when you can see the smoke…!”
Digital certificates expert Tim Callan added that the popularity of DevOps-friendly architectures like containerization, virtualization and cloud has greatly increased the number of certificates the enterprise needs while radically decreasing their average lifespan. “That means many more expiration events, much more administration time required, and greatly increased risk of a failed renewal,” he said.
Read more of this story at Slashdot.