The Internet Avoided a Minor Disaster Last Week

A tiny backend bug at Let’s Encrypt almost broke millions of websites. A five-day scramble ensured it didn’t.

The Internet Avoided a Minor Disaster Last Week

This is a story about something that could have gone wrong on the internet this week but instead turned out mostly OK. How often can you say that?

Around 9 o’clock on the East Coast on Friday, February 28, bad news arrived on the doorstep of Let’s Encrypt. An arm of the nonprofit Internet Security Research Group, Let’s Encrypt is a so-called certificate authority that lets websites implement encrypted connections at no cost. A CA parcels out digital certificates that essentially vouch that a website isn't an imposter. That cryptographic guarantee is the backbone of HTTPS, the encrypted connections that keep anyone from intercepting or spying on your interactions with websites.

Those certificates expire after a set amount of time; Let's Encrypt caps its certificates at 90 days, at which point a site operator has to renew. It's a largely automated process, but if a site doesn't have an active certificate, your browser will notice and may not load the page you're trying to visit at all.

Think of it sort of like updating the registration on your car every year. If your tags expire, you'll get pulled over.

Let's Encrypt's work is technical and happens in the background. But in a few short years it has helped make the internet much more secure on a fundamental level. Plenty of companies offer security certificates; Let’s Encrypt just took the audacious step of making them free. A week ago, it issued its billionth certificate.

But that ubiquity also means that when a pebble drops in the middle of Let’s Encrypt’s pond, the ripples can travel a long way. On February 28, the pebble was a bug that threatened to effectively render 3 million sites nonfunctional in a matter of days.

This is a story about something that could have gone wrong on the internet this week but instead turned out mostly OK. How often can you say that?

Around 9 o’clock on the East Coast on Friday, February 28, bad news arrived on the doorstep of Let’s Encrypt. An arm of the nonprofit Internet Security Research Group, Let’s Encrypt is a so-called certificate authority that lets websites implement encrypted connections at no cost. A CA parcels out digital certificates that essentially vouch that a website isn't an imposter. That cryptographic guarantee is the backbone of HTTPS, the encrypted connections that keep anyone from intercepting or spying on your interactions with websites.

Those certificates expire after a set amount of time; Let's Encrypt caps its certificates at 90 days, at which point a site operator has to renew. It's a largely automated process, but if a site doesn't have an active certificate, your browser will notice and may not load the page you're trying to visit at all.

Think of it sort of like updating the registration on your car every year. If your tags expire, you'll get pulled over.

Let's Encrypt's work is technical and happens in the background. But in a few short years it has helped make the internet much more secure on a fundamental level. Plenty of companies offer security certificates; Let’s Encrypt just took the audacious step of making them free. A week ago, it issued its billionth certificate.

But that ubiquity also means that when a pebble drops in the middle of Let’s Encrypt’s pond, the ripples can travel a long way. On February 28, the pebble was a bug that threatened to effectively render 3 million sites nonfunctional in a matter of days.