Security Researchers Lose Faith in HTTP Public Key Pinning – InApps 2022

A few years ago, Google security engineers proposed a method for webmasters to protect their users from hackers who might try to impersonate HTTPS websites by using fraudulently obtained SSL certificates. The security mechanism, known as HTTP Public Key Pinning (HPKP) or certificate pinning, is now an internet standard, but some security researchers are having second thoughts about its efficiency.

It turns out that certificate pinning can cause more harm than good because it’s hard to configure and getting it wrong can leave websites inaccessible. On top of that, hackers can also potentially abuse it for ransomware-like attacks.

The idea behind HPKP — that websites could tell browsers exactly what certificates they should expect from them — came about in response to a series of incidents in which hackers managed to obtain SSL certificates for websites they didn’t own, either by tricking certificate authorities or by compromising their infrastructure.

Trust Issues

The security of the internet’s public key infrastructure relies on the trust placed by browser and operating system makers in organizations known as certificate authorities (CAs). These organizations are allowed to issue publicly-trusted SSL certificates for websites after performing domain name ownership checks. CAs have to follow industry-accepted policies and are subject to periodic external audits, but things sometimes do go wrong for a variety of reasons.

One problem is that are tens of certificate authorities spread around the world and many of them also have authorized resellers that can issue certificates on their behalf. Some CAs are managed by commercial companies, while others are run by government-backed institutions. Some are large and address the global market, while others are small and address specific regional needs. This means that their security posture and level of technical know-how also varies widely.

Theoretically, CAs use automated and manual controls that should prevent them from issuing certificates to people who don’t own or are in control of the websites for which those certificates have been requested. But over the years there have been many cases where CAs got hacked and attackers obtained fraudulent certificates for high-profile websites, or where CA employees unintentionally issued fraudulent certificates or where attackers found and exploited loopholes in their domain ownership verification systems.

The bottom line is that for a domain name owner there is no guarantee that someone else won’t obtain a valid SSL certificate for their website without their knowledge. Such a certificate would enable man-in-the-middle attacks, where attackers in a position to intercept traffic between users and a HTTPS website could decrypt the traffic on the fly or could serve a fake version of that website.

How does HTTP Public Key Pinning Solve This?

As its name implies, HPKP allows website owners to tell browsers to make a record — a pin — of the public key that corresponds to their website’s certificate and to only accept certificates with that key on subsequent visits for a specified period of time.

This is a somewhat simplified explanation because the public key doesn’t necessarily have to be that of the leaf certificate — the end-entity certificate of the website — but can be the key of any certificate in the website’s certificate chain. This includes any intermediate CA the leaf certificate ties back to, or the root CA — the topmost link in the chain.

This provides more flexibility and, for example, allows website owners to tell browsers to accept any valid certificates for their websites as long as those certificates are issued by a specific certificate authority that they know they’ll be using for the foreseeable future.

A website’s HPKP policy and pinned public key are defined through special server response headers. In practice, things are even more complicated because to be valid, an HPKP policy needs to define at least one backup key that doesn’t correspond to any certificates in the chain. This secondary key can be used to generate a new certificate when the primary one expires or needs to be replaced, at which point a new backup key is also generated.

So What’s the Problem?

“The problem with HPKP is that it can be quite a complex idea to get your head around and requires a perfect deployment otherwise things can go wrong,” security researcher Scott Helme, a past advocate for HPKP adoption, said in a recent blog post titled “I’m giving up on HPKP.”

One of the most common problems is that the secret keys corresponding to the pinned public keys can be accidentally deleted, compromised or otherwise lost. In that case, the website owner could obtain a new certificate with a different key, but browsers won’t accept it so the website will become inaccessible to past visitors until the HPKP policy expires, which can be a very long time. According to Helme, this has come to be known among security professionals as HPKP suicide.

If the primary certificate expires and the owner forgets to generate a new certificate using one of the backup keys and instead generates a certificate with a new key and adds it to the HPKP list, visitors will also be locked out. This happened to SmashingMagazine.com last year.

Even worse, if attackers compromise a server they could intentionally modify the HPKP policy and add keys that they control in a way that the website owner can only restore access to the website if the attackers provide their secret key. This was described in a presentation at the DEF CON hacker conference in 2016 and could enable ransomware-like attacks.

Helme is not alone in becoming disillusioned with HPKP. Ryan Sleevi, a software engineer at Google and one of the authors of the HPKP standard, recently tweeted that: “Pinning is terrible — and harms the ecosystem more than helps, as we’ve seen. It was a bad thing to standardize.”

When someone jokingly offered him scotch to learn more about it, he replied: “No scotch needed to get me to apologize for my sins and the painful lessons learned. I actively discourage it now, even with ecosystem risk.”

Ivan Ristic, the former head of the Qualys SSL Labs and author of the Bulletproof SSL and TLS book, also expressed his concerns about HPKP in a detailed blog post last year, declaring it dead.

“I think that, ultimately, HPKP requires too much effort and that only a small number of sites will ever deploy it,” Ristic said. “At the same time, it can be — in the current form — used as a powerful weapon against everybody. The irony of HPKP is that it’s not going to be used by many sites (because it’s too taxing), but that it can be used against the long tail of millions of small sites who are not even aware that HPKP exists.”

The good and the bad news

The good news is that HPKP was standardized fairly recently, though in 2015, it is only supported in Google Chrome, Mozilla Firefox and Opera, so it has not yet seen significant adoption across the Web. Furthermore, while the HPKP standard allows users to create long-lived HPKP policies, Chrome and Firefox only honor observed pins for 60 days, to limit the impact of potential mistakes.

According to a recent scan performed by Helme, around 277,000 of the Alexa Top Million websites currently use HTTPS and only 3,500 have HPKP enabled. Nevertheless, there was a significant jump in adoption since February when a similar scan detected only 500 HPKP-enabled websites across the Alexa Top 1 Million.

The bad news is that even among those 3,500 websites that currently use HPKP, there are plenty of examples of bad deployments, according to Helme.

“Not just a little wrong either, totally wrong, to the point where the browser wouldn’t accept the policy and it’s a good job too because it’s the only thing saving them,” he said.

The bottom line is that unless you really know what you’re doing and you are ready to commit to carefully planned certificate key rotations, you should probably stay away from HPKP; especially since there are other mechanisms that partially solve the same problem.

One such mechanism is Certificate Transparency, a framework increasingly adopted by certificate authorities that involves publishing all issued certificates in a public log. This allows for certificates to be more easily audited and for any mis-issued certificates to be quickly identified.

Another mechanism is a DNS record called Certification Authority Authorization (CAA) that all certificate authorities will have to start honoring later this year. This DNS record can be used by domain owners to specify which CAs are allowed to issue certificates for their domain names.

Google is a sponsor of InApps.

Feature image via Pixabay.