Cloudbleed: Breakdown of Cloudflare's Memory Leak
On Thursday, February 23rd, Cloudflare announced a serious bug in its caching infrastructure that caused uninitialized memory to be printed on a number of its customers’ websites. This information included sensitive data such as passwords, cookies, tokens, private messages, and while it believes the bug was limited to roughly a thousand websites, it caused sensitive data to be dumped from potentially any Cloudflare reverse proxy customer. Some observers have stated this issue has similarities with “Heartbleed” and have thus referred to it as “Cloudbleed.”
Vulnerability Background
The issue was caused by a bug in some of Cloudflare’s enterprise product features, such as websites that had one of the following features enabled:
- Email Obfuscation
- Server-side Excludes
- Automatic HTTPS Rewrites
The component that these features had in common was that they parsed the HTML of a web page to obfuscate its data, or perform some other action. Unfortunately, due to an error in the parsing logic when attempting to read HTML that was corrupted, there was a chance that a buffer overrun would occur and output the contents of uninitialized memory back onto the web page.
Cloudflare has stated that the buggy code that caused this issue went live on September 22, 2016, when they released their Automatic HTTPS Rewrite feature. This was the first opportunity for all customer sensitive data to be exposed. Due to the nature of the bug, the sensitive data that was revealed was agnostic to the particular website where it was posted and it could have come from any of Cloudflare’s reverse proxy customers. Any person who had discovered those URLs from that point on would’ve have been able to monitor them and scrape the memory contents of the Cloudflare caching infrastructure.
When Google Project Zero reported the issue to Cloudflare, it acted quickly to prevent further information from being leaked. Since this sensitive data was exposed on public websites, web crawlers such as Google, Bing, and Yahoo had already cached those pages in their search indexes. Cloudflare and Google reached out to some of these services to have those pages removed before announcing the vulnerability. Unfortunately, since Cloudflare is not aware of all caching services, some of these pages may still be retained by other parties and other historical archives.
Since Cloudflare is terminating HTTPS connections for these customers, data that would normally be otherwise protected with encryption using TLS/SSL had a chance of being exposed. The researcher who discovered this issue indicated he observed:
“.. private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We're talking full https requests, client IP addresses, full responses, cookies, passwords, keys ..”
Cloudflare subsequently analyzed the cached contents provided by Google and others and reached out to approximately 150 of its customers it was able to concretely identify. Since this information has been exposed for several months, it’s difficult for Cloudflare to be fully comprehensive in identifying all impacted customers.
Affected Companies
Bitsight Security Ratings customers have the ability to filter their portfolios on specific Cloudflare products and service providers used by their monitored companies. To be able to determine whether a monitored company is potentially affected by this issue, navigate to the Portfolio page and choose "Cloudflare Hosting" under the “Product” filters. The results will include any company that has a website or resource using that Cloudflare product. Customers using Bitsight Discover can find more detailed information about the company resources potentially affected, such as the website, IP addresses, domains impacted, and more.