How encryption evolved to protect us from ISPs

Why ISPs monitor our traffic

ISPs are usually large telecommunications companies that manage the networks – digital subscriber line (DSL), cable, fiber optic, satellite, etc. – that facilitate ‘the information superhighway’ of internet traffic. ISPs also distribute modems and routers (usually an all-in-one box) that enable us to use the internet on multiple devices at home or elsewhere. It is through this infrastructure that ISP monitoring takes place.

It’s important to note that there are a few legitimate reasons as to why an ISP might monitor our traffic. Here are a few examples:

• Service quality – ISPs allocate bandwidth to optimize service based on use. For example, streaming and online gaming require high speed, uninterrupted connections, so they’re given a higher priority. Simpler web activities like browsing or sending emails, which aren’t as sensitive to minor hiccups or delays, are given a lower priority.

• Security – ISPs monitor traffic for signs of malicious activities like malware distribution, phishing attacks, and DDoS (Distributed Denial of Service) attacks. They do this primarily to keep their user base secure and intact, but can also market security upgrades and products.

• Customer support – With a clear overview of user home networks, devices, and traffic patterns, ISP customer support can solve issues faster – and cheaper. They can often remotely access ISP-provided routers as well.

• Regulation – ISPs can be legally obliged to pass user data to law enforcement in certain cases and are required to monitor traffic for illegal activity.

• Targeted advertising – You stream movies? Oh, you need a 4K TV! ISPs build user profiles based on web activity, then upsell products to you or pass your profile to data brokers for targeted advertising.

There are cases when ISPs sell your data. A 2021 Federal Trade Commission report found that, in the US: “Even though several ISPs promise not to sell consumer personal data, they allow it to be used, transferred, and monetized by others, and hide disclosures about such practices in the fine print of their privacy policies.”

How ISPs monetize our data. Source: FTC

What stops ISPs from collecting your data?

1. Regulatory requirements

• The EU’s GDPR tightly controls how ISPs collect, store, and process personal data, which generally ensures a higher level of privacy for users.

• The US is lacking in this area, with no broad federal legislation in place, resulting in a state-by-state patchwork of privacy laws.

• Australia, Brazil, Canada, the EFTA countries, Japan, South Korea, and Switzerland have all enacted data protection regulations.

2. Encryption

In the old days (the wild ‘90s), there was none – ISPs could see everything. Except for some e-commerce and banking services, encryption was almost non-existent. Then in 1995, Taher Elgamal of Netscape developed Secure Sockets Layer (SSL) to secure transactions. This innovation started us down the long and winding road of encryption protocols and their eventual wide scale adoption.

A brief history of SSL to TLS

Secure Sockets Layer (SSL) was developed by Netscape, the pioneering web browser developer, as a protocol to secure transactions. SSL 2.0 was the first version released to the public in 1995. SSL 3.0, which fixed many of the vulnerabilities found in SSL 2.0, came in 1996. The groundwork was laid for future internet security protocols.

Transport Layer Security (TLS) was introduced in 1999 as TLS 1.0 by the Internet Engineering Task Force. Since then, TLS has been the internet’s security standard, undergoing multiple updates and improvements. TLS 1.2, released in 2008, added support for stronger encryption algorithms and was widely adopted for its enhanced security features.

TLS 1.3 arrived in 2018. With a simplified “handshake” process, fewer interactions were needed between client and server to authenticate one another and establish a secure connection. Boasting faster and more robust cryptographic algorithms, TLS 1.3 was a big step forward in speed, security, and privacy.

As of February 2024, 99.9% of the 150,000 most popular websites support TLS 1.2. 67.8% support TLS 1.3, and that number is growing every day.

Timeline of SSL to current day.

SNI: Scaling up the internet

Server Name Indication (SNI), an extension to TLS introduced in 2003, massively scaled up the internet’s hosting capacity. By specifying the target hostname during the “Client Hello” message (the first step in the TLS handshake), multiple HTTPS websites or services could now share a single IP address. With IPv4 addresses running out at the time (total exhaustion occurred in 2011), this was essential to keeping the internet up and running.

SNI was integrated with the QUIC protocol in 2021, boosting performance and security further. But a problem remained. SNI is unencrypted and exposes the hostname (website) that the client is trying to connect to. This issue was highlighted when certain governments including South Korea's began using SNI filtering as a more precise means of censorship and surveillance. SNI’s purpose had been abused by ISPs and governments to collect data.

ESNI, ECH: Final piece of the security puzzle – or not?

So along came Encrypted Server Name Indication (ESNI). Introduced in 2018, it aimed to do exactly what it says on the tin: encrypting SNI. But it would only serve as a stopgap. Cloudflare, the web services company who helped develop the standard, said: “While ESNI took a significant step forward, it falls short of our goal of achieving full handshake encryption. Apart from being incomplete — it only protects SNI — it is vulnerable to a handful of sophisticated attacks.”

Most recently in line was Encrypted Client Hello (ECH) with the more ambitious goal of encrypting the entire Client Hello message. Cloudflare rolled out ECH as a TLS 1.3-exclusive extension in September 2023, but disabled it the following month to address “a number of issues”. A re-release is planned for 2024.

However, even with ECH in place, privacy concerns won’t fully go away. ECH doesn’t fully circumvent traffic analysis or ‘sniffing’ techniques that can reveal metadata like connection times, duration, packet sizes, and more – enough to start a basic user profile for tracking. And users' IP addresses are still always exposed when online. The Internet Protocol routes online traffic, and the client-server model for data transmission wouldn’t work without visible IP addresses.

DNS: Falling short in privacy

Closely related to the IP routing system is the Domain Name System (DNS), known as ‘the internet’s phone book.’ DNS maps domain names to IP addresses. When you type a domain name like www.example.com into your browser search bar, the browser has to find out the domain’s corresponding IP address in order to request the domain’s content for you. To do this, your computer first sends a request to a DNS server, which returns the domain’s IP address (e.g. 142.250.105.100). Without this system, your browser wouldn’t know where to go.

The problem is, ISPs often run their own DNS servers to take a peek as these requests are filled. ISP-provided routers come preconfigured to direct your DNS queries to their proprietary servers. And if ISPs control a DNS server, they can effectively block the use of Encrypted Client Hello by not including ECH configurations in the HTTPS resource records returned to clients.

Protocols like DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT), which encrypt DNS requests, offer solutions to this issue. Not to be outdone, ISPs started operating their own DoH services, controlling DNS settings, and limiting configuration changes. Some providers even argued that DoH is not in the consumer’s interest. Remember: if the ISP runs the DoH service, they can see your online activities.

Even without using DNS or connecting to the wider internet, ISP-managed routers can collect information about the devices connected to them. They can track the unique Media Access Control (MAC) of each device. MAC allows devices to communicate on a local network segment, with the data being openly visible to anyone on the same network. ISPs use software on their routers to capture, fingerprint, and identify devices and their MAC addresses.

What can we do while we wait for ‘total’ encryption?

There are a few things you can take care of.

1. Be aware if you use an ISP managed router

Did it arrive at your door, perhaps with a technician ready to install it, after you signed up? Then it’s managed by the ISP, or at least set to their favored default configurations. Log in to the router, change the default password, and make sure you’re using at least WPA2 encryption. Keep in mind that if you’re using wifi calling (WhatsApp, Facetime, etc), your speech travels through these devices – another reason to fortify your network security.

2. Use a trustworthy DNS server

Look for public, privacy-focused public DNS servers. For example, Cloudflare DNS (1.1.1.1) doesn’t log DNS traffic, doesn’t save your IP address, and doesn’t sell user data to advertisers.

3. Use a VPN

Virtual private networks (VPNs) can protect your online activity by encrypting traffic going from your device to a VPN server. This server then handles your internet requests, shielding them from ISP surveillance. This protection extends to DNS queries if you use the VPN's DNS server. Of course, using a VPN transfers your trust from the ISP to the VPN provider. That’s why no logs VPNs are among the best ways for keeping yourself safe and secure online today.