Most developers have written code that sends a HTTP request and reads the response, many have also sent HTTPS requests. This article aims to explain the difference between the two, and how HTTPS works in detail. The article assumes some prior knowledge of client server comminications, and basic encryption concepts, such as public and private keys.
HTTP and HTTPS
HTTP is a protocol, it defines the format of communications between a client and a server, including the request and response, to ensure that both parties understand each other.
When a client sends a HTTP request to a server, it is sent in plain text, meaning any query parameters to the request, headers and contents are sent in plain text. In addition, the server will send back a response over the same channel, meaning the response will also be plain text.
HTTP is therefore open to Man in the Middle attacks, meaning a third party could easily eavesdrop on the communication, or worse, pretend to be the client or the server.
HTTPS is another such protocol, the ‘S’ here stands for secure. People are generally aware that HTTPS is the better option. But why and what does it really mean?
When a server receives an initial client request, over HTTPS, the two parties perform something called cipher suite negotiation. The client supports a certain set of encryption ciphers, as does the server, the two must agree on which to use in order to encrypt further communication.
The client sends the server a list of the ciphers it supports, normally in order of preference. The server examines the list and determines which of the ciphers, if any it will use. If both parties support the same cipher, then good news … we can start encrypting.
Not all ciphers are secure
It’s worth noting that the client and server supporting the same cipher does not mean that the cipher is secure. Take RC4 for example. Many servers and clients will still use the RC4 cipher in some cases, despite it being shown to be insecure. When building an application, be sure to check which ciphers you intend to support. It is often a trade off between browser compatibility and security, so be conscious of this when choosing your applications supported browsers.
HTTPS and Certificates
HTTPS requests and responses are encrypted in transport between the client and the server, in order to do this, the server requires a certificate. This certificate is sent to the client when first contact is made, for example, when you load https://github.com.
The certificate is used to identify the server, as a particular domain name and company. A certificate authority (CA) signs this certificate saying that they trust these details to be correct. The certificate binds the domain name to a public key, which will be used as part of the encryption process.
Handshake and Certificate exchange
Once the server knows it can perform encryption with the client, and the cipher has been agreed, it must hand over it’s certificate for inspection by the client.
The client will check the domain name of the certificate, if it matches the address of the domain name to which the client is talking to then the client will check the certificate against its trusted CAs. If the certificate was signed by a trusted CA then the client will take the public key provided by the certificate, and use it to start communicating securely.
So, given the servers certificate has been validated by the client, are we are now ready to start sending encrypted requests? Well not just yet, the client and server need to perform a handshake in order to generate a shared encryption key.
The specifics of this handshake vary depending on the protocol used, and I am am not going to go into the details in this article. Possible handshake protocols are SSL (secure sockets layer) and TLS (transport layer security), these terms are often used analogously. Many think that the reason for this change is because the two protocols are vastly different, however, the actual reason appears to be a legal one. TLS should be seen as the natural progression of SSL, in fact many people call TLS 1.0, SSL 3.1.
At the moment of writing TLS 1.2 is the preferred protocol for the handshake, there is a good and detailed explanation of how this works here.
Certificate Authorities and Root Certificates
As an aside, an important thing to remember here is that the process described above relies on the CA being trusted by the client. Certificate authorities have a root certificate, this certificate is used to sign further certificates. Browsers normally come with these root certificates installed, so that they can identify fraudulent certificates that have not been correctly signed. If a CA becomes compromised, a browser may remove the root certificate from it’s trusted set and thus will report any sites trying to communicate over HTTPS with a certificate signed with this root certificate as insecure. This is important as the certificate is what lets us know that our information is going to who we intend it to go to, if we can’t trust the certificate then we can’t be sure where our information is going.
In the same way that the browser has a copy of the CA root certificates, and does not need to rely on anyone else to check the authenticity of a certificate, a client can keep a copy of a servers certificate. This is known as certificate pinning.
If a client pins the certificate of a server it means that it does not require a CA to check the authenticity of a server’s certificate. Instead, it can just check whether the received certificate matches the certificate it already has. This means that a client can say that it will only trust a particular certificate, or set of certificates. This is much more secure than having to trust a CA, as you have direct control over who you trust and when to revoke that trust.
For example, a mobile app may ship with the certificate for the domain that it’s API is hosted on, meaning it will only trust communications from the API’s domain. Google Chrome also uses a variation of this mechanism.
So after cipher suite negotiation, certificate exchange, and a handshake, the two parties are able to communicate over an encrypted channel. At any point it is possible for some part of the process to become insecure, whether the cipher, or the CA, or the handshake protocol itself. It is therefore always worth being vigilant and updating your products when new vulnerabilities or more secure technologies are found.