ESPeriments with Nginx (Part III)

Ave!
This is the concluding part of the ESPeriments with Nginx series, where we’ll talk about testing out the HTTPS bits and wiring things up on the ESP8266 end to use HTTPS also.
First off, we’ll need to test out the HTTPS API endpoint. Before doing this, I’ll assume the server (web) app is running and the Nginx proxy has been suitably configured (and includes the HTTPS directives as discussed in the previous post). Essentially, we’ll need to simulate HTTP and HTTPS requests to the API endpoint and see the response behavior and codes. There are an eclectic mix of tools to do this sort of work (e.g Postman, etc), but we’ll be using Fiddler by Telerik, which (IMHO) is a solid piece of kit.
We’ll use Fiddler’s Composer tab to create a suitable request to the API endpoint. Essentially, the server (web) app is designed to receive a POST request whose payload is the JSON representation of a request object – which should contain a field called “Tag” for purposes of illustration. Here’s a screenshot of Fiddler showing the relevant data and options having been entered.

Clicking on the “Execute” button to the right of the screen will attempt to execute the request. So let’s do that.

Oho! Looking at the request pane to the left, we can see that the return code is not/was not 200 as expected. Rather, we got a 405, which in HTTP speak means “Method not allowed”. This is on account of the Nginx configuration as discussed in the previous post, where we’d configured such behavior. Essentially, we want all requests to the API endpoint to be over HTTPS. So we set the request type to HTTPS by editing the text in the address bar – we change “http://192.168.67.128/api” to “https://192.168.67.128/api/”. That should ideally do the trick. Hitting execute again, this dialog box pops out:

Now this is an undesirable state of affairs. From the dialog box, it is stated that: “The server xxxxxxxx presented a certificate that did not validate, due to RemoteCertificateNameMismatch, RemoteCertificateChainErrors”. This is troubling because when we write code for SSL validation on the ESP8266, any errors will probably cause the request to fail, leaving us unable to verify the validity of any RFID tags swiped. Ideally, we want to have no errors at all. However, the truth is the RemoteCertificateNameMismatch is more of a concern so we’ll make that top priority.
The error message provided does provide some insight into the problem though. A “RemoteCertificateNameMismatch” seems to imply that some name somewhere isn’t quite right. Its not plain English though, so we’ll try some simple sleuthing debugging to try and figure out an exact explanation. After some research, I found out that using the command “curl –kv <secure url>” would perform an SSL connection even if the remote host didn’t use a trusted CA (since we’re using a self-signed certificate we need this) [k], and give verbose results [v], which would provide much more informative error messages, so let’s give that a whirl. In this case, the URL will be "https://192.168.67.128/api/". Thus executing the command curl –kv https://192.168.67.128/api/ yields the following output:

And there we have it: “common-name: xxxxxxxxxx does not match ‘192.168.67.128’” . This probably arose during the time we were generating the certificate using the openssl tool and were asked the common name of the server. Setting this to the IP address or DNS name of the server should do the trick. Because this is a local LAN deployment, we’ll use the IP address only. So, what we need to do is to regenerate the certificate as described earlier, taking care to enter the IP address of the server as the common name when prompted. Once that’s done, we need to replace the old certificates (which we stored in /etc/nginx/ssl) with the newly-generated ones. You can delete the old ones and move the new ones in. Just be sure the names are consistent with the filenames specified in the Nginx configuration file for the site, so ensure you name the new certificate and key appropriately. Doing so, we can reattempt to run curl –kv https://192.168.67.128/api/ with bated breath:

Voila! No more SSL errors. We can now focus on the ESP bit.
The default ESP8266HTTPClient that comes with the ESP8266 Arduino SDK supports SSL out of the box. Now, since the ESP is a resource-constrained device, it is infeasible to use traditional verification methods i.e storing basically every major CA’s public key and using it to verify each certificate. So, another approach using certificate “fingerprints” was adopted. Certificate fingerprints are simply cryptographic hashes of the contents of an SSL certificate. Therefore, a certificate should have a unique fingerprint. Any alteration to that certificate or perhaps the use of a different certificate will have a different fingerprint, and that mismatch may be used to detect that the remote party is not in fact, the genuine remote party. More details about fingerprints are available here.
In essence, what we’re trying to do here is this: we will generate the fingerprint of our certificate and store this sole fingerprint on the ESP8266. Next, for every HTTPS request we make, we will generate the fingerprint of the certificate returned by our supposed auth server and compare it to the fingerprint we’d previously generated. A match means its the same/real auth server and we go from there. A mismatch implies that something isn’t right and we abort the request.
We begin by generating the certificate fingerprint using this command: openssl x509 -noout -fingerprint -sha1 -inform pem -in [certificate-file.crt] . Naturally, [certificate-file.crt] is replaced with the path to your .crt file, which might be /etc/nginx/ssl/foobar.crt, for example. On running the command, you get some output that looks like this:
SHA1 Fingerprint:=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
We’re interested in the xx:xx:xx…bits. You could store it as a string in the ESP code, for example, or maybe #define it as a string i.e wrap it in quotes. Most importantly, in either case, replace the colons with spaces. The Arduino library expects the fingerprint to be space-delimited, not colon-delimited.
Next, after #including the <ESP8266HTTPClient.h> library, we’ll write fingerprint verification code as follows:
String SSL_FINGERPRINT = “DE AD BE EF FE ED CA FE DE AD BE EF FE ED CA FE DE AD BE EF”;
HTTPClient http;
if (http.begin(“https://192.168.67.128/api/”,SSL_FINGERPRINT)) {
//go ahead with request
}
else {
//run. away. fast.
}
And that’s it folks! This hasn’t been the best of series and has been fraught with many issues, delays and all. But I hope at the end you’ve picked up a thing or two and can go forth, wrapped up in SSL protections!
Hopefully be seeing you again soon. Until then, stay safe