MAB Labs has been working on a project to integrate a proprietary implementation of TLS into the Amazon FreeRTOS’ network stack on an ESP32. As with any non-trivial effort, we faced issues in the porting effort. However, some of the issues we faced were network-related. Specifically, when communicating with the AWS backend using the ESP32, the TLS layer was not happy. To determine the underlying cause, we wanted to capture a network trace (or a “packet capture” – “pcap” for short) of the interactions between the ESP32 and AWS.
Unfortunately, it isn’t straightforward to extract packet data from the ESP32. We had to think outside of the box and leverage some of our past experience. We had done some work on modifying OpenWRT for another client and thought we could use it for this purpose. Our plan was to install OpenWRT on a Raspberry Pi 3 we had laying around and connect it to our ISP modem. Then, once we confirmed that a valid Internet connection was present via the Raspberry Pi, we could configure the ESP32 to connect to the OpenWRT SSID, run the MQTT demo example in the AFR codebase that we were using to validate the TLS port, and capture traffic on the wireless interface (the specific interface is important, which we’ll get to later). Finally, we could send over the pcap to our client to help debug further.
Raspberry Pi 3B Setup and Configuration
The first step was to find the appropriate version of OpenWRT and install it on the Raspberry Pi 3B. This step was straightforward, since there is an excellent guide on OpenWRT’s website here. Since we use our Raspberry Pi 3B for other projects, we used a different SD card to load OpenWRT. This way, we didn’t have to worry about losing any other work.
After plugging in the SD Card with OpenWRT into the Raspberry Pi 3B, we connected a laptop to the Ethernet port of the Raspberry Pi (since the wireless interface isn’t enabled on start). After we were assigned an IP address, we pointed our web browser to 192.168.1.1 to perform two critical steps. First, we had to enable the wireless interface and configure it for the internal LAN. The initial configuration of OpenWRT is shown below, where we navigated to the IP address of the configuration page, clicked on the Network drop down, and clicked on Interfaces (Note: Always set your root password!):
We followed the directions on this forum post to configure OpenWRT as a “normal router”.
Then we installed tcpdump on the Raspberry Pi by executing the following commands on the terminal (by sshing into the 192.168.1.1 IP address from the laptop above);
root@OpenWrt:~# opkg update root@OpenWrt:~# opkg install tcpdump
Finally, we captured packets on the wireless (or “WLAN”) interface of the Raspberry Pi by executing the following command:
root@OpenWrt:~# tcpdump -i wlan0 -s 65535 -w "<capture_file>
“<capture_file>.pcap” is the name of the file where you want to store the captured packets. The filename is usually given a “.pcap” extension so that it can be opened directly in Wireshark (a packet analysis tool).
This step is straightforward. If you’ve followed the steps on Amazon’s instructions here to configure the ESP32 to run FreeRTOS and communicate with your IoT core, you’ll notice that a few files have been automatically generated in your clone of the repo, which are shown below:
The file that we changed is “demos/include/aws_clientcredential.h”. Specifically, we searched for the words SSID and PASSWORD within that file, commented out the macro definitions for our existing access point, and updated them with our OpenWRT accesss point:
Then we rebuilt FreeRTOS, flashed it to the ESP32, ran the demo example (with tcpdump still running in OpenWRT), and noted the IP address that was assigned to the ESP32:
Analysis in Wireshark
After we ran the example and recreated the error condition, we halted the packet capture in OpenWRT (by simply hitting Ctrl-C on the Raspberry Pi). We then extracted the capture file (over the SSH connection using SCP), and opened it in Wireshark. We wanted to filter out the traffic so that we could focus only on the ESP32. To do that, we simply configured the filter in Wireshark to only display packets that have a source or destination IP address that matches what the ESP32 reported above:
This is why we wanted to capture traffic on the WLAN interface. If we had captured traffic on the Ethernet interface (the connection to our cable modem), we would have captured all traffic going out to the Internet, and it would have been difficult to extract the relevant traffic to and from the ESP32.
MAB Labs had to think outside of the box to debug issues faced while working on the ESP32, since there was no straightforward and efficient way to capture packet data on the ESP32. We leveraged OpenWRT and a Raspberry Pi to intercept traffic to and from the ESP32 to help debug our issue. We captured traffic on the wireless interface of the Raspberry Pi to allow us to filter traffic in Wireshark and focus only on the packets sent and received by the ESP32.
If you need any help with your IoT needs, schedule a conversation by clicking here and completing the form.