Mutual TLS (mTLS) Explained: When the Server Also Verifies the Client

root · March 9, 2026, 6:15pm

Cryptography for IoT Hackers - Part 6

Series Index

Part	Title	Status
1	What TLS Actually Does (And Why Your IoT Device Needs It)	Published
2	Certificates From Scratch The Trust Chain Nobody Explains	Published
3	Setting Up Your Own CA and Issuing Certs with OpenSSL	Published
4	ESP32 + TLS: Your First Secure MQTT Connection	Published
5	What the TLS Handshake Looks Like on the Wire (Wireshark Lab)	Published
6 - You are here	Mutual TLS (mTLS) Explained: When the Server Also Verifies the Client	Published
7	MITM Attack on a TLS IoT Device
8	Cert Pinning The Fix, and How Attackers Bypass It
9	Embedded Crypto Pitfalls: Hardcoded Keys and Weak RNG
10	Breaking mTLS: Stolen Certs and Certificate Confusion
11	Secure Provisioning How to Get Certs Onto Devices Safely
12	Building a Hardened ESP32 TLS Client Checklist and Final Lab
13	Secure OTA Updates Why Your Update Channel Is an Attack Surface

Before You Read This

You need Parts 1 through 5 before this one.

Quick recap of where we are. In Part 3 we created our own Root CA and issued real certificates using OpenSSL. In Part 4 we got the ESP32 talking to a Mosquitto broker over TLS on port 8883 encrypted, authenticated. In Part 5 we opened Wireshark and watched the TLS handshake happen live on the wire, packet by packet.

Everything was working. The ESP32 was verifying the broker. Encrypted messages were arriving. Life was good.

But there was a problem hiding in the setup. And Part 6 is where we find it and fix it.

The Problem With Regular TLS

Let me ask you something before we go anywhere.

In Part 4 and Part 5 when the ESP32 connected to the broker did the broker ever ask the ESP32 “who are you?” Did the broker check any certificate from the ESP32 side?

No. It did not.

The broker just let the connection in. Any device that showed up on port 8883 with a working TLS setup got connected. The broker only checked one thing is this a valid TLS connection? If yes, come in.

Now think about a real IoT deployment. You have 1000 ESP32 sensors in the field sending temperature data to your broker. A hacker gets hold of your rootCA.crt maybe they dumped firmware from one device, maybe they sniffed it off the wire (remember from Part 5, the certificate is visible in Wireshark). They know your broker’s IP.

They write their own sketch. Same broker IP. Same rootCA.crt. They flash it onto their own ESP32.

What happens when they connect?

The broker lets them in. Because the broker never asked “are you one of MY devices?”

They can now publish fake sensor data to your broker. Your entire system is feeding on attacker-controlled data.

That is the gap. And mTLS is the fix.

What mTLS Actually Is

mTLS stands for mutual TLS. The “mutual” part means both sides verify each other.

In regular TLS, only the client checks the server’s certificate. The server does not check anything about the client.

In mTLS, the server also demands a certificate from the client. If the client cannot present a valid certificate signed by the trusted CA the broker refuses the connection. Full stop.

The hacker has no certificate signed by your CA. They get kicked out at the door.

How the Handshake Changes Regular TLS vs mTLS

This is important to understand before touching any config or code. Let me walk through both side by side.

Regular TLS (what we had in Part 4 and Part 5):

1. ESP32  → Broker : Client Hello  (here are my supported cipher suites)
2. Broker → ESP32  : Server Hello + server.crt  (here is my identity)
3. ESP32 verifies server.crt using rootCA.crt ✅
4. Both sides do ECDHE → session key generated
5. Encrypted connection open. Broker never checked ESP32 identity.

mTLS (what we are building now):

1. ESP32  → Broker : Client Hello  (here are my supported cipher suites)
2. Broker → ESP32  : Server Hello + server.crt + "send me your certificate too"
3. ESP32 verifies server.crt using rootCA.crt ✅
4. ESP32  → Broker : esp32.crt  (here is MY identity card)
5. Broker → ESP32  : "prove you own that certificate — sign this random number"
6. ESP32 signs the challenge using esp32.key → sends signature back
7. Broker verifies signature using public key inside esp32.crt ✅
8. Both sides do ECDHE → session key generated
9. Encrypted connection open. Both sides verified.

Step 2 now has one extra thing Certificate Request. That is the broker saying “I need your cert too.”

Steps 4, 5, 6, 7 are completely new. That is the mTLS addition.

You will actually see all of this in Wireshark later in this blog the Certificate Request, the Certificate packet from the ESP32, and the Certificate Verify packet.

Why Does the ESP32 Need Both esp32.crt AND esp32.key?

This is a question I had when I first set this up.

The broker receives esp32.crt. That is just a file. Anyone can copy a certificate file. What stops a hacker from grabbing esp32.crt off the wire (it travels in plaintext, just like server.crt does) and presenting it as their own?

The answer is the challenge-response step.

Think of it like this. Anyone can print a fake ID card with your name on it. But if someone asks you to prove the ID is yours by doing something only you can do like signing a document with your handwriting a fake ID fails that test.

In mTLS, the broker sends a random number (the challenge). The ESP32 signs it using esp32.key. The broker verifies the signature using the public key that is inside esp32.crt. If the signature matches only the real owner of esp32.key could have done that.

The private key never leaves the ESP32. It just proves ownership by signing something. That is the whole point.

So:

esp32.crt = identity card (who you are)
esp32.key = proof of ownership (you really are that person)

Both are needed. One without the other is useless.

What You Need for This Part

Everything from Parts 3, 4, and 5 still set up and working
Your ~/my-ca folder with rootCA.crt, rootCA.key, server.crt, server.key
Mosquitto running on Kali with TLS config from Part 4
ESP32 DevKit
Wireshark on Kali

Step 0 - IP Changed? Reissue the Server Certificate

This caught me off guard. I moved to a different network and my Kali IP changed from 192.168.28.67 to 192.168.0.105.

In Part 3 when we created server.crt we added a SAN (Subject Alternative Name) with the Kali IP. The ESP32 connects using an IP address, and when the broker sends its certificate, the ESP32 checks does this certificate belong to this IP?

If the IP changed, the SAN no longer matches. You will get an SSL handshake failure.

Check what IP is in your current server cert:

openssl x509 -in ~/my-ca/server.crt -noout -text | grep -A5 "Subject Alternative"

If the IP shown does not match your current Kali IP you need to reissue. Here is how.

Why do we use SAN and not just the CN?

The CN (Common Name) in our certificate says iotsec.local that is a hostname. The ESP32 connects using an IP address, not a hostname. Modern TLS does not allow IP matching against the CN it only allows it in the SAN field. So we put the IP in the SAN so the ESP32 can verify “yes, this certificate belongs to 192.168.0.105.”

Reissue the server certificate with the correct IP:

# Step 1 — Create SAN config with new IP
cat > ~/my-ca/server_ext.cnf << EOF
subjectAltName=IP:192.168.0.105,DNS:iotsec.local
EOF

# Step 2 — Generate new CSR (reusing existing server key)
openssl req -new -key ~/my-ca/server.key -out ~/my-ca/server.csr \
  -subj "/C=NP/O=IoTSec/CN=iotsec.local"

If you get a permission error on server.key — fix ownership first:

sudo chown iotsec:iotsec ~/my-ca/server.key

# Step 3 — Sign with rootCA using new SAN
openssl x509 -req -in ~/my-ca/server.csr \
  -CA ~/my-ca/rootCA.crt \
  -CAkey ~/my-ca/rootCA.key \
  -CAcreateserial \
  -out ~/my-ca/server.crt \
  -days 365 \
  -extfile ~/my-ca/server_ext.cnf

Verify it:

openssl x509 -in ~/my-ca/server.crt -noout -text | grep -A5 "Subject Alternative"

You should see your current Kali IP in the output.

Step 1 - Create the ESP32 Client Certificate

In regular TLS, only the broker had a certificate. Now the ESP32 needs one too.

Same process as Part 3 when we created the server certificate. Three steps.

Step 1 - Generate ESP32 private key:

openssl genrsa -out ~/my-ca/esp32.key 2048

What this does: generates a 2048-bit RSA private key for the ESP32. This is the key the ESP32 will use to sign the broker’s challenge.

Step 2 - Create a CSR (Certificate Signing Request):

openssl req -new -key ~/my-ca/esp32.key \
  -out ~/my-ca/esp32.csr \
  -subj "/C=NP/O=IoTSec/CN=esp32-client"

What this does: creates a request saying “I am esp32-client, please sign my certificate.” The CN here is esp32-client just an identifier. The broker will see this name when the ESP32 connects.

Step 3 - Sign with our rootCA:

openssl x509 -req -in ~/my-ca/esp32.csr \
  -CA ~/my-ca/rootCA.crt \
  -CAkey ~/my-ca/rootCA.key \
  -CAcreateserial \
  -out ~/my-ca/esp32.crt \
  -days 365

What this does: our Root CA signs the ESP32’s certificate. Now the broker can verify it because the broker trusts our Root CA.

Notice we did NOT add a SAN here. The ESP32 client certificate does not need one SAN is for server certificates where the client needs to verify an IP or hostname. Client certificates just need an identity (the CN).

Check your ~/my-ca folder now. You should have:

rootCA.key      - Root CA private key (never leaves this machine)
rootCA.crt      - Root CA certificate (trusted by everyone in our setup)
server.key      - Broker's private key
server.crt      - Broker's certificate (signed by rootCA)
esp32.key       - ESP32's private key (will go into the sketch)
esp32.crt       - ESP32's certificate (will go into the sketch)

Step 2 - Update Mosquitto Config to Require Client Certificates

This is the smallest change in the whole blog. One word.

Open the TLS config:

sudo nano /etc/mosquitto/conf.d/tls.conf

Your current config looks like this:

allow_anonymous true
listener 8883
cafile /home/iotsec/my-ca/rootCA.crt
certfile /home/iotsec/my-ca/server.crt
keyfile /home/iotsec/my-ca/server.key
require_certificate false

Change the last line:

require_certificate true

That single change tells Mosquitto from now on, any client that connects must present a valid certificate signed by our rootCA.crt. No certificate, no entry.

Save and exit (Ctrl+X, Y, Enter).

Now fix the server key permissions so Mosquitto can read it:

sudo chown mosquitto:mosquitto ~/my-ca/server.key
sudo chmod 640 ~/my-ca/server.key

Restart Mosquitto:

sudo systemctl restart mosquitto
sudo systemctl status mosquitto

You should see Active: active (running). If it fails, run this to see the actual error:

sudo mosquitto -c /etc/mosquitto/mosquitto.conf

Step 3 - Test mTLS From the Terminal First

Before touching the ESP32, always test the broker from the terminal. If the broker has a problem, you want to find it here not after flashing the ESP32 and spending 20 minutes wondering why it is not connecting.

Open two terminals.

Terminal 1 - Subscribe with client certificate:

mosquitto_sub -h 192.168.0.105 -p 8883 \
  --cafile ~/my-ca/rootCA.crt \
  --cert ~/my-ca/esp32.crt \
  --key ~/my-ca/esp32.key \
  -t test/topic

Breaking down the flags:

Flag	What it does
`-h 192.168.0.105`	Broker IP - connect to this machine
`-p 8883`	Port - TLS MQTT port
`--cafile rootCA.crt`	Use this to verify the broker’s certificate
`--cert esp32.crt`	This is MY certificate - send it to the broker
`--key esp32.key`	This is MY private key - proves I own that cert
`-t test/topic`	Subscribe to this topic

The --cert and --key flags are the two new additions compared to Part 4. That is the entire mTLS difference on the client side.

Terminal 2 - Publish with client certificate:

mosquitto_pub -h 192.168.0.105 -p 8883 \
  --cafile ~/my-ca/rootCA.crt \
  --cert ~/my-ca/esp32.crt \
  --key ~/my-ca/esp32.key \
  -t test/topic -m "hello from mTLS"

If Terminal 1 shows hello from mTLS broker is working correctly.

Step 4 - Prove the Broker Is Actually Enforcing mTLS

This is the important test. Try connecting without any client certificate:

mosquitto_sub -h 192.168.0.105 -p 8883 \
  --cafile ~/my-ca/rootCA.crt \
  -t test/topic

You should see:

Error: The connection was lost.

That error means the broker said no. No certificate presented, connection refused.

This is exactly what happens to the hacker now. They can reach the broker. The TLS handshake starts. But when the broker asks “send me your certificate” and they have nothing connection dropped.

Step 5 - Update the ESP32 Sketch for mTLS

In Part 4 the sketch had one certificate hardcoded rootCA.crt. That was the only thing needed for regular TLS.

For mTLS, two more things go into the sketch:

esp32.crt - the ESP32’s identity certificate to send to the broker
esp32.key - the private key to sign the broker’s challenge

Get the content of all three files:

cat ~/my-ca/rootCA.crt
cat ~/my-ca/esp32.crt
cat ~/my-ca/esp32.key

Now here is the full sketch. Read through the comments the new lines are clearly marked:

#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <PubSubClient.h>

// --- WiFi credentials ---
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";

// --- Broker ---
const char* mqtt_server = "192.168.0.105";  // your Kali IP
const int mqtt_port = 8883;

// --- Root CA (same as Part 4) ---
// ESP32 uses this to verify the broker's certificate
const char* rootCA = \
"-----BEGIN CERTIFICATE-----\n" \
"YOUR_ROOT_CA_CERT_HERE\n" \
"-----END CERTIFICATE-----\n";

// --- ESP32 Client Certificate (NEW for mTLS) ---
// ESP32 sends this to the broker as its identity card
const char* esp32_cert = \
"-----BEGIN CERTIFICATE-----\n" \
"YOUR_ESP32_CERT_HERE\n" \
"-----END CERTIFICATE-----\n";

// --- ESP32 Private Key (NEW for mTLS) ---
// ESP32 uses this to sign the broker's challenge and prove ownership
const char* esp32_key = \
"-----BEGIN PRIVATE KEY-----\n" \
"YOUR_ESP32_KEY_HERE\n" \
"-----END PRIVATE KEY-----\n";

WiFiClientSecure espClient;
PubSubClient client(espClient);

void connectWiFi() {
  Serial.print("Connecting to WiFi");
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nWiFi connected");
  Serial.println(WiFi.localIP());
}

void connectMQTT() {
  while (!client.connected()) {
    Serial.print("Connecting to MQTT broker...");
    if (client.connect("ESP32-mTLS")) {
      Serial.println("Connected!");
      client.publish("test/topic", "hello from ESP32 mTLS");
    } else {
      Serial.print("Failed, rc=");
      Serial.println(client.state());
      delay(3000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  connectWiFi();

  // Part 4 had only this one line:
  espClient.setCACert(rootCA);

  // mTLS adds these two new lines:
  espClient.setCertificate(esp32_cert);  // send our cert to broker
  espClient.setPrivateKey(esp32_key);    // prove we own it

  client.setServer(mqtt_server, mqtt_port);
  connectMQTT();
}

void loop() {
  client.loop();
}

The entire mTLS change in the sketch is two lines in setup():

espClient.setCertificate(esp32_cert);
espClient.setPrivateKey(esp32_key);

That is it. Everything else is identical to Part 4.

Flash the sketch. Open Serial Monitor at 115200 baud. Expected output:

.....
WiFi connected
192.168.0.107
Connecting to MQTT broker...Connected!

On the Kali terminal running mosquitto_sub you should see:

hello from ESP32 mTLS

Step 6 - Watch mTLS in Wireshark

Now the fun part. Let’s see exactly what changed on the wire compared to Part 5.

Set your filter:

tcp.port == 8883

Start capture, then reboot the ESP32 to trigger a fresh handshake from the beginning.

Here is what you will see:

20   TCP      ESP32 → Broker   SYN
21   TCP      Broker → ESP32   SYN, ACK
22   TCP      ESP32 → Broker   ACK
23   TLSv1.2  ESP32 → Broker   Client Hello
24   TCP      Broker → ESP32   ACK
25   TLSv1.2  Broker → ESP32   Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
26   TCP      ESP32 → Broker   ACK
28   TLSv1.2  ESP32 → Broker   Certificate
30   TLSv1.2  ESP32 → Broker   Client Key Exchange
34   TLSv1.2  ESP32 → Broker   Certificate Verify
35   TLSv1.2  ESP32 → Broker   Change Cipher Spec
36   TLSv1.2  ESP32 → Broker   Encrypted Handshake Message
40   TLSv1.2  Broker → ESP32   New Session Ticket, Change Cipher Spec, Encrypted Handshake Message
42   TLSv1.2  ESP32 → Broker   Application Data

Now compare this to the Part 5 capture. Two things are different.

Difference 1 - Packet 25: Certificate Request appears

In Part 5, packet 25 was:

Server Hello, Certificate, Server Key Exchange, Server Hello Done

In Part 6, packet 25 is:

Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done

That one extra thing Certificate Request is the broker saying “I need your certificate too.” That single addition in the handshake is what makes it mTLS.

Difference 2 — Packets 28 and 34 are completely new

In Part 5 these packets did not exist at all.

Packet	What it is	What is happening
28	Certificate	ESP32 sending `esp32.crt` to the broker “here is my identity card”
34	Certificate Verify	ESP32 signing the broker’s challenge with `esp32.key` — “prove I own that cert”

If you click on packet 25 and expand it in Wireshark, you will see the Certificate Request section. The broker is listing which CAs it trusts and our IoTSec Root CA will be in that list.

If you click on packet 28, you will see the ESP32’s certificate being transmitted — CN = esp32-client.

If you click on packet 34, you will see the Certificate Verify the actual signature the ESP32 generated using esp32.key.

This is the challenge-response we talked about earlier, live on the wire.

What an Attacker Sees Now vs Before

In Part 5 we made a table of what is visible on the wire. Let’s update it for mTLS:

What	Visible to attacker?	Notes
Connection exists	Yes	TCP packets
IP addresses	Yes	IP headers are plaintext
Port 8883	Yes	Port numbers are plaintext
Broker’s certificate	Yes	Certificates are public by design
ESP32’s certificate	Yes	Also public CN = esp32-client
Which cipher was selected	Yes	Server Hello is plaintext
Session key	No	Never sent on wire ECDHE
MQTT message content	No	Encrypted with AES-256
ESP32 private key	No	Never leaves the device

The attacker can see that esp32-client connected to the broker. They can even grab the esp32.crt file right off the wire. But without esp32.key they cannot sign the challenge. The broker will reject them.

The Hardcoded Private Key Problem

Here is something important to think about.

We hardcoded esp32.key directly into the Arduino sketch. That is convenient for a lab. But think about what that means in the real world.

If an attacker physically gets hold of your ESP32 and reads the flash memory they have esp32.key. With that key they can impersonate your device to the broker. The entire mTLS protection is gone.

This is called a firmware extraction attack. It is one of the most common attacks on IoT devices. The fix involves secure key storage, hardware security modules, and secure provisioning which is exactly what Part 9 and Part 11 cover.

For now just know: hardcoding private keys in firmware is fine for a learning lab. In production, never do this.

Common Errors and What They Mean

Error: The connection was lost when connecting without a cert

This is correct behavior. The broker is enforcing require_certificate true. No cert = no entry.

MQTT connection failed, rc=-2

Broker not reachable. Check the IP in your sketch matches your current Kali IP. Run ifconfig on Kali to confirm.

SSL handshake failed

Two possible causes. Either rootCA.crt in the sketch does not match what the broker is using, or the broker IP changed and the SAN in server.crt no longer matches. Check both.

Mosquitto fails to start after config change

Most likely a permissions issue on server.key. Mosquitto runs as the mosquitto user. If you changed ownership earlier, fix it:

sudo chown mosquitto:mosquitto ~/my-ca/server.key
sudo chmod 640 ~/my-ca/server.key

certificate verify failed

The ESP32 cannot verify the broker’s certificate. Either rootCA.crt in the sketch is wrong, or the IP in the SAN does not match what the ESP32 is connecting to.

Quick Knowledge Check

Three questions. Answer these before moving to Part 7.

1. In regular TLS, only the client verifies the server. What does mTLS add?

The server also demands a certificate from the client. If the client cannot present one signed by the trusted CA, connection is refused.

2. The ESP32 sends esp32.crt to the broker. Why does it also need esp32.key?

Because anyone can copy a certificate file. The private key proves ownership the ESP32 uses it to sign the broker’s challenge. Only the real owner of the private key can produce a valid signature.

3. You can see esp32.crt in Wireshark. Is that a problem?

No. Certificates are public documents, designed to be shared. The sensitive part is esp32.key and that never leaves the ESP32.

What Just Happened - The Full Picture

Let me put it all together one more time.

Before this part, the broker had an open door. Any device that spoke TLS could connect. We proved this by just removing --cert and --key from mosquitto_sub the connection worked fine.

Now the broker checks identity. Only devices with a certificate signed by our Root CA can connect. We proved this by trying to connect without a cert and getting Error: The connection was lost.

We watched the new handshake steps in Wireshark Certificate Request from the broker, Certificate from the ESP32, Certificate Verify from the ESP32. Three new packets that were not there in Part 5.

And we identified the real-world weakness a private key hardcoded in firmware is a problem if someone can read the flash. That is the thread Part 9 and Part 11 will pick up.

What Is Coming in Part 7

Part 7 is where we actually attack the setup.

We are going to do a MITM attack Man In The Middle. We will put Kali between the ESP32 and the broker, intercept the traffic, and see what TLS protects against and what it does not.

This is the part where all the theory becomes real. You built the defence. Now you attack it.