Learn Zig Series (#86) - TLS via C Interop

avatar

Learn Zig Series (#86) - TLS via C Interop

zig.png

What will I learn?

  • Why you should NEVER hand-roll your own TLS, and what to do instead;
  • How to link a battle-tested C crypto library (OpenSSL) into a Zig build;
  • How @cImport turns OpenSSL's headers into callable Zig functions;
  • How to wrap the raw SSL_CTX/SSL handles in a clean Zig struct with defer cleanup;
  • How to translate OpenSSL's error queue into honest Zig error values;
  • How to drive a real TLS 1.3 handshake and read/write encrypted bytes;
  • How to layer TLS underneath the HTTP/2 frame reader we built last episode;
  • How to test code that depends on a C library without needing a live server.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • An installed Zig 0.14+ distribution (download from ziglang.org);
  • OpenSSL 3.x development headers installed (libssl-dev on Ubuntu);
  • The ambition to learn Zig programming.

Difficulty

  • Intermediate

Curriculum (of the Learn Zig Series):

Learn Zig Series (#86) - TLS via C Interop

Solutions to Episode 85 Exercises

At the end of episode 85 I left you three exercises on the HTTP/2 frame layer -- a PING handler, RST_STREAM support, and a frame logger. They all reuse the FrameHeader, FrameType, Flags, Frame, FrameReader and Stream types we built last time, so keep that file open next to this one.

Exercise 1: A PING frame handler

const std = @import("std");
// Reuses FrameHeader, FrameType, Flags, Frame from episode 85.

pub const PING_PAYLOAD_LEN = 8;

/// Build the ACK reply for a received PING: echo the exact 8 opaque bytes back,
/// on stream 0, with the ACK flag set.
pub fn encodePingAck(opaque_data: []const u8, out: []u8) !usize {
    if (opaque_data.len != PING_PAYLOAD_LEN) return error.FrameSizeError;
    const header = FrameHeader{
        .length = PING_PAYLOAD_LEN,
        .frame_type = .ping,
        .flags = Flags.ACK,
        .stream_id = 0,
    };
    header.encode(out[0..9]);
    @memcpy(out[9..][0..PING_PAYLOAD_LEN], opaque_data);
    return 9 + PING_PAYLOAD_LEN;
}

/// Handle an incoming PING. If it is itself an ACK, it answers a ping WE sent --
/// ignore it. Otherwise reply. Returns bytes written to `out`, or 0 for "nothing".
pub fn handlePing(frame: Frame, out: []u8) !usize {
    if (frame.header.stream_id != 0) return error.ProtocolError;
    if (frame.payload.len != PING_PAYLOAD_LEN) return error.FrameSizeError;
    if (Flags.has(frame.header.flags, Flags.ACK)) return 0;
    return try encodePingAck(frame.payload, out);
}

test "ping is echoed with ACK set" {
    const incoming = Frame{
        .header = .{ .length = 8, .frame_type = .ping, .flags = 0, .stream_id = 0 },
        .payload = &[_]u8{ 1, 2, 3, 4, 5, 6, 7, 8 },
    };
    var out: [17]u8 = undefined;
    const n = try handlePing(incoming, &out);
    const reply = try FrameHeader.parse(out[0..n]);
    try std.testing.expect(Flags.has(reply.flags, Flags.ACK));
    try std.testing.expectEqualSlices(u8, incoming.payload, out[9..n]);
}

The whole trick is recognising that a PING with the ACK bit already set is a response, not a request -- echoing those would create an infinite ping-pong between two endpoints. The 8-byte payload is opaque, meaning we copy it verbatim and never interpret it; the sender uses it to match replies to requests (handy for round-trip latency measurement).

Exercise 2: RST_STREAM in the state machine

/// An RST_STREAM frame carries a 4-byte error code and tears the stream down
/// from ANY state. These methods slot in next to onRecv from episode 85.
pub fn onReset(self: *Stream) void {
    self.state = .closed;
}

/// A frame arriving on a stream we already reset must be rejected up front.
pub fn onRecvChecked(self: *Stream, frame_type: FrameType, end_stream: bool) !void {
    if (self.state == .closed) return error.StreamClosed;
    try self.onRecv(frame_type, end_stream);
}

pub fn parseRstError(payload: []const u8) !u32 {
    if (payload.len != 4) return error.FrameSizeError;
    return std.mem.readInt(u32, payload[0..4], .big);
}

test "open stream can be reset to closed" {
    var stream = Stream{ .id = 1, .state = .open };
    stream.onReset();
    try std.testing.expectEqual(StreamState.closed, stream.state);
    try std.testing.expectError(error.StreamClosed, stream.onRecvChecked(.data, false));
}

onReset is brutally simple on purpose -- RST_STREAM is the "I am done with this stream, right now, no matter where we were" signal. The interesting part is onRecvChecked: once a stream is closed, any further frame on it (except a couple the spec tolerates in a small race window) is a StreamClosed error. Making that the default keeps a buggy or hostile peer from reviving a dead stream.

Exercise 3: A frame logger

const std = @import("std");

fn frameTypeName(t: FrameType) []const u8 {
    return switch (t) {
        .data => "DATA",
        .headers => "HEADERS",
        .priority => "PRIORITY",
        .rst_stream => "RST_STREAM",
        .settings => "SETTINGS",
        .push_promise => "PUSH_PROMISE",
        .ping => "PING",
        .goaway => "GOAWAY",
        .window_update => "WINDOW_UPDATE",
        .continuation => "CONTINUATION",
        _ => "UNKNOWN", // forward-compatible: never crash on a new type
    };
}

/// Read a captured HTTP/2 byte stream from a file and print one line per frame.
/// Generate input with: curl --http2-prior-knowledge -v http://localhost:8080/
pub fn logFrames(path: []const u8, writer: anytype) !void {
    var file = try std.fs.cwd().openFile(path, .{});
    defer file.close();

    var reader = FrameReader{};
    var chunk: [4096]u8 = undefined;
    while (true) {
        const n = try file.read(&chunk);
        if (n == 0) break;
        try reader.feed(chunk[0..n]);
        while (try reader.next()) |frame| {
            try writer.print("{s} stream={d} len={d} flags=0x{x:0>2}\n", .{
                frameTypeName(frame.header.frame_type),
                frame.header.stream_id,
                frame.header.length,
                frame.header.flags,
            });
        }
    }
}

The key detail is that frameTypeName matches the _ arm of our non-exhaustive enum, so an unknown frame type prints UNKNOWN instead of panicking. That mirrors exactly what a conformant endpoint must do: ignore frame types it does not understand. Note that I did NOT use @tagName here, because @tagName on an unnamed value of a non-exhaustive enum is illegal behaviour -- a manual switch is the safe way.

At the very end of episode 85 I wrote that HPACK and "securing all of this with TLS (you rarely see plaintext HTTP/2 in the wild)" were the natural next steps. Well -- here we are again ;-) Today we tackle the TLS half, and we are going to do it the way grown-ups do it in production: by wrapping a C library through Zig's C interop, which we first explored back in episodes 27 and 28.

The first rule of TLS: don't write your own

Let me be blunt, because this matters. You do not implement TLS yourself. Not the handshake, not the record layer, not the certificate validation, none of it. TLS is a sprawling protocol with twenty-five years of accumulated attack history -- Heartbleed, BEAST, POODLE, the lot -- and every one of those bugs lived in code written by people far more careful than a tutorial author rushing to make a deadline. The cryptographic primitives are unforgiving: a timing leak in your AES-GCM tag comparison hands an attacker your session, and you will never see it in a unit test.

So the correct move is to lean on a library that thousands of paranoid engineers have already hardened: OpenSSL, BoringSSL, or LibreSSL. They are written in C, which means this episode is really about something more general than TLS -- it's about how Zig consumes a serious, real-world C library cleanly. TLS just happens to be the most worthwhile example I can think of.

Having said that, Zig's standard library does ship an experimental pure-Zig TLS client (std.crypto.tls), and it's genuinely impressive. But it's a client only, it targets a subset of cipher suites, and it moves with the compiler. For a server, for full protocol coverage, or for "I need this to interop with every weird CDN on the planet", wrapping a mature C library is still the pragmatic choice. That is what we'll build.

Linking OpenSSL into the build

Everything starts in build.zig (episode 15). To call OpenSSL we need libc and the two OpenSSL shared libraries -- ssl (the protocol) and crypto (the primitives):

// build.zig
const exe = b.addExecutable(.{
    .name = "tls-client",
    .root_source_file = b.path("src/main.zig"),
    .target = target,
    .optimize = optimize,
});
exe.linkLibC();             // OpenSSL needs the C runtime
exe.linkSystemLibrary("ssl");
exe.linkSystemLibrary("crypto");
b.installArtifact(exe);

linkLibC() is mandatory the moment you touch a C library -- it pulls in malloc, the symbol that OpenSSL allocates with, plus the dynamic loader glue. linkSystemLibrary asks the system linker to find libssl.so and libcrypto.so via pkg-config or the standard search paths. On Ubuntu you need libssl-dev installed so the headers and the .so symlinks exist.

Now the import side. In episode 27 we met @cImport, which runs Zig's built-in C translator over a set of headers and hands you back a normal Zig namespace:

const std = @import("std");
const c = @cImport({
    @cInclude("openssl/ssl.h");
    @cInclude("openssl/err.h");
});

After this, every OpenSSL function is reachable as c.SSL_new, every constant as c.TLS1_2_VERSION, every opaque type as c.SSL_CTX. The translation happens at compile time, it's cached, and it understands the macros and typedefs in those headers. This is the part that makes Zig such a comfortable C citizen -- no hand-written bindings, no extern fn declarations to keep in sync. Bam, jonguh!

Wrapping the raw handles

OpenSSL's object model has two layers worth knowing. An SSL_CTX is a factory holding configuration shared across many connections -- the protocol version range, the certificate trust store, session caches. An SSL is a single live connection, created from a context. The C idiom is: configure the context once, spin up an SSL per socket.

Raw, that's a pile of c.SSL_CTX_new / c.SSL_new / c.SSL_free calls with manual cleanup that's painfully easy to get wrong. Let's wrap it in a Zig struct so defer handles teardown and so the messy pointer-or-null returns become Zig errors:

pub const TlsError = error{
    ContextInit,
    HandshakeFailed,
    WantRead,
    WantWrite,
    ConnectionClosed,
    SyscallError,
    SslError,
};

pub const TlsClient = struct {
    ctx: *c.SSL_CTX,
    ssl: *c.SSL,
    socket: std.posix.socket_t,

    pub fn init(host: [:0]const u8, socket: std.posix.socket_t) TlsError!TlsClient {
        const method = c.TLS_client_method();
        const ctx = c.SSL_CTX_new(method) orelse return error.ContextInit;
        errdefer c.SSL_CTX_free(ctx);

        // Refuse anything older than TLS 1.2 -- the old versions are broken.
        _ = c.SSL_CTX_set_min_proto_version(ctx, c.TLS1_2_VERSION);
        // Verify the server's certificate chain against the system trust store.
        c.SSL_CTX_set_verify(ctx, c.SSL_VERIFY_PEER, null);
        if (c.SSL_CTX_set_default_verify_paths(ctx) != 1) return error.ContextInit;

        const ssl = c.SSL_new(ctx) orelse return error.ContextInit;
        errdefer c.SSL_free(ssl);

        // Bind this SSL object to our already-connected TCP socket fd.
        if (c.SSL_set_fd(ssl, @intCast(socket)) != 1) return error.ContextInit;
        // SNI: tell the server which virtual host we want. Without it, most CDNs
        // hand back the wrong certificate and the verify step fails.
        _ = c.SSL_set_tlsext_host_name(ssl, host.ptr);
        // Turn on hostname checking inside the X.509 verification itself.
        _ = c.SSL_set1_host(ssl, host.ptr);

        return .{ .ctx = ctx, .ssl = ssl, .socket = socket };
    }

    pub fn deinit(self: *TlsClient) void {
        _ = c.SSL_shutdown(self.ssl); // best-effort close_notify
        c.SSL_free(self.ssl);
        c.SSL_CTX_free(self.ctx);
    }
};

Two things here are pure Zig payoff. First, orelse return error.ContextInit converts OpenSSL's "returns NULL on failure" convention into a real error in one line -- no if (ptr == NULL) boilerplate. Second, errdefer gives us transactional cleanup: if SSL_new succeeds but SSL_set_fd fails, the errdefer c.SSL_CTX_free(ctx) still runs and frees the context, while the success path skips it. Getting that right by hand in C is exactly where leaks breed.

Nota bene: host is typed [:0]const u8 -- a sentinel-terminated slice from episode 16. OpenSSL wants a NUL-terminated char* for the SNI name, and that type guarantees the terminator is there without an allocation or a copy.

Turning OpenSSL's error queue into Zig errors

C libraries signal failure with magic return values, and OpenSSL is especially baroque about it: most calls return 1 for success and <= 0 for trouble, but to learn what went wrong you call SSL_get_error with the original return value. The answer tells you whether the connection wants more bytes, wants to write, closed cleanly, or genuinely failed. Let me factor that mapping into a pure function -- which, as you'll see in the testing section, is the bit that makes this whole thing unit-testable:

fn mapSslError(code: c_int) TlsError {
    return switch (code) {
        c.SSL_ERROR_WANT_READ => error.WantRead,
        c.SSL_ERROR_WANT_WRITE => error.WantWrite,
        c.SSL_ERROR_ZERO_RETURN => error.ConnectionClosed, // clean TLS close
        c.SSL_ERROR_SYSCALL => error.SyscallError,
        else => error.SslError,
    };
}

fn lastError(self: *TlsClient, ret: c_int) TlsError {
    return mapSslError(c.SSL_get_error(self.ssl, ret));
}

pub fn handshake(self: *TlsClient) TlsError!void {
    const ret = c.SSL_connect(self.ssl);
    if (ret != 1) return self.lastError(ret);
}

SSL_connect drives the whole handshake -- it sends the ClientHello, processes the server's certificate, validates the chain against the trust store we configured, runs the key exchange, and derives session keys. When it returns 1, you have an encrypted channel. When it doesn't, mapSslError tells you whether you simply need to wait for more socket data (WantRead, normal on a non-blocking socket) or whether the certificate failed to verify (SslError, which you must NOT ignore -- that's the entire point of TLS).

The WantRead / WantWrite distinction is the thing people trip over. On a non-blocking socket, OpenSSL doesn't block waiting for bytes; it returns "I want to read more" and expects you to wait on the fd (via poll/epoll from earlier episodes) and call again. Surfacing those as distinct Zig errors means the caller's event loop can switch on them instead of guessing.

Reading and writing encrypted bytes

Once the handshake is done, SSL_write and SSL_read behave almost like the plain socket write/read we've used since episode 21 -- except every byte is transparently encrypted and decrypted. The wrappers mostly translate the <= 0 sentinel into our error set:

pub fn write(self: *TlsClient, data: []const u8) TlsError!usize {
    const ret = c.SSL_write(self.ssl, data.ptr, @intCast(data.len));
    if (ret <= 0) return self.lastError(ret);
    return @intCast(ret);
}

pub fn read(self: *TlsClient, buf: []u8) TlsError!usize {
    const ret = c.SSL_read(self.ssl, buf.ptr, @intCast(buf.len));
    if (ret <= 0) {
        const err = self.lastError(ret);
        // A clean TLS shutdown is reported as 0 bytes, like EOF on a socket.
        if (err == error.ConnectionClosed) return 0;
        return err;
    }
    return @intCast(ret);
}

Notice the asymmetry in read: a clean close (SSL_ERROR_ZERO_RETURN) becomes a return of 0, matching the convention that 0 bytes means end-of-stream -- the same convention our HTTP and frame readers already rely on. Mapping it that way means existing read loops just work without special-casing TLS.

Here's the whole thing fetching a page over HTTPS, TCP connect (episode 21) first, TLS layered on top:

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const stream = try std.net.tcpConnectToHost(allocator, "example.com", 443);
    defer stream.close();

    var tls = try TlsClient.init("example.com", stream.handle);
    defer tls.deinit();
    try tls.handshake();

    const request =
        "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n";
    _ = try tls.write(request);

    var buf: [4096]u8 = undefined;
    const out = std.io.getStdOut();
    while (true) {
        const n = try tls.read(&buf);
        if (n == 0) break;
        try out.writeAll(buf[0..n]);
    }
}

That defer tls.deinit() paired with defer stream.close() is the layered-resource pattern done right: TLS shuts down first (sending its close_notify), then the TCP socket closes underneath it, in the correct reverse order Zig guarantees for defer.

TLS is just a transport: feeding the HTTP/2 reader

Here is where last episode and this one click together. TLS sits below the application protocol -- above it, nothing changes. So the FrameReader and Connection from episode 85 don't care whether their bytes came from a plain socket or a decrypted TLS stream. We just swap the byte source:

/// Pump TLS-decrypted bytes straight into the HTTP/2 frame reader from ep 85.
pub fn pumpFrames(tls: *TlsClient, reader: *FrameReader, conn: *Connection) !void {
    var buf: [16384]u8 = undefined;
    while (true) {
        const n = try tls.read(&buf);
        if (n == 0) break;
        try reader.feed(buf[0..n]);
        while (try reader.next()) |frame| {
            try conn.handleFrame(frame);
        }
    }
}

This is the real-world shape of an HTTP/2 client: TCP at the bottom, TLS in the middle (HTTP/2 in browsers is negotiated via the ALPN extension during the handshake, almost always over TLS), and our frame state machine on top. Each layer is independently testable and independently swappable. That separation is not an accident of our design -- it's the layering the protocols themselves were built around.

Testing code that depends on a C library

Testing TLS honestly is awkward, because a true end-to-end test needs a live peer with a valid certificate -- not something you want firing on every zig build test. So you split the problem. The parts that are pure logic get unit tests; the parts that need a network get a separate, opt-in integration test.

That's precisely why I pulled mapSslError out into a standalone function earlier. It has zero dependency on a live SSL object, so we can hammer it directly:

test "ssl error codes map to the right zig errors" {
    try std.testing.expectEqual(error.WantRead, mapSslError(c.SSL_ERROR_WANT_READ));
    try std.testing.expectEqual(error.WantWrite, mapSslError(c.SSL_ERROR_WANT_WRITE));
    try std.testing.expectEqual(error.ConnectionClosed, mapSslError(c.SSL_ERROR_ZERO_RETURN));
    try std.testing.expectEqual(error.SyscallError, mapSslError(c.SSL_ERROR_SYSCALL));
    try std.testing.expectEqual(error.SslError, mapSslError(c.SSL_ERROR_SSL));
}

For the live path, gate it behind an environment variable so CI can choose to run it: open a real connection to a known-good host, complete the handshake, and assert it succeeded. The lesson generalises far beyond TLS -- whenever you wrap a C library, push as much of your own logic as possible into pure functions, and keep the thin "actually call into C" layer as small and as dumb as you can. The pure part is where your bugs will be, and the pure part is what tests cheaply.

One more testing tip: run your TLS code under zig build test with the address sanitizer or under Valgrind occasionally. Because OpenSSL allocates with C's malloc, Zig's GeneralPurposeAllocator leak detector won't see those allocations -- a forgotten SSL_free is invisible to it. The defer/errdefer discipline above is your real defence, but a periodic Valgrind run catches the case you missed.

Performance considerations

Two costs dominate. The first is the handshake: a full TLS 1.3 handshake involves asymmetric crypto (an ECDHE key exchange plus a signature verification) and one network round-trip before any data flows. That's milliseconds, which is enormous next to the microseconds of the symmetric crypto that follows. The fix is to not do it twice: enable session resumption so a returning client skips the expensive part. With TLS 1.3 you ask for a session cache on the context and OpenSSL handles the tickets:

// On the SSL_CTX, before creating connections:
_ = c.SSL_CTX_set_session_cache_mode(ctx, c.SSL_SESS_CACHE_CLIENT);

The second cost is the record layer doing AES-GCM or ChaCha20-Poly1305 on every byte. On any modern x86 or ARM chip this is hardware-accelerated (AES-NI), so it's nearly free -- OpenSSL picks the accelerated path automatically. The thing you control is buffer sizing: read in chunks of 16 KB or so (a TLS record maxes at ~16 KB), because calling SSL_read for tiny amounts wastes a function-call-and-decrypt cycle per call. The 16 KB buffer in pumpFrames above is sized exactly for that reason. Nota bene: don't disable certificate verification "for speed" -- it saves you almost nothing and throws away the entire security guarantee. I've seen that done in production and it makes me want to cry quit some.

How this compares to C, Rust, and Go

In C, you'd write essentially what we wrapped -- raw OpenSSL -- but without errdefer, without the sentinel-slice safety, and with every NULL check by hand. The OpenSSL API is the same; the difference is purely how much rope you're handed to hang yourself with. Decades of CVEs in C TLS glue code (not in OpenSSL itself, but in applications using it) came from exactly the cleanup and error-handling mistakes Zig's defer makes hard.

In Go, crypto/tls is a pure-Go implementation in the standard library, and it is genuinely lovely -- tls.Dial("tcp", "example.com:443", cfg) and you're done, no C, no linking, cross-compiles trivially. The price is that you're locked into Go's runtime and its cipher choices, and when you need a feature the Go team hasn't prioritised, you wait.

In Rust, the modern answer is rustls, a pure-Rust TLS stack with no OpenSSL dependency, memory-safe by construction, and increasingly the default in the ecosystem. It's arguably the best-engineered option of the lot. Rust also can wrap OpenSSL (the openssl crate) when it must interop with the C world, much as we did here.

Where does Zig land? Right in its honest middle: the pure-Zig std.crypto.tls is coming along but isn't yet a complete OpenSSL replacement, so for serious work you wrap the C library -- and Zig makes wrapping C less painful than any language I've used. You get exact control over buffers and allocation, no runtime, trivial cross-compilation of your own code, and @cImport doing the binding work for free. Having said that -- for a quick HTTPS client where you don't care about the bytes, Go's batteries-included approach will get you there in three lines, and that's fine too ;-)

Where this is heading

We can now wrap any TCP socket in a verified, encrypted TLS channel, and we've seen that the layers above it -- HTTP/1.1, HTTP/2 frames -- don't have to change one line to run secured. That's a big deal, because it unlocks the protocols that real-time, interactive web apps are built on: the ones where the server pushes data to the client whenever it likes, over a long-lived connection, secured the same wss:// way your browser does it. We've now got every piece that sits underneath such a protocol -- sockets, framing, state machines, and encryption -- so the next step is to build the upgrade dance that turns an ordinary HTTPS request into a persistent, bidirectional channel. The C interop and binary-parsing muscles you trained over the last few episodes are exactly the ones we'll flex.

The frame reader, the state machine, and now the TLS wrapper -- they're not separate tutorials, they're the layers of one real networking stack you've been assembling brick by brick.

Exercises

  1. Add ALPN negotiation. Before the handshake, call SSL_set_alpn_protos to advertise h2 and http/1.1, and after a successful handshake read back the chosen protocol with SSL_get0_alpn_selected. Wrap both in your TlsClient and write a small program that connects to a real HTTPS host and prints which protocol the server picked. (This is how a client decides between HTTP/2 and HTTP/1.1 in practice.)

  2. Make the client non-blocking. Put the socket in non-blocking mode (recall O_NONBLOCK from the I/O episodes), then handle error.WantRead and error.WantWrite from handshake and read/write by waiting on the fd with poll before retrying. Prove it works by driving two TLS connections from a single thread without either one blocking the other.

  3. Build a tiny HTTPS server. Use TLS_server_method instead of the client method, load a self-signed certificate and key with SSL_CTX_use_certificate_file and SSL_CTX_use_PrivateKey_file, accept a TCP connection (episode 51), wrap it with SSL_accept instead of SSL_connect, and serve a one-line response. Generate the test cert with openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1.

Bedankt en tot de volgende keer!

@scipio

stem stemsocial steemstem zig programming
0
0
0.000
1 comments
avatar

Congratulations @scipio! You have completed the following achievement on the Hive blockchain And have been rewarded with New badge(s)

You received more than 40000 upvotes.
Your next target is to reach 45000 upvotes.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

0
0
0.000
Reply