Learn Rust Series (#58) - Build Scripts (build.rs) and Generating Code at Build Time

avatar

Learn Rust Series (#58) - Build Scripts (build.rs) and Generating Code at Build Time

rust-banner.png

What will I learn

  • You will learn what build.rs is: a Rust program Cargo compiles and runs before building your crate;
  • how a build script talks back to Cargo through a small line-based protocol printed on stdout;
  • how to emit custom cfg flags, inject environment variables, and generate Rust source into OUT_DIR;
  • how the crate pulls that generated code in with include!, and why rerun-if-changed is the difference between fast and maddeningly slow builds;
  • when a build script is genuinely the right tool, and when a plain const, a macro, or a cfg flag is the simpler answer.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu, with Cargo;
  • An installed Rust toolchain (via rustup, from rustup.rs);
  • The previous fifty-seven episodes, especially crates, cfg, and file I/O;
  • The ambition to learn systems programming from the ground up.

Difficulty

  • Intermediate

Curriculum (of the Learn Rust Series):

Learn Rust Series (#58) - Build Scripts (build.rs) and Generating Code at Build Time

Last episode we spent our time on decisions the compiler already knows how to make: #[cfg] gates on a platform, a feature the user switched on, a --release flag. All of that is static knowledge -- predicates fixed before anyone touches the keyboard. But every now and then a crate needs to do something that no amount of cfg can express, because the answer is not known in advance and has to be computed on the machine doing the build. Is a particular C library installed, and where? What is the current git commit hash? How do you turn a 40,000-line data file into a Rust lookup table without hand-typing 40,000 lines? Those questions cannot be answered by a predicate; they need real code to run at build time.

Cargo's answer is build.rs, an ordinary Rust program you drop at your crate root. Cargo notices it, compiles it as a standalone executable, and runs it before compiling your actual crate. The script does its work and then communicates back to Cargo by printing lines on stdout -- a tiny, line-oriented protocol where every meaningful line begins with cargo:. It is one of the more advanced corners of the ecosystem, and once you understand it, a lot of "magic" crates stop being magic: bindgen, prost, cc, vergen, all of them are just build scripts doing exactly what we are about to learn ;-)

Solutions to Episode 57 Exercises

Episode 57 was conditional compilation with cfg, and all three exercises were about compiling code in or out. Here they are in full.

Exercise 1 asked for two target_os-gated versions of line_ending() -- "\r\n" on Windows, "\n" otherwise -- with a not(...) fallback arm so it compiles on every platform, printed from main:

#[cfg(target_os = "windows")]
fn line_ending() -> &'static str { "\r\n" }

#[cfg(not(target_os = "windows"))]
fn line_ending() -> &'static str { "\n" }

fn main() {
    print!("first line{}second line{}", line_ending(), line_ending());
}

The pair of arms tiles the whole space of targets: Windows matches the first, everything else matches the second. From the compiler's point of view there is only ever one line_ending in the build, so there is no duplicate-definition error, and no target is left without a definition.

Exercise 2 wanted a pretty feature declared in a [features] table, gating a show(v: i32) helper, with a fallback that prints plainly:

// Cargo.toml:
//   [features]
//   pretty = []          # a bare switch, no extra dependencies

#[cfg(feature = "pretty")]
fn show(v: i32) {
    println!("+--------+\n| {v:^6} |\n+--------+"); // fancier output under `--features pretty`
}

#[cfg(not(feature = "pretty"))]
fn show(v: i32) {
    println!("value: {v}"); // plain baseline output
}

fn main() {
    show(42);
}

Because features are set by whoever depends on you, exactly one show survives: the fancy one under cargo build --features pretty, the plain one otherwise. The two definitions must be additive in spirit -- turning pretty on adds nicer output, it never removes the function or changes its signature.

Exercise 3 combined two ideas: cfg!(debug_assertions) inside an ordinary if, plus cfg_attr adding a Clone derive to a struct only under a serde feature, confirming the baseline still compiles:

#[cfg_attr(feature = "serde", derive(Clone))] // only derives Clone when `serde` is on
#[derive(Debug)]
struct Config {
    verbose: bool,
}

fn main() {
    let c = Config { verbose: true };
    if cfg!(debug_assertions) {
        println!("debug build, extra checks active: {c:?}");
    } else {
        println!("release build: {c:?}");
    }
}

The cfg!(debug_assertions) call folds to a compile-time true or false, but both branches are compiled and type-checked, which is exactly why both must be valid code. The cfg_attr line reads as "if serde is on, behave as if #[derive(Clone)] were written here; otherwise, as if this line were absent" -- so the baseline build never pays for a derive it did not ask for. Right, homework cleared. Now, build scripts.

What build.rs is

A build script is a file named build.rs sitting in your crate root, right next to Cargo.toml, with its own fn main. You do not have to register it anywhere -- Cargo detects the file by name, compiles it as a separate little program, and runs it before it compiles your crate proper. Its whole reason to exist is side effects and communication with Cargo, and that communication happens purely by printing lines that start with cargo::

// build.rs -- Cargo compiles and runs this BEFORE the crate itself
fn main() {
    // probe something about the build environment, then tell Cargo what we found
    let target_features = std::env::var("CARGO_CFG_TARGET_FEATURE").unwrap_or_default();
    let has_avx = target_features.split(',').any(|f| f == "avx");

    if has_avx {
        println!("cargo:rustc-cfg=has_avx"); // makes #[cfg(has_avx)] true in the crate
    }
    println!("cargo:rerun-if-changed=build.rs");
}

Notice there is nothing exotic in that file -- it is plain Rust, using std exactly the way any program does. What makes it special is only when Cargo runs it and that Cargo is listening to its stdout. Everything the script needs to know about the build arrives as environment variables that Cargo sets before launching it: CARGO_CFG_TARGET_FEATURE, CARGO_CFG_TARGET_OS, PROFILE ("debug" or "release"), OUT_DIR (a private scratch directory just for this build), CARGO_PKG_VERSION, HOST, TARGET, and quite some more. The script reads those, does its work, and speaks back through cargo: lines. That is the entire model, and the rest of this episode is just learning which cargo: lines exist and what each one does.

Emitting a custom cfg the crate can react to

The first and simplest instruction is cargo:rustc-cfg=NAME, which sets a custom cfg flag the compiler then honours across your whole crate. Once the script above has printed cargo:rustc-cfg=has_avx, the crate can gate code on has_avx exactly as if it were a built-in predicate -- and, as we drilled last episode, always with a fallback so it compiles either way:

#[cfg(has_avx)]
fn backend() -> &'static str { "AVX fast path" }

#[cfg(not(has_avx))]
fn backend() -> &'static str { "portable path" }

fn main() {
    // "portable path" unless build.rs detected AVX and set has_avx
    println!("using the {}", backend());
}

This is precisely how a crate can ship an optimised code path that only compiles on hardware that supports it, without the user configuring a single thing. The build script does the sniffing; the source code reacts to the result. Compared to the alternative -- forcing every user to pass --features avx by hand, and getting angry bug reports when they forget -- letting the machine decide at build time is a much nicer experience. Having said that, keep the number of build-script-emitted cfgs small; each one is a hidden switch that a reader of your crate cannot see just by looking at Cargo.toml.

Generating Rust source into OUT_DIR

Now we reach the reason build scripts really earn their keep: code generation. The pattern is that the script writes a .rs file into the private directory Cargo hands it via OUT_DIR, and the crate then pulls that file into its own source with the include! macro. Here is a script that generates a small lookup table of squares -- trivial as data, but the mechanism is identical whether you are emitting five entries or fifty thousand:

// build.rs
use std::env;
use std::fs;
use std::path::Path;

fn main() {
    let out_dir = env::var("OUT_DIR").expect("OUT_DIR is set by Cargo for build scripts");
    let dest = Path::new(&out_dir).join("table.rs");

    let mut code = String::from("pub const SQUARES: [u32; 5] = [");
    for i in 0..5u32 {
        code.push_str(&format!("{}, ", i * i));
    }
    code.push_str("];\n");

    fs::write(&dest, code).expect("failed to write generated table");
    println!("cargo:rerun-if-changed=build.rs");
}

The file it writes is nothing more than a chunk of ordinary Rust text. It has no idea it was generated -- to the compiler it is just source. Here is the kind of thing that lands in OUT_DIR, shown standalone so you can see there is no trickery to it:

// $OUT_DIR/table.rs -- written by build.rs, then include!d by the crate
pub const SQUARES: [u32; 5] = [0, 1, 4, 9, 16];

And the crate consumes it with include!, which splices the file's contents in at exactly the point of the macro, as if you had typed them there yourself. The idiom is include!(concat!(env!("OUT_DIR"), "/table.rs")), where env! reads the OUT_DIR variable at compile time (not at runtime, mind you) and concat! glues the path together:

// src/lib.rs in the real crate
include!(concat!(env!("OUT_DIR"), "/table.rs")); // brings SQUARES into scope here

pub fn square(i: usize) -> Option<u32> {
    SQUARES.get(i).copied()
}

fn main() {
    println!("{:?}", square(3)); // Some(9), straight out of the generated table
}

That is the whole loop: script writes source, crate includes it, SQUARES becomes a perfectly normal item you can call, document, and test. The payoff shows up when the table is enormous or derived from something else -- a Unicode data file, a list of country codes, a generated parser -- where hand-writing it would be both tedious and a place for typos to hide. Nota bene: always write into OUT_DIR and never back into your own src/, because OUT_DIR lives under target/ and gets cleaned up properly, whereas scribbling into src/ pollutes your repository and confuses version control.

Setting environment variables the crate reads at compile time

The next instruction, cargo:rustc-env=KEY=VALUE, injects an environment variable that your crate can read with the env! macro during its own compilation. This is the standard, blessed way to stamp a binary with build metadata -- the profile it was built in, a version string, a timestamp, a git hash:

// build.rs
fn main() {
    let profile = std::env::var("PROFILE").unwrap_or_default();
    println!("cargo:rustc-env=BUILD_PROFILE={profile}"); // e.g. "release"

    let version = std::env::var("CARGO_PKG_VERSION").unwrap_or_else(|_| "0.0.0".to_string());
    println!("cargo:rustc-env=FULL_VERSION=v{version}"); // e.g. "v0.1.0"
}

On the crate side, env!("BUILD_PROFILE") and env!("FULL_VERSION") are resolved at compile time into string literals baked straight into the binary, so there is zero runtime cost to reading them -- they are as cheap as any &'static str:

fn main() {
    // both were injected by build.rs via cargo:rustc-env, read at compile time
    println!("built in the {} profile", env!("BUILD_PROFILE"));
    println!("version {}", env!("FULL_VERSION"));
}

The distinction between env! and std::env::var matters here and trips people up. env!("X") is a macro that runs during compilation and fails the build if X is not set; it gives you a &'static str fixed at compile time. std::env::var("X") is a function that reads the process environment while the program runs and returns a Result. For build stamping you want the former, because you want the value frozen into the binary at build time, not re-read from whatever environment the user happens to run it in.

Linking native libraries

Build scripts are also how a Rust crate links against a native C library. The script prints cargo:rustc-link-lib= to name a library and cargo:rustc-link-search= to tell the linker where to look, and Cargo forwards both to the final link step:

// build.rs -- tell the linker to pull in a system library
fn main() {
    // link against the system zlib (libz); the linker resolves the actual .so/.dylib/.lib
    println!("cargo:rustc-link-lib=z");
    // add a directory to the linker's search path, if the library lives somewhere custom
    println!("cargo:rustc-link-search=native=/usr/local/lib");
    println!("cargo:rerun-if-changed=build.rs");
}

This is the foundation under crates like cc (which compiles a bundled .c file and links it in) and the whole family of *-sys crates that wrap system libraries. You will rarely hand-write these lines yourself -- you lean on cc or pkg-config to figure out the right flags -- but it is good to know that underneath the abstraction it is nothing more than a build script printing cargo: lines. The magic bottoms out in plain text on stdout, which I find rather reassuring ;-)

Controlling when the script re-runs

Here is the directive that separates a pleasant build from an infuriating one. By default, Cargo re-runs a build script whenever anything in the package changes, which means a script that takes a few seconds turns every trivial edit into a slow rebuild. The rerun-if-changed and rerun-if-env-changed directives narrow that down to the script's actual inputs, so a script that reads schema.json only re-runs when that file (or a relevant environment variable) changes:

// build.rs
fn main() {
    // re-run ONLY when these specific inputs change, not on every source edit
    println!("cargo:rerun-if-changed=schema.json");
    println!("cargo:rerun-if-changed=build.rs");
    println!("cargo:rerun-if-env-changed=TARGET");

    // ... read schema.json, generate code into OUT_DIR ...
}

There is a subtle rule to internalise here: the moment you emit even one rerun-if-changed line, you opt out of the default "re-run on any change" behaviour entirely, and Cargo will only re-run the script when one of the inputs you listed changes. That is a double-edged sword. Get the list right and your builds are fast and correct. Forget to list an input the script actually reads, and you get the nastier failure mode: stale generated code that silently ignores a changed file, so you edit schema.json, rebuild, and nothing happens -- because you never told Cargo that schema.json was an input. Getting these directives wrong is the classic cause of both "why is my whole project rebuilding for no reason?" and its evil twin, "why is my change not taking effect?". List every file and every env var the script reads, and no others.

A worked example: stamping a build

Let us tie the threads together into something you might genuinely ship: a build script that stamps the binary with its version and profile, and also generates a tiny helper the crate can call. First the script, which uses both rustc-env and OUT_DIR codegen in one go:

// build.rs
use std::env;
use std::fs;
use std::path::Path;

fn main() {
    // 1. stamp metadata as compile-time env vars
    let version = env::var("CARGO_PKG_VERSION").unwrap_or_else(|_| "0.0.0".into());
    let profile = env::var("PROFILE").unwrap_or_default();
    println!("cargo:rustc-env=APP_VERSION={version}");
    println!("cargo:rustc-env=APP_PROFILE={profile}");

    // 2. generate a small function into OUT_DIR
    let out_dir = env::var("OUT_DIR").expect("OUT_DIR set by Cargo");
    let dest = Path::new(&out_dir).join("banner.rs");
    let code = format!(
        "pub fn banner() -> &'static str {{ \"myapp {version} ({profile})\" }}\n"
    );
    fs::write(&dest, code).expect("write banner.rs");

    // 3. only re-run when the build script itself changes
    println!("cargo:rerun-if-changed=build.rs");
}

And the crate that consumes all three outputs -- the two env vars and the generated banner function -- reads like perfectly ordinary code, with no hint that half of it was manufactured moments before compilation:

// src/main.rs
include!(concat!(env!("OUT_DIR"), "/banner.rs")); // brings `banner()` into scope

fn main() {
    // env! reads the build-script-injected variables at compile time
    println!("version : {}", env!("APP_VERSION"));
    println!("profile : {}", env!("APP_PROFILE"));
    println!("{}", banner()); // the generated helper
}

The reader of main.rs sees a banner() call and two env! lookups and thinks nothing of them, which is exactly right -- a good build script is invisible from the outside. All the environment-probing and code-writing happened one phase earlier, and what the crate compiles against is just source and string literals.

When not to reach for build.rs

Build scripts are powerful, but they are not free. Every one of them adds a compile-and-run step to your clean build, complicates your crate for anyone reading it, and becomes a thing that can fail on someone else's machine in ways plain source never does. So before you write one, ask the honest question: would a plain const, a macro_rules! table (episodes back when we did macros), or a cfg flag do the same job? A handful of constants do not need code generation -- just type them. A compile-time computation over known values might be a const fn in stead of a script. A platform choice is a #[cfg], which we just spent a whole episode on.

Reserve build.rs for genuine build-time work that nothing simpler can express: probing the host system, linking or compiling native code, or generating source that is too large or too data-driven to hand-write and keep in sync. Used for the right job it is indispensable -- there is simply no other way to turn a 40,000-entry data file into a Rust table without it. Used for the wrong one, it is a maintenance tax you keep paying on every build. The best build script, like the best unsafe block, is the one you were careful enough not to need.

How Go, C and Python approach build-time codegen

A glance sideways sharpens the picture, as always. Go has go generate, a command that scans your source for //go:generate comment directives and runs whatever tool they name -- a code generator, a stringer, a protobuf compiler. The crucial difference from Rust is that go generate is a manual, separate step: you run it yourself, it writes .go files that you then commit to your repository, and a normal go build never re-runs it. Cargo's build.rs, by contrast, runs automatically as part of the build and writes into a scratch directory you do not commit. Two philosophies -- Go checks generated code in and regenerates on demand, Rust regenerates transparently and keeps it out of the repo:

//go:generate stringer -type=Color
// run `go generate ./...` by hand; the generated file is committed to the repo

C and C++ have no language-level notion of a build script at all; the moral equivalent lives entirely in the build system. A Makefile or a CMakeLists.txt runs a code generator, a configure script probes the system for available libraries, and the results feed into the compile. It works, but it is a separate language and a separate mental model bolted onto the side of your project -- exactly the kind of glue Cargo folds into one tool. Python, being interpreted, mostly does at runtime what Rust does at build time: need a lookup table generated from a data file? You just read the file and build the dict when the program starts. There is no compile step to hook, so "code generation" in Python usually means either runtime construction or a separate script you run to emit .py files by hand. Seen against those three, build.rs is the sweet spot again: automatic like nothing in the C world, transparent unlike go generate, and genuinely compile-time unlike Python -- one Rust program, run by one tool, feeding results straight back into the same build.

Wrapping up

So here is the shape of it. A build script is a build.rs at your crate root with its own fn main; Cargo compiles and runs it before your crate, feeding it information through environment variables (OUT_DIR, PROFILE, CARGO_CFG_*, CARGO_PKG_VERSION, and friends) and listening to the cargo: lines it prints back. With cargo:rustc-cfg it emits a custom cfg your source can gate on; with cargo:rustc-env it injects a variable your crate reads at compile time through env!; with cargo:rustc-link-lib and cargo:rustc-link-search it wires in native libraries. Its most powerful trick is generating Rust source into OUT_DIR and pulling it in with include!(concat!(env!("OUT_DIR"), "/file.rs")), which is how the big codegen crates do their work. And cargo:rerun-if-changed / rerun-if-env-changed are the directives that keep builds both fast and correct -- list every input the script reads, and no others. Above all, reach for a build script only when a const, a macro, or a cfg genuinely cannot do the job, because every script is a step you pay for on every clean build.

We now have a crate that can test itself, benchmark itself, split into a workspace, compile conditionally, and even generate parts of itself at build time. That is a lot of machinery. What we have not yet talked about is whether the code we write in between all that machinery actually looks the way seasoned Rust programmers expect it to look -- the naming, the idioms, the small stylistic choices that mark code as fluent rather than merely correct. Rust ships tooling that has opinions about exactly that, and turning those opinions loose on your own code is a surprisingly good way to learn the language's taste. Next time we let the tools grade our style ;-)

Exercises

  1. Write a build.rs that emits cargo:rustc-cfg=fast_path only when a chosen environment variable is set (read it with std::env::var), then consume fast_path in the crate with #[cfg(fast_path)] plus a #[cfg(not(fast_path))] fallback, printing which path is active from main. Remember to emit cargo:rerun-if-env-changed= for the variable you read.
  2. Write a build.rs that generates a pub const GREETINGS: [&str; 3] array into OUT_DIR by building the source as a String, then include it in the crate with include!(concat!(env!("OUT_DIR"), "/greetings.rs")) and print all three entries.
  3. Use cargo:rustc-env to expose a BUILD_ID string from build.rs (derive it from CARGO_PKG_VERSION and PROFILE), read it in the crate with env!("BUILD_ID"), and in a comment explain why env! is the right choice here rather than std::env::var.

Bedankt en tot de volgende keer! ;-)

@scipio

stem stemsocial steemstem rust programming
0
0
0.000
0 comments