Learn Rust Series (#24) - Blanket Implementations & the Newtype Pattern

avatar

Learn Rust Series (#24) - Blanket Implementations & the Newtype Pattern

rust-banner.png

What will I learn

  • You will learn what a blanket implementation is and how it implements a trait for a whole family of types at once;
  • how the standard library leans on blanket impls, for example giving ToString to everything that is Display, and Into to everything that has a From;
  • the newtype pattern in depth: type safety, upheld invariants, and the orphan-rule dodge from last episode seen as a design pattern in its own right;
  • how to build type-safe wrappers -- distinct id types that the compiler refuses to let you mix up;
  • why newtypes are genuinely zero-cost, and how Deref (episode 19) restores the inner type's methods when you want them back.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • An installed Rust toolchain (via rustup, from rustup.rs);
  • The previous twenty-three episodes, especially traits & generics (episode 8), From/Into (episode 21), the orphan rule (episode 23) and Deref (episode 19);
  • The ambition to learn systems programming from the ground up.

Difficulty

  • Beginner

Curriculum (of the Learn Rust Series):

Learn Rust Series (#24) - Blanket Implementations & the Newtype Pattern

At the very end of episode 23, once the orphan rule was behind us, I promised two things for today. First, that the little Wrapper struct we built to dodge the orphan rule was "far more than a workaround" -- a genuine design pattern that lets you attach meaning, invariants and new behaviour to a plain inner value. Second, that there was "a companion trick for writing one impl that covers a whole family of types at once". Those two promises are the whole episode: the newtype pattern and the blanket implementation. Together they are, I reason, the clearest demonstration in the language of how much leverage the trait system hands you for how little code.

The reason I like teaching them side by side is that they push in opposite directions. A blanket impl spreads outward: you write a single impl and a whole set of types light up with new behaviour at once. A newtype narrows inward: you wrap exactly one type to make it safer, stricter or trait-implementable. One is a fire hose, the other a scalpel. And both, delightfully, are zero-cost -- the compiler resolves them away and you pay nothing at runtime. Let's clear last episode's homework first, as always, and then get into it ;-)

Solutions to Episode 23 Exercises

Episode 23 was the orphan rule, coherence, and the newtype-plus-Deref escape hatch. Three exercises, and here is each one with full, runnable code.

Exercise 1 asked you to define a local trait Loud with a method shout(&self) -> String, and implement it for both i32 and &str so that 42.shout() returns "42!!!" and "hi".shout() returns "HI!!!":

trait Loud {
    fn shout(&self) -> String;
}

impl Loud for i32 {
    fn shout(&self) -> String {
        format!("{self}!!!")
    }
}

impl Loud for &str {
    fn shout(&self) -> String {
        format!("{}!!!", self.to_uppercase())
    }
}

fn main() {
    println!("{}", 42.shout());   // 42!!!
    println!("{}", "hi".shout()); // HI!!!
}

This is the extension trait case from last episode: Loud is your trait, so you may implement it for as many foreign types as you like, i32 and &str included. The i32 impl just formats the number with three exclamation marks; the &str impl upper-cases first and then adds them. Two foreign types, one local trait, zero orphan-rule trouble.

Exercise 2 wanted you to first try impl std::fmt::Display for Vec<String> directly (which fails with error E0117, the orphan impl), then fix it the idiomatic way with a newtype Lines that prints each string on its own line:

use std::fmt;

struct Lines(Vec<String>);

impl fmt::Display for Lines {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        for line in &self.0 {
            writeln!(f, "{line}")?;
        }
        Ok(())
    }
}

fn main() {
    let poem = Lines(vec![
        String::from("roses are red"),
        String::from("violets are blue"),
        String::from("the borrow checker"),
        String::from("is watching you"),
    ]);
    print!("{poem}");
}

Because Lines is a struct you own, implementing Display on it is perfectly legal -- this is the "own the type" case. Notice the writeln!(f, ...)? inside the loop: each writeln! returns a fmt::Result, and the ? bails out early if writing ever fails, which is exactly the error-propagation habit we picked up in episode 6. We finish by returning Ok(()).

Exercise 3 asked you to add a Deref impl to that Lines newtype, targeting Vec<String>, and then call .len() and .iter() on a Lines value directly, without ever touching .0:

use std::fmt;
use std::ops::Deref;

struct Lines(Vec<String>);

impl Deref for Lines {
    type Target = Vec<String>;
    fn deref(&self) -> &Vec<String> {
        &self.0
    }
}

impl fmt::Display for Lines {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        for line in self.iter() {
            writeln!(f, "{line}")?;
        }
        Ok(())
    }
}

fn main() {
    let lines = Lines(vec![
        String::from("one"),
        String::from("two"),
        String::from("three"),
    ]);
    println!("count: {}", lines.len()); // 3, reached through Deref
    print!("{lines}");
}

The Deref impl forwards a &Lines to a &Vec<String>, so lines.len() and self.iter() both resolve to Vec's own methods via deref coercion. I even used self.iter() inside the Display impl itself, in stead of the clunkier self.0.iter(). Right, homework cleared -- now the two patterns I promised you ;-)

Blanket implementations

A blanket impl is an implementation written on a generic type parameter carrying a trait bound. Its shape is impl<T: SomeBound> MyTrait for T, and read out loud it says "every type T that satisfies SomeBound automatically gets MyTrait". Here is the smallest possible one:

use std::fmt::Display;

trait Describe {
    fn describe(&self) -> String;
}

impl<T: Display> Describe for T {
    fn describe(&self) -> String {
        format!("the value is {self}")
    }
}

fn main() {
    println!("{}", 42.describe());      // the value is 42
    println!("{}", "hello".describe()); // the value is hello
    println!("{}", 3.5.describe());     // the value is 3.5
}

Look at what you got for one impl. Integers, string slices and floats all received a working describe method, without a single line written for any of them individually. The reason is right there in the bound: every one of those types implements Display, and the blanket impl says any Display type is a Describe type. You did not enumerate the types -- you described the property they must have, and the compiler applied your impl to everything that qualifies, now and forever, including types that do not exist yet.

This is not some exotic trick reserved for library authors; the standard library is built out of blanket impls, and you have been enjoying them since episode 2 without knowing their name. The clearest one is ToString. Ever wondered why every printable value has a .to_string() method? Because std contains a single blanket impl, morally impl<T: Display> ToString for T. Implement Display for your own struct and you get .to_string() thrown in for free -- one impl in the standard library reaching every Display type on Earth.

The blanket impl you already relied on

Here is the one that should make you sit up, because you met it in episode 21 and did not realise it was a blanket impl. Remember how I told you to always implement From, never Into, because implementing one hands you the other automatically? That "automatically" is a blanket impl living in the standard library. Roughly, it reads:

// This is (approximately) what the standard library contains.
// You never write this yourself -- it is already there, once, for all types.
//
// impl<T, U> Into<U> for T
// where
//     U: From<T>,
// {
//     fn into(self) -> U {
//         U::from(self)
//     }
// }

struct Celsius(f64);
struct Fahrenheit(f64);

impl From<Celsius> for Fahrenheit {
    fn from(c: Celsius) -> Fahrenheit {
        Fahrenheit(c.0 * 9.0 / 5.0 + 32.0)
    }
}

fn main() {
    let f: Fahrenheit = Celsius(100.0).into(); // .into() exists for FREE
    println!("{}", f.0);                        // 212
}

We wrote exactly one From<Celsius> for Fahrenheit impl, and .into() sprang into existence on Celsius with no further work. That is the blanket impl impl<T, U: From<T>> Into<U> for T doing its job: it covers every (T, U) pair for which a From exists. When I said in episode 21 "implement From and you get Into for free", the free part was a blanket impl all along. Now you know the machinery behind the magic.

Blanket impls and coherence

One thing that trips people up, and it ties straight back to last episode. A blanket impl impl<T: Bound> MyTrait for T is enormously greedy -- it potentially claims every type. So the compiler will usually not let you also write a second, more specific impl like impl MyTrait for i32 alongside it, because that specific type is already covered by the blanket, and two impls for one pair would violate the coherence rule we learned in episode 23 (at most one impl per trait-and-type pair, across the whole program). Rust's stable trait system has no "the more specific impl wins" override for the general case, so in practice a blanket impl is an all-or-nothing commitment: you cover the whole family by property, or you write per-type impls, but you do not casually mix the two. It is the same coherence principle from last episode, just viewed from the generic side of the fence.

The newtype pattern: type safety

Now the opposite move -- narrowing instead of spreading. A newtype is nothing more than a tuple struct with a single field wrapping another type. We built one last episode to escape the orphan rule, but its very first job, and the one you will use daily, is safety: giving two values the same underlying representation but distinct types, so the compiler physically stops you mixing them up.

struct UserId(u64);
struct ProductId(u64);

fn load_user(id: UserId) -> String {
    format!("loading user {}", id.0)
}

fn main() {
    let user = UserId(1);
    let product = ProductId(99);

    println!("{}", load_user(user));
    // load_user(product); // compile ERROR: expected UserId, found ProductId
    let _ = product;
}

Both UserId and ProductId are just a u64 at runtime, but to the type checker they are as different as String and bool. Passing a ProductId where a UserId is expected is a hard compile error, not a runtime surprise. Contrast this with the naive alternative, where both ids are plain u64: there, load_user(product_id) compiles happily and silently loads the wrong record, and you find out in production when a customer sees somebody else's data. In any codebase juggling a dozen kinds of identifier -- user ids, order ids, session tokens, all secretly u64 or String -- newtypes turn a whole class of mix-up into a category of bug the compiler simply refuses to build. That is an enormous, and enormously cheap, win.

The newtype pattern: upheld invariants

A newtype can do more than label a value; it can guarantee something about it. The trick is to make the inner field private and expose only a checked constructor, so that every single value of the type is valid by construction:

struct NonEmpty(String);

impl NonEmpty {
    fn new(s: String) -> Option<NonEmpty> {
        if s.is_empty() {
            None
        } else {
            Some(NonEmpty(s))
        }
    }
    fn get(&self) -> &str {
        &self.0
    }
}

fn main() {
    let name = NonEmpty::new(String::from("Rust")).unwrap();
    println!("{}", name.get()); // Rust
    println!("empty rejected? {}", NonEmpty::new(String::new()).is_none()); // true
}

The constructor new returns an Option: Some for a non-empty string, None for the empty one. Because that constructor is the only way to build a NonEmpty (the field is private, so nobody can write NonEmpty(whatever) from outside the module), anywhere a NonEmpty turns up in your program, you know -- without checking, without an assertion, without a defensive if -- that its string is not empty. The guarantee is baked into the type. This style has a slogan in the Rust world: "parse, don't validate". Instead of validating the same string over and over at every function that touches it, you validate once at the boundary, convert into a type that encodes the guarantee, and let the type system carry that proof around for you. Fewer checks, fewer bugs, and the compiler enforcing your invariants for you.

Here is a second flavour of the same idea, a bounded number:

struct Percentage(u8);

impl Percentage {
    fn new(value: u8) -> Option<Percentage> {
        if value <= 100 {
            Some(Percentage(value))
        } else {
            None
        }
    }
    fn value(&self) -> u8 {
        self.0
    }
}

fn main() {
    let half = Percentage::new(50).unwrap();
    println!("{}%", half.value());              // 50%
    println!("valid 150? {}", Percentage::new(150).is_some()); // false
}

A Percentage can never hold 150, because new rejects anything above 100. Every function downstream that accepts a Percentage gets that guarantee for free -- no revalidation, no doubt. You have encoded a business rule directly into the type system, and the compiler now enforces it on your behalf everywhere.

Restoring the inner methods with Deref

The one honest cost of a newtype is ergonomic: the wrapper hides the inner type's methods behind self.0. When the wrapper is genuinely a transparent stand-in for its inner value, you can bring those methods back with Deref, exactly as we did with Lines in the homework and with Wrapper last episode:

use std::ops::Deref;

struct Meters(f64);

impl Deref for Meters {
    type Target = f64;
    fn deref(&self) -> &f64 {
        &self.0
    }
}

fn main() {
    let distance = Meters(9.0);
    println!("{}", distance.sqrt()); // 3  -- f64::sqrt reached through Deref
    println!("{}", *distance + 1.0); // 10 -- explicit deref to the inner f64
}

Because Meters derefs to f64, calling distance.sqrt() coerces through to f64::sqrt, and *distance gives you the raw number to add to. But -- and this is the judgement call -- you should reach for Deref on a newtype deliberately, not reflexively. For a safety newtype like UserId, you almost certainly do not want Deref, because exposing all of u64's arithmetic (user_id * 2, user_id - other_id) would quietly undo the very safety you paid for. For a transparent unit wrapper like Meters, where you genuinely want the number's behaviour and the type is really just a label saying "these f64s are metres", Deref is convenient and honest. The question to ask yourself is always: is this wrapper here to restrict the inner type, or merely to label it? Restrict, and skip Deref, exposing only the few methods you bless. Label, and Deref is fair game.

Zero cost, and why that matters

Now the point that makes all of this practical rather than academic: a newtype has no runtime overhead whatsoever. A UserId(u64) occupies exactly as much memory as a bare u64, with no tag, no pointer, no indirection. You can prove it to yourself in three lines:

struct UserId(u64);

fn main() {
    println!("{} vs {}",
        std::mem::size_of::<UserId>(), // 8
        std::mem::size_of::<u64>());   // 8
}

Eight bytes versus eight bytes. The wrapper exists purely at compile time, to give the value a distinct identity in the type checker's eyes; once the program is compiled, the UserId and the u64 are indistinguishable in memory. So every guarantee we bought -- the mixed-id safety, the upheld invariants, the "parse, don't validate" proofs -- is genuinely free. That is the whole reason idiomatic Rust reaches for newtypes so readily: they cost nothing at runtime and convert whole classes of logic bug into compile errors. It is one of those rare deals where the safe option is also the fast option, and you would be a little daft not to take it.

How other languages handle this

Since a good chunk of you came through the Learn Python Series first (as did I, having taught Python for years), it is worth a quick look sideways. Both of today's patterns exist elsewhere, but weaker.

Python's typing module has a NewType, and it is a pale cousin of Rust's:

from typing import NewType

UserId = NewType("UserId", int)
ProductId = NewType("ProductId", int)

def load_user(uid: UserId) -> str:
    return f"loading user {uid}"

load_user(ProductId(99))  # a type CHECKER (mypy) flags this...
                          # ...but Python itself runs it without complaint

The crucial difference is when the check happens. Python's NewType is a hint for external tools like mypy; at runtime UserId is just int, and the interpreter will cheerfully run load_user(ProductId(99)) if you skip the type checker. Rust's newtype is enforced by the compiler -- the program with the mix-up does not build, full stop. And the invariant trick (NonEmpty, Percentage) has no real Python equivalent at all: Python cannot stop code from constructing an "invalid" object, only convention and runtime assertions can.

Blanket impls, meanwhile, are Rust's flavour of what Haskell calls typeclasses -- "any type with property X automatically gains behaviour Y". Go has nothing comparable: its interfaces are satisfied structurally, but you cannot say "every type implementing fmt.Stringer also gets method Z for free". So three points on a line again, the same shape we saw last episode: Python maximally flexible and unchecked, Go simple and restrictive, Rust powerful and checked at compile time. I know which trade-off I want when correctness actually matters ;-)

What did we actually learn?

  • A blanket implementation has the shape impl<T: Bound> MyTrait for T -- it grants a trait to every type satisfying the bound at once, by describing a property rather than listing types.
  • The standard library is full of them: ToString for every Display type, and the Into you got "for free" back in episode 21 was the blanket impl impl<T, U: From<T>> Into<U> for T.
  • Because a blanket impl claims a whole family, it collides with per-type impls under the coherence rule from episode 23 -- it is largely an all-or-nothing commitment.
  • A newtype is a single-field tuple struct that buys three things at zero runtime cost: type safety (distinct ids the compiler will not let you swap), upheld invariants (private field plus checked constructor = "parse, don't validate"), and a home for foreign trait impls (last episode's orphan-rule dodge).
  • Deref restores the inner type's methods when the wrapper is a transparent label, but you deliberately skip it when the wrapper's job is to restrict the inner type.

The thread through this whole stretch has been the trait system handing you leverage: one impl covering many types, one wrapper encoding a guarantee the compiler then enforces everywhere. Next we push further into the trait system's quieter corners -- into traits that carry no methods at all, that exist purely as promises the compiler reads about a type: can it be copied bit for bit, can it be sent between threads, does it even have a known size. Those marker traits are the load-bearing walls behind an awful lot of what you have been doing since episode 3, and it is high time we looked at them directly. One thing at a time ;-)

Exercises

Three exercises as always, gentle to chewier. Full solutions open the next episode, so genuinely have a go first -- typing it yourself is where it sticks.

  1. Write a blanket impl impl<T: std::fmt::Debug> PrettyPrint for T where PrettyPrint is your own trait with a method pretty(&self) -> String that returns the value formatted with {:?} inside square brackets, e.g. [42] or ["hi"]. Call .pretty() on an integer, a string slice and a Vec<i32> to confirm one impl covered all three.
  2. Build a Meters(f64) newtype and a separate Feet(f64) newtype, plus a function describe(m: Meters) -> String. Confirm that passing a Feet value to describe is a compile error, and that passing a Meters works. (This is the type-safety case: same inner f64, distinct types.)
  3. Write a Temperature newtype wrapping f64 whose constructor new returns Option<Temperature> and rejects anything below -273.15 (absolute zero). Add a Deref to f64 so you can call .round() on a Temperature directly, then reason in a comment about whether exposing all of f64 via Deref is a good idea for this particular type.

Bedankt en tot de volgende keer! ;-)

@scipio

stem stemsocial steemstem rust programming
0
0
0.000
0 comments