Learn Rust Series (#19) - Deref, DerefMut & Deref Coercion

avatar

Learn Rust Series (#19) - Deref, DerefMut & Deref Coercion

rust-banner.png

What will I learn

  • You will learn how the Deref trait makes a type behave like a reference to another type;
  • how deref coercion silently converts &String to &str, &Vec<T> to &[T], and &Box<T> to &T;
  • how method resolution follows Deref so wrapped types inherit the inner type's methods for free;
  • how DerefMut does the same for mutable access, and why it needs Deref as a supertrait;
  • why you should implement Deref only for genuine smart pointers, not for general convenience.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • An installed Rust toolchain (via rustup, from rustup.rs);
  • The previous eighteen episodes, especially smart pointers from episode 12 and traits from episode 8;
  • The ambition to learn systems programming from the ground up.

Difficulty

  • Beginner

Curriculum (of the Learn Rust Series):

Learn Rust Series (#19) - Deref, DerefMut & Deref Coercion

At the end of last episode I promised we would look at how a type can quietly stand in for the thing it wraps, so that a smart pointer feels like the value inside it. That is today. And here is the slightly unsettling part: you have been leaning on this feature in every single episode without noticing it was there. It is why you can hand a &String to a function that only asks for &str, why calling a Vec method works straight through a Box, and why Rc and Arc back in episode 12 and 13 felt so transparent. The whole trick is one small trait, Deref, plus a compiler convenience called deref coercion -- and once you see it, a pile of "eh, it just works" moments collapses into a single clear rule ;-)

Having said that, before we open the new topic we clear last episode's homework, as always.

Solutions to Episode 18 Exercises

Episode 18 was operator overloading with std::ops: how a + b is really a.add(b), how the result type is an associated Output, and how Mul<Rhs> carries a generic parameter so one type can multiply against several right-hand types. There were three exercises, and here is each one with full code you can paste and run.

Exercise 1 asked you to implement Neg for Vec2 so that -v flips the sign of both components. Neg is a unary operator, so its neg method takes only self and returns the negated value:

use std::ops::Neg;

#[derive(Debug, Clone, Copy)]
struct Vec2 { x: f64, y: f64 }

impl Neg for Vec2 {
    type Output = Vec2;
    fn neg(self) -> Vec2 { Vec2 { x: -self.x, y: -self.y } }
}

fn main() {
    println!("{:?}", -Vec2 { x: 1.0, y: -2.0 }); // Vec2 { x: -1.0, y: 2.0 }
}

The key insight is that a unary operator is the same idea as a binary one, minus a hand: there is no other parameter, just self going in and a fresh Vec2 coming out. -v desugars to v.neg() exactly the way a + b desugared to a.add(b).

Exercise 2 wanted a second Mul impl on Vec2, this time Mul<Vec2> returning an f64 dot product, coexisting with the scaling Mul<f64> from the episode body. This is the whole point of Mul carrying a generic right-hand parameter:

use std::ops::Mul;

#[derive(Debug, Clone, Copy)]
struct Vec2 { x: f64, y: f64 }

impl Mul<f64> for Vec2 {
    type Output = Vec2;
    fn mul(self, s: f64) -> Vec2 { Vec2 { x: self.x * s, y: self.y * s } }
}
impl Mul<Vec2> for Vec2 {
    type Output = f64;
    fn mul(self, o: Vec2) -> f64 { self.x * o.x + self.y * o.y }
}

fn main() {
    let a = Vec2 { x: 1.0, y: 2.0 };
    let b = Vec2 { x: 3.0, y: 4.0 };
    println!("{:?}", a * 3.0); // Vec2 { x: 3.0, y: 6.0 } (scaling)
    println!("{}", a * b);     // 11 (dot product)
}

Notice that Output differs between the two impls: scaling yields a Vec2, the dot product yields an f64. The compiler picks the right impl purely from the right-hand operand type, and both live happily on the same struct. That is the generic-parameter half of episode 18 doing real work.

Exercise 3 asked for an IndexMut<(usize, usize)> impl on Grid so you could assign into a cell with g[(0, 0)] = 99. IndexMut builds on Index, and its index_mut returns &mut Self::Output, which is what makes the left-hand-side assignment legal:

use std::ops::{Index, IndexMut};

struct Grid { cells: Vec<i32>, width: usize }

impl Index<(usize, usize)> for Grid {
    type Output = i32;
    fn index(&self, (r, c): (usize, usize)) -> &i32 { &self.cells[r * self.width + c] }
}
impl IndexMut<(usize, usize)> for Grid {
    fn index_mut(&mut self, (r, c): (usize, usize)) -> &mut i32 { &mut self.cells[r * self.width + c] }
}

fn main() {
    let mut g = Grid { cells: vec![0; 4], width: 2 };
    g[(0, 1)] = 99;
    println!("{}", g[(0, 1)]); // 99
}

The one-sentence why: g[(0, 1)] = 99 is really *g.index_mut((0, 1)) = 99, and because index_mut handed back a &mut i32 pointing straight into the vector, the assignment writes through to the real cell. The grid has to be mut, of course. Right -- homework cleared, on to Deref ;-)

The Deref trait

Deref lets you customise what the dereference operator * does for your own type. A type that implements it can be treated as if it were a reference to some inner Target type. Let us build the classic teaching example, a hand-rolled box, so nothing is hidden:

use std::ops::Deref;

struct MyBox<T>(T);

impl<T> MyBox<T> {
    fn new(x: T) -> MyBox<T> { MyBox(x) }
}

impl<T> Deref for MyBox<T> {
    type Target = T;
    fn deref(&self) -> &T { &self.0 }
}

fn main() {
    let b = MyBox::new(5);
    assert_eq!(5, *b);        // works because of our Deref impl
    println!("{}", *b);       // 5
}

The interesting line is *b. Our MyBox is just a tuple struct wrapping a T; on its own, * would have no meaning for it. But because we implemented Deref, the compiler rewrites *b into *(b.deref()) behind your back. The deref method hands back a &T, and the built-in * on that plain reference reads the value out. Note the shape of deref: it takes &self and returns a reference &Target, never the value by itself. If it returned T by value it would have to move the inner data out on every dereference, which would be both wrong and impossible for most types. This tiny trait, type Target plus a deref that lends out a reference, is the foundation everything else in this episode is built on.

Deref coercion: the quiet convenience

Here is where it stops being a curiosity and starts being the thing you actually feel every day. When you pass a reference to a function, and the reference type does not match the expected parameter type, but it implements Deref to a type that eventually does, the compiler quietly inserts as many deref calls as it takes to line the types up. This is deref coercion:

use std::ops::Deref;

struct MyBox<T>(T);
impl<T> Deref for MyBox<T> {
    type Target = T;
    fn deref(&self) -> &T { &self.0 }
}

fn hello(name: &str) {
    println!("hello, {name}");
}

fn main() {
    let m = MyBox(String::from("Rust"));
    hello(&m); // &MyBox<String> -> &String -> &str, all inserted for you
}

Look at hello(&m). We have a &MyBox<String> in hand, but hello wants a &str. The compiler chains two coercions to bridge that gap: first &MyBox<String> to &String via our own Deref, then &String to &str via the standard library's Deref for String. It keeps applying deref until it either matches or runs out of options. Without this feature you would have to write something ugly like hello(&(*m)[..]) at every call site, and every wrapper type you ever made would leak its plumbing into every function that touched it. Deref coercion is precisely what makes smart pointers feel transparent -- you pass the wrapper, and the compiler unwraps it as far as the target needs.

And note the crucial detail: this coercion happens at compile time, entirely in the type checker. There is no runtime cost, no reflection, no hidden allocation. The finished machine code is exactly what you would have written by hand with the explicit derefs, so the convenience is genuinely free.

Methods come along for free

Coercion does not only fire when you pass arguments; it also drives method resolution, and this is where wrappers earn their keep. When you write value.some_method(), Rust looks for some_method on the type itself, then on its & and &mut forms, and if it still has not found it, it follows Deref to the target and looks there too, repeating until it either finds the method or gives up. Combined with DerefMut (the mutable sibling), a thin wrapper around a Vec can be pushed to and measured as if it were a Vec:

use std::ops::{Deref, DerefMut};

struct Wrapper<T>(T);

impl<T> Deref for Wrapper<T> {
    type Target = T;
    fn deref(&self) -> &T { &self.0 }
}
impl<T> DerefMut for Wrapper<T> {
    fn deref_mut(&mut self) -> &mut T { &mut self.0 }
}

fn main() {
    let mut w = Wrapper(vec![1, 2, 3]);
    w.push(4);               // Vec::push, reached through DerefMut
    println!("{}", w.len()); // Vec::len, reached through Deref -> 4
    println!("{:?}", *w);    // [1, 2, 3, 4]
}

w.push(4) is not a method on Wrapper -- we never wrote one. The compiler cannot find push on Wrapper, so it follows DerefMut to the inner Vec<i32> and calls Vec::push there. w.len() resolves the same way through the immutable Deref. The distinction between the two is exactly the borrow you need: push mutates, so it travels through DerefMut; len only reads, so Deref suffices. That is the whole reason DerefMut exists as a seperate trait and requires Deref as its supertrait -- you cannot hand out a &mut path to the inner value unless you can already hand out a & path to it.

Why String is &str and Vec is &[T]

This is the payoff moment, because it explains two facts you have quietly relied on since the early episodes. String implements Deref<Target = str>, so any &String coerces to &str. Vec<T> implements Deref<Target = [T]>, so any &Vec<T> coerces to a slice &[T]. That is not a special case baked into the language -- it is the exact same Deref mechanism you just implemented for MyBox, only living in the standard library:

fn takes_slice(items: &[i32]) -> i32 {
    items.iter().sum()
}

fn main() {
    let v = vec![1, 2, 3, 4];
    let total = takes_slice(&v); // &Vec<i32> coerces to &[i32]
    println!("{}", total);       // 10
}

So the standard advice you have heard me repeat -- "accept &str over &String, and &[T] over &Vec<T> in your function signatures" -- is not some arbitrary style rule. Those slice types are the more general borrow: a &[i32] can come from a Vec, from an array, from another slice, from anything contiguous. And deref coercion means your callers pay absolutely nothing to hand you their owned Vec where you asked for a slice, because the compiler inserts the coercion silently. Accepting the general type costs the caller zero and buys you a function that works with more inputs. This is exactly why save_tasks in our episode 14 to-do project took a slice rather than a &Vec, and now you know the machinery that made that painless.

Read through, write through

To see both directions of the coercion sitting in one place, here is a little Stack that wraps a Vec and is read and mutated purely through Deref/DerefMut, without a single method of its own:

use std::ops::{Deref, DerefMut};

struct Stack<T>(Vec<T>);

impl<T> Deref for Stack<T> {
    type Target = Vec<T>;
    fn deref(&self) -> &Vec<T> { &self.0 }
}
impl<T> DerefMut for Stack<T> {
    fn deref_mut(&mut self) -> &mut Vec<T> { &mut self.0 }
}

fn main() {
    let mut s = Stack(vec![10, 20]);
    s.push(30);                  // through DerefMut
    let top = s.last().copied(); // through Deref
    println!("{:?} top={:?}", *s, top); // [10, 20, 30] top=Some(30)
}

Every operation here -- push, last, and the *s dereference in the print -- reaches the inner Vec through coercion. The Stack type contributes nothing but a name and a wrapper. That is either wonderfully convenient or a warning sign, depending entirely on why you wrapped the Vec in the first place, which brings us to the one piece of judgement this episode really wants you to keep.

How this looks from Python and Go

A lot of you come to this series from Python (as I did, having taught it for years), so it is worth seeing where deref coercion has an analogue and where it simply does not. Python has no dereference operator to overload, but it does have __getattr__, a hook that fires when an attribute is not found on an object -- and people use it to forward methods to a wrapped object, which is spiritually close to what Deref does for method resolution:

class Wrapper:
    def __init__(self, inner):
        self._inner = inner
    def __getattr__(self, name):
        return getattr(self._inner, name)  # forward anything we don't have

w = Wrapper([1, 2, 3])
w.append(4)          # forwarded to the list
print(len(w._inner)) # 4

__getattr__ forwards append to the inner list the way Deref forwards push to the inner Vec. The difference is the by-now-familiar one: Python resolves this at runtime, so a typo in the method name blows up only when that line executes, while Rust's coercion is resolved entirely by the type checker before the program ever runs. And Python's version happily forwards everything, which is exactly the kind of "fake inheritance" Rust wants you to think twice about.

Go takes the opposite stance and gives you embedding, which is explicit and checked, but deliberately not a coercion:

type Inner struct{}

func (Inner) Hello() string { return "hi from inner" }

type Wrapper struct {
    Inner // embedded: Wrapper promotes Inner's methods
}

func main() {
    w := Wrapper{}
    println(w.Hello()) // promoted from the embedded Inner
}

Go promotes Inner's methods onto Wrapper at compile time, which looks like the same convenience, but there is no *w that suddenly becomes an Inner, and no coercion of *Wrapper to *Inner when you pass it around. Rust keeps the two ideas separate: Deref is specifically the "I am a pointer to that" relationship, and it is meant for pointer-like types, not for borrowing another type's methods because it saves typing. Which is the whole moral of the next section.

Use Deref only for smart pointers

Deref is genuinely tempting to abuse, precisely because it forwards methods so smoothly. You could implement Deref to make a User type expose every method of some inner Account, faking a kind of inheritance Rust deliberately does not have. Resist that with both hands. Deref coercion is meant to signal exactly one thing: "this type is a smart pointer to that type." When you stretch it into general delegation, method resolution becomes a genuine riddle, because a reader looking at user.close() cannot tell whether close lives on User or was silently reached through some Deref to Account. The methods appear out of nowhere, and error messages start pointing at types the reader never mentioned.

The standard library holds itself to this rule with iron discipline. Box<T>, Rc<T>, Arc<T>, String, Vec<T>, RefCell's guards, MutexGuard -- every type in std that implements Deref is genuinely a pointer or an owner that wraps exactly one inner value it is standing in for. None of them use it to borrow an unrelated type's API for convenience. So the test is simple: if the phrase "is a pointer to" honestly describes your type, Deref is the right tool. If you only want the inner methods without the pointer relationship, write the forwarding methods you actually want by hand, or expose the inner value through a plain accessor. It is a few more keystrokes and a great deal less confusion for whoever reads the code next -- very possibly you, six months from now.

What did we actually learn?

  • Deref customises the * operator: *b is rewritten by the compiler into *(b.deref()), where deref returns a &Target reference into the wrapped value, never the value by move.
  • Deref coercion is the compile-time convenience that inserts deref calls until types line up, so &MyBox<String> flows to &String to &str on its own; it has zero runtime cost.
  • Coercion also drives method resolution: a method not found on the wrapper is looked up on the Deref target, which is why a Vec wrapper gets push and len for free. DerefMut handles the mutable path and requires Deref as a supertrait.
  • String: Deref<Target = str> and Vec<T>: Deref<Target = [T]> are the exact mechanism behind "prefer &str over &String, and &[T] over &Vec<T>" -- the slice is the general borrow, and callers pay nothing to coerce into it.
  • Implement Deref only for genuine smart pointers, never for general method-borrowing, because using it as fake inheritance makes method resolution and error messages confusing; the standard library follows this rule without exception.

The thread across these last few episodes is that Rust's trait system keeps handing you deliberate, sharp-edged tools: associated types, operators, and now the pointer relationship itself, each one a single trait doing one honest job. Next time we stay right here with what makes a smart pointer smart and look at the other end of a value's life -- the precise, predictable moment Rust cleans it up and runs your teardown code -- but one thing at a time ;-)

Exercises

Three exercises, gentle to chewier as always. Full solutions open the next episode, so have a real go first, because typing this stuff yourself is where it actually sticks.

  1. Give MyBox<T> a Deref impl that also prints "deref called" inside deref, then call a &str function with a MyBox<String> and count from the output how many times coercion invokes deref for a single call. Explain in a comment why it is the number you see.
  2. Build a Stack<T> wrapping a Vec<T> with both Deref and DerefMut, then use push, pop, and len on it directly -- without writing any of those methods yourself -- and print the stack after a couple of pushes and a pop.
  3. Write a function taking &[u8], then call it with a Vec<u8> (via &v), a &Vec<u8>, and an array reference &[1u8, 2, 3], confirming all three coerce to the slice. Add a one-line comment on which Deref impl does the work for the Vec case.

Bedankt voor het lezen, en tot de volgende keer! ;-)

@scipio

stem stemsocial steemstem rust programming
0
0
0.000
0 comments