The Why and How of Rust Declarative Macros

In order to prepare to conduct a technical interview of a potential future co-worker, I decided to try to solve the problem we would be presenting to the candidate. I chose to do it in Rust (even though we don't use Rust on my team) so that I could approach the problem with a fresh perspective and potentially learn some new things about my favorite programming language.

It turns out revisiting an old problem using a dramatically different programming language will teach you a lot! I wrote four different solutions using different approaches and patterns, which helped me better understand Rust's standard library and how to write more "Rusty" code. In addition it prepared me to better understand what the interviewee might do in the interview so I can ask good questions to see how they think.

But the biggest thing I learned through this exercise was how to write Rust declarative macros, which this post is all about.

Why Use Declarative Macros

I've never worked with a language that uses macros before and reading about them has always scared me a little. Code that writes other code, but with special syntax? Yikes. Since meta-programming can become extremely complicated, I've never reached for it to solve any problems, but I stumbled across a good opportunity to dive into it when writing tests for my interview answers!

While testing my potential solutions, I found myself repeating the same exact lines of code over and over, with minor variations:

// Tweak the test array to check the different conditions in each test
let test_array = vec![3, 3, 4, 2, 4, 2, 4, 4];

let first_result = find_answer_1(&test_array);
let second_result = find_answer_2(&test_array);
let third_result = find_answer_3(&test_array);
let fourth_result = find_answer_4(&test_array);
// Add another line here in every test when a new function is made

assert_eq!(4, *first_result.unwrap());
assert_eq!(4, *second_result.unwrap());
assert_eq!(4, *third_result.unwrap());
assert_eq!(4, *fourth_result.unwrap());
// Add another line here in every test when a new function is made

In addition, whenever I added another solution to the problem I had to update multiple lines in every test case. It was becoming a real headache, and it only got worse with every new problem solution function I wrote.

Thankfully, declarative macros are the perfect tool for writing repetitive code with minor variations!

Now instead of writing all those lines for each test, I only needed to do the following to test each case:

test_find_answer_functions!(
    // The answer based on the list below
    4,
    // The test's input data
    &[3, 3, 4, 2, 4, 4, 2, 4, 4],
    // The names of the functions I want to test
    find_answer_1,
    find_answer_2,
    find_answer_3,
    find_answer_4
    // Add another function name here whenever it's created
);

Isn't that dramatically better? It's extendable too, so when I get an itch to write another solution to the problem in the future, I can quickly tack it onto the end of the macro's arguments and it will also get tested.

How to Write Declarative Macros

So now that we've seen how a declarative macro can simplify writing code, let's dig into how to write them. The following code block is the final macro I came up with, along with a copious number of comments describing the syntax, since there are some different symbols used compared to writing regular Rust that you may not be familiar with:

// macro_rules! is the macro used to create declarative macros
//   test_find_answer_functions is the name of this macro
macro_rules! test_find_answer_functions {
    // Match macro usage where None is the expected output
    //   (matches the literal None, is not macro syntax)
    // - $test_array:expr - array of values to search for the answer
    //   (expr means any expression, e.g. vec![1, 2, 3])
    // - $function:ident - name of the function to test against
    //   (ident means an identifier, i.e. the function's name)
    // - $(__),+ - repeat 1 or more times
    (None, $test_array:expr, $($function:ident),+) => {
        // syntax for repeating based on the number of functions provided
        $(
            // Call function with test data, assert the result is None
            assert_eq!(None, $function($test_array));
        )*
    };
    // Match macro usage where generic type T is the expected output
    // - $answer:expr - value of T we expect to be the answer
    // - $test_array:expr - same array as the None branch
    // - $function:ident - same list of functions as the None branch
    //
    ($answer:expr, $test_array:expr, $($function:ident),+) => {
        // syntax for repeating based on the number of functions provided
        $(
            // Call function with test data and assert the result is the answer
            assert_eq!($answer, *$function($test_array).unwrap());
        )*
    };
}

Like I said before, macros are a bit weird. It's got a whole "who programs the programs" vibe to it that requires you to think about your code's structure differently, so I definitely ran into some issues when making the macro that wrote my tests for me.

If you ever want to try writing your own Rust declarative macros, you'll find a few of the roadblocks I faced written out below so you can avoid them yourself:

Issue 1

The first issue I ran into is that I didn't have a clear idea of what an :expr or an :ident was, so I was getting some weird errors. After reading through the metavariables section of the Rust Reference (which is a deep dive into the inner workings of Rust), I found my problem. I was treating my function name as an expr instead of an ident. Turns out expr is any valid Rust expression, like the value I wanted to test and the list of values to test against, and ident is any identifier, like the names I gave my functions. Little facepalm moment there, but solved easily enough.

Issue 2

The second issue was dealing with the two different patterns of test code. Some of my tests expected a result to be found, while some expected no solution to the test data provided. This led to two different assertions:

// For data with an answer
let test_array = vec![3, 3, 4, 2, 4, 2, 4, 4];
let first_result = find_answer_1(&test_array);
assert_eq!(4, *first_result.unwrap());

// For data without an answer
let test_array = vec![3, 3, 4, 2, 4, 2];
let first_result = find_answer_1(&test_array);
assert_eq!(None, first_result);

That pesky dereference (*) and .unwrap() in the example with an answer is totally different than the second example, where we only have to check the first_result Option to see if it's None!

Thankfully, Rust declarative macros support the same powerful pattern matching that Rust uses. In the macro code above, you'll see two different cases. One for None and the other for Some result.

And that order is important! When writing patterns, you want to start with the most specific at the top so that it's matched against before its more general version. Since None is a macro metavariable of type expr, putting the second pattern first would mean None matched that more generic pattern. I spent a few minutes stuck there until I remembered that particular rule of pattern matching.

Issue 3

Finally, I fought the borrow checker for a bit, since I couldn't easily tell what the final output of my macro would be. Rather than randomly throw * or & into my macro, I decided to finally figure out how to view the compiled code with the following command:

rustc --pretty expanded -Z unstable-options src/lib.rs --test

Now that's a bit more complicated than I would prefer for a debugging command, but it makes sense once it's broken down:

  • rustc is the Rust compiler
  • In order to use the --pretty expanded flag to preserve spacing after compilation, -Z unstable-options is required
  • -Z unstable-options requires the nightly compiler (which can be turned on for a single workspace using rustup override set nightly)
  • src/lib.rs is the name of the file to compile, which is the one I'm writing my code in
  • --test means to compile the test code, which I needed since my macros are only used in the tests

Unfortunately, that final command expands all macros, including the final code for things like assert_eq! and the #[test] attributes on the tests themselves, so it took me a little bit of digging to find my specific macro code. But once I found my macro's output, I could clearly see the borrow checker problem and fix it!

Why Not Use Plain Rust?

I could've written a solution using plain ol' Rust, avoiding macros entirely. The main reason I didn't was I simply forgot that was an option and finished the macro before I remembered that you could pass functions to other functions (which I absolutely love to do!).

I decided to write up a solution using functions. In the end I still feel like macros are a better fit in terms of ergonomics. I had to write two different functions, one for the Some case (when a solution is found) and another for the None case (when there is no solution):

fn test_find_answer_functions_some<T, F>(answer: T, data: &[T], funcs: Vec<F>)
where
    T: Eq + Hash + Debug,
    F: FnOnce(&[T]) -> Option<&T>,
{
    for func in funcs {
        assert_eq!(answer, *func(data).unwrap())
    }
}

fn test_find_answer_functions_none<T, F>(answer: Option<&T>, data: &[T], funcs: Vec<F>)
where
    T: Eq + Hash + Debug,
    F: FnOnce(&[T]) -> Option<&T>,
{
    for func in funcs {
        assert_eq!(answer, func(data));
    }
}

I could've written a single function that handles both scenarios by passing in an option as the answer parameter for the Some case, but that led to me writing this absolutely hideous answer argument as seen below:

test_find_answer_functions(
    Some(&4),
    &[3, 3, 4, 2, 4, 4, 2, 4, 4],
    [
        find_majority_element_two_loop,
        find_majority_element_two_iter,
        find_majority_element_one_iter,
        find_majority_element_counting,
    ]
    .to_vec(),
);

Wrapping the answer in a Some instead of a naked 4 like I could do with the macro was just too much for my perfectionist brain to handle.

In order to match the prettier user-friendly macro syntax, two functions were required. Even then, I had an ugly .to_vec() that I couldn't get rid of (although I'm sure there's a way to do so if I spent a little more time on it):

test_find_answer_functions_some(
    4,
    &[3, 3, 4, 2, 4, 4, 2, 4, 4],
    [
        find_majority_element_two_loop,
        find_majority_element_two_iter,
        find_majority_element_one_iter,
        find_majority_element_counting,
    ]
    .to_vec(),
);

In addition to the less than ideal user interface, the function approach requires a bit more of a heavy lift on the runtime side of things. A macro literally prints out code, and that resulting code can be optimized by the compiler. The functional approach happens dynamically at runtime, so there's a bit of a performance cost there. This toy example isn't concerned with performance, but it is something to consider when making a decision between the two approaches.

Wrapping Up

By this point you should have a high-level understanding of what Rust declarative macros are, where they might be helpful, and some potential issues you might face when writing your own.

While my example was a little contrived, it was a real-world usage of macros that made my life as a programmer a little bit easier and got me excited to look for more substantial opportunities to use macros in the future.

Further details regarding declarative and other macros can be found in the official Rust book, which is one of the best resources out there for learning the language and should be on the reading list of anyone wanting to become proficient in Rust. If you found this article interesting, I think you'd really enjoy the book. It's one of the best examples of approachable technical writing I've ever come across.


If you have questions or comments, feel free to reach out to me through any of the methods on my About Me page, or leave a comment in my guestbook!


You'll only receive email when they publish something new.

More from Lane Sawyer๐ŸŒน
All posts