Enhancing Test Stability with Fixtures in Rust

No more awkward test cases. Here's one fixture-based testing framework that cleaned up my Rust unit tests.

In this article:

• 🔬 We'll learn the limitations of #[test] in Rust.
• 🪄 We'll explore an alternative to writing cleaner unit tests with fixtures.
• 🏗 We'll refactor the unit tests in one of my open-source projects!

Let's go.

The Fixture Problem

Test fixtures are very effective in producing repeatable tests. A test fixture can be a constant or function that encapsulates a test's dependency.

I wanted to create a few test fixtures for my unit tests in one of my open-source projects called Voy. It's a WebAssembly semantic engine written in Rust. What it does is extract features using machine learning models, build an index, and provide a query function that enables users to search the index based on meaning and semantics.

Here's a quick demo:

You can find the Voy's repository on GitHub! Feel free to try it out. The repository includes examples that you can see how to use Voy in different frameworks.

While I was still working on the feature extraction, I needed pre-generated embeddings to test the index and query part of the engine. The fixture for embeddings looks like this:

pub static EMBEDDING: [[f32; 768]; 6] = [
    [
        0.01960003957247733,
        -0.03651725347725505,
        0.03361894761373059,
        // ...
    ]

I created a static variable for 6 embeddings in an array. Each item is a 768-dimensional vector that represents a sentence.

To use the fixture, I simply imported it into my test module. My unit test looks like this:

use super::engine_fixtures::{EMBEDDING, CONTENT, QUESTION};
use crate::engine::{add, index, remove, search, Query};
use crate::{EmbeddedResource, Resource};

fn get_resource(k: usize) -> Resource {
     let embeddings = EMBEDDING
         .iter()
         .take(k)
         .enumerate()
         .map(|(i, x)| EmbeddedResource {
             id: i.to_string(),
             title: CONTENT.get(i).unwrap().to_string(),
             url: "".to_owned(),
             embeddings: x.to_vec(),
         })
         .collect();
     Resource { embeddings }
 }

#[test]
 fn it_returns_vector_search_result() {
     let resource: Resource = get_resource(6);
     let index = index(resource).unwrap();

     let query = Query::Embeddings(QUESTION.to_vec());
     let result = search(&index, &query, 1).unwrap();

     assert_eq!(result.get(0).unwrap().title, CONTENT[0]);
     assert_eq!(result.get(1).unwrap().title, CONTENT[1]);
     assert_eq!(result.get(2).unwrap().title, CONTENT[2]);
     assert_eq!(result.get(3).unwrap().title, CONTENT[4]);
     assert_eq!(result.get(4).unwrap().title, CONTENT[5]);
     assert_eq!(result.get(5).unwrap().title, CONTENT[3]);
 }

I created another 2 fixtures for the sentences that generated the embeddings and a question embedding to perform the query.

The test case worked. The fixture produced repeatable results. However, it seems a little off. It's not very clean:

• I needed a helper function to "get_resource" in the test case to initiate the fixture.
• The fixtures were not encapsulated in the unit test and the usages were scattered.

So I started looking for a better way to write the test.

Introducing The rstest Crate

rstest makes it very easy to write unit tests with fixtures. All you need to do is:

Defining fixtures

#[fixture]
pub fn fixture() -> u32 { 42 }

Replacing #[test] with #[rstest] macro.

#[rstest]
fn should_success(fixture: u32) {
    assert_eq!(fixture, 42);
}

You can check out rstest's repository to find more ways to write test fixtures.

Now let's use rstest to refactor the fixtures above. I'll simply create a function to return the static variable. Like this:

use rstest::fixture;

#[fixture]
pub fn embedding_fixture() -> [[f32; 768]; 6] {
    EMBEDDING
}

We can refactor the "get_resource" helper function as a fixture too:

#[fixture]
pub fn resource_fixture() -> Resource {
    let content = content_fixture();
    let embeddings = embedding_fixture()
        .iter()
        .enumerate()
        .map(|(i, x)| EmbeddedResource {
            id: i.to_string(),
            title: content.get(i).unwrap().to_string(),
            url: "".to_owned(),
            embeddings: x.to_vec(),
        })
        .collect();
    Resource { embeddings }
}

Since each fixture functions are just functions, we can use them inside another fixture function like we see in the "resource_fixture" function above.

To use the fixtures in a test case, we'll need to import the fixtures and inject them as parameters:

use fixtures::*;
use rstest::*;

#[rstest]
fn it_returns_vector_search_result(
    resource_fixture: Resource,
    question_fixture: [f32; 768],
    content_fixture: [&'static str; 6],
) {
    let index = index(resource_fixture).unwrap();
    let query = Query::Embeddings(question_fixture.to_vec());
    let result = search(&index, &query, 6).unwrap();

    assert_eq!(result.get(0).unwrap().title, content_fixture[0]);
    assert_eq!(result.get(1).unwrap().title, content_fixture[1]);
    assert_eq!(result.get(2).unwrap().title, content_fixture[2]);
    assert_eq!(result.get(3).unwrap().title, content_fixture[4]);
    assert_eq!(result.get(4).unwrap().title, content_fixture[5]);
    assert_eq!(result.get(5).unwrap().title, content_fixture[3]);
}

Make sure the parameters have the same names as the fixtures. rstest uses the names to inject and initiate the fixtures in the test functions.

Let's run "cargo test" to see if it works.

It does!

Final Thoughts

The rstest has more convenient features. For example, you can write multiple test cases with #[case]:

use rstest::rstest;

#[rstest]
#[case(0, 0)]
#[case(1, 1)]
#[case(2, 1)]
#[case(3, 2)]
#[case(4, 3)]
fn fibonacci_test(#[case] input: u32, #[case] expected: u32) {
    assert_eq!(expected, fibonacci(input))
}

You can also test Future!

use rstest::*;
#[fixture]
async fn base() -> u32 { 42 }

#[rstest]
#[case(21, async { 2 })]
#[case(6, async { 7 })]
async fn my_async_test(#[future] base: u32, #[case] expected: u32, #[future] #[case] div: u32) {
    assert_eq!(expected, base.await / div.await);
}

These are just a few examples I took from the rstest. It has more fixture-based features that help us to structure cleaner unit tests with test fixtures. I really enjoyed it.

Voy is an open-source semantic search engine in WebAssembly. I created it to empower more projects to build semantic features and create better user experiences for people around the world. Voy follows several design principles:

• 🤏 Tiny: Reduce overhead for limited devices, such as mobile browsers with slow networks or IoT.
• 🚀 Fast: Create the best search experience for the users.
• 🌳 Tree Shakable: Optimize bundle size and enable asynchronous capabilities for modern Web API, such as Web Workers.
• 🔋 Resumable: Generate portable embeddings index anywhere, anytime.
• ☁️ Worldwide: Run a semantic search on CDN edge servers.

It's available on npm. You can simply install it with your favorite package manager and you're ready to go.

# with npm
npm i voy-search

# with Yarn
yarn add voy-search

# with pnpm
pnpm add voy-search

Give it a try and I'm happy to hear from you!

Want to Connect?