• Miguel Ojeda's avatar
    rust: support running Rust documentation tests as KUnit ones · a66d733d
    Miguel Ojeda authored
    Rust has documentation tests: these are typically examples of
    usage of any item (e.g. function, struct, module...).
    
    They are very convenient because they are just written
    alongside the documentation. For instance:
    
        /// Sums two numbers.
        ///
        /// ```
        /// assert_eq!(mymod::f(10, 20), 30);
        /// ```
        pub fn f(a: i32, b: i32) -> i32 {
            a + b
        }
    
    In userspace, the tests are collected and run via `rustdoc`.
    Using the tool as-is would be useful already, since it allows
    to compile-test most tests (thus enforcing they are kept
    in sync with the code they document) and run those that do not
    depend on in-kernel APIs.
    
    However, by transforming the tests into a KUnit test suite,
    they can also be run inside the kernel. Moreover, the tests
    get to be compiled as other Rust kernel objects instead of
    targeting userspace.
    
    On top of that, the integration with KUnit means the Rust
    support gets to reuse the existing testing facilities. For
    instance, the kernel log would look like:
    
        KTAP version 1
        1..1
            KTAP version 1
            # Subtest: rust_doctests_kernel
            1..59
            # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
            ok 1 rust_doctest_kernel_build_assert_rs_0
            # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
            ok 2 rust_doctest_kernel_build_assert_rs_1
            # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
            ok 3 rust_doctest_kernel_init_rs_0
            ...
            # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
            ok 59 rust_doctest_kernel_types_rs_2
        # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
        # Totals: pass:59 fail:0 skip:0 total:59
        ok 1 rust_doctests_kernel
    
    Therefore, add support for running Rust documentation tests
    in KUnit. Some other notes about the current implementation
    and support follow.
    
    The transformation is performed by a couple scripts written
    as Rust hostprogs.
    
    Tests using the `?` operator are also supported as usual, e.g.:
    
        /// ```
        /// # use kernel::{spawn_work_item, workqueue};
        /// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
        /// # Ok::<(), Error>(())
        /// ```
    
    The tests are also compiled with Clippy under `CLIPPY=1`, just
    like normal code, thus also benefitting from extra linting.
    
    The names of the tests are currently automatically generated.
    This allows to reduce the burden for documentation writers,
    while keeping them fairly stable for bisection. This is an
    improvement over the `rustdoc`-generated names, which include
    the line number; but ideally we would like to get `rustdoc` to
    provide the Rust item path and a number (for multiple examples
    in a single documented Rust item).
    
    In order for developers to easily see from which original line
    a failed doctests came from, a KTAP diagnostic line is printed
    to the log, containing the location (file and line) of the
    original test (i.e. instead of the location in the generated
    Rust file):
    
        # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
    
    This line follows the syntax for declaring test metadata in the
    proposed KTAP v2 spec [1], which may be used for the proposed
    KUnit test attributes API [2]. Thus hopefully this will make
    migration easier later on (suggested by David [3]).
    
    The original line in that test attribute is figured out by
    providing an anchor (suggested by Boqun [4]). The original file
    is found by walking the filesystem, checking directory prefixes
    to reduce the amount of combinations to check, and it is only
    done once per file. Ambiguities are detected and reported.
    
    A notable difference from KUnit C tests is that the Rust tests
    appear to assert using the usual `assert!` and `assert_eq!`
    macros from the Rust standard library (`core`). We provide
    a custom version that forwards the call to KUnit instead.
    Importantly, these macros do not require passing context,
    unlike the KUnit C ones (i.e. `struct kunit *`). This makes
    them easier to use, and readers of the documentation do not need
    to care about which testing framework is used. In addition, it
    may allow us to test third-party code more easily in the future.
    
    However, a current limitation is that KUnit does not support
    assertions in other tasks. Thus we presently simply print an
    error to the kernel log if an assertion actually failed. This
    should be revisited to properly fail the test, perhaps saving
    the context somewhere else, or letting KUnit handle it.
    
    Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
    Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
    Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
    Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
    Signed-off-by: default avatarMiguel Ojeda <ojeda@kernel.org>
    Reviewed-by: default avatarDavid Gow <davidgow@google.com>
    Signed-off-by: default avatarShuah Khan <skhan@linuxfoundation.org>
    a66d733d
rustdoc_test_builder.rs 3.25 KB