plug-in

Two days ago in the study of coco plug-in system, we often say to interface-oriented development rather than implementation-oriented, plug-in this thing, like the middleware in the back-end framework, we follow the framework defined interface to implement middleware, which can also be considered a plug-in, we have many mechanisms to achieve “Pre-compiler plug-in”, but like coco to compile and release the binary program, what is the way to allow users to define plug-ins to complement the functionality? You can’t just insert the user’s code during runtime and then recompile the whole program, right? Is there a way to load the user’s library files at runtime? Yes, this technique is called Dynamic Loading.

Dynamic Loading

In coco with dlopen this crate implements dynamic loading, but in Rust dynamic loading seems to have to unsafe, just in the implementation of the plug-in mechanism for coco some moment, I suddenly had a new idea.

WebAssembly

WebAssembly is a new way of coding that runs in modern web browsers - it is a low-level assembly-like language with a compact binary format that can run with near-native performance and provides a compilation target for languages such as C / C ++ so that they can run on the web. It is also designed to coexist with JavaScript, allowing the two to work together. –MDN

WebAssembly was originally designed to run in the browser, but just like JS’s node, WebAssembly now has browser-independent runtimes such as wasmtime and wasmer, and they both provide helper libraries embedded in each major language, which means I can freely call functions in wasm binaries in the languages supported by this runtime? Try it!

wasmtime

I tried to make a demo with wasmtime, first creating a new crate.

1

cargo new adder --lib

Open the src/lib.rs file and write a simple summation function.

1
2
3
4

#[no_mangle]
pub extern "C" fn adder(a: i32, b: i32) -> i32 {
    a + b
}

no_mangle tells the Rust compiler not to change the function name for subsequent calls. Also modify the Cargo.toml file.

1
2

[lib]
crate-type = ['cdylib']

Then you can compile it with this command.

1
2
3

# rustup target add wasm32-wasi
# 如果没有设置target要用上面的命令设置下
cargo build --target wasm32-wasi

Now you can find the corresponding adder.wasm file in the target directory under the project.

Next, create a crate: adders.wasm.

1

cargo new wasm_test

In main.rs, write the following code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

use std::error::Error;
use wasmtime::*;

fn main() -> Result<(), Box<dyn Error>> {
    let engine = Engine::default();
    let store = Store::new(&engine);
    let module = Module::from_file(&engine, "adder.wasm")?;
    let instance = Instance::new(&store, &module, &[])?;
    let adder = instance
        .get_func("adder")
        .expect("adder was not an exported function");
    let adder = adder.get2::<i32, i32, i32>()?;
    let result = adder(2, 4)?;
    println!("result is {}", result);
    Ok(())
}

Don’t forget to introduce the wasmtime dependency, but running it now will report an error straight away.

1

Error: wrong number of imports provided, 0 ! = 4

After debugging, I found that the error was in the creation of the instance, and browsing the documentation I found that if there are imports dependencies in wasm, then the third argument here cannot be an empty array, but an array of all dependencies, but where did such a simple function get the dependencies? The wasm file was converted to text format by the wasm2wat tool, then searched by grep, and sure enough, some dependencies were automatically added after compilation.

1
2
3
4
5

❯ wasm2wat adder.wasm | grep import
  (import "wasi_snapshot_preview1" "fd_write" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview18fd_write17ha0aef7cef0a152b0E (type 6)))
  (import "wasi_snapshot_preview1" "environ_sizes_get" (func $__wasi_environ_sizes_get (type 2)))
  (import "wasi_snapshot_preview1" "proc_exit" (func $__wasi_proc_exit (type 0)))
  (import "wasi_snapshot_preview1" "environ_get" (func $__wasi_environ_get (type 2)))

Because the target of our compilation is wasm32-wasi, wasi is WebAssembly System Interface, a standardized WebAssembly system interface, and the code given by wasmtime in the section explaining how to use it in Rust is applicable to wasm32- unknown-unknown (I have to say that the quality of the documentation is not very good), here if you change the compilation target to wasm32-unknown-unknown can be run directly.

But not to change the compilation target will have to slightly modify the program, according to the search issue, I changed the code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

use std::error::Error;
use wasi_cap_std_sync::WasiCtxBuilder;
use wasmtime::*;
use wasmtime_wasi::Wasi;

fn main() -> Result<(), Box<dyn Error>> {
    let engine = Engine::default();
    let store = Store::new(&engine);

    let mut linker = Linker::new(&store);
    let wasi = Wasi::new(
        &store,
        WasiCtxBuilder::new()
            .inherit_stdio()
            .inherit_args()?
            .build()?,
    );
    wasi.add_to_linker(&mut linker)?;

    let module = Module::from_file(&engine, "adder.wasm")?;
    let instance = linker.instantiate(&module)?;
    let adder = instance
        .get_func("adder")
        .expect("adder was not an exported function");
    let adder = adder.get2::<i32, i32, i32>()?;
    let answer = adder(1, 7)?;
    println!("the answer is {}", answer);
    Ok(())
}

Wasmtime currently supports five languages, wasmer supports more, so users can write plugins in C/C++ or Go, and we can call them in Rust programs and don’t have to write unsafe.

Summary

With WebAssembly we can call libraries written in other languages in Rust and vice versa, WebAssembly becomes an intermediate language or virtual machine, e.g. in Python, due to the dynamic typing feature, it can be used like this

1
2
3
4
5

import wasmtime.loader
import adder  # 直接引入adder.wasm


print(adder.adder(1, 7))

I am very optimistic about the future of WebAssembly, the emergence of a standalone runtime, making WebAssembly a general public language runtime that implements “Run any code on any client”.