What is Tokio-Trace Rust Library?

What is Tokio-Trace Rust Library?

RustConf 2019 - Tokio-Trace: Scoped, Structured, Async-Aware Diagnostics by Eliza Weisman

tokio-trace is a new set of Rust libraries that provide primitives for recording scoped, structured, and async-aware diagnostics. Unlike traditional logging, tokio-trace emits structured diagnostics that model the contextual and causal relationships between between events. tokio-trace was designed by the tokio project to solve problems with logging in asynchronous applications, but may be used in any Rust codebase. This talk presents the motivation and influences behind tokio-trace, introduces its core concepts, and demonstrates how it can be used.


tokio-trace is a framework for instrumenting Rust programs to collect structured, event-based diagnostic information.

In asynchronous systems like Tokio, interpreting traditional log messages can often be quite challenging. Since individual tasks are multiplexed on the same thread, associated events and log lines are intermixed making it difficult to trace the logic flow. tokio-trace expands upon logging-style diagnostics by allowing libraries and applications to record structured events with additional information about temporality and causality — unlike a log message, a span in tokio-trace has a beginning and end time, may be entered and exited by the flow of execution, and may exist within a nested tree of similar spans. In addition, tokio-trace spans are structured, with the ability to record typed data as well as textual messages.

The tokio-trace crate provides the APIs necessary for instrumenting libraries and applications to emit trace data.

Core Concepts

The core of tokio-trace's API is composed of Events, Spans, and Subscribers. We'll cover these in turn.


Span represents a period of time during which a program was executing in some context. A thread of execution is said to enter a span when it begins executing in that context, and to exit the span when switching to another context. The span in which a thread is currently executing is referred to as the current span.

For example:

extern crate tokio_trace;

use tokio_trace::Level;

span!(Level::TRACE, "my_span").enter(|| { // perform some work in the context of my_span... });

Spans form a tree structure — unless it is a root span, all spans have a parent, and may have one or more children. When a new span is created, the current span becomes the new span's parent. The total execution time of a span consists of the time spent in that span and in the entire subtree represented by its children. Thus, a parent span always lasts for at least as long as the longest-executing span in its subtree.

// this span is considered the "root" of a new trace tree:
span!(Level::INFO, "root").enter(|| {
    // since we are now inside "root", this span is considered a child
    // of "root":
    span!(Level::DEBUG, "outer_child").enter(|| {
        // this span is a child of "outer_child", which is in turn a
        // child of "root":
        span!(Level::TRACE, "inner_child").enter(|| {
            // and so on...

In addition, data may be associated with spans. A span may have fields — a set of key-value pairs describing the state of the program during that span; an optional name, and metadata describing the source code location where the span was originally entered.

// construct a new span with three fields:
//  - "foo", with a value of 42,
//  - "bar", with the value "false"
//  - "baz", with no initial value
let my_span = span!(Level::INFO, "my_span", foo = 42, bar = false, baz);

// record a value for the field "baz" declared above: my_span.record("baz", &"hello world");

When to use spans

As a rule of thumb, spans should be used to represent discrete units of work (e.g., a given request's lifetime in a server) or periods of time spent in a given context (e.g., time spent interacting with an instance of an external system, such as a database).

Which scopes in a program correspond to new spans depend somewhat on user intent. For example, consider the case of a loop in a program. Should we construct one span and perform the entire loop inside of that span, like:

span!(Level::TRACE, "my loop").enter(|| {
    for i in 0..n {
        // ...

Or, should we create a new span for each iteration of the loop, as in:

for i in 0..n {
    span!(Level::TRACE, "my loop", iteration = i).enter(|| {
        // ...

Depending on the circumstances, we might want to do either, or both. For example, if we want to know how long was spent in the loop overall, we would create a single span around the entire loop; whereas if we wanted to know how much time was spent in each individual iteration, we would enter a new span on every iteration.


An Event represents a point in time. It signifies something that happened while the trace was executing. Events are comparable to the log records emitted by unstructured logging code, but unlike a typical log line, an Event may occur within the context of a Span. Like a Span, it may have fields, and implicitly inherits any of the fields present on its parent span.

For example:

// records an event outside of any span context:
event!(Level::INFO, "something happened");

span!(Level::INFO, "my_span").enter(|| { // records an event within "my_span". event!(Level::DEBUG, "something happened inside my_span"); });

Essentially, Events exist to bridge the gap between traditional unstructured logging and span-based tracing. Similar to log records, they may be recorded at a number of levels, and can have unstructured, human-readable messages; however, they also carry key-value data and exist within the context of the tree of spans that comprise a trace. Thus, individual log record-like events can be pinpointed not only in time, but in the logical execution flow of the system.

Events are represented as a special case of spans — they are created, they may have fields added, and then they close immediately, without being entered.

In general, events should be used to represent points in time within a span — a request returned with a given status code, n new items were taken from a queue, and so on.


As Spans and Events occur, they are recorded or aggregated by implementations of the Subscriber trait. Subscribers are notified when an Event takes place and when a Span is entered or exited. These notifications are represented by the following Subscriber trait methods:

  • observe_event, called when an Event takes place,
  • enter, called when execution enters a Span,
  • exit, called when execution exits a Span

In addition, subscribers may implement the enabled function to filter the notifications they receive based on metadata describing each Span or Event. If a call to Subscriber::enabled returns false for a given set of metadata, that Subscriber will not be notified about the corresponding Span or Event. For performance reasons, if no currently active subscribers express interest in a given set of metadata by returning true, then the corresponding Span or Event will never be constructed.


First, add this to your Cargo.toml:

tokio-trace = "0.1"

Next, add this to your crate:

extern crate tokio_trace;

Spans are constructed using the span! macro, and then entered to indicate that some code takes place within the context of that Span:

// Construct a new span named "my span" with trace log level.
let span = span!(Level::TRACE, "my span");
span.enter(|| {
    // Any trace events in this closure or code called by it will occur within
    // the span.
// Dropping the span will close it, indicating that it has ended.

Events are created using the event! macro, and are recorded when the event is dropped:

use tokio_trace::Level;
event!(Level::INFO, "something has happened!");

Users of the log crate should note that tokio-trace exposes a set of macros for creating Events (trace!debug!info!warn!, and error!) which may be invoked with the same syntax as the similarly-named macros from the log crate. Often, the process of converting a project to use tokio-trace can begin with a simple drop-in replacement.

Let's consider the log crate's yak-shaving example:

extern crate tokio_trace;
use tokio_trace::{field, Level};
pub fn shave_the_yak(yak: &mut Yak) {
    // Create a new span for this invocation of shave_the_yak, annotated
    // with  the yak being shaved as a field on the span.
    span!(Level::TRACE, "shave_the_yak", yak = field::debug(&yak)).enter(|| {
        // Since the span is annotated with the yak, it is part of the context
        // for everything happening inside the span. Therefore, we don't need
        // to add it to the message for this event, as the log crate does.
        info!(target: "yak_events", "Commencing yak shaving");

    loop {
        match find_a_razor() {
            Ok(razor) => {
                // We can add the razor as a field rather than formatting it
                // as part of the message, allowing subscribers to consume it
                // in a more structured manner:
                info!({ razor = field::display(razor) }, "Razor located");
            Err(err) => {
                // However, we can also create events with formatted messages,
                // just as we would for log records.
                warn!("Unable to locate a razor: {}, retrying", err);


You can find examples showing how to use this crate in the examples directory.

In libraries

Libraries should link only to the tokio-trace crate, and use the provided macros to record whatever information will be useful to downstream consumers.

In executables

In order to record trace events, executables have to use a Subscriber implementation compatible with tokio-trace. A Subscriber implements a way of collecting trace data, such as by logging it to standard output.

Unlike the log crate, tokio-trace does not use a global Subscriber which is initialized once. Instead, it follows the tokio pattern of executing code in a context. For example:

extern crate tokio_trace;

let my_subscriber = FooSubscriber::new();

tokio_trace::subscriber::with_default(my_subscriber, || { // Any trace events generated in this closure or by functions it calls // will be collected by my_subscriber. })

This approach allows trace data to be collected by multiple subscribers within different contexts in the program. Alternatively, a single subscriber may be constructed by the main function and all subsequent code executed with that subscriber as the default. Any trace events generated outside the context of a subscriber will not be collected.

The executable itself may use the tokio-trace crate to instrument itself as well.

The tokio-trace-nursery repository contains less stable crates designed to be used with the tokio-trace ecosystem. It includes a collection of Subscriber implementations, as well as utility and adapter crates.

In particular, the following tokio-trace-nursery crates are likely to be of interest:

  • tokio-trace-futures provides a compatibility layer with the futures crate, allowing spans to be attached to Futures, Streams, and Executors.
  • tokio-trace-fmt provides a Subscriber implementation for logging formatted trace data to stdout, with similar filtering and formatting to the env-logger crate.
  • tokio-trace-log provides a compatibility layer with the log crate, allowing log Records to be recorded as tokio-trace Events within the trace tree. This is useful when a project using tokio-trace have dependencies which use log.

Crate Feature Flags

The following crate feature flags are available:

  • A set of features controlling the static verbosity level.
  • log causes trace instrumentation points to emit log records as well as trace events. This is inteded for use in libraries whose users may be using either tokio-trace or log.
tokio-trace = { version = "0.1", features = ["log"] }


pub use self::span::Span;


dispatcher Dispatches trace events to Subscribers.c

event Events represent single points in time during the execution of a program.

field Structured data associated with Spans and Events.

level_filters Trace verbosity level filtering.

span Spans represent periods of time in the execution of a program.

subscriber Collects and records trace data.


debug Constructs an event at the debug level.

debug_span Constructs a span at the debug level.

error Constructs an event at the error level.

error_span Constructs a span at the error level.

event Constructs a new Event.

info Constructs an event at the info level.

info_span Constructs a span at the info level.

span Constructs a new span.

trace Constructs an event at the trace level.

trace_span Constructs a span at the trace level.

warn Constructs an event at the warn level.

warn_span Constructs a span at the warn level.


Dispatch Dispatch trace data to a Subscriber.

Event Events represent single points in time where something occurred during the execution of a program.

Level Describes the level of verbosity of a span or event.

Metadata Metadata describing a span or [event].


Subscriber Trait representing the functions required to collect trace data.

Value A field value of an erased type.

Further Reading

Rust for Weld, a High Performance Parallell JIT Compiler

JavaScript to Rust, bringing it to Electron, adding WASM to an existing React

What is class fixes and class breaks in Rust Language?

Building a Retro Computer in Embedded Rust Language


Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Rust Casting, Shadowing, Consts and Static

A mix of topics of casting, shadowing, constants and static variables inside the Rust Programming Language. This Rust programming language tutorial series is aimed at easing your training step by step.

Rust Zürichsee, April: Gerhard Bräunlich - PyO3 and Rust in action

This presentation was the first experiment with livestreaming of the Rust Zürisee meetup group in Switzerland. Gerhard is sharing parts of his story migratin...

Prevent Breaking Code Changes in Future Releases using `non exhaustive` enums in Rust

Hello everyone, recently I have come across a feature in Rust, known as non_exhaustive. It was introduced in Rust 1.40.0 . This attribute prevents source code-breaking changes in projects downstream.

Rust vs Go - Which Is More Popular?

Rust vs Go - Which Is More Popular - Go and Rust are two of the hottest compiled programming languages. I develop in Go full-time and love it, and I'm learning more about Rust recently - its an exc

Rust 2020: Rust at Wasmer

We love Rust at Wasmer! We’ll overview how Rust development in 2019 has impacted us as a small company using a lot of advanced Rust features for our WebAssembly runtime and tools, what we’re thankful for in Rust this year, the problems we face, and what we would like to see in the future.