Fifthtry

Observer: Observability for Rust

Observer lets you observe the execution of rust code.

Observer is a replacement for logging, statsd/metrics and newrelice/open-tracing.

v0.1 is already in use in production at Acko, and supports new relic.

Concepts

Observer monitors a program or an app.

Some apps, eg servers have concept of request, where we are interested in performance of the request, and not how fast the program ends.

In such cases Context represents a single request. If you are writing say a program that say processes a video file frame by frame, the context could be frame.

Say in case of http request, Context is created right after the http request comes in, and then we call different functions.

Functions in rust can be annotated with a macro: observed, which will track performance of that invocation of function in the context.

We also have a file, that describes the type of observed fields, so observe_fields are type checked at compile time. It’s not recommended to modify the type signature of observed fields, as in some context, its best to store each span in row in a database with proper column types representing each observed field, and changing type may need schema change.

#[observed]
fn observe_me_too() -> Result<i32> {
    observe_field("foo1", 32);
    Ok(12)
}

When an “observed” function is called, the observer keeps track of how long the function took, and what observed functions the observed function itself called, creating a tree, or a flame graph, with timing.

Each call of an observed function creates what we call “span”. The span is a tree of spans, and context contains an array of spans.

Further we have concept of observe_field(), which associates data with appropriate span.

observed” functions be with or without result, by default without. If with_result is used, the span is aware that it is a “failable” span, keeps track of success/failure of the function.

planned v0.2

The primary goal of v0.2 is to refactor out new relic code from observer into a separate crate. Instead we will create a trait that must be implemented by different backend.

observer::Observer struct

This struct has to be created the beginning of an app/program.

It all backends must be passed to observer when creating it.

pub struct Observer {
  // private fields
}

impl Observer {
    pub fn new(backends: [impl Backend]) -> Observer {
       // ...
       // this will iterate through all passed Backends
       // and call their .app_started() method.
    }

    pub fn create_context(&self, context_id: &str) {
        // could return Context, or could keep in 
        // thread-local, not yet decided

        // this will call .context_started() of all
        // backends registered in new().
    }

    pub fn end_context(&self) {}
}

observer::Backend trait

Not yet sure what the name of the trait should be, it has to have the following methods:

trait Backend {
    fn app_started()
    fn app_ending()

    fn context_created()
    fn context_ended()

    fn span_created()
    fn span_log()
    fn span_ended()
}

app_started() and app_ending()

These methods will be called when the program starts and when program is about to end.

They should be used for setup/cleanup for external libraries for example.

context_created() and context_ended()

These methods are called when context is created or ended. This would be a good place to log things to logging backend, or send data to statsd, or new-relic etc.

span_created() span_ended()

These are called around span, when its created and ended. If you want to do function level timing in statsd, this can be used.

span_field()

When a observe_field() is called, this method is invoked.

Table Of Content

Immobile v2

this-ambiguation

Link Log

August 2020

July 2020

June 2020

May 2020

April 2020

March 2020

February 2020

January 2020

Recommendations

Books Have Read / Recommend

Product Management Books

Badass: Making Users Awesome

Movies

Five Cs of An Organisation

Success and failure of encryption

Open Source

Observer: Observability for Rust

Realm: Web Development Framework Using Rust and Elm

MartD: Server To Browser Messages

On Writing And Formats Of Written Communications

Rust Stuff

Rust feature flags

Why is diesel not compatible with async?

Making Postgres Only Diesel Code To Also Support Sqlite

Rust Git2’s Concepts

Git Hash And Build Date In Rust Build

Systray Only Native App In Rust

Software and Tools I Use Often

IPFS

DNS Over HTTPS Controversy

The Patel Motel Cartel

Standalone Complex

Awesome

January 2020

Word Of The Day

Monkey

Positions

ViM

Emacs

Nix On OSX Catalina

Postgres: WAL / Logical Decoding

Postgres: Listen-Notify

Wisdom

Rules

Go All The Way

SSH Commands

Lovelace

Sorry

Nu Shell

SHA256 vs SHA224

Pronouns Bad

Ghost

Web Components

Early Return