Fifthtry

Quick Start Realm Tutorial

In this tutorial we will setup an app, to get a breadth first understanding on web application development using Realm.

In this tutorial we will be building a web-app that shows a count of how many times a link has been clicked. It uses sqlite as database. The web-app is full stack, “single page application”.

Please ensure you have read about the motivation behind Realm before proceeding.

Enter Nix

Realm is in Rust, Elm, and uses Python for database modelling (and migration). We have dependency on PostgreSQL libraries, do some scripting in zsh, have dependency on openssl and so on.

Setting all tools, ensuring they are of compatible versions etc is hard to do, especially if development happens across Linux and Mac.

We use Nix to solve this problem.

You can install Nix by running:

curl https://nixos.org/nix/install | sh

If you are using Catalina, read this.

realm-starter

In this tutorial we will be starting with a pre-configured project, realm-starter.

Start by cloning it:

git clone https://github.com/amitu/realm-starter.git

First order of business is to get nix to install all dependencies we have (Rust, Python, Elm etc). We also do not want to have any reliance on dependencies installed on your host system for maximum isolation and reproducibility.

We will be using nix-shell, in pure mode for this:

cd realm-starter
nix-shell --pure --run zsh

The way realm-starter is setup, the previous step will install all nix dependencies, and then install all python and some rust dependencies.

It will take a few minutes. This is a one time setup, subsequent launches would be lot faster.

Every time you want to work on the project, you must enter the nix-shell using the above command.

Creating Sqlite Database

In the realm-starter project, and in Realm projects in general, we use Django’s ORM for database modelling and database schema migrations.

We will be writing models.py to define our model, you can find a pre-created, bare-bones model in dj/hello/models.py:

from django.db import models


class Counter(models.Model):
    count = models.IntegerField(default=0)

Every time we modify our models.py, it is recommended to run recreatedb_sqlite (or recreatedb_pg depending on if we are using sqlite or Postgres).

$ recreatedb_sqlite
    -> deleting existing db
    -> running django migration
Operations to perform:
  Apply all migrations: hello
Running migrations:
  Applying hello.0001_initial... OK
    -> creating pristine.db
    -> generating diesel.toml
django_migrations  hello_counter
    -> updating schema.rs
    -> all done

This creates diesel schema file, src/schema.rs:

table! {
    hello_counter (id) {
        id -> Integer,
        count -> Integer,
    }
}

The generated schema.rs file must be checked-in in git, and must be updated every time we change models.py.

Bulding Elm

All our elm code lives in a folder named frontend.

We use doit for building things in Realm. It comes pre-configured in realm-starter. To build the whole project (Main, Test & Storybook) we can simply run:

doit elm
.  elm:test
.  elm:storybook
.  elm:iframe
.  elm:main
.  elm:index

We will look at our Elm file in detail a bit later on.

Running Our App

We are there, we can run our app using cargo run:

cargo run
   Compiling realm_tutorial v0.1.0 (/Users/amitu/PycharmProjects/realm-starter)
    Finished dev [unoptimized + debuginfo] target(s) in 2m 37s
     Running `target-nix/debug/realm_tutorial`
Listening on http://0.0.0.0:3000

You can now visit 127.0.0.1:3000 to visit our sweet app. Feel free to click on “Go Up” link and see the counter go up.

Walk Through: src/main.rs

Lets take a walk though the main essential elements of the project we just created.

Our project starts in src/main.rs:

use realm::base::*;
realm::realm! {middleware}

pub fn middleware(ctx: &realm::Context) -> realm::Result {
    let conn = sqlite::connection()?;
    let in_: In0 = realm::base::In::from(&conn, ctx);

    route(&in_)
}

pub fn route(in_: &In0) -> realm::Result {
    let mut input = in_.ctx.input()?;

    match in_.ctx.pm() {
        t if realm::is_realm_url(t) => realm::handle(in_, t, &mut input),
        ("/increment/", _) => realm_tutorial::routes::increment::get(in_),
        _ => realm_tutorial::routes::index::get(in_),
    }
}

Where is our beloved main() function you ask?!

Our realm::realm!{} macro generates the main() for us. realm!{} a pretty bare bones:

#[macro_export]
macro_rules! realm {
    ($e:expr) => {
        pub fn main() {
            realm::realm_serve!($e)
        }
    };
}

As you can see, it simply delegates responsibility to realm::realm_serve! macro. Both realm! and realm_serve! macros expect a “middleware”, a rust function with takes &realm::Context and returns realm::Result.

middleware()

The middleware is a function that is called on every request. It gets request data, and its job is to construct the response.

pub fn middleware(ctx: &realm::Context) -> realm::Result {
    let conn = sqlite::connection()?;
    let in_: In0 = realm::base::In::from(&conn, ctx);

    route(&in_)
}

In our middleware we are getting a sqlite connection, and constructing an realm::In object. realm::In object is convenient holder that keeps realm::Context and diesel::Connection, and is the first argument of a lot of rust functions.

realm::In itself is parametric type, and for now we are using bool, but we will learn about what is the use of the parameter later on.

The middleware() delegates to route() the actual job of handling the http request.

route()

Realm does not come with any custom routing library, largely because Rust is so awesome at pattern matching.

pub fn route(in_: &In0) -> realm::Result {
    let mut input = in_.ctx.input()?;

    match in_.ctx.pm() {
        t if realm::is_realm_url(t) => realm::handle(in_, t, &mut input),
        ("/increment/", _) => realm_tutorial::routes::increment::get(in_),
        _ => realm_tutorial::routes::index::get(in_),
    }
}

Realm does come with a helper for extracting data from http requests, realm::RequestConfig, which is the type of input above. In this program we are not yet extracting any parameter from request, and are simply going to rely on request path, so we will get to realm::RequestConfig later on.

realm::Context::pm(): this is a helper that returns the tuple: (&str, Method), the request path (minus the query string) and the requested http method. We are matching on this tuple.

In the match we are first checking if it is a realm::is_realm_url(), if so, we are letting realm::handle() it.

Realm comes with some request handlers, the one we are most interested right now is a static file serving handler, which intercepts all GET requests starting with /static/ and serves them mapping it to a folder named static in current directory.

Next we have a clause for path "/increment/", and ignoring the method, which we are passing to realm_tutorial::routes::increment::get(). This is a recommended structure of a realm project, to put all routes in a module names routes.

Finally we are sending all other requests to realm_tutorial::routes::index::get() function.

Walk Through: src/routes/index.rs

Lets look at our first route, index.rs:

pub use realm::base::*;
pub use realm::Page as RealmPage;

#[realm_page(id = "Pages.Index")]
struct Page {
    message: String,
    count: i32,
}

pub fn get(in_: &In0) -> realm::Result {
    Page {
        message: "hello world".to_string(),
        count: crate::db::get_count(in_)?,
    }
    .with_title("Welcome")
}

pub fn redirect(in_: &In0) -> realm::Result {
    get(in_).map(|r| r.with_url(crate::reverse::index()))
}

Much is going on here, let’s focus one by one on different components.

struct Page

In realm, the primary job of a “route” is to return an object that implements realm::Page trait. Realm then figures out what to do with that object, convert it to HTML, or JSON, or SPA app etc.

Lets see it in action. The default is to return “server-rendered-html”, if the user agent is a crawler/bot or program (eg curl/python):

curl http://127.0.0.1:3000
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <title>Welcome</title>
        <meta name="viewport" content="width=device-width" />
        <script id="data" type="application/json">

        </script>
        <style>p {margin: 0}</style>
    </head>
    <body>
        <h1>Hello SSR</h1>

<p>Message from server: hello world.</p>

<p>Count: 0. <a href="/increment/">Go Up</a></p>
        <div id="main"></div>
    </body>
</html>

This is the page you want google to consume. It has invoked a template named templates/pages/index.html, matching with the id added above, and rendering it by passing the Page defined above.

<h1>Hello SSR</h1>

<p>Message from server: {{ message }}.</p>

<p>Count: {{ count }}. <a href="/increment/">Go Up</a></p>

Rust on compile time ensures the fields {{ message }} and {{ count }} exist in our Page struct (and have appropriate type, not demonstrated here).

We have other “modes”, eg html:

curl "http://127.0.0.1:3000/?realm_mode=html"
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <title>Welcome</title>
        <meta name="viewport" content="width=device-width" />
        <script id="data" type="application/json">
{
  "id": "Pages.Index",
  "config": {
    "count": 0,
    "message": "hello world"
  },
  "title": "Welcome",
  "url": "/?realm_mode=html",
  "replace": null,
  "redirect": null
}
        </script>
        <style>p {margin: 0}</style>
    </head>
    <body>
        
        <div id="main"></div><script src='/static/elm.js'></script>
    </body>
</html>

This is what we return to browsers. It does not contain the pre-rendered content, instead it contains JSON representation of the Page, and a link to our javascript: /static/elm.js.

Then we have the api mode:

curl "http://127.0.0.1:3000/?realm_mode=api" 
{
  "count": 0,
  "message": "hello world"
}

In case you want to consume your endpoint as an API.

Coming back to our struct:

#[realm_page(id = "Pages.Index")]
struct Page {
    message: String,
    count: i32,
}

It is declaring we are going to return a string “message”, and an int “count”. Further it has declared that on frontend, we should use an elm module Pages.Index to render this.

Lets see how are we constructing this struct:

pub fn get(in_: &In0) -> realm::Result {
    Page {
        message: "hello world".to_string(),
        count: crate::db::get_count(in_)?,
    }
    .with_title("Welcome")
}

We have hardcoded the message, but are computing the count using crate::db::get_count() function, which will look at briefly.

Further we are using the .with_title() trait method, provided by realm::Page trait, to convert it to realm::Result, the title is the title of the HTML page, as you can inspect in the output above.

Walk Through: src/db.rs

We have put aside the code that interacts with database in this module:

pub use realm::base::*;

pub fn increment(in_: &In0) -> Result<()> {
    // elided 
}

pub fn get_count(in_: &In0) -> Result<i32> {
    use crate::schema::hello_counter;
    use diesel::prelude::*;

    if let Ok(count) = hello_counter::table
        .select(hello_counter::count)
        .first(in_.conn)
    {
        return Ok(count);
    };

    diesel::insert_into(hello_counter::table)
        .values(hello_counter::count.eq(0))
        .execute(in_.conn)?;

    Ok(0)
}

Not much happening here, beyond a few basic calls to database using diesel. What is really cool - this code would stop compiling if you modify models.py and say rename column named count to say the_count - instead of you encountering a runtime error.

Walk Through: src/routes/increment.rs

Lets look at src/routes/increment.rs to elucidate one more point:

pub use realm::base::*;

pub fn get(in_: &In0) -> realm::Result {
    crate::db::increment(in_)?;
    crate::routes::index::redirect(in_)
}

Here we have not defined any struct, as we do not want a new page, we have decided in this API/page, we will show the same content as we showed in index.

Such routes are “action” routes in Realm. Liking something, posting a comment, creating a todo etc, are all actions. They usually involve some data validation, database changes, and return some “page” that should be shown to user after the action has taken place. In this case we want to show the page with incremented count to user, which is in index page.

We have invoked a function crate::routes::index::redirect(), lets review it:

pub fn redirect(in_: &In0) -> realm::Result {
    get(in_).map(|r| r.with_url(crate::reverse::index()))
}

This function is calling get() method of crate::routes::index, and then calling with_url: this we are doing because in the browser, we do not want users to see the /increment/, but /, as users may reload the page, and can accidentally perform the action.

Further note: in this case we have called .with_url(), this adds a new “history” in browser, if you click on “Go Up” a few times, you will see that history has index page a number of times, which may or may not be what you are looking for. If you do not want history to be added, you can call .with_replace() instead.

Lets look at it via curl:

curl "http://127.0.0.1:3000/increment/?realm_mode=api"
{
  "count": 1,
  "message": "hello world"
}

curl "http://127.0.0.1:3000/increment/?realm_mode=api"
{
  "count": 2,
  "message": "hello world"
}

Count is going up, and we are getting the index’s output.

Walk Through: frontend/Pages/Index.elm

Lets move on to frontend now. For now the entire frontend is one Elm file: frontend/Pages/Index.elm.

As discussed before, each “route” in Realm, “maps” to one “Elm App”. Pages.Index is one such “Elm App”. Each Elm App in Realm are created using Realm.app method:

module Pages.Index exposing (main)

import Browser as B
import Element as E
import Element.Events as EE
import Element.Font as EF
import Json.Decode as JD
import Realm as R


main =
    R.app app


app : R.App Config Config ()
app =
    R.App config R.init0 R.update0 R.sub0 document

Realm Apps Realm.App are “parametrised” over three types, the “Config” for the app, the “Model” of the app, and the “Msg” of the app.

Config represents the data returned by our sever, an exact translation of Page struct. Here is how it looks like:

type alias Config =
    { message : String
    , count : Int
    }

In our case we have used Config as both “Config” and “Model”, as Model is used to represent “frontend app state”, and currently we do not have any app state.

Further, since our app currently does not have any interaction etc, we have used () as our Msg type.

Lets take a look at the view, whose job is to create our DOM tree based current “frontend app state”:

view : Config -> E.Element (R.Msg ())
view c =
    E.column [ E.centerX, E.centerY, E.spacing 20 ]
        [ E.el [ E.centerX, EF.italic ]
            (E.text "Hello Elm")
        , E.text ("Message from server: " ++ c.message)
        , E.row [ E.spacing 10, E.alignRight ]
            [ E.text ("Count: " ++ String.fromInt c.count)
            , E.link []
                { url = "/increment/"
                , label = E.text "Go Up"
                }
            ]
        ]

Not much happening here, we have a vertically and horizontally aligned column, which contains three rows, first row with static message, italicised, second row with message from server, and third row is a row with two elements, current count, and a link.

We are using the wonderful elm-ui, so you can do fronted css etc without hating the world.

This is it for now. You can move to how to guides to learn more.

Feel free to create an issue if you find any issue with this tutorial.

A note: first time you do a commit, pre-commit hooks that come with starter template, will take a lot of time, minutes, doing cargo check, subsequent commits would be faster, but be on a watch out: during pre-commit checks, upstaged changes are stashed, and if you are editing code waiting for commit to happen in background you may put repository in strange state when pre-commit tries to unstash the stashed change and if it conflicts with your edits.

Table Of Content

What is Realm?

A Bit On Motivation

Routing is Hard

What does Realm do?

Backend Data And Type Safety

Tutorial

Quick Start Realm Tutorial

In Depth Tutorial (not ready)

Nix
Shell
Doit
Hello Rust
Hello Elm
Hello Static Files
Hello Server Side Rendering
Pre-Commit Hooks

Routing, Request And Response

Frontend, Data, Navigation, And APIs

How To Guides

File Upload

Backend: S3 File Upload
Authenticated File Serving
Frontend: Uploading Files From Elm

How to use storybook?

How to implement “loading..”?

Docs

Realm.In

Realm.Storybook.Story

realm::In

realm::Context

realm::Result

realm::RequestConfig

Environment Variables

Internals - Only for Realm Developers, not Users

“Realm DATA”
iFrame Controller
Shutdown Routine

Change Log

Get Realm Starter Working

Transparent Offline Feature

How to make http requests in Realm?

Development

Tutorial: ToDo App

Enhance Realm Starter

Double Load Issue

Deploy To Heroku Button

End failure

Realm-Starter Github Template

Proposal: Activity Store

Proposal: Bundling

Proposal: Retry On Network Error

Storybook: Editable JSON

Storybook: Notes

Storybook: Reference

Backlog

Readings

Change Log

How to Publish

Testing

Code Snippets

Skip rustfmt For Some Section

Close Modal Dialog When Clicked Outside

Ignoring Lints In Python

Ignoring Lints (clippy and rustc warnings) In Rust

Handle DateTime in Rust & Elm

Handle CiText value read in Rust