Fifthtry

Quick Start Realm Tutorial

In this tutorial we will setup an app, to get a breadth first understanding on web application development using Realm.

In this tutorial we will be building a web-app that shows a count of how many times a link has been clicked. It uses sqlite as database. The web-app is full stack, “single page application”.

Please ensure you have read about the motivation behind Realm before proceeding.

Enter Nix

Realm is written in Rust, Elm, and uses Python for database modelling (and migration). We have dependency on PostgreSQL libraries, do some scripting in zsh, have dependency on openssl and so on.

Setting all tools, ensuring they are of compatible versions etc is hard to do, especially if development happens across Linux and Mac.

We use Nix to solve this problem.

You can install Nix by running:

curl https://nixos.org/nix/install | sh

If you are using Catalina, read this.

realm-starter

In this tutorial we will be starting with a pre-configured project, fifthtry/realm-starter.

Start by cloning it:

git clone https://github.com/fifthtry/realm-starter.git

First order of business is to get nix to install all dependencies we have (Rust, Python, Elm etc). We also do not want to have any reliance on dependencies installed on your host system for maximum isolation and reproducibility.

We will be using nix-shell, in pure mode for this:

cd realm-starter
nix-shell --pure --run zsh

The way realm-starter is setup, the previous step will install all nix dependencies, and then install all python and some rust dependencies when run for the first time. This is a one time setup, subsequent launches would be lot faster.

Since nix caches the dependencies installed, spinning up new realm projects doesn’t require re-downloads when new environment is needed for those projects.

Every time you want to work on the project, including git commits, you must enter the nix-shell using the above command. There are a few pre-commit hooks that check the code before its committed and they rely on nix-shell for dependency.

Creating Sqlite Database

In the realm-starter project, and in Realm projects in general, we use Django’s ORM for database modelling and database schema migrations.

We will be writing models.py to define our model, you can find a pre-created, bare-bones model in dj/hello/models.py:

from django.db import models


class Counter(models.Model):
    count = models.IntegerField(default=0)

Every time we modify our models.py, it is recommended to run recreatedb_sqlite (or recreatedb_pg depending on if we are using sqlite or Postgres).

$ recreatedb_sqlite
    -> deleting existing db
    -> running django migration
Operations to perform:
  Apply all migrations: hello
Running migrations:
  Applying hello.0001_initial... OK
    -> creating pristine.db
    -> generating diesel.toml
django_migrations  hello_counter
    -> updating schema.rs
    -> all done

This creates diesel schema file, src/schema.rs:

table! {
    hello_counter (id) {
        id -> Integer,
        count -> Integer,
    }
}

The generated schema.rs file must be checked-in in git, and must be updated every time we change models.py.

Bulding Elm

All our elm code lives in a folder named frontend.

We use doit for building things in Realm. It comes pre-configured in realm-starter. To build the whole project (Main, Test & Storybook) we can simply run:

doit elm
.  elm:test
.  elm:storybook
.  elm:iframe
.  elm:main
.  elm:index

We will look at our Elm code in detail later on.

Running Our App

Now that we have everything setup, we can run our app using cargo run:

cargo run
   Compiling realm_tutorial v0.1.0 (/Users/amitu/PycharmProjects/realm-starter)
    Finished dev [unoptimized + debuginfo] target(s) in 2m 37s
     Running `target-nix/debug/realm_tutorial`
Listening on http://0.0.0.0:3000

You can now visit 127.0.0.1:3000 to visit our sweet app.

Feel free to play with the “Go Up” link and see the counter go up.

You can also use the command serve which is an alias to cargo run -- --test to run the server in test mode. This is the most common way of launching the server while developing a Realm application.

Walk Through: src/main.rs

Lets take a walk though the essential elements of the project we just created.

Our project starts in src/main.rs:

use realm::base::*;
realm::realm! {middleware}

pub fn http404(msg: &str) -> realm::Result {
    use realm::Page;
    realm_tutorial::not_found::not_found(msg).with_title(msg)
}

pub fn middleware(ctx: &realm::Context) -> realm::Result {
    observer::create_context("middleware");
    let conn = sqlite::connection()?;
    let in_: In0 = realm::base::In::from(&conn, ctx);

    realm::end_context(&in_, route(&in_), |_, m| http404(m))
}

pub fn route(in_: &In0) -> realm::Result {
    let mut input = in_.ctx.input()?;

    match in_.ctx.pm() {
        t if realm::is_realm_url(t) => realm::handle(in_, t, &mut input),
        ("/increment/", _) => realm_tutorial::routes::increment::get(in_),
        ("/", _) => realm_tutorial::routes::index::get(in_),
        _ => http404("Page not found"),
    }
}

Where is our beloved main() function you ask?!

Our realm::realm!{} macro generates the main() for us. A bare bones realm!{} looks like:

#[macro_export]
macro_rules! realm {
    ($e:expr) => {
        pub fn main() {
            realm::realm_serve!($e)
        }
    };
}

As you can see, it simply delegates responsibility to realm::realm_serve! macro. Both realm! and realm_serve! macros expect a “middleware”, a rust function with takes &realm::Context and returns realm::Result.

http404()

pub fn http404(msg: &str) -> realm::Result {
    use realm::Page;
    realm_tutorial::not_found::not_found(msg).with_title(msg)
}

This function is an example of a realm::Page. It returns the 404 page defined in src/not_found.rs.

.with_title(msg) sets the title of the page to the string passed to the function.

middleware()

The middleware is a function that is called on every request. It gets request data, and its job is to construct the response.

pub fn middleware(ctx: &realm::Context) -> realm::Result {
    observer::create_context("middleware");
    let conn = sqlite::connection()?;
    let in_: In0 = realm::base::In::from(&conn, ctx);

    realm::end_context(&in_, route(&in_), |_, m| http404(m))
}

In our middleware we are getting a sqlite connection, and constructing an realm::In object. realm::In object is convenient holder that keeps realm::Context and diesel::Connection, and is the first argument of a lot of rust functions.

realm::In itself is parametric type, and for now we are using bool, but we will learn about what is the use of the parameter later on.

The middleware() delegates to route() the actual job of handling the http request.

route()

Realm does not come with any custom routing library, largely because Rust is so awesome at pattern matching.

pub fn route(in_: &In0) -> realm::Result {
    let mut input = in_.ctx.input()?;

    match in_.ctx.pm() {
        t if realm::is_realm_url(t) => realm::handle(in_, t, &mut input),
        ("/increment/", _) => realm_tutorial::routes::increment::get(in_),
        ("/", _) => realm_tutorial::routes::index::get(in_),
        _ => http404("Page not found"),
    }
}

Realm does come with a helper for extracting data from http requests, realm::RequestConfig, which is the type of input above. In this program we are not yet extracting any parameter from request, and are simply going to rely on request path, so we will get to realm::RequestConfig later on.

realm::Context::pm(): this is a helper that returns the tuple: (&str, Method), the request path (minus the query string) and the requested http method. We are matching on this tuple.

In the match we are first checking if it is a realm::is_realm_url(), if so, we are letting realm::handle() it.

Realm comes with some request handlers, the one we are most interested right now is a static file serving handler, which intercepts all GET requests starting with /static/ and serves them mapping it to a folder named static in current directory.

Next we have a clause for path "/increment/", and ignoring the method, which we are passing to realm_tutorial::routes::increment::get(). This is a recommended structure of a realm project, to put all routes in a module names routes.

Next we have the case for the root URL "/", here too we ignore the method and pass the request to realm_tutorial::routes::index::get() which handles the index route.

Finally we are sending all other requests to http404() function which displays the 404 page as discussed above. You can view the 404 page by visiting any route other than "/increment" and "/".

src/routes/index.rs

Lets look at our first route, index.rs:

pub use realm::base::*;
pub use realm::{Or404, Page as RealmPage};

#[realm_page(id = "Pages.Index")]
struct Page {
    message: String,
    count: i32,
}

pub fn get(in_: &In0) -> realm::Result {
    Page {
        message: "hello world".to_string(),
        count: crate::db::get_count(in_)?,
    }
    .with_title("Welcome")
}

pub fn redirect(in_: &In0) -> realm::Result {
    get(in_).map(|r| r.with_url(crate::reverse::index()))
}

Much is going on here, let’s focus one by one on different components.

struct Page

In realm, the primary job of a “route” is to return an object that implements realm::Page trait. Realm then figures out what to do with that object, convert it to HTML, or JSON, or SPA app etc.

Lets see it in action. The default is to return “server-rendered-html”, if the user agent is a crawler/bot or program (eg curl/python):

curl http://127.0.0.1:3000
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <title>Welcome</title>
        <meta name="viewport" content="width=device-width" />
        <script id="data" type="application/json">

        </script>
        <style>p {margin: 0}</style>
    </head>
    <body>
        <h1>Hello SSR</h1>

<p>Message from server: hello world.</p>

<p>Count: 0. <a href="/increment/">Go Up</a></p>
        <div id="main"></div>
    </body>
</html>

This is the page you want google to consume. It has invoked a template named templates/pages/index.html, matching with the id added above, and rendering it by passing the Page defined above.

<h1>Hello SSR</h1>

<p>Message from server: {{ message }}.</p>

<p>Count: {{ count }}. <a href="/increment/">Go Up</a></p>

Rust on compile time ensures the fields {{ message }} and {{ count }} exist in our Page struct (and have appropriate type, not demonstrated here).

We have other “modes”, eg html:

curl "http://127.0.0.1:3000/?realm_mode=html"
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <title>Welcome</title>
        <meta name="viewport" content="width=device-width" />
        <script id="data" type="application/json">
{
  "id": "Pages.Index",
  "config": {
    "count": 0,
    "message": "hello world"
  },
  "title": "Welcome",
  "url": "/?realm_mode=html",
  "replace": null,
  "redirect": null
}
        </script>
        <style>p {margin: 0}</style>
    </head>
    <body>
        
        <div id="main"></div><script src='/static/elm.js'></script>
    </body>
</html>

This is what we return to browsers. It does not contain the pre-rendered content, instead it contains JSON representation of the Page, and a link to our javascript: /static/elm.js.

Then we have the api mode:

curl "http://127.0.0.1:3000/?realm_mode=api" 
{
  "count": 0,
  "message": "hello world"
}

In case you want to consume your endpoint as an API.

Coming back to our struct:

#[realm_page(id = "Pages.Index")]
struct Page {
    message: String,
    count: i32,
}

It is declaring we are going to return a string “message”, and an int “count”. Further it has declared that on frontend, we should use an elm module Pages.Index to render this.

Lets see how are we constructing this struct:

pub fn get(in_: &In0) -> realm::Result {
    Page {
        message: "hello world".to_string(),
        count: crate::db::get_count(in_)?,
    }
    .with_title("Welcome")
}

We have hardcoded the message, but are computing the count using crate::db::get_count() function, which will look at briefly.

Further we are using the .with_title() trait method, provided by realm::Page trait, to convert it to realm::Result, the title is the title of the HTML page, as you can inspect in the output above.

The function redirect is discussed later with src/routes/increment.rs.

Walk Through: src/db.rs

We have put aside the code that interacts with database in this module:

pub use realm::base::*;

pub fn increment(in_: &In0) -> Result<()> {
    // elided 
}

pub fn get_count(in_: &In0) -> Result<i32> {
    use crate::schema::hello_counter;
    use diesel::prelude::*;

    if let Ok(count) = hello_counter::table
        .select(hello_counter::count)
        .first(in_.conn)
    {
        return Ok(count);
    };

    diesel::insert_into(hello_counter::table)
        .values(hello_counter::count.eq(0))
        .execute(in_.conn)?;

    Ok(0)
}

Not much happening here, beyond a few basic calls to database using diesel. What is really cool - this code would stop compiling if you modify models.py and say rename column named count to say the_count - instead of you encountering a runtime error.

src/routes/increment.rs

Lets look at src/routes/increment.rs to elucidate one more point:

pub use realm::base::*;

pub fn get(in_: &In0) -> realm::Result {
    crate::db::increment(in_)?;
    crate::routes::index::redirect(in_)
}

Here we have not defined any struct, as we do not want a new page, we have decided in this API/page, we will show the same content as we showed in index.

Such routes are “action” routes in Realm. Liking something, posting a comment, creating a todo etc, are all actions. They usually involve some data validation, database changes, and return some “page” that should be shown to user after the action has taken place. In this case we want to show the page with incremented count to user, which is in index page.

We have invoked a function crate::routes::index::redirect(), lets review it:

pub fn redirect(in_: &In0) -> realm::Result {
    get(in_).map(|r| r.with_url(crate::reverse::index()))
}

This function is calling get() method of crate::routes::index, and then calling with_url: this we are doing because in the browser, we do not want users to see the /increment/, but /, as users may reload the page, and can accidentally perform the action.

Further note: in this case we have called .with_url(), this adds a new “history” in browser, if you click on “Go Up” a few times, you will see that history has index page a number of times, which may or may not be what you are looking for. If you do not want history to be added, you can call .with_replace() instead.

Lets look at it via curl:

curl "http://127.0.0.1:3000/increment/?realm_mode=api"
{
  "count": 1,
  "message": "hello world"
}

curl "http://127.0.0.1:3000/increment/?realm_mode=api"
{
  "count": 2,
  "message": "hello world"
}

Count is going up, and we are getting the index’s output.

Frontend of the app

The frontend of the application is written in Elm. Realm develops a few features on top of Elm for better productivity and faster development. (The batteries included approach!)

Walkthrough frontend/Routes.elm

The simple frontend/Routes.elm file contains routes for the application. Notice that these are the same as that defined in the frontend. As discussed before, each “route” in Realm, “maps” to one “Elm App”. Pages.Index is one such “Elm App”.

module Routes exposing (..)


index : String
index =
    "/"


increment : String
increment =
    "/increment/"

The actual frontend that is displayed when you visit the / index page is covered in frontend/Pages/Index.elm.

Walkthrough frontend/Pages/Index.elm

This file contains all the code for the frontend. It is a bit long so we will go through it in parts. You can open it to see how these parts fit together.

Before discussing the functions and attributes, lets discuss the types first:

type alias Config =
    { message : String
    , count : Int
    }


type alias Model =
    { config : Config }


type Msg
    = NoOp

Realm Apps Realm.App are “parametrised” over three types, the “Config” for the app, the “Model” of the app, and the “Msg” of the app.

Config represents the data returned by our sever, an exact translation of Page struct.

We have defined Config that is a record with message and count. The Model type defines the structure of the record for our model that will store the data of config and any of the state data that we would like to track. We have kept it simple and just stored the config as is for now. The Msg type is a way of declaring which type of messages are we going to entertain in our application. As we do not need any messages for this app, we have set it to NoOp.

Every Realm Page has a main function which returns the Realm app. Each Elm App in Realm are created using Realm.app method.

app : R.App Config Model Msg
app =
    R.App config init update subscriptions document
    
main =
	R.app app

The app has a few interesting things:

config

config : JD.Decoder Config
config =
    JD.map2 Config
        (JD.field "message" JD.string)
        (JD.field "count" JD.int)

This function is the decoder for the response sent by the server in Json into the initial Config which will be stored in Model. Here we have the message and count defined to decode to a string and int respectively.

init

init : R.In -> Config -> ( Model, Cmd (R.Msg Msg) )
init _ c =
    ( { config = c }, Cmd.none )

This function takes in a Realm:In (which it ignores) and the Config and returns us the model of the application, also with an Elm command. For now, we do not use commands in this application, this the Command returned is set to Cmd.none.

update

update : R.In -> Msg -> Model -> ( Model, Cmd (R.Msg Msg) )
update _ _ m =
    ( m, Cmd.none )

This function is used to update the model of our app whenever a message is received from the elm runtime. Here too we ignore any messages received and return the model as it is.

You might wonder: Then how does the Go Up button work? In this example, the Go Up button links to the /increment/ path. Whenever the Go Up button is clicked, the browser navigates to /increment/ from where the server receives the request, increments the number in the DB and redirects to the homepage which shows the updated count.

subscriptions

subscriptions : R.In -> Model -> Sub (R.Msg Msg)
subscriptions _ _ =
    Sub.none

This function is responsible for messages that are not generated directly by the user and may include: websocket, page scroll, HTTP request status change and so on. As we don’t have any of these messages in our app, we have returned the Sub.none.

document

document : R.In -> Model -> B.Document (R.Msg Msg)
document in_ =
    view >> R.document in_

This is the function that generates the Document visible on the frontend. Well not quite exactly, it actually relies on the view function to get the user interface.

view

view : Model -> E.Element (R.Msg Msg)
view m =
    let
        c =
            m.config
    in
    E.column [ E.centerX, E.centerY, E.spacing 20 ]
        [ E.el [ E.centerX, EF.italic ]
            (E.text "Hello Elm")
        , E.text ("Message from server: " ++ c.message)
        , E.row [ E.spacing 10, E.alignRight ]
            [ E.text ("Count: " ++ String.fromInt c.count)
            , E.link []
                { url = "/increment/"
                , label = E.text "Go Up"
                }
            ]
        ]

This is the main code that generates our UI! This takes in a Model tyoee and returns an Elm-UI Element which can generate a message (as denoted by (R.Msg Msg) in the type signature). Here we extract the config as a variable c and use it in Elm-UI’s elements to create our simple interface.

Note that E.text takes in a single String and basically gives us a div with the string as it’s child. We use the result and pass it to E.el to apply style attributes to it. E.column takes in a set of attributes and a list of elements and displays them in a column. And E.row works similar to E.column but displays it in a row. E.link creates an anchor link to the URL with the label set to “Go Up”.

You will notice that there are other files in the frontend folder, these contain sample codes for writing tests and storybook demos. You can check them out to see how they work. Also, a sample 404 page is also included so that you don’t have to worry about it in the beginning.

We are using the wonderful elm-ui, so you can do fronted css etc without hating the world.

This is it for now. You can move to how to guides to learn more.

Feel free to create an issue if you find any issue with this tutorial.

A note: first time you do a commit, pre-commit hooks that come with starter template, will take a lot of time, minutes, doing cargo check, subsequent commits would be faster, but be on a watch out: during pre-commit checks, upstaged changes are stashed, and if you are editing code waiting for commit to happen in background you may put repository in strange state when pre-commit tries to unstash the stashed change and if it conflicts with your edits.

Table Of Content

What is Realm?

A Bit On Motivation

Routing is Hard

What does Realm do?

Backend Data And Type Safety

Tutorial

Quick Start Realm Tutorial

In Depth Tutorial (not ready)

Nix
Shell
Doit
Hello Rust
Hello Elm
Hello Static Files
Hello Server Side Rendering
Pre-Commit Hooks

Routing, Request And Response

Frontend, Data, Navigation, And APIs

How To Guides

File Upload

Backend: S3 File Upload
Authenticated File Serving
Frontend: Uploading Files From Elm

How to use storybook?

How to implement “loading..”?

Docs

Realm.In

Realm.Storybook.Story

realm::In

realm::Context

realm::Result

realm.magicSlice

realm::RequestConfig

Environment Variables

Internals - Only for Realm Developers, not Users

“Realm DATA”
iFrame Controller
Shutdown Routine
Testing Internals

Change Log

Get Realm Starter Working

Transparent Offline Feature

How to make http requests in Realm?

Development

Tutorial: ToDo App

Realm Testing

Enhance Realm Starter

Double Load Issue

Deploy To Heroku Button

End failure

Realm-Starter Github Template

Proposal: Tracker And Visit

Proposal: Activity Store

Proposal: Bundling

Proposal: Retry On Network Error

Storybook: Editable JSON

Storybook: Notes

Storybook: Reference

Backlog

Readings

Change Log

How to Publish

Testing

Code Snippets

Skip rustfmt For Some Section

Close Modal Dialog When Clicked Outside

Ignoring Lints In Python

Ignoring Lints (clippy and rustc warnings) In Rust

Handle DateTime in Rust & Elm

Handle CiText value read in Rust

Transport Enum Type to and fro Rust/Elm through JSON