Fifthtry

Authenticated File Serving

If you want to expose the uploaded files, you can directly use S3 or you may want to configure AWS Cloud Front or some other CDN. But these methods do not allow access control, and the only works if uploaded files are public.

Public CDNs are a great thing to have for optimising website load time, so they should be considered and used as long as possible.

If you have files that are private in nature, one way to retain the benefit of public CDN and still have some privacy is by making the URLs hard to guess. This is privacy by obfuscation, and for some applications may be good enough.

Now we are coming to proper authenticated file serving.

Reverse Proxies

Lets talk about nginx for a bit, why do people put nginx in front of application servers? Its because backend services are generally resource heavy compared to nginx, and because browsers can be slow (unintentionally: slow network, or intentionally: as denial of service attack).

Consider a HTTP request, when nginx is put as a reverse proxy before backend, the request goes to nginx, and nginx then proxies the request to backend. Nginx holds the connection from client side, and waits for backend to respond. Nginx to backend request is fast, they are usually on same machine, or in same internal network.

Which means as soon as backend has generated the response, it is downloaded by nginx over a high speed connection, and then nginx slowly feeds the response to client at whatever speed client can consume. This frees up the backend as soon as possible for the next connection.

First Attempt At Authentication

With reverse proxy in place, first solution could be for our backend to accept all requests, authenticate the request, fetch the content of the uploaded file from store, eg S3, and serve the content to nginx. Nginx will then feed the file to client.

The problem is that our backend is still doing a lot of work, it is downloading the content from S3, and then feeding it to nginx, and if it GBs of content, it has to be done in a streaming fashion, which our backend may not support (as realm currently, as of 25th June, 2020 does not). Further the CPU requirement, the RAM requirement etc would still be there, and whatever milliseconds backend is downloading from S3 is the millisecond wasted.

There is one more concern, do we want to download the file from S3 on every http request, or should we have caching? Of course we should have caching, but then our backend has to implement caching too.

Letting NGinx Download From S3

Nginx already has support for both doing streaming S3 download, and caching. How do we plug authentication into nginx is the only question.

We have two different ways we can let backend (rust) do authentication, and let nginx take care of cached S3 download: auth_request directive and internal directive along with X-Accel-Redirect header.

Table Of Content

What is Realm?

A Bit On Motivation

Routing is Hard

What does Realm do?

Backend Data And Type Safety

Tutorial

Quick Start Realm Tutorial

In Depth Tutorial (not ready)

Nix
Shell
Doit
Hello Rust
Hello Elm
Hello Static Files
Hello Server Side Rendering
Pre-Commit Hooks

Routing, Request And Response

Frontend, Data, Navigation, And APIs

How To Guides

File Upload

Backend: S3 File Upload
Authenticated File Serving
Frontend: Uploading Files From Elm

How to use storybook?

How to implement “loading..”?

Docs

Realm.In

Realm.Storybook.Story

realm::In

realm::Context

realm::Result

realm::RequestConfig

Environment Variables

Internals - Only for Realm Developers, not Users

“Realm DATA”
iFrame Controller
Shutdown Routine

Change Log

Get Realm Starter Working

Transparent Offline Feature

How to make http requests in Realm?

Development

Tutorial: ToDo App

Enhance Realm Starter

Double Load Issue

Deploy To Heroku Button

End failure

Realm-Starter Github Template

Proposal: Activity Store

Proposal: Bundling

Proposal: Retry On Network Error

Backlog

Readings

Change Log

How to Publish

Testing

Code Snippets

Skip rustfmt For Some Section

Close Modal Dialog When Clicked Outside

Ignoring Lints In Python

Ignoring Lints (clippy and rustc warnings) In Rust