If you want to expose the uploaded files, you can directly use S3 or you may want to configure AWS Cloud Front or some other CDN. But these methods do not allow access control, and the only works if uploaded files are public.
Public CDNs are a great thing to have for optimising website load time, so they should be considered and used as long as possible.
If you have files that are private in nature, one way to retain the benefit of public CDN and still have some privacy is by making the URLs hard to guess. This is privacy by obfuscation, and for some applications may be good enough.
Now we are coming to proper authenticated file serving.
Lets talk about nginx for a bit, why do people put nginx in front of application servers? Its because backend services are generally resource heavy compared to nginx, and because browsers can be slow (unintentionally: slow network, or intentionally: as denial of service attack).
Consider a HTTP request, when nginx is put as a reverse proxy before backend, the request goes to nginx, and nginx then proxies the request to backend. Nginx holds the connection from client side, and waits for backend to respond. Nginx to backend request is fast, they are usually on same machine, or in same internal network.
Which means as soon as backend has generated the response, it is downloaded by nginx over a high speed connection, and then nginx slowly feeds the response to client at whatever speed client can consume. This frees up the backend as soon as possible for the next connection.
With reverse proxy in place, first solution could be for our backend to accept all requests, authenticate the request, fetch the content of the uploaded file from store, eg S3, and serve the content to nginx. Nginx will then feed the file to client.
The problem is that our backend is still doing a lot of work, it is downloading the content from S3, and then feeding it to nginx, and if it GBs of content, it has to be done in a streaming fashion, which our backend may not support (as realm currently, as of 25th June, 2020 does not). Further the CPU requirement, the RAM requirement etc would still be there, and whatever milliseconds backend is downloading from S3 is the millisecond wasted.
There is one more concern, do we want to download the file from S3 on every http request, or should we have caching? Of course we should have caching, but then our backend has to implement caching too.
Nginx already has support for both doing streaming S3 download, and caching. How do we plug authentication into nginx is the only question.
We have two different ways we can let backend (rust) do authentication, and let nginx take care of cached S3 download:
auth_request directive and
internal directive along with
rustfmtFor Some Section