orderly remotes

Rich FitzJohn


Orderly supports a centralised workflow, where a central server holds the canonical version of a report. In order to make this workflow work, we need a number of things:

To make this workflow work, we have developed some additional software.

First, we have a docker-based server system called OrderlyWeb which can be deployed onto a server. We will communicate with this server over https using a package orderlyweb which will follow orderly onto CRAN.


The OrderlyWeb server software provides a number of things

  1. A persistent copy of R running orderly that can run reports as needed using the orderly.server::orderly_runner function
  2. A web server that exposes a user-friendly web portal showing versions of orderly reports and an http api (this application is written in Kotlin)
  3. A reverse proxy used to secure the application with https

A diagram may help show how the pieces fit together

Getting OrderlyWeb installed requires a little work, and you will need access to an appropriate server running docker, our python-based deployment tool, and TLS certificates for the server that you are using. We recommend using vault for storing certificates and any secrets used in deployment.

OrderlyWeb will use GitHub for authentication, the setup for which is described here.

Configuring orderly to talk to OrderlyWeb

Once OrderlyWeb is running, we can tell orderly about it by adding an appropriate setting to the orderly_config.yml file. For example, if you had one testing and one production instance

    driver: orderlyweb::orderlyweb_remote
      host: testing.example.com
      port: 443
      token: $GITHUB_TOKEN
    driver: orderlyweb::orderlyweb_remote
      host: production.example.com
      port: 443
      token: $GITHUB_TOKEN

where $GITHUB_TOKEN is an environment variable that holds a GitHub token with “user” scope. The testing remote is listed first, and so will be the default one used in any remote-using command.

Interacting with the remote server

Several commands can interact with the remote server:

Future developments

We hope that this system will become easier to deploy in future. We use it internally on two projects with orderly archives that hold gigabytes of analyses, and it has evolved to meet our needs. If you think it might work for you but can’t get started, please get in touch.

Deploying a remote server

We recommend using HashiCorp’s vault for storing secrets (of which several are used here), and our tooling is designed to make this easy.

You will need to set up and/or gather a few things beforehand:

Create an orderly-web configuration; see here for an example and here for a highly annotated example. A somewhat stripped down version of this looks like:

  addr: https://vault.dide.ic.ac.uk:8200
    method: github

container_prefix: orderly_web
network: orderlyweb_nw

  orderly: orderly_volume
  proxy_logs: orderlyweb_proxy_logs
  css: orderlywebweb_css

    repo: yourorg
    name: orderly
    tag: latest
    source: clone
    url: git@github.com:yourorg/orderly-reports
    public: VAULT:secret/deploy-key:public
    private: VAULT:secret/deploy-key:private

    repo: vimc
    name: orderly-web
    tag: master
    migrate: orderlyweb-migrate
    admin: orderly-web-user-cli
    css-generator: orderly-web-css-generator
  url: https://orderly.example.com
  dev_mode: false
  port: 8888
  name: Outputs
  email: you@example.com
    github_org: yourorg
    github_team: yourteam
      id: VAULT:secret/oauth/real:id
      secret: VAULT:secret/oauth/real:secret
    fine_grained: false
    montagu: false

  enabled: true
    certificate: VAULT:secret/proxy/ssl_certificate:value
    key: VAULT:secret/ncov/ssl_private_key:value
  hostname: ncov.dide.ic.ac.uk
  port_http: 80
  port_https: 443
    repo: vimc
    name: orderly-web-proxy
    tag: master

You need to install the orderly-web deployment scripts from PyPi

pip3 install --user orderly-web

With all that done, running

orderly-web start .

should bring everything up!