Deployment guide#

This guide explains how to deploy and manage AiiDAlab servers for multiple users. See AiiDAlab launch for how to run the AiiDAlab docker container locally (e.g. for testing purposes).

Multi-user deployment#

Single-server deployment#

For medium-sized deployments (a handful of users), an AiiDAlab multi-user server can deployed on a single (virtual or bare-metal) machine.

Since deploying a multi-user server requires additional packages, such as JupyterHub, DockerSpawner, Apache HTTP Server, and Docker, we provide the aiidalab-server Ansible role which automates the setup of the server. Please see the corresponding git repository for more information.

We also provide instructions for deploy with microk8s in a single-server setup. This is a good option if you want to deploy AiiDAlab on a single machine, but expect a moderate amount of users (<50).

Full kubernetes deployment#

If you are expecting a large number of users (>50), consider deploying AiiDAlab on a scalable Kubernetes cluster. We provide instructions and deployment k8s scripts for this use case.

Maintaining an app registry#

By default, AiiDAlab is configured to use the AiiDAlab application registry maintained by the AiiDAlab team for installing new apps. We encourage organizations to use this registry and register their apps there, unless they have specific reasons for needing to maintain their own registry. Here we describe how to maintain and publish a dedicated AiiDAlab apps registry.

Create the new registry

Tip

Instead of creating a new repository from scratch, you can also fork the official AiiDAlab registry repository and adjust it to your needs.

To create a registry, first make sure to install the aiidalab package on the machine that you want to build the registry on.

$ pip install aiidalab

Note

For testing, you could build the registry on a running AiiDAlab instance, in this case the aiidalab package is already installed.

Next, create a directory for your registry repository, e.g., with:

~$ mkdir my-aiidalab-registry
~$ cd my-aiidalab-registry/
~/my-aiidalab-registry$

Then create two files: a apps.yaml and a categories.yaml file. The first one contains our index of registered applications and the second one the available categories for apps in this registry.

The definition for entries in the apps.yaml file are described in detail in the documentation on app registration. Example:

apps.yaml#

 hello-world:
   releases:
     - "git+https://github.com/aiidalab/aiidalab-hello-world@master:"

The categories.yaml file is a dictionary where, the key of each entry is the category’s id and the value consist of a title and a description field. Example:

categories.yaml#

 classical:
   description: Apps for performing calculations based on classical/empirical force
     fields.
   title: Classical
 quantum:
   description: Apps for performing quantum-mechanical calculations.
   title: Quantum

Build the new registry

Make sure to switch into the directory in which you previously created the apps.yaml and categories.yaml files, then build the registry with:

~/my-aiidalab-registry$ aiidalab registry build

By default, this will create the registry website and API pages in the ./build/ directory.

You can check whether the registry was successfully built by opening the ./build/index.html page directly in your browser or by inspecting the ./build/api/v1/apps_index.json file.

Serve the new registry

Note

The official AiiDAlab registry repository is automatically published on GitHub pages via a GitHub actions integration. If you forked the repository, it should automatically publish the registry under your GitHub pages domain.

The registry is generated via static HTML pages and can therefore be easily published with any standard web server. For a quick test, you could use the Python built-in web server, with:

~/my-aiidalab-registry$ cd ./build/
~/my-aiidalab-registry/build$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ..

This will launch a simple web server, which is reachable via the address: http://0.0.0.0:8000.

You can test whether the registry is reachable by executing:

~$ curl localhost:8000/api/v1/apps_index.json

Tip

You can use ngrok to temporarily server the registry over the internet for testing.

First, install ngrok, then start your local web server as described above, and in a separate terminal run ngrok http 8000. This will give you a public address that you can use as the base URL for your registry address.

Configure AiiDAlab to use the new registry

To instruct AiiDAlab to use a different registry, you can either create a configuration file called aiidalab.toml in the user’s home directory or set the AIIDALAB_REGISTRY environment variable. The former is especially suitable for testing, while the latter is probably the better approach to specify a dedicated registry organization-wide.

Configuration file

To instruct an AiiDAlab instance to use this registry, simply logon to AiiDAlab, and then create a file called aiidalab.toml in the home directory, with the following content:

~/aiidalab.toml#

registry = "http://localhost:8000/api/v1"

Where you replace the URL with the one where you serve the newly created registry.

Environment variable (with Docker)

The registry can be specified by setting the AIIDALAB_REGISTRY environment variable. For example, to pass the variable when starting the container, add the following argument:

-e AIIDALAB_REGISTRY=http://localhost:8000/api/v1

To verify that the new registry is being used, open the terminal and run:

$ aiidalab info
AiiDAlab, version 21.10.0
Apps path:      /home/aiida/apps
Apps registry:  http://localhost:8000/api/v1

The value behind “Apps registry” should point to the just configured address.

Test the new registry

Try to search for registered applications by opening the App Store in AiiDAlab (), or by listing the registered apps (and their releases) on the command line with:

~$ aiidalab search
Collecting apps and releases... Done.
hello-world==v1.1.0

Troubleshooting#

Slow I/O#

When running AiiDAlab on disks through OpenStack’s block storage, observe the following command for a few minutes:

watch -n 0.1 "ps axu| awk '{print \$8, \"   \", \$11}' | sort | head -n 10"

Almost all processes should be in the S state. If a process stays in the D state for a longer time, it is most likely waiting for slow I/O.