Quarto on HPC

What is Quarto?

From the Quarto webpage, it is

An open-source scientific and technical publishing system

Quarto can be used to create documentation pages, scientific notebooks, professional presentations, and websites (like this one!).

This system is useful for creating notes, particularly these containing Python/R code. And in contrast with jupyter notebooks, it offers a wide variety of ways to share these notes.

Install Quarto on a remote server (HPC)

In this post, two options are presented to install Quarto on a HPC server. The first option installs Quarto in a conda virtual environment. You can use this option if you don’t need or want to perform additional configuration and/or you are planning to use a single virtual environment for authoring your documents. The second option installs Quarto in your home directory. That works great from creating code-less documents, to authoring documents that require different virtual environments (one per document).

Option 1. Install Quarto in a conda virtual environment

For this option you should have conda installed in your home directory. I recommend using miniforge if you don’t have conda already.

Activate the virtual environment from where you will use Quarto, i.e. conda activate my-venv (change my-venv with the actual name of your environment).

Note

If you don’t have a virtual environment, start by creating one instead of installing packages in the base environment.

Install quarto from the conda-forge channel as follows.

conda install -c conda-forge quarto

You are ready to render and preview documents from the command line!

Option 2. Install Quarto on your home directory

Download the Quarto source code for Linux x86 Tarball from their official website into your home directory.

You can download the source code by copying the address link of the download, and using wget to download it directly into your home directory.

Tip

Download the source code with wget using the following command (change X.X.X to the latest version available).

wget https://github.com/quarto-dev/quarto-cli/releases/download/vX.X.X/quarto-X.X.X-linux-amd64.tar.gz

Tip

If you are installing it in your PC, you can use the installer that matches your OS.

Decompress the source code in a location of your preference within your home directory, like /home/$USER/opt/.

Tip

You might need to create a /home/$USER/opt directory if it doesn’t already exist.

mkdir /home/$USER/opt

Tip

Decompress the source code with tar by executing the following command (change the version X.X.X to what you are using first).

tar -C ~/opt -xvzf quarto-X.X.X-linux-amd64.tar.gz

Create a symbolic link to the Quarto executable inside your /home/$USER/bin directory.

Tip

You might need to create a /home/$USER/bin directory if there is not one already as follows.

mkdir /home/$USER/bin

In general, your /home/$USER/bin directory should already be in your PATH environment variable, even if the directory didn’t already exist.

Tip

Create a symbolic link to the quarto executable with the following command (again, change the version X.X.X first).

ln -s /home/$USER/opt/quarto-X.X.X/bin/quarto /home/$USER/bin/quarto

Check that quarto is installed by executing the quarto --version command.

Render and preview documents

You can render and preview your Quarto markdowns by using the following command from the directory where your .qmd files are located.

quarto preview --no-browser --no-watch-inputs

That will render your documents from the markdowns. For html, revealjs, or websites formats, it will start a local server and provide a link that you can paste in your browser to visualize the result.

Authoring documents with (Python) code cells

To create documents that contain executable Python cells you need jupyter installed. If you already have jupyter in your base environment, check if it is being detected by Quarto with the following command.

quarto check jupyter

That shows the jupyter kernel that will be used to execute the Python code cells in your documents.

Caution

If you don’t already have jupyter installed in your base environment, Quarto will prompt you to install it.

Path: /home/$USER/miniforge-pypy3/bin/python
Jupyter: (None)

Jupyter is not available in this Python installation.
Install with conda install jupyter

However, you can use jupyter from an existing conda virtual environment instead of installing it in base. In case you don’t have jupyter installed, activate your virtual environment and install jupyter as follows (replace my-venv with your environment name).

conda activate my-venv
conda install -c conda-forge jupyter

Then override the QUARTO_PYTHON variable to point to the python executable in that virtual environment as follows (replace my-venv with the name of your virtual environment).

export QUARTO_PYTHON=[...]/envs/my-venv/bin/python

Replace [...] in the above command to the absolute path of your conda installation. In my case it looks like /home/$USER/miniforge-pypy3. You can also add that command into your ~/.bashrc file, so your virtual environment is selected by default when you open a new terminal.

Select the jupyter kernel

Note

If you are using Option 1 for installing Quarto in a conda environment, you won’t need to specify the kernel since it will use the one available in that virtual environment. So, you can skip this step.

If you are using Quarto with multiple virtual environments, you can select a specific jupyter kernel for each document. To do that, add jupyter: my-venv to the header of your .qmd file or replace the exiting jupyter: pythonX option if it is already set. Then, add the jupyter kernel of that virtual environment to the list of available kernels with the following commands.

conda activate my-venv
python -m ipykernel install --user --name "my-venv"

Cache jupyter code cells

Every time you render a document with Quarto, all code blocks are executed. In the case your code blocks are time-consuming, you might want to use the cache option in quarto documents. The cache option allows to execute cells only when a cell in the document is modified.

Note

That means that modifying the text around cells won’t trigger the execution of the code cells. However, if at least one cell is modified, all cells will be executed.

To use the cache option, install jupyter-cache as follows.

conda activate my-venv
conda install -c conda-forge jupyter-cache

Note

If you overrode the QUARTO_PYTHON variable with a virtual environment, install jupyter-cache only in that environment.

Integration with VSCode

You can author Quarto markdowns using VSCode, so you have a nice interface with code highlighting. If you have access to the HPC server, you can use the vscode module to create documents. That is particularly useful when creating jupyter-like notebooks, since you can execute the code directly on the HPC environment.

Install the Quarto extension

The VScode Quarto extension adds markdowns highlighting and commands autocompletion for authoring .qmd files. It can render and preview the quarto markdown documents by hitting Ctrl Shift K (Cmd Shift K on Mac). You can search for the Quarto extension in the Extensions tab (Ctrl Shift X/Cmd Shift X on Mac).

Important

Restart VSCode to make sure that the extension recognizes the Quarto installation in your virtual environments.

Warning

If you followed Option 1 to install Quarto in a virtual environment the keyboard shortcut might not work. That happens because installing Quarto through conda won’t make it visible to where the extension expects it to be.

Additional resources

Now that you have Quarto installed in your home directory, you can start creating your own markdown .qmd documents. The Quarto authoring guides are the perfect place for learning!