Computing Resources

Sarmata Group Compute

Network Architecture

The computing infrastructure consists of the login node antemurale.acs.uni-duesseldorf.de and two compute servers sarmatia.acs.uni-duesseldorf.de and tujestpolin.acs.uni-duesseldorf.de. All external access must go through the login node.

Server Descriptions

  • antemurale: The login node, accessible from the external network. All user connections must authenticate through this server first.
  • sarmatia: Compute server accessible only from the internal network via slurm from the login node. Two 4090 GPUs.
  • tujestpolin: Compute server accessible only from the internal network via slurm from the login node. Up to six 5090 GPUs.
  • rema: Development server accessible only from the internal network via ssh from the login node. One 4070 GPU.

Usage

Slurm

In order to get a console or a job running, first log in to antemurale.

For a console, use:

ssh antemurale
srun --partition=gpu --gres=gpu:1 --pty bash

To submit a job for running in the background, either use a shell script:

#!/bin/bash
#SBATCH --job-name=my_training
#SBATCH --partition=gpu
#SBATCH --gres=gpu:2          # request 2 GPUs
#SBATCH --cpus-per-task=8
#SBATCH --mem=32G
#SBATCH --time=12:00:00
#SBATCH --output=logs/%j.out
#SBATCH --error=logs/%j.err

python train.py

or directly use the sbatch command:

sbatch --gres=gpu:2 --wrap="python train.py"

If you want to request a specific GPU, use:

sbatch --gres=gpu:rtx4090:1 --wrap="python train.py"

or

sbatch --gres=gpu:rtx5090:1 --wrap="python train.py"

A few useful commands:

squeue             # view job queue
scancel <jobid>    # cancel a job
sinfo              # node/partition status
sacct              # job accounting history

Currently, direct access to the compute nodes is possible via SSH as well, but it is not allowed to start jobs there. All compute must go through slurm! Jobs not started through slurm will be killed.

For the development server you can login via SSH. This is for development only: You can run short scripts that use the GPU, have Jupyter notebooks running here (if they do not take up the whole memory and make the GPU unusable for others), etc. No compute runs are allowed here.

Slurm Prioritization

Slurm prioritization is configured to use the fairshare mechanism. Slurm is configured with two account types: student and PhDs. Currently, Bachelor, Master and project students all together have 50% share, while PhD students have the other 50% share. Each user gets a fraction of the {student|phd} account share corresponding to how many other users this account has.

User Management

Users are centrally managed. For changing the password, you must use a custom command as follows only on the login node antemurale.

ssh antemurale # Password must be changed on the login node
/usr/local/bin/sarmata-change-password

You may have to wait up to an hour until the changed password is synced to the other servers.

File Storage

All files are stored centrally and you can access them from all compute nodes. Home datafolders might have a 1TB quota.

Discord Channel

For questions and requests use the Discord channel.

Setting Up SSH Key Authentication

SSH key authentication provides secure, password-free access to the servers. This section describes the complete setup process.

Step 1: Generate SSH Keys on Your Local Machine

If you don’t already have SSH keys, generate them:

# Generate an ED25519 key (recommended, modern)
ssh-keygen -t ed25519

# Or generate an RSA key (traditional, widely supported)
ssh-keygen -t rsa -b 4096 -N ""

When prompted:

  • Press Enter to accept the default file location (~/.ssh/id_ed25519 or ~/.ssh/id_rsa)
  • For the passphrase, either press Enter twice for no passphrase, or enter a passphrase for additional security

Step 2: Copy Your Public Key to the Login Node

Display your public key:

cat ~/.ssh/id_ed25519.pub
# or
cat ~/.ssh/id_rsa.pub

Log in to antemurale with your password:

ssh your_username@antemurale.acs.uni-duesseldorf.de

Add your public key to the authorized keys file:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys

Paste your public key (the entire line from the previous step), save, and set correct permissions:

chmod 600 ~/.ssh/authorized_keys

Exit and test the connection. You should now be able to log in without a password.

Configuring SSH for Direct Access

The SSH configuration file allows you to define connection shortcuts and automatically tunnel through the login node to reach compute servers.

Edit or create ~/.ssh/config on your local machine:

nano ~/.ssh/config

Add the following configuration:

# Login Node
Host antemurale
    HostName antemurale.acs.uni-duesseldorf.de
    User your_username
    IdentityFile ~/.ssh/id_ed25519

# Development Server rema
Host rema
    HostName rema.acs.uni-duesseldorf.de
    User your_username
    ProxyJump antemurale
    IdentityFile ~/.ssh/id_ed25519

Replace your_username with your actual username on the servers.

Python Environment

  • Installation instructions for uv, how to install packages. No apt.
  • Weights & Biases for logging.
  • tmux for running jobs while being disconnected.