Skip to content

Hardware Overview

The Future Technologies Partition is currently split into two different cluster installations with different hardware components, different cluster management software stacks and different login nodes.

FTP is a distributed memory parallel computer consisting of multiple individual servers called "nodes". Nodes are divided into two clusters, one supports ARM instruction set and the other supports x86. Each node has either two Intel Xeon processors or up to two ARM processors, at least 32 GB of local memory, local NVMe or SATA SSD disks and two high-performance network adapters. All nodes are connected by an extremely fast, low-latency InfiniBand 4X HDR interconnect. In addition two large parallel file systems are connected to FTP.

The operating system installed on every node is Red Hat Enterprise Linux (RHEL) 8.x. On top of this operating system, a set of (open source) software components like Slurm has been installed. Some of these components are of special interest to end users and are briefly discussed here. Others are mostly just of importance to system administrators and are thus not covered by this documentation.

The different server systems in FTP have different roles and offer different services.

Login Nodes

The login nodes are the only nodes directly accessible to end users. These nodes can be used for interactive logins, file management, software development and interactive pre- and postprocessing. Two nodes are dedicated as login nodes.

Compute Nodes

The majority of the nodes (25 out of 29) are dedicated to computations. These nodes are not directly accessible to users. Instead the calculations have to be submitted to a so-called batch system. The batch system manages all compute nodes and executes the queued jobs depending on their priority and as soon as the required resources become available. A single job may use hundreds of compute nodes and many thousand CPU cores at once.

Administrative Service Nodes

Some nodes provide additional services like resource management, external network connections, monitoring, security etc. These nodes can only be accessed by system administrators.

FTP-A64 ARM cluster

Login node ARM-A100 A64FX nodes Dual Socket ARM Altra Max
No. Nodes 1 4 8 6
CPU 1x Ampere Altra Q80-33 1x Ampere Altra Q80-33 1x Fujitsu A64FX 2x Ampere Altra Max M128-30
Total Cores 80 80 48 256
CPU Clock 3.3 GHz 3.3 GHz 1.8 GHz 3 GHz
Instruction Set ARMv8.2+ ARMv8.2+ ARMv8.2-A + SVE ARMv8.2+
Memory 512 GiB 512 GiB 32 GiB 512 GiB
Accelerator Type - NVIDIA A100 PCIe 40GB - -
No. Accelerators - 2 - -
Disk (useable) 890 GB 890 GB 372 GB 1.8 TB
Disk type SATA-SSD SATA-SSD NVMe SATA-SSD
InfiniBand 200 GBit/s BlueField 2 200 GBit/s BlueField 2 100GBit/s ConnectX-6 200 GBit/s ConnectX-6

FTP-X86 cluster

Login node Intel Cascade Lake + NVIDIA V100 AMD EPYC Milan + Graphcore AMD EPYC Milan + MI100 AMD EPYC Milan + MI210 AMD EPYC Milan + MI250
No. Nodes 1 2 1 1 1 2
CPU 2x Intel Xeon Gold 6230 2x Intel Xeon Gold 6230 2x AMD EPYC 7543 2x AMD EPYC 7543 2x AMD EPYC 7543 2x AMD EPYC 7713
Sockets 2 2 2 2 2 2
Total Cores 40 40 64 64 64 128
Total Threads 40 40 128 128 128 128
CPU Base Clock 2.1 GHz 2.1 GHz 2.8 GHz 2.8 GHz 2.8 GHz 2.0 GHz
Instruction Set x86_64 x86_64 x86_64 x86_64 x86_64 x86_64
Memory 192 GB 192 GB 512 GB 512 GB 512 GB 1 TB
Accelerator type - NVIDIA V100 Graphcore IPU-M2000 AMD MI100 AMD MI210 AMD MI250
No. Accelerators - 2 4 (=16 IPUs) 4 4 4
Accelerator TFLOPS (FP64) - 7 - 11,5 22,6 45,3
Accelerator TFLOPS (FP32) - 14 - 23 22,6 45,3
Accelerator TFLOPS (FP16) - 28,2 250 184,6 181 362,1
Accelerator Mem b/w (GB/s) - 900 1228 1600 3276
Disk 890GB 890 GB 3 TB 3 TB 3 TB 1.8 TB
Disk type NVMe NVMe NVMe NVMe NVMe NVMe

Interconnect

An important component of FTP is the InfiniBand 4x HDR 200 GBit/s interconnect. All nodes are attached to this high-throughput, very low-latency (~ 1 microsecond) network. InfiniBand is ideal for communication intensive applications and applications that e.g. perform a lot of collective MPI communications.


Last update: September 27, 2023