Skip to content

Remote Visualization

Remote visualization enables you to run graphical applications on HoreKa in an interactive way, while making use of all its resources (including hardware-based 3D rendering on the GPUs) and only transfering the resulting, pre-rendered images to your local system. This means you neither have to move the data out of HoreKa, nor have local access to a desktop system that would be powerful enough to visualize that data.

We officially support two different methods for remote visualization:

  • JupyterHub Desktop (currently limited to software rendering for OpenGL)

  • The custom start_vnc_desktop script that supports hardware-accelerated rendering for OpenGL and advanced options

It is also possible to set up a VNC server manually, but this is largely out of the scope of this documentation and we can only provide limited support through the support channels. Basic information about manual setups can be found in the last chapter of this document.

Info

All our officially supported remote visualization solutions currently use TurboVNC as the VNC server and VirtualGL or Mesa3D for hardware-accelerated/software-based OpenGL rendering. While most software will usually run just fine, please note that these software packages might not support each and every X11 and OpenGL feature/and or extension. Therefore specific applications might not work at all or require additional steps.

JupyterHub Desktop

This solution allows you to get access to a remote desktop through a common web browser. No dedicated client installation is required. The default desktop environment is currently GNOME Shell, the remote desktop therefore supports all normal X11-based Linux GUI applications running on the cluster and also supports OpenGL.

Please note that Jupyter Desktop currently only supports software-based OpenGL rendering on the CPU and doesn't use any GPUs allocated on the node for hardware acceleration. While software rendering can be quite powerful on modern CPUs, this means that the JupyterHub Desktop offered on HoreKa is currently not suitable for visualization of large 3D models.

The "Desktop" launch button on JupyterHub

To log into the JupyterHub system and start a Desktop session, please follow the instructions in the JupyterHub section of this documentation, make sure to select the JupyterLab-Basemodule jupyter/extensions in the resource selection dialogue, and then click on Desktop in the created JupyterHub Notebook.

The JupyterHub Desktop session will continue to run on the cluster if the browser window is closed, and will automatically be reconnected on the next JupyterHub login. Please note that this also means that if you don't close the desktop session before closing the browser window, it will keep consuming resources until the initial runtime limit choosing during resource selection in the JupyterHub session is reached and the session is terminated by the batch system.

start_vnc_desktop

start_vnc_desktop is a custom-made script developed specifically for the HPC systems at SCC. The aim is to provide the same functionality as the JupyterHub Desktop session, but support more advanced use cases (e.g. hardware-accelerated OpenGL rendering) while still taking away most of the complexity involved with allocating resources on the cluster, setting up a password-protected Virtual Network Computing (VNC) server inside this session, maybe setting up hardware-accelerated OpenGL rendering, starting a full-fledged GNOME Shell desktop and then connecting to this VNC server with a client running on your desktop system.

The start_vnc_desktop script has to be executed on a login node and supports several command line options (the complete list can be queried with start_vnc_desktop --help):

Option Meaning
-n Number of nodes to allocate
--ppn Number of processors to allocate on each node
--num-gpu Number of GPUs to allocate on each node
-q Batch partition/queue to submit the job to
-t Walltime limit for the session
--hw-rendering Use hardware-accelerated OpenGL rendering on GPU(s)
--sw-rendering Force software-based OpenGL rendering on GPU(s)

By default the script will allocate a single node without any GPUs. So to start a simple session, just run start_vnc_desktop without any parameters.

Remote desktop environment with hardware-accelerated applications

As soon as the requested resources have been allocated, start_vnc_desktop will set up everything necessary on the allocated visualization node(s) and output instructions on how to connect to the new desktop session using a VNC client. Since VNC clients differ in functionality and functionality, the output is split into multiple parts and contains both quick instructions for officially supported clients (currently TurboVNC) and generic information for other clients.

Session lifetime

start_vnc_desktop currently uses the salloc command to allocate the visualization nodes. This means that after the script has set up everything completely and output instructions on how to connect, you will be dropped into a shell on one of the allocated compute nodes and not back on the login node! If you log out of this shell, the allocated job and therefore the whole remote desktop will be terminated. And if you want to log out of HoreKa completely, you therefore have to type exit twice.

The currently recommended client is TurboVNC. Contrary to some other popular VNC clients, it is not included with most Linux distributions and therefore almost always has to be downloaded and installed manually on all operating systems. But on the other hand it is one of the few clients that have a built-in gateway feature that supports SSH as a gateway out of the box.

The gateway feature is necessary because the compute node the remote desktop session is running on cannot be directly reached from outside the cluster system, but only through one of the login nodes. Before the VNC connection can be established, a port forwarding via SSH to one of the login nodes has to be created. TurboVNC automates this step.

The steps for connecting to the running remote desktop session on HoreKa depend on the operating system and setup on the client side:

  • Linux Terminal (preferred): This is the quickest and most convenient option of all. Open a Terminal and execute the command line provided by the start_vnc_desktop script in the "Hint for TurboVNC Viewer users (command line)" section, which should look something like /opt/TurboVNC/bin/vncviewer ExtSSH=1 Via=user@hk.scc.kit.edu Server=<server>:1 Password=<password> SecurityTypes=VNC. After execution, TurboVNC uses your local SSH client to log into HoreKa, so you will see the familiar prompts for your One-Time Password and service password, followed by the prompt for the one-time VNC password generated by the start_vnc_desktop script, and then the VNC connection will start.

  • Linux/Windows/Mac GUI: Start the TurboVNC client using the application launcher and then follow the instructions described in the "Hint for TurboVNC Viewer users (GUI)" section. Fill in all the required fields (see image below) in the GUI client and then click on Connect. You will then see the familiar prompts for your One-Time Password and service password, followed by the prompt for the one-time VNC password generated by the start_vnc_desktop script, and then the VNC connection will start.

  • Windows/Mac Terminal: The command line mentioned in the "Hint for TurboVNC Viewer users (command line)" section also works on Windows, Mac and possibly other platforms supported by TurboVNC, but you have to replace the path to the binary with the location where TurboVNC has been been installed (which differs a lot due to system language settings and choices during installation). For example on an English-language Windows installation, the path is usually "C:\Program Files\TurboVNC\vncviewer.bat".

Internal vs. external TurboVNC SSH client

Depending on your TurboVNC client version and operating system, the GUI might not work as expected. Especially you may want or even have to force it to use the operating system's own SSH client instead of its internal one. Disabling the internal SSH client is not possible using the GUI, it can only be done on the command line by passing ExtSSH=1 as can be seen in the Linux Terminal example above.

Important TurboVNC dialogues

Other clients

The information the start_vnc_desktop outputs gives for TurboVNC is also the same information that is necessary for other VNC clients. Since other clients usually don't have the built-in SSH gateway feature, you have to set up SSH port forwarding manually by performing the following steps:

  1. Run start_vnc_desktop and wait until it has allocated the resources and set everything up.

  2. Set up port forwarding manually by opening an additional terminal on your desktop and running ssh <user>@hk.scc.kit.edu -L 5901:<server>:5901. Replace the <user> with the SSH user value and <server> with the VNC server value from the output of start_vnc_desktop. You will then see the familiar prompts for your One-Time Password and service password and be logged into HoreKa. But because of the -L parameter, this SSH connection has now also created a port forwarding between your local system and the compute node where the VNC server runs in the background, with the login node acting as a forwarder.

  3. Start your VNC Client and use localhost:5901 as the server address. When asked for a password, enter the one-time password generated by start_vnc_desktop and mentioned under VNC Authentication in its output.

  4. Keep the SSH connection running until after you close the VNC client, otherwise the connection will drop.

If you are getting error messages like Connection refused from your VNC client or see messages like channel 3: open failed: connect failed: Connection refused, make sure that all parameters are correct.

Manual VNC server setup

We can only provide very limited support for all setups that don't use either JupyterHub Desktop or start_vnc_desktop in combination with TurboVNC, but if you have already set up your own VNC servers on other systems everything should work the same on HoreKa.

You can load the vis/turbovnc to get a vncserver binary. The section on VNC clients and port forwarding in this documentation can be of further help.

For security reasons, and also to avoid unintentional mistakes by successfully connecting to a different VNC server than you own, we ask you to always set up password authentication for your VNC servers.


Last update: September 28, 2023