Remote visualization enables you to run graphical applications on HoreKa in an interactive way, while making use of all its resources (including hardware-based 3D rendering on the GPUs) and only transfering the resulting, pre-rendered images to your local system. This means you neither have to move the data out of HoreKa, nor have local access to a desktop system that would be powerful enough to visualize that data.
We officially support two different methods for remote visualization:
JupyterHub Desktop (currently limited to software rendering for OpenGL)
start_vnc_desktopscript that supports hardware-accelerated rendering for OpenGL and advanced options
It is also possible to set up a VNC server manually, but this is largely out of the scope of this documentation and we can only provide limited support through the support channels. Basic information about manual setups can be found in the last chapter of this document.
All our officially supported remote visualization solutions currently use TurboVNC as the VNC server and VirtualGL or Mesa3D for hardware-accelerated/software-based OpenGL rendering. While most software will usually run just fine, please note that these software packages might not support each and every X11 and OpenGL feature/and or extension. Therefore specific applications might not work at all or require additional steps.
This solution allows you to get access to a remote desktop through a common web browser. No dedicated client installation is required. The default desktop environment is currently GNOME Shell, the remote desktop therefore supports all normal X11-based Linux GUI applications running on the cluster and also supports OpenGL.
Please note that Jupyter Desktop currently only supports software-based OpenGL rendering on the CPU and doesn't use any GPUs allocated on the node for hardware acceleration. While software rendering can be quite powerful on modern CPUs, this means that the JupyterHub Desktop offered on HoreKa is currently not suitable for visualization of large 3D models.
To log into the JupyterHub system and start a Desktop session, please follow the instructions in the JupyterHub section of this documentation, make sure to select the JupyterLab-Basemodule
jupyter/extensions in the resource selection dialogue, and then click on Desktop in the created JupyterHub Notebook.
The JupyterHub Desktop session will continue to run on the cluster if the browser window is closed, and will automatically be reconnected on the next JupyterHub login. Please note that this also means that if you don't close the desktop session before closing the browser window, it will keep consuming resources until the initial runtime limit choosing during resource selection in the JupyterHub session is reached and the session is terminated by the batch system.
start_vnc_desktop is a custom-made script developed specifically for the HPC systems at SCC. The aim is to provide the same functionality as the JupyterHub Desktop session, but support more advanced use cases (e.g. hardware-accelerated OpenGL rendering) while still taking away most of the complexity involved with allocating resources on the cluster, setting up a password-protected Virtual Network Computing (VNC) server inside this session, maybe setting up hardware-accelerated OpenGL rendering, starting a full-fledged GNOME Shell desktop and then connecting to this VNC server with a client running on your desktop system.
start_vnc_desktop script has to be executed on a login node and supports several command line options (the complete list can be queried with
|Number of nodes to allocate
|Number of processors to allocate on each node
|Number of GPUs to allocate on each node
|Batch partition/queue to submit the job to
|Walltime limit for the session
|Use hardware-accelerated OpenGL rendering on GPU(s)
|Force software-based OpenGL rendering on GPU(s)
By default the script will allocate a single node without any GPUs. So to start a simple session, just run
start_vnc_desktop without any parameters.
As soon as the requested resources have been allocated,
start_vnc_desktop will set up everything necessary on the allocated visualization node(s) and output instructions on how to connect to the new desktop session using a VNC client. Since VNC clients differ in functionality and functionality, the output is split into multiple parts and contains both quick instructions for officially supported clients (currently TurboVNC) and generic information for other clients.
start_vnc_desktop currently uses the
salloc command to allocate the visualization nodes. This means that after the script has set up everything completely and output instructions on how to connect, you will be dropped into a shell on one of the allocated compute nodes and not back on the login node! If you log out of this shell, the allocated job and therefore the whole remote desktop will be terminated. And if you want to log out of HoreKa completely, you therefore have to type
Recommended client: TurboVNC¶
The currently recommended client is TurboVNC. Contrary to some other popular VNC clients, it is not included with most Linux distributions and therefore almost always has to be downloaded and installed manually on all operating systems. But on the other hand it is one of the few clients that have a built-in gateway feature that supports SSH as a gateway out of the box.
The gateway feature is necessary because the compute node the remote desktop session is running on cannot be directly reached from outside the cluster system, but only through one of the login nodes. Before the VNC connection can be established, a port forwarding via SSH to one of the login nodes has to be created. TurboVNC automates this step.
The steps for connecting to the running remote desktop session on HoreKa depend on the operating system and setup on the client side:
Linux Terminal (preferred): This is the quickest and most convenient option of all. Open a Terminal and execute the command line provided by the
start_vnc_desktopscript in the "Hint for TurboVNC Viewer users (command line)" section, which should look something like
/opt/TurboVNC/bin/vncviewer ExtSSH=1 Viaemail@example.com Server=<server>:1 Password=<password> SecurityTypes=VNC. After execution, TurboVNC uses your local SSH client to log into HoreKa, so you will see the familiar prompts for your One-Time Password and service password, followed by the prompt for the one-time VNC password generated by the
start_vnc_desktopscript, and then the VNC connection will start.
Linux/Windows/Mac GUI: Start the TurboVNC client using the application launcher and then follow the instructions described in the "Hint for TurboVNC Viewer users (GUI)" section. Fill in all the required fields (see image below) in the GUI client and then click on Connect. You will then see the familiar prompts for your One-Time Password and service password, followed by the prompt for the one-time VNC password generated by the
start_vnc_desktopscript, and then the VNC connection will start.
Windows/Mac Terminal: The command line mentioned in the "Hint for TurboVNC Viewer users (command line)" section also works on Windows, Mac and possibly other platforms supported by TurboVNC, but you have to replace the path to the binary with the location where TurboVNC has been been installed (which differs a lot due to system language settings and choices during installation). For example on an English-language Windows installation, the path is usually
Internal vs. external TurboVNC SSH client
Depending on your TurboVNC client version and operating system, the GUI might not work as expected. Especially you may want or even have to force it to use the operating system's own SSH client instead of its internal one. Disabling the internal SSH client is not possible using the GUI, it can only be done on the command line by passing
ExtSSH=1 as can be seen in the Linux Terminal example above.
The information the
start_vnc_desktop outputs gives for TurboVNC is also the same information that is necessary for other VNC clients. Since other clients usually don't have the built-in SSH gateway feature, you have to set up SSH port forwarding manually by performing the following steps:
start_vnc_desktopand wait until it has allocated the resources and set everything up.
Set up port forwarding manually by opening an additional terminal on your desktop and running
ssh <user>@hk.scc.kit.edu -L 5901:<server>:5901. Replace the
<user>with the SSH user value and
<server>with the VNC server value from the output of
start_vnc_desktop. You will then see the familiar prompts for your One-Time Password and service password and be logged into HoreKa. But because of the
-Lparameter, this SSH connection has now also created a port forwarding between your local system and the compute node where the VNC server runs in the background, with the login node acting as a forwarder.
Start your VNC Client and use
localhost:5901as the server address. When asked for a password, enter the one-time password generated by
start_vnc_desktopand mentioned under VNC Authentication in its output.
Keep the SSH connection running until after you close the VNC client, otherwise the connection will drop.
If you are getting error messages like
Connection refused from your VNC client or see messages like
channel 3: open failed: connect failed: Connection refused, make sure that all parameters are correct.
Manual VNC server setup¶
We can only provide very limited support for all setups that don't use either JupyterHub Desktop or
start_vnc_desktop in combination with TurboVNC, but if you have already set up your own VNC servers on other systems everything should work the same on HoreKa.
You can load the
vis/turbovnc to get a
vncserver binary. The section on VNC clients and port forwarding in this documentation can be of further help.
For security reasons, and also to avoid unintentional mistakes by successfully connecting to a different VNC server than you own, we ask you to always set up password authentication for your VNC servers.