*[DediGPU](https://dedigpu.com/) — Markdown mirror of [https://dedigpu.com/templates](https://dedigpu.com/templates) · updated 2026-09-02 · index for LLMs: [llms.txt](https://dedigpu.com/llms.txt) · everything: [llms-full.txt](https://dedigpu.com/llms-full.txt)*

Templates

# Your stack, running before you log in.

Pick an operating system and a template on the order form. Drivers, toolkits and the service are installed before first boot; serving templates pull the model you name. Root over SSH from the first minute, and nothing you cannot replace.

[Order a server](https://dedigpu.com/signup) [Templates and ports, in the docs](https://dedigpu.com/docs/templates-and-ports)

- 14 templates, 5 operating systems
- Model pre-loaded on serving templates
- systemd services, root everywhere

Operating systems

## Five systems, driver included.

| System | Notes | Cards | Licence | Supported until |
| --- | --- | --- | --- | --- |
| **Ubuntu 24.04 LTS** | Kernel 6.8 · the default for every template | NVIDIA, AMD · sxm, pcie, consumer | included | April 2029 |
| **Ubuntu 22.04 LTS** | Kernel 5.15 HWE · for stacks pinned to CUDA 11.8 / 12.1 | NVIDIA, AMD · sxm, pcie, consumer | included | April 2027 |
| **Debian 12** | Bookworm · minimal, driver from the vendor repository | NVIDIA, AMD · sxm, pcie, consumer | included | June 2028 |
| **Rocky Linux 9** | RHEL-compatible · Slurm, enterprise and HPC stacks | NVIDIA, AMD · sxm, pcie, consumer | included | May 2032 |
| **Windows Server 2022** | Standard · RDP on :3389 · licence billed at our rate | NVIDIA · pcie, consumer | $28/mo | October 2031 |

NVIDIA driver 570 with CUDA 12.8 on Linux, ROCm 6.4 on MI300X, Studio or data-centre driver on Windows. [Drivers and pinned toolkits](https://dedigpu.com/docs/operating-systems).

Base system

## The driver and nothing else.

- ### Bare OS + driver
  NVIDIA driver 570 with the CUDA 12.8 toolkit, or ROCm 6.4 on AMD. Nothing else running.
  - **Listens on:** nothing
  - **Runs on:** NVIDIA · AMD · Linux · Windows
  - **First check:** `nvidia-smi`

Frameworks

## Frameworks with the toolchain in place.

- ### PyTorch 2.7
  CUDA 12.8 or ROCm 6.4 build, cuDNN, torchvision, torchaudio, Jupyter Lab on :8888.
  - **Listens on:** Jupyter Lab :8888
  - **Runs on:** NVIDIA · AMD · Linux
  - **First check:** `python3 -c 'import torch; print(torch.cuda.device_count())'`
- ### TensorFlow 2.18 + JAX 0.5
  CUDA 12.8, XLA, Keras 3, Jupyter Lab on :8888.
  - **Listens on:** Jupyter Lab :8888
  - **Runs on:** NVIDIA · Linux
  - **First check:** `python3 -c 'import jax; print(jax.devices())'`
- ### Jupyter Lab
  Lab on :8888 with PyTorch, JAX and TensorFlow kernels side by side.
  - **Listens on:** Jupyter Lab :8888
  - **Runs on:** NVIDIA · Linux
  - **First check:** `jupyter server list`

LLM serving

## An API on a port, model already pulled.

- ### vLLM 0.9 *+ model*
  OpenAI-compatible server on :8000, tensor-parallel across every card, model pulled before first boot.
  - **Listens on:** OpenAI-compatible API :8000
  - **Runs on:** NVIDIA · AMD · Linux
  - **First check:** `curl localhost:8000/v1/models`
- ### SGLang 0.4 *+ model*
  Structured generation with RadixAttention on :30000, model pulled before first boot.
  - **Listens on:** SGLang API :30000
  - **Runs on:** NVIDIA · Linux
  - **First check:** `curl localhost:30000/health`
- ### Text Generation Inference 3 *+ model*
  Hugging Face TGI on :8080 with the Messages API, model pulled before first boot.
  - **Listens on:** TGI API :8080
  - **Runs on:** NVIDIA · AMD · Linux
  - **First check:** `curl localhost:8080/info`
- ### Ollama + Open WebUI *+ model*
  Ollama on :11434 and a chat UI on :3000. Name an Ollama tag such as llama3.1:70b and it is pulled first.
  - **Listens on:** Ollama API :11434, Open WebUI :3000
  - **Runs on:** NVIDIA · AMD · Linux
  - **First check:** `ollama list`

Diffusion and rendering

## Diffusion, video and rendering.

- ### ComfyUI
  ComfyUI on :8188 with ComfyUI-Manager, Flux and SDXL loaders. No checkpoints bundled — bring yours.
  - **Listens on:** ComfyUI :8188
  - **Runs on:** NVIDIA · Linux · Windows
  - **First check:** `systemctl status comfyui`
- ### Stable Diffusion WebUI Forge
  Forge on :7860 with ControlNet and the usual extensions.
  - **Listens on:** Forge :7860
  - **Runs on:** NVIDIA · Linux · Windows
  - **First check:** `systemctl status forge`
- ### Render node
  Blender 4.4 headless with Cycles and OptiX, ready for your Octane, Redshift or V-Ray licence.
  - **Listens on:** nothing
  - **Runs on:** NVIDIA · Linux · Windows
  - **First check:** `blender -b --version`

Orchestration

## Containers, Kubernetes, Slurm.

- ### Docker + GPU toolkit
  Docker 27 with the compose plugin and the NVIDIA Container Toolkit, or the ROCm runtime on AMD.
  - **Listens on:** nothing
  - **Runs on:** NVIDIA · AMD · Linux
  - **First check:** `docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi`
- ### Kubernetes node (k3s)
  Single-node k3s with the NVIDIA GPU Operator. Join it to your cluster or run it alone.
  - **Listens on:** Kubernetes API :6443
  - **Runs on:** NVIDIA · Linux
  - **First check:** `kubectl get nodes -o wide`
- ### Slurm worker
  slurmd 24.11 with Pyxis and Enroot; give us your controller address after boot and it joins.
  - **Listens on:** nothing
  - **Runs on:** NVIDIA · Linux
  - **First check:** `sinfo`

Pre-loaded model

## Name it on the order form. It is there at first boot.

vLLM, SGLang, TGI and Ollama take a model and a precision when you order. The weights are pulled while the server is provisioned, and the order form checks the fit — parameters × bytes per parameter × 1.2 must be at most the server's memory — so a 70B model at BF16 is refused on a single 80 GB card rather than failing at boot.

| Precision | When | Memory |
| --- | --- | --- |
| **BF16 / FP16** | full quality · 2 bytes per parameter | 2 B/param |
| **FP8** | near-lossless · 1 byte · Ada, Hopper, Blackwell, MI300X | 1 B/param |
| **INT8** | 1 byte per parameter · any card | 1 B/param |
| **INT4 (AWQ / GPTQ)** | 0.5 byte per parameter · smallest footprint | 0.5 B/param |

Under the hood

## A service you own.

```
systemctl status vllm            # or comfyui, jupyter, ollama, tgi, sglang, forge
journalctl -u vllm -f            # logs
nano /opt/dedigpu/vllm/env       # MODEL, DTYPE, MAX_MODEL_LEN, API_KEY…
systemctl restart vllm
```

Weights cache under `/opt/dedigpu/models` on the local NVMe. Change the model in the env file and restart; the [docs](https://dedigpu.com/docs/preloading-a-model) cover gated repositories and quantised checkpoints.

Questions

## About templates.

**Can I change or remove a template after boot?**

Yes. Each template is a systemd service under /opt/dedigpu/<template> with its arguments in an env file. Stop it, edit it, replace it, or re-image the server with another template from the console.

**Which templates accept a pre-loaded model?**

vLLM, SGLang, TGI and Ollama. You name a Hugging Face repository or an Ollama tag and a precision on the order form; the weights are pulled before first boot and the order form refuses a model that would not fit in memory.

**Are the ports open to the Internet?**

Yes, there is no firewall in front of the server. Jupyter and ComfyUI start with a token or password shown on the server page; vLLM, SGLang, TGI and Ollama start without authentication. Reach them through an SSH tunnel or put a key in front before you share the address.

**Which templates run on AMD MI300X?**

Bare ROCm, PyTorch, vLLM, TGI, Ollama and Docker with the ROCm runtime. TensorFlow, SGLang, the diffusion templates, k3s and Slurm are NVIDIA only for now.

**Which templates run on Windows Server?**

Bare driver, ComfyUI, Stable Diffusion WebUI Forge and the render node, on PCIe and workstation cards. Frameworks and serving templates are Linux only.

## Pick a card, pick a template, name the model.

The server comes up with the API answering. Billed at cost, one term at a time.

[See the servers](https://dedigpu.com/gpu) [Create an account](https://dedigpu.com/signup)