The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
ggmlR is GPU-first: Vulkan is auto-detected at install time and used by default when available. All APIs (sequential, functional, autograd, ONNX) fall back to CPU transparently when no GPU is present.
if (ggml_vulkan_available()) {
cat("Vulkan is available\n")
ggml_vulkan_status() # print device list and properties
} else {
cat("No Vulkan GPU — running on CPU\n")
}
n <- ggml_vulkan_device_count()
cat("Vulkan device count:", n, "\n")# Low-level device registry (all backends including CPU)
ggml_backend_load_all()
n_dev <- ggml_backend_dev_count()
for (i in seq_len(n_dev)) {
dev <- ggml_backend_dev_get(i - 1L) # 0-based
name <- ggml_backend_dev_name(dev)
desc <- ggml_backend_dev_description(dev)
mem <- ggml_backend_dev_memory(dev)
cat(sprintf("[%d] %s — %s\n", i, name, desc))
cat(sprintf(" %.1f GB free / %.1f GB total\n",
mem["free"] / 1e9, mem["total"] / 1e9))
}Full example: inst/examples/device_discovery.R
ggml_vulkan_status() prints a summary and also
returns it invisibly, as
list(available, n_devices, devices) — so a script can
branch on the result without re-querying with
ggml_vulkan_available() and
ggml_vulkan_device_count().
st <- ggml_vulkan_status()
if (st$available && st$n_devices > 0) {
# Each entry is a list: index, name, free_memory, total_memory
nm <- vapply(st$devices, function(d) d$name, character(1))
cat("Vulkan devices:", paste(nm, collapse = ", "), "\n")
} else {
cat("No Vulkan device — CPU only\n")
}# Select GPU (falls back to CPU if unavailable)
device <- tryCatch({
ag_device("gpu")
"gpu"
}, error = function(e) {
message("GPU not available, using CPU")
"cpu"
})
cat("Active device:", device, "\n")After ag_device("gpu"), all subsequent
ag_param and ag_tensor calls allocate on the
GPU. Switch back with ag_device("cpu").
Weights, Adam moments, gradients and forward activations stay in
device buffers across a training step, so a step makes 4 host/device
crossings instead of 10, and the Adam update does not touch the host at
all. This is the default and changes nothing numerically. Transformers
train on the GPU too: ag_flash_attention() computes all
heads in one fused op and its gradient in one more, on both CPU and
Vulkan. See vignette("autograd-engine"), sections 3.1 and
11.
if (device == "gpu") {
ag_dtype("f16") # half-precision on Vulkan GPU
# ag_dtype("bf16") # bfloat16 — falls back to f16 on Vulkan automatically
} else {
ag_dtype("f32") # full precision on CPU
}
cat("Active dtype:", ag_default_dtype(), "\n")bf16 → f16 fallback is automatic on Vulkan because bf16
is not natively supported in GLSL shaders.
if (ggml_vulkan_available()) {
mem <- ggml_vulkan_device_memory(0L)
cat(sprintf("GPU memory: %.1f MB free / %.1f MB total\n",
mem$free / 1e6, mem$total / 1e6))
}ggmlR supports multiple Vulkan devices via the backend scheduler.
dp_train() distributes data across replicas
automatically:
n_gpu <- ggml_vulkan_device_count()
cat(sprintf("Using %d GPU(s)\n", n_gpu))
# dp_train handles multi-GPU internally — see vignette("data-parallel-training")For low-level multi-GPU scheduler usage see
inst/examples/multi_gpu_example.R.
The high-level ggml_fit() API picks up the Vulkan
backend automatically — no extra configuration needed:
data(iris)
x_train <- scale(as.matrix(iris[, 1:4]))
y_train <- model.matrix(~ Species - 1, iris)
model <- ggml_model_sequential() |>
ggml_layer_dense(64L, activation = "relu", input_shape = 4L) |>
ggml_layer_dense(3L, activation = "softmax") |>
ggml_compile(optimizer = "adam", loss = "categorical_crossentropy")
# Training runs on GPU if Vulkan is available
model <- ggml_fit(model, x_train, y_train, epochs = 50L,
batch_size = 32L, verbose = 0L)On a Vulkan device with fp16 support, matrix multiplies keep their tiles in f16 by default. A single row goes through an exact f32 kernel, larger batches through the tiled one, so predictions carry a relative error around 1e-3 that depends on the batch size. Training is not hurt by noise of that size; comparisons against a reference, tests and reproducible evaluation are.
predict_precision = "f32" in ggml_compile()
applies to every prediction of the model;
matmul_precision = "f32" applies to one
ggml_predict() (or a trainer’s $predict())
call and wins over the model’s setting. The result then matches the CPU
to about 1e-6. Training graphs are not affected. The f32 tiles cost
throughput — about a seventh of the f16 speed for large multiplies on an
RX 9070 under RADV; inst/examples/benchmark_coopmat.R
measures it on your device. The CPU backend is exact either way.
p_def <- ggml_predict(model, x_train, batch_size = 32L)
p_f32 <- ggml_predict(model, x_train, batch_size = 32L,
matmul_precision = "f32")
cat(sprintf("max |default - f32|: %.2e\n", max(abs(p_def - p_f32))))ggml_ssm_conv() and ggml_ssm_scan() have
Vulkan backward shaders, so a Mamba block trains entirely on the GPU —
for d_state 128 or 256 and Mamba-2 shapes; other shapes
compute the backward on the CPU. The backward kernels of the RWKV ops
and ggml_gated_linear_attn() are CPU-only.
inst/examples/mamba_train_demo.R trains a block end to end;
the Vulkan backward shaders brought it from 21.7 s to 7.7 s.
# Weights loaded to GPU once at load time
model_onnx <- onnx_load("model.onnx", device = "vulkan")
# Repeated inference — no weight re-transfer
for (i in seq_len(100L)) {
out <- onnx_run(model_onnx, list(input = batch[[i]]))
}Against ONNX Runtime on the 15 reference ONNX Model Zoo models, the
quantised MaskRCNN-12-int8 matches bit for bit (max|d| = 0)
on Vulkan as on the CPU. The others agree within 1e-3 on Vulkan, except
five transformer models (XCiT, CaiT, BERT, RoBERTa, GPT-NeoX) at 1e-3 to
7e-3, from long float reductions summed in a different order. See
vignette("onnx-import") for which operators run on the
GPU.
Vulkan support is compiled in automatically when
libvulkan-dev and glslc are detected at
install time. SIMD (AVX2/AVX512) for the CPU fallback requires an
explicit flag:
# Default install (Vulkan auto-detected, CPU fallback without SIMD)
R CMD INSTALL .
# With CPU SIMD acceleration
R CMD INSTALL . --configure-args="--with-simd"
# Also enable the hard-exit path for multi-GPU scripts (see below)
R CMD INSTALL . --configure-args="--with-simd --enable-hard-exit"On Windows R ignores configure.args, so set the
equivalent environment variables before installing from source:
Sys.setenv(GGML_USE_SIMD = "1"),
Sys.setenv(GGML_VK_HARD_EXIT = "1").
--enable-hard-exit is off by default
and is only relevant to standalone multi-GPU scripts.
It controls whether ggml_vulkan_shutdown(hard = TRUE) may
call _exit() to skip the exit-time Vulkan loader teardown,
which can otherwise flakily segfault after such a script has
printed its results. The released package omits it because CRAN policy
forbids a package from terminating the user’s R session; in a default
build hard = TRUE tears Vulkan down normally and emits a
warning rather than silently doing nothing. See
vignette("multi-gpu") for the full story.
To check what was compiled in:
cat(ggml_version(), "\n")
ggml_vulkan_status() # shows "Vulkan not available" if not compiled in
ggml_vulkan_hard_exit_available() # TRUE only with --enable-hard-exitag_device() and ag_dtype() set
package-level state that outlives this document: anything running later
in the same R session keeps the GPU and the f16 default unless it is put
back. Restore it once the GPU section is done.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.