TL;DR
Linux users with NVIDIA GPUs can now repurpose VRAM as swap space using a user-space daemon that leverages CUDA APIs. This approach extends available memory on hybrid laptops without kernel changes, but performance varies with workload.
Linux users with NVIDIA GPUs can now use VRAM as swap space through a new user-space daemon that communicates with the GPU via CUDA APIs. This development allows systems to extend memory capacity without kernel modifications, addressing hardware limitations in hybrid laptops where VRAM is often unused.
The approach involves a small daemon that allocates VRAM by calling CUDA APIs and exposes it as a block device using the Network Block Device (NBD) protocol over a Unix socket. The kernel’s built-in NBD driver then maps this device as a swap space, making the VRAM accessible as high-priority swap memory. This method does not require kernel modules or modifications, and it survives driver updates due to its user-space design.
The implementation has been tested on an AMD/ATI + RTX 3070 laptop running Pop!_OS with kernel 6.17, with 7 GB allocated as swap, resulting in a total addressable memory of approximately 46 GB—including zram and SSD swap—tripling the default capacity. The process relies on CUDA’s cuMemcpyHtoD and cuMemcpyDtoH functions, which work on any CUDA-capable NVIDIA GPU without special permissions, sidestepping limitations encountered with the NVIDIA peer-to-peer (P2P) API, which is restricted to Quadro and data center GPUs.
Why It Matters
This development is significant because it offers a practical way to utilize otherwise idle VRAM for swap, especially in laptops with soldered memory and limited upgrade options. It effectively increases available system memory, which can improve performance in memory-intensive workloads, reduce system crashes due to insufficient RAM, and optimize hardware utilization without hardware upgrades.
PNY NVIDIA RTX A6000
- CUDA Cores: Double-speed FP32 processing for workflows
- RT Cores: Up to 2X throughput for ray tracing
- Tensor Cores: Up to 5X AI training throughput
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
Hybrid laptops often have integrated AMD or Intel graphics for display output, with discrete NVIDIA GPUs sitting idle most of the time. Traditionally, VRAM on these GPUs remains unused for general system memory purposes because of driver restrictions, especially on consumer GeForce cards. Prior attempts to access VRAM directly via NVIDIA’s P2P API or ioremap methods failed due to driver restrictions and hardware limitations. This new approach leverages CUDA APIs and user-space tools to bypass these barriers, inspired by recent discussions on Hacker News about repurposing GPU VRAM for swap in Linux environments.
“The NBD approach sidesteps all of the limitations imposed by the NVIDIA driver, allowing any CUDA-capable GPU to serve as swap without kernel modifications.”
— Developer of the nbd-vram project
“Using VRAM as swap effectively triples available memory on my hybrid laptop, with minimal performance impact for sporadic tasks.”
— Test user on Hacker News
ICY DOCK 8 Bay 2.5 SAS/SATA HDD/SSD Hot Swap Backplane Mobile Rack Enclosure for 2 x 5.25 Bay (2 x MiniSAS HD) | ToughArmor MB508SP-B
- Drive Compatibility: Supports 8x 2.5” SATA/SAS drives
- Drive Size: Fits 5-15mm height drives
- Drive Bays: Uses 2x 5.25” external bays
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
While the method has been demonstrated successfully, its performance for sustained high-throughput workloads is limited compared to NVMe SSDs, and benchmarks show slower sequential transfer speeds. The long-term stability and impact on GPU lifespan are not yet fully understood. Compatibility with all NVIDIA GPU models and driver versions remains to be confirmed, and the approach may face limitations on certain hardware configurations or driver updates.
As an affiliate, we earn on qualifying purchases.
What’s Next
Developers plan to refine the daemon for better performance, expand testing across different GPU models, and integrate user-friendly configuration tools. Future updates may include automatic management of VRAM swap sizing and improved power management features. Wider adoption depends on community testing and feedback, with potential inclusion in Linux distributions as an experimental feature.
Network Block Device (NBD) Linux
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Can I use this method with any NVIDIA GPU?
It works with CUDA-supported NVIDIA GPUs, including consumer RTX and GTX cards. However, compatibility with specific models and driver versions is still being tested.
Does using VRAM as swap harm my GPU?
Long-term effects are not yet fully known. The method uses CUDA APIs without kernel modifications, but intensive or prolonged use as swap could impact GPU lifespan or performance. Users should monitor GPU temperatures and stability.
How does the performance compare to traditional swap methods?
Benchmarks indicate slower sequential transfer speeds than NVMe SSDs, but for sporadic, small I/O operations, latency is significantly lower, making it suitable for certain workloads.
Is this approach officially supported by NVIDIA or Linux distributions?
No, this is an experimental, community-developed solution. It is not officially supported and should be used with caution.
Source: Hacker News
