It's fairly straightforward to add GPU support when running on the host, but LlamaGPT runs inside a Docker container, and that's where it gets a bit challenging.

It shouldn't, nVidia provides a CUDA Docker plugin that lets you expose your GPU to the container, and it works quite well.

See above if you're interested in that. It does work quite well, even with nested virtualization (WSL2).

I am, thanks!