sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

docs(coding-agent): restructure NVIDIA RTX llama.cpp prerequisites

Browse source
This commit is contained in:
Christian Klotz 2026-10-13 15:01:17 +01:00
parent 6f2f164ab7
commit b9ab2a6238
No known key found for this signature in database
1 changed files with 8 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

17
packages/coding-agent/docs/nvidia-rtx-llama-cpp.md
View file

@ -2,17 +2,16 @@
[NVIDIA Local AI Inference Recipes](https://nvidia.github.io/local-ai-inference-recipes/index.html) provides llama.cpp settings tuned for specific NVIDIA hardware and models. Each recipe gives a single-model `llama-server` command. Pi works with the llama.cpp router, which can serve and manage several models, so this guide converts the recipe command into a router preset.
The example below uses Windows PowerShell. See [Run Pi on Windows](windows.md) for native Windows and WSL setup, and [Local Models with llama.cpp](llama-cpp.md) for Pi's complete router and model-management reference.
The examples below use Windows PowerShell. See [Local Models with llama.cpp](llama-cpp.md) for Pi's complete router and model-management reference.
## Choose a recipe
## Before you begin
Open the [recipe catalog](https://nvidia.github.io/local-ai-inference-recipes/index.html), select your hardware, operating system, and model, then choose **llama.cpp**. Check the prerequisites and memory requirements. Keep the recipe's launch command for the conversion below.
1. Open the [recipe catalog](https://nvidia.github.io/local-ai-inference-recipes/index.html), select your hardware, operating system, and model, then choose **llama.cpp**. Check the prerequisites and memory requirements. Keep the recipe's launch command for the conversion below.
2. Install llama.cpp with the recipe's setup instructions. The recipe's settings depend on its build, for example CUDA support or speculative decoding options.
3. Run `llama-server --help` and check that it lists `--models-preset`. Without it, the build cannot load the recipe as a router preset. Use a newer build that still meets the recipe's requirements.
4. Install Pi with the [Quickstart](quickstart.md). On native Windows, also see [Run Pi on Windows](windows.md).
## Install llama.cpp
Follow the recipe's setup instructions to install llama.cpp. Use a current build with router and model-preset support.
## Create a model preset
## Convert the recipe to a preset
A preset lists the settings for one or more models. Each section name becomes the model ID shown in Pi. Save the preset next to `llama-server`, or pass its full path when starting the router.
@ -90,7 +89,7 @@ If the recipe requires a Hugging Face token, set `$env:HF_TOKEN` in the same Pow
Use the [router health checks](llama-cpp.md#troubleshooting) to confirm that the server is ready. In Windows PowerShell, use `curl.exe` rather than `curl`, which PowerShell maps to a different command. The model is ready when `/models` lists `qwen3.8-27b` with the status `loaded`.
## Connect Pi
## Use the model in Pi
Follow [Configure Pi](llama-cpp.md#configure-pi) and connect to `http://127.0.0.1:8080`. The preset section name is the model ID, so select `qwen3.8-27b` in `/model` for this example.