From b9ab2a6238d2c91cf0c6ebf836dfd05883f9db51 Mon Sep 17 00:00:00 2001 From: Christian Klotz Date: Tue, 13 Oct 2026 15:01:17 +0100 Subject: [PATCH] docs(coding-agent): restructure NVIDIA RTX llama.cpp prerequisites --- .../coding-agent/docs/nvidia-rtx-llama-cpp.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/packages/coding-agent/docs/nvidia-rtx-llama-cpp.md b/packages/coding-agent/docs/nvidia-rtx-llama-cpp.md index 6a6163034..9ea812a55 100644 --- a/packages/coding-agent/docs/nvidia-rtx-llama-cpp.md +++ b/packages/coding-agent/docs/nvidia-rtx-llama-cpp.md @@ -2,17 +2,16 @@ [NVIDIA Local AI Inference Recipes](https://nvidia.github.io/local-ai-inference-recipes/index.html) provides llama.cpp settings tuned for specific NVIDIA hardware and models. Each recipe gives a single-model `llama-server` command. Pi works with the llama.cpp router, which can serve and manage several models, so this guide converts the recipe command into a router preset. -The example below uses Windows PowerShell. See [Run Pi on Windows](windows.md) for native Windows and WSL setup, and [Local Models with llama.cpp](llama-cpp.md) for Pi's complete router and model-management reference. +The examples below use Windows PowerShell. See [Local Models with llama.cpp](llama-cpp.md) for Pi's complete router and model-management reference. -## Choose a recipe +## Before you begin -Open the [recipe catalog](https://nvidia.github.io/local-ai-inference-recipes/index.html), select your hardware, operating system, and model, then choose **llama.cpp**. Check the prerequisites and memory requirements. Keep the recipe's launch command for the conversion below. +1. Open the [recipe catalog](https://nvidia.github.io/local-ai-inference-recipes/index.html), select your hardware, operating system, and model, then choose **llama.cpp**. Check the prerequisites and memory requirements. Keep the recipe's launch command for the conversion below. +2. Install llama.cpp with the recipe's setup instructions. The recipe's settings depend on its build, for example CUDA support or speculative decoding options. +3. Run `llama-server --help` and check that it lists `--models-preset`. Without it, the build cannot load the recipe as a router preset. Use a newer build that still meets the recipe's requirements. +4. Install Pi with the [Quickstart](quickstart.md). On native Windows, also see [Run Pi on Windows](windows.md). -## Install llama.cpp - -Follow the recipe's setup instructions to install llama.cpp. Use a current build with router and model-preset support. - -## Create a model preset +## Convert the recipe to a preset A preset lists the settings for one or more models. Each section name becomes the model ID shown in Pi. Save the preset next to `llama-server`, or pass its full path when starting the router. @@ -90,7 +89,7 @@ If the recipe requires a Hugging Face token, set `$env:HF_TOKEN` in the same Pow Use the [router health checks](llama-cpp.md#troubleshooting) to confirm that the server is ready. In Windows PowerShell, use `curl.exe` rather than `curl`, which PowerShell maps to a different command. The model is ready when `/models` lists `qwen3.8-27b` with the status `loaded`. -## Connect Pi +## Use the model in Pi Follow [Configure Pi](llama-cpp.md#configure-pi) and connect to `http://127.0.0.1:8080`. The preset section name is the model ID, so select `qwen3.8-27b` in `/model` for this example.