Ollama (pass 2) (llm)
Summary
ExpandThe `llm/` package is responsible for spawning LLM inference runner subprocesses, allocating model layers across available GPUs, and proxying completion, embedding, and tokenization requests to the runner over a local HTTP interface. The architecture separates the legacy `llamaServer` (backed by the llama.cpp C bindings) from the newer `ollamaServer` (using a native tokenizer), both inheriting from a shared `llmServer` base that manages subprocess lifecycle, port allocation, and request routing.
The package demonstrates solid engineering practices in several areas: a semaphore-based concurrency limiter prevents overloading the runner, the `StatusWriter` captures and surfaces runner errors for diagnostics, the `Close()` method properly acquires a lock before freeing the llama model, and the `WaitUntilRunning` loop includes stall detection with progress-based timer resets. The GPU layer assignment algorithm uses a sophisticated iterative approach with backoff to handle memory allocation failures gracefully.
The primary area of concern is the subprocess communication channel, which uses unauthenticated HTTP on localhost. On multi-user systems, any local process can interact with the runner by discovering the port. Additionally, a TOCTOU race condition in port allocation could allow a local attacker to intercept runner communications. A logic bug in the completion token repeat detection could cause silent incomplete responses. These issues are addressable without architectural changes, and remediation is recommended before deployment on shared infrastructure.
Findings
1 issue identified
TOCTOU Race Condition in Runner Port Allocation Enables Local Interception
llm/server.go:StartRunner
View Details
Description
StartRunner allocates a TCP port by binding to port 0, reading the assigned port, then closing the listener before the subprocess starts. Between l.Close() and the subprocess binding, a local attacker can race to bind the same port and intercept all runner communications including prompts, completions, and model loading requests.
Impact
A local attacker can intercept all model prompts and responses, inject arbitrary completions, or cause denial of service. On multi-user systems, this requires no elevated privileges.
Recommendation
Pass the listener file descriptor to the subprocess instead of closing and rebinding, or implement a shared authentication token to mitigate the impact of a stolen port.
Fix
This is a partial fix noting the TOCTOU issue. The complete fix requires either passing the listener fd to the subprocess or implementing a shared bearer token (M-1) to authenticate the connection even if the port is hijacked.
llm/server.go
Conclusion
ExpandThe llm/ package demonstrates competent Go engineering with thoughtful handling of GPU memory allocation, subprocess lifecycle management, and graceful error recovery. The iterative layout algorithm with backoff for memory allocation is well-designed, and the use of semaphore-based concurrency limiting prevents resource exhaustion at the application level. The StatusWriter pattern for capturing runner diagnostics and the proper mutex-guarded model cleanup in Close() reflect attention to correctness.
The most pressing concern is the unauthenticated, race-prone localhost HTTP channel used for all parent-to-runner communication. On single-user desktop systems (Ollama’s primary deployment target), the practical risk is low. However, on shared servers, containers, or CI environments — where multiple users or processes share a network namespace — a local attacker can intercept or inject model data. Adding a per-session bearer token to the runner HTTP interface would address both the TOCTOU race (H-1) and the missing authentication (M-1) with a single, low-complexity change.
The Ollama (pass 2) llm/ package is suitable for single-user desktop deployment in its current form. For deployment on shared or multi-tenant infrastructure, implementing runner authentication is strongly recommended before exposure. The token repeat logic bug (L-1) should be addressed as a reliability improvement regardless of deployment context.
Legal Disclaimer: This report covers the code submitted for analysis. It does not account for infrastructure, deployment configuration, third-party dependencies, or changes made after the audit date. Automated analysis may produce false positives or miss context-dependent vulnerabilities. audited.xyz provides this report “as is” without warranty of any kind.