Two design improvements for local AI deployments — trusted HTTP endpoints and clearer Mode definitions

[unfall103-debug]

Environment

• SLM version: 3.8.14
• Deployment: LXC/Docker containers on private LAN
• LLM: local OpenAI-compatible API (llama.cpp)
• Embedding: local OpenAI-compatible API (Qwen3-Embedding)
• Reranker: local rerank API (llama.cpp)
• All services communicate through internal HTTP endpoints

---

Issue 1: Trusted private LAN HTTP endpoints rejected for reranker

Symptom

Running:

slm soft-prompts

returns:

Remote reranker not started — retrieval.cross_encoder_endpoint must use HTTPS for non-loopback hosts.
Plain HTTP is allowed only for localhost/loopback.
Reranking is DISABLED.

Manual verification

The reranker endpoint itself works correctly:

curl -X POST http://192.168.x.x:8041/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{"query":"test","documents":["hello world"]}'

The endpoint returns valid reranking scores.

Root cause

The reranker service is not the problem. The issue is the security validation layer rejecting all non-loopback HTTP endpoints.

In local AI deployments, internal HTTP is commonly used between trusted services:

LXC containers

Docker containers

Homelab servers

llama.cpp

vLLM

OpenAI-compatible local inference servers


These endpoints are normally not exposed to the public internet.

Suggested improvement

Keep HTTPS as the default security policy, but provide an explicit opt-in mechanism for trusted local networks:

Examples:

allow_trusted_http_endpoints: true

CIDR whitelist for private networks

Per-service override:

embedding.allow_http

reranker.allow_http



This preserves security defaults while supporting advanced local deployments.


---

Issue 2: Mode definitions should describe user experience, not vendors

The current Mode design appears to mix user workflow and backend technology.

For example, Ollama is currently treated as the definition of local AI, but in advanced local AI deployments many users choose different backends:

llama.cpp

vLLM

OpenAI-compatible inference servers

custom model stacks


Reasons include:

multi-GPU support

deployment flexibility

hardware optimization

wider model selection


The following classification is based on the user deployment experience, not the technical implementation.


---

Proposed Mode redesign

A — Mathematical Mode

No LLM dependency

Pure mathematical / algorithmic memory processing

Maximum privacy

Lowest resource usage

No external model required


B — Default / Easy Local Mode

One-click local installation experience

SLM manages local dependencies automatically

Ollama + recommended models as the default implementation

Designed for users who want a simple local AI memory system


C — Custom Mode

User-controlled AI infrastructure

User selects Provider / Endpoint / Model

OpenAI-compatible APIs

llama.cpp

vLLM

Custom Ollama deployments

Local API servers

Cloud APIs


Designed for advanced users who already manage their own AI stack.


---

Why these changes matter

Both issues are related to the same long-term direction:

SLM should become a true "bring your own models" memory system.

Beginners should have a simple Ollama-based path.

Advanced users should be able to connect their own llama.cpp/vLLM infrastructure.

Non-English users should be able to choose specialized embedding and reranking models.

The system should avoid vendor assumptions while keeping safe defaults for normal users.

[varun369]

Thank you for the detailed deployment context and the concrete examples. We agree with the direction: SuperLocalMemory should support model-neutral, bring-your-own infrastructure while keeping a simple local path for users who want it.

This is active design work. V4 intentionally requires HTTPS for non-loopback reranker endpoints because recall queries, candidate memory text, and any configured bearer credential cross that same boundary. We are preserving that default while developing a deliberate advanced local-deployment path with explicit operator intent and a clearly bounded trust model for private infrastructure.

We are also refining the mode language around the user experience and operating model rather than treating a particular vendor as the definition of local AI. Keeping this issue open as the tracking thread for both pieces of work; we will post the proposed configuration and documentation contract here before changing the default behavior.

[varun369]

4.0.6 ships Issue 1. Issue 2 is not done, so this stays open, scoped to that.

Issue 1 — private-LAN HTTP endpoints: fixed (7b7f1ff)

retrieval.trust_plain_http_lan (default true) allows plain HTTP to numeric RFC1918 / ULA / link-local addresses — the same posture already applied to a loopback reranker. Set it to false and every non-loopback host requires HTTPS again, for hardened or shared-colocation deployments.

Still refused, deliberately: public IPs, carrier-grade NAT (100.64.0.0/10), and bare hostnames. Hostnames are excluded because DNS is mutable and is not a trust boundary — reranker.internal can be repointed without your config changing.

The threat model is written into the code rather than implied: trusting a private address does not stop an attacker already on that LAN. The flag means "this LAN is under my control and I accept that risk", not "RFC1918 traffic is secure".

Your http://192.168.x.x:8041/v1/rerank endpoint is accepted as configured.

Issue 2 — mode vocabulary: not done

You are right, and it is still wrong. Mode B reads "Smart Local — Local Ollama LLM (Phi-3, Llama 3.2)" and Mode C is described as a cloud provider. That makes one vendor the definition of local AI, which is exactly what you objected to.

The capability is in better shape than the wording. Mode C already accepts any OpenAI-compatible endpoint, and the Settings pane will set Mode C with a custom endpoint and no API key — so llama.cpp and vLLM work today. But the one-click "Mode C" button still refuses without a key (mode_c_requires_api_key), which is a real obstacle if you never had a key to give it. So the description misleads and the two paths disagree with each other.

Staying open for the mode language and that quick-switch guard.

[varun369]

Revisiting this against v4.1.0. Issue 1 is fixed; issue 2 is partly addressed.

Issue 1 — trusted private-LAN HTTP endpoints: fixed

Plain HTTP is now accepted for numeric RFC1918 addresses, controlled by retrieval.trust_plain_http_lan, which defaults to true. Verified just now against your exact case:

ACCEPTED  trust=True   http://192.168.1.50:8041/v1/rerank   <- your endpoint
REJECTED  trust=False  http://192.168.1.50:8041/v1/rerank   <- hardened mode
ACCEPTED  trust=True   http://10.0.0.5:8041/v1/rerank
REJECTED  trust=True   http://example.com/v1/rerank         <- public host, still refused
ACCEPTED  trust=True   http://127.0.0.1:8041/v1/rerank

The threat model is written into the code where the decision is made: an attacker already on your LAN can still MITM plain HTTP, so this is allowed on the assumption that the LAN is under your control, and trust_plain_http_lan=false restores the strict behaviour for zero-trust environments. Bare hostnames are still refused even when they resolve to a private IP — DNS is not accepted as proof of locality — so use a numeric address, as in your example.

Issue 2 — clearer Mode definitions: partly

4.1.0 makes every surface state what a mode can and cannot do, and why, rather than quietly doing less:

• Mode A now says it has no language model, and names Mode B and Mode C as the way to get one.
• Mode C with no API key set now reports the model as unavailable and names the next step (slm provider set). Previously it reported it as available while every model-backed feature was about to fall back to assembling from your notes.
• A configured model that does not respond is reported differently from one that was never configured — different problems, different next steps.

That is the surface half of what you asked for. If your ask was really about the documentation defining the modes, say so and I will treat that as its own issue rather than assuming.

Leaving this open for issue 2. If the mode messaging in 4.1.0 covers what you needed, comment and I will close it.

Thank you for the report — the reranker one in particular was precise enough to act on directly.