A calm path from request to result
The public edge is the product boundary. It authenticates the caller, validates the model and request shape, applies route-specific limits, and hands work to private queues. Provider addresses, GPU identities, credentials, and content stay behind that boundary.
ClientsOpenAI · Ollama · SDKs
→
RassyMind typed edgeauth · catalog · limits
→
authenticated Bifrost queuesbounded admission
→
Private lanes
4 chat lanesembeddingrerankASR + TTS
Channels are the capabilities
A channel is the client-facing contract: the route, request shape, authentication, and safety limits. Pick a channel by what you need to do; pick a lane by the kind of work inside that channel.
Ollama chat
Ollama-compatible chat for installed clients, using the same typed catalog underneath.Embeddings
One normalized 4,096-dimension contract for documents, queries, and code retrieval.Retrieval
Private ingest, query, and delete operations with provenance-only results and bounded caches.Reranking
Second-stage relevance scoring on the dedicated rerank lane.Speech in
Streaming-capable speech-to-text on the isolated ASR lane.Speech out
Streaming-capable text-to-speech with voice cloning off by default.Catalog
Authenticated discovery of canonical IDs and compatibility aliases; no provider details.Lanes are the work profiles
A lane is a named public capability backed by one verified provider and one assigned card. Aliases preserve compatibility, while the canonical names make intent clear.
| Public lane | Best for | Runtime contract | Compatibility aliases |
|---|---|---|---|
| rassy-mind | general · operator · research · agent-tools · complex-reasoning | qwen3.6:27b-q4_K_M | rassy-agent |
| rassy-code | coding · repository-work · architecture · debugging · code-tools | qwen3.6:27b-q4_K_M | rassy-codex, rassy-architect |
| rassy-fast | normal-chat · quick-code · structured-output · light-tools | qwen3.5:9b-q4_K_M | rassy-general, rassy-codex-lite |
| rassy-utility | summarize · classify · route · compact · rewrite · background | qwen3.5:9b-q4_K_M | rassy-worker, rassy-worker-code, rassy-summarizer |
| rassy-embed | document-embedding · query-embedding · code-retrieval | Qwen/Qwen3-Embedding-8B-GGUF:Q8_0 | rassy-embed-query, qwen3-embed, qwen3-embed-query |
| rassy-rerank | bounded-second-stage-reranking | Qwen/Qwen3-Reranker-4B | none |
| rassy-stt | speech-to-text · live-transcription · long-audio | Qwen/Qwen3-ASR-1.7B | none |
| rassy-tts | text-to-speech · live-voice · expressive-voice | Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice | rassy-audio |
Logical lane selection
rassy-mind handles general, research, operator, and complex reasoning work. rassy-code is shaped for repositories, architecture, debugging, and code tools. rassy-fast favors normal chat and quick structured work. rassy-utility handles compact background jobs. Generic callers—including Cheshire Cat—start on rassy-fast; explicit programming work stays on rassy-code, while operator and deep-reasoning work stays on rassy-mind.
Physical lane discipline
The public name resolves through the locked catalog to a provider such as rm-mind or rm-code. Eligible 27B requests may use bounded load-aware selection between equivalent 27B providers; 9B overflow is limited to eligible routine work. Fallback preserves the catalog route when private telemetry is unavailable.
What happens to a request
Authenticate and validate
The edge checks the bearer key, method, JSON shape, model alias, capability, body size, context, output ceiling, and route-specific deadline. Rejections are counted by safe reason, never by content.
Estimate the work
In memory, the edge estimates prompt size from request structure and combines it with the requested output ceiling. The estimator is conservative and content-free: it does not persist or log the prompt.
Admit fairly
Session-bound traffic stays coherent. A 30-second active-caller window lets one client use normal capacity while multiple active clients share newly released capacity without a hidden waiting queue.
Route and stream
Adaptive 27B selection considers provider load, deferred work, local reservations, and estimated work. Bounded 9B overflow remains limited to routine requests; cancellation releases all reservations.
Return a compatible result
The edge translates the private provider response into the requested OpenAI or Ollama wire format. Provider errors become generic, actionable HTTP errors without leaking upstream bodies.
Decision sequence
authenticate→catalog + limits→estimate work→session lookup→fair admission→lane + Bifrost→stream / release
Control plane and data plane
The control plane decides what is allowed and where it goes. The data plane carries the request through an authenticated transport and isolated providers. The public page exposes the contract; the admin console exposes bounded operational state.
Client / adminAPI calls · operations
Edge policyauth · catalog · work · fairness · sessions
Bifrost transportprivate queues · provider protocol
Providers + storesmodel lanes · SQLite · Qdrant
Retrieval that stays bounded
RassyMind keeps retrieval explicit so quality work does not become an unbounded memory or latency problem.
embed query→dense top 40+lexical top 40→rank fusion→batched rerank top 20→deduplicate + pack→return top 10
Documents and queries use the same normalized 4096-dimensional embedding contract. Identical concurrent queries share one embedding, fusion, and rerank operation; the reranker scores its candidate set as one padded batch rather than performing one model call per passage. Completed results live only in a small, short-lived, corpus-generation-scoped cache. Ingest and delete invalidate it. Metrics expose counts and timings only, never queries, vectors, source IDs, or result text.
Efficiency is a quality feature
The fastest useful answer is not the one with the largest prompt. It is the one that spends compute on the most relevant evidence and avoids repeating work.
Reuse before recompute
Repeated conversation prefixes can be reused by the llama.cpp prompt cache. The two 27B lanes keep one verified slot each, so cache reuse lowers prompt-processing work without pretending that one GPU can safely serve unlimited parallel generations.
Pack evidence, not noise
Context selection removes duplicate passages, respects an explicit token budget, and places the strongest evidence at the edges of the packed context. This keeps related evidence readable while reducing the chance that a long middle section buries the answer.
Batch the expensive step
Hybrid retrieval casts a wide first net, then the dedicated reranker scores candidates together. This preserves the recall-oriented dense-plus-lexical design while reducing repeated tokenizer and GPU launch overhead.
Measure the trade
Admin telemetry separates requests, failures, rejections, cancellations, rolling health windows, and per-route latency. Changes are kept only when real endpoint checks show acceptable latency, quality, cancellation, and resource behavior.
Evidence is a separate, explicit pipeline
Retrieval never silently changes a chat prompt. A trusted caller chooses when to query, whether to request content, and how much evidence to pack into its own context.
Ingest and query lifecycle
source→normalize + chunk→exact embedding cache→Qdrant + SQLite FTS5→dense / lexical→RRF→batched rerank→dedupe + position-aware pack
Each source is canonical in SQLite and can produce deterministic rassymind-chunker-v1 chunks with source, ordinal, heading, character and line provenance. Qdrant stores one bounded point per chunk; SQLite remains authoritative for content and stale-point rejection.
Default query
Returns ranked identifiers, scores, and provenance without content. This keeps ordinary retrieval responses small and prevents accidental evidence disclosure.
Opt-in evidence
Set include_content: true when a trusted caller needs passages. This is the explicit opt-in evidence pack: context_budget_tokens selects deduplicated, edge-balanced evidence without summarizing or exceeding the budget.
The dials that shape behavior
These are operational heuristics and safety bounds, not hidden model behavior. They are centralized, bounded, and intended to be tuned from measured evidence.
Admission and fairness
- input quantum
- 2,048 estimated tokens per input work unit
- output quantum
- 1,024 output tokens per output work unit
- output weight
- 2×, because decode is sequential
- 27B capacity
- 18 aggregate reservations across equivalent lanes
- 9B capacity
- 9 aggregate reservations across fast/utility
- fairness window
- 30 seconds of recent activity
- fast overflow
- Routine, tool-free requests only; input ≤8,192 estimated tokens and output ≤2,048
Model lane contracts
- 27B context
- 65,536 tokens; max output 8,192
- 27B lanes
rassy-mindandrassy-code, one production model per GPU- 9B context
- 32,768 tokens
- fast output
- 4,096 maximum
- utility output
- 2,048 bounded overflow ceiling
- selection
- Load, deferred work, reservations, work estimate, affinity, and round-robin tie breaks
Retrieval quality and cost
- chunker
rassymind-chunker-v1; target 1,050 and hard maximum 1,500 estimated tokens- overlap
- 100 estimated tokens, bounded at structural boundaries
- candidate sets
- Dense top 40 + FTS5 lexical top 40
- rerank
- Top 20, then return at most 10
- query cache
- 128 entries, 60-second TTL, invalidated on corpus mutation
- embedding cache
- 100,000 exact packed float32 entries; 4,096 dimensions
State and transport
- session state
- Opaque hash, 30-minute idle TTL, 4,096-entry hard bound, persisted atomically at the edge state mount
- body limit
- 64 MiB, including supported audio uploads
- deadlines
- Chat 30m; embeddings/retrieval 5m; rerank 2m; ASR/TTS 10m
- pooling
- 64 max idle connections, 16 per host, 90-second idle timeout
- shutdown
- Graceful edge shutdown closes idle upstream connections and persists bounded state
Use the familiar API surface
Every operational route requires a bearer key. The public page explains the contract without becoming a discovery endpoint for internal services. Operators can open the authenticated admin console for aggregate usage, lane reservations, fairness state, alerts, and key management.
curl https://ollama.rasies.com/v1/models \
-H 'Authorization: Bearer YOUR_KEY'
curl https://ollama.rasies.com/v1/chat/completions \
-H 'Authorization: Bearer YOUR_KEY' \
-H 'Content-Type: application/json' \
-d '{"model":"rassy-mind","messages":[{"role":"user","content":"Hello"}]}'
# The same catalog is available to Ollama-compatible clients:
curl https://ollama.rasies.com/api/tags \
-H 'Authorization: Bearer YOUR_KEY'Private by construction
Content stays out of telemetry
The gateway and caller services do not log prompts, retrieved text, tool arguments, transcripts, audio, or reasoning. Operational metrics are aggregate-only: counts, durations, queue state, and bounded rejection reasons.
Capabilities fail closed
Image generation is disabled. No paired GPUs, dynamic replacement, hidden memory injection, automatic personal-memory injection, or unrestricted host tools are available. Unsupported models and unavailable capabilities are rejected before provider work.
Health is liveness, not inference
Anonymous health and readiness answer only whether the service is alive and whether required dependencies are available. Models, metrics, retrieval, media, and administration remain authenticated.
Qualification remains honest
Candidate 2.2.1 is deployed with compiled identity verified. API conformance 18/18, voice qualification, failure handling, rollback, hardware, drift, and model attestation gates pass. Structured 27B capability remains fail-closed pending provider qualification. The current release status is NOT RELEASE-QUALIFIED. Remaining evidence gaps are listed below so operational progress is never confused with a formal release claim. A rollback restores the prior edge route and bind without deleting state or changing the locked GPU map.
- 27B tool and structured schema capabilities are explicitly unqualified and fail closed; differential probes pass only their rejection guardrails.
- 27B structured/tool capability is not qualified and correctly returns capability_unqualified; production remains parallel=1, the prior executable rollback is verified, Qwen3.8, parallel=2, voice consolidation, and elastic reclamation remain unpromoted because exact production evidence is absent.