Platone is a hybrid inference architecture voice RAG you can discuss with, balancing latency, cost, and privacy.
Retrieval is hybrid and reranked, with coverage judged before every answer. You can discuss with your KBs and when the documents fall short, it warns you and escalates to a web search or an unguarded opinion only after an explicit "yes". Persona, right acronym pronunciation, and other smart features are set per knowledge base, and the user can talk over it mid-answer and interrupt.
Component placement, model choice, and fallback behavior are decided from measured results — not defaults or vendor reputation.
01
How Platone's enhanced RAG works
A naive RAG retrieves the top matches and pastes them into the prompt, even when they don't answer the question. Platone checks coverage first.
TurnQuestion + history
→
CloudRewrite → standalone query
→
EdgeHybrid search
→
EdgeRerank
→
CloudCoverage check
The coverage check returns one of three verdicts:
Covered
Grounded answer, streamed straight from the documents.
Partial
Answers what's covered, then offers to go further for the rest.
Not covered
Retries once with a broader search. Still nothing → what happens next depends on the KB's trust policy.
Not agentic. This cascade is fixed application logic — not the model calling its own tools — so every path is deterministic, logged, and reproducible.
02
Response policies
On "not covered", Platone can still continue the conversation — via a web search or an opinion. This is a per-knowledge-base setting, with different configurations available:
Grounded
Answers stay grounded in the KB's own documents. Offers a web search when they fall short — never an opinion. Default policy.
Hybrid
Offers both — web search or opinion. The user picks one.
Advisor
Offers an opinion. Never searches the web.
Strict
No offer of any kind — abstains cleanly. Even the offer itself is a risk on some corpora.
03
Architecture — hybrid by design
Latency-critical and privacy-sensitive stages run on the edge GPU; reasoning-heavy generation runs on cloud EU inference. Raw audio never leaves the edge — only text crosses the boundary.
Placement follows latency and privacy. STT, embeddings, reranking, VAD and TTS run locally with deterministic latency. Rewriting and generation run on cloud EU inference, AI Act–compliant, where a larger model is worth the round-trip.
04
Features
Persona sets how Platone talks; the KB's trust policy (previous section) sets how far it can go. Neither is hard-coded — both live with the corpus, read per turn, no restart.
Persona — how it's said
tecnicodivulgativoconsulenziale
Same grounding, sources and coverage gate across all three — persona changes phrasing, never sourcing.
Pronunciation control
A deterministic layer rewrites acronyms and product codes into a pronounceable form before synthesis. An admin page lets you type a rendering and hear it instantly, per knowledge base.
Beyond the knowledge base
Web search and "advisor" opinions are opt-in per turn, never blended into a documented answer, and always announce themselves as such.
Barge-in — interrupt mid-answer
Speaking over Platone cancels the answer stream immediately — wired by hand into the pipeline, not inherited from a stock service.
Decision path is visible per turn. A dedicated page shows the path a turn took — score, verdict, retries, offer, choice — including branches not taken.
05
Technology stack
Edge node
RTX 4070 SUPER 12GB · Ubuntu 24.04
OrchestrationFastAPI + Pipecat, systemd
STTQwen3‑ASR‑1.7B
Embeddingsbge‑m3, 1024‑dim
Rerankerbge‑reranker‑v2‑m3
Vector DBQdrant (k3s/containerd)
TTSQwen3‑TTS, cloned voice
Cloud — Regolo.ai
EU inference, OpenAI‑compatible API
RewriteLlama‑3.3‑70B‑Instruct
Expert / judgegemma4‑31b (streaming)
Advisor branchglm5.2‑beta
External / client
consent‑gated, no lock‑in
Web searchTavily (default)
ClientBrowser, WebRTC P2P
06
Key architectural decisions
Model routing by workload
ProblemIngest, expert answers and rewriting have opposite needs.
ChoiceConfigurable per-role routing instead of one model for everything.
ResultExpert response time fell from 5–11s to 0.15s p50.
Edge/cloud placement by latency & privacy
ProblemAlmost every stage could technically run in either place.
ChoiceLatency/privacy-sensitive stages stay on-edge; generation runs in the cloud.
ResultSame rule reused for every feature added since.
External dependencies behind one interface
ProblemProviders change; wiring directly to one is fragile.
ChoiceOne function per capability — provider swap confined to one adapter file.
ResultTTS swapped four times, STT once — pipeline untouched.
Abstention as a shipped feature
ProblemAlways answering — even wrong — isn't trustworthy for support.
CauseThe rewriting model kept dropping the topic entity.
FixMoved rewriting to a larger cloud model — recall back to 0.975.
Transcript silently rewrote technical terms
SymptomEnglish terms inside Italian speech came out wrong — no visible error, just failed retrieval.
CauseSTT pinned to Italian, "corrected" unfamiliar terms.
FixSwapped in a code-switching ASR model — pipeline untouched.
Faster voice engine failed silently
SymptomMic worked, nothing got transcribed.
CauseNew TTS engine shared the 12GB GPU with STT — first turn ran out of memory.
FixRoot-caused and documented; frozen for higher-VRAM hardware.
Web fallback returned thin answers
SymptomWeb-escalated turns came back weak.
CauseSearch engine returned only preview snippets, not full page content.
FixSwitched engines — 10/10 vs 8/10 accuracy, latency down ~45%.
Lesson: on shared cloud inference, latency shifts with provider load — re-measure before blaming the model.
08
What's next
Teaching new pronunciations to the KB
Pronunciation control (see Features) can fix how a word sounds — but it has to be told which words. Every new knowledge base starts with none taught, so someone listens and corrects the ones that come out wrong.
Validating retrieval anchors
An indexing bug meant AI-generated Q&A "anchors" never reached the index — retrieval held up fine without them. An A/B test will decide whether they're worth re-enabling.
Streaming voice (blocked on hardware)
Would cut time-to-first-audio from ~2.2s to 0.1s and fix choppy intonation between sentences — but doesn't fit today's 12GB GPU alongside STT. Frozen for higher-VRAM hardware.
09
Some measured results
Measured on the corpus indexed at the time of testing, not on a benchmark suite.
~0.1sexpert TTFT, p50
~125msretrieval + rerank, end‑to‑end
32/35real voice queries, end‑to‑end
35/35altered facts caught by judge
0.924R@3, full synthetic golden (n=2079)
9.1/12GBedge GPU, steady state
First-pass retrieval alone solved 27/35 real queries; the retry recovered 5 more. Off-topic questions were reliably declined across every difficulty level.