POC — Private AI for small businesses

PMI AI Gateway

A private AI workspace for Italian small businesses: one instance per company, open models hosted in Italy, conversations that stay on the company's own server.

Employees chat with top open-weight models, ask questions about their documents, upload an Excel file and get it back analysed or modified (Word and PowerPoint are next, with the same flow), and search the web when the model decides it's needed. Knowledge bases and saved prompts can be shared with colleagues. Italian by default, from the interface to the chat titles. Privacy is not an add-on: no conversation logs, admin chat reading switched off in the interface (one API exception, documented below), automatic deletion after 90 days.

Built on Open WebUI. The value is not the software — it's the two dozen defaults changed, each one measured, justified and written down.

01

One instance per company

No shared multi-tenant platform: every customer gets its own small VM on a European cloud, its own Regolo account and its own keys. A spent budget or a revoked key stops one customer, never the others.

EmployeeBrowser
→https
Customer VMCaddy · TLS
→
Customer VMOpen WebUI · chats, files, vectors
→prompt
Regolo · ItalyChat · task · compaction · embedding
Customer VMOpen WebUI
→query only
External · USAWeb search · Tavily
·
EmployeeBrowser · Python (Pyodide) for Excel
Customer VM — EU cloud, 4 vCPU / 8 GB, no GPU Regolo.ai — Italian datacenters, zero data retention External — the only hop outside the EU
What crosses which border, said plainly. Conversations and files live only in the customer's VM. Prompts reach Regolo in Italy and are not stored there. The one exception is web search: the search query — written by the model from the chat — goes to a US provider. That is why the in-app notice says where models run and where chats are kept, and never promises "everything stays in the EU".
02

What the user sees

Hundreds of models in the picker become three, named after the company. Each one was measured before being exposed: real context window, reasoning or not, behaviour without an output limit.

PMI AI

glm5.2 · default

The quality model, with reasoning. Web search on by default; the model decides when to use it.

PMI AI Small

qwen3.8-27b

The economical model for everyday tasks. Also summarises long chats when the context fills up.

PMI AI Excel

glm5.2 + code interpreter

Opens with the code interpreter on and web search off. Reads, computes, modifies and returns the file.

Hidden, still working. Titles, tags and follow-ups run on a separate non-reasoning model (gemma4-31b), hidden from the menu — a reasoning model there would spend tokens thinking about a three-word title.

03

Company knowledge

Open WebUI's built-in RAG, wired to Regolo. Users build their own knowledge bases — contracts, price lists, procedures — share them with colleagues, and recall them in any chat. Answers come from the company's documents, not from the model's memory.

UploadPDF · Word · text
→
Customer VMSplit · ~1000-char chunks
→text
Regolo · ItalyEmbed · Qwen3‑Embedding‑8B
→vectors
Customer VMStore · vector database
ChatQuestion
→
Regolo · ItalyEmbed the question
→
Customer VMFind · 20 closest chunks
→text
Regolo · ItalyAnswer · chat model

Personal or shared

Every user can build knowledge bases and share them with colleagues or groups. Public sharing stays off.

Kept until removed

Knowledge base documents don't expire with the 90-day purge — chat attachments do. Both are stated in the privacy notice.

What it doesn't do (yet)

No OCR: scanned PDFs and images aren't read. No reranker. Files up to 25 MB.

Documents stay home; only the indexing travels. Text goes to Regolo in Italy to become vectors — a numeric fingerprint of its meaning — and comes back. Vectors and documents live on the customer's server. Changing the embedding model means re-indexing every knowledge base, so it's chosen once, on purpose.
04

Defaults changed

Open WebUI's defaults aren't mistakes: they were made for local models with small contexts and zero marginal cost. On paid APIs with large-context, tool-calling models, the same defaults leave value on the table — or open a budget with no ceiling.

AreaDefaultPMI AI Gateway
Imagemoving tagpinned by digest, upgrades tested on the template first
Modelshundreds, new ones hidden by per-model grants3 visible, everyone sees all of them
Output lengthnone (provider injects 200k on one model)16,000 tokens on every model
Contextno compactioncompaction per model at ~75% of real input window
Tool iterations256 — no real brake40
Web searchoff, or a fixed pipelineone tool, on by default, model-driven
Embeddingslocal MiniLM, 384‑dim, EnglishQwen3‑Embedding‑8B, 4096‑dim, multilingual
Uploadsno size limit25 MB per file, enforced in app and proxy
Sign-upopen, users pendingclosed; accounts created by us, or Microsoft SSO
Session4 weeks7 days
Admin and chatsadmin reads and exports all chatsread and export off; usage statistics kept
Auditoffaccesses and admin actions, no content, kept 3 months
Retentionchats and files forever90 days from last use, pinned chats excluded
Languagebrowser language, titles in EnglishItalian interface, titles always in Italian
Community sharingon — chats can leave the instanceoff
05

Features

What a small company actually asks for: a chat that is safer than the free tools employees already use, that works on their files, and that someone else keeps running.

Office files in chat — Excel today, Word and PowerPoint next

Excel: liveWord · PowerPoint: plannedruns in the browser

The model never does arithmetic in its head: it writes Python, which runs in the user's browser (Pyodide), not on the server. A dedicated model knows the traps — totals rows, Excel Tables to extend, files with macros or charts to warn about. Word and PowerPoint will follow the same seamless flow: upload, ask, get the file back.

Privacy by design

no chat logs90-day retentionaudit without content

No log of conversations. The admin can't browse employees' chats from the interface. Accesses and admin changes are audited — who, when, from where — never what was asked.

One web search tool

Two overlapping ways to search the web became one tool, on by default. The model decides when to search; the user doesn't have to remember a switch. Crawl and deep-research tools with no upper bound are removed server-side.

Shared knowledge and prompts

Users build their own knowledge bases and saved prompts and share them with colleagues or groups. Public sharing stays off.

Italian first

Interface in Italian by default, chat titles always in Italian, and a chat that opens with "ciao" is titled "Nuova conversazione" instead of an English "Empty Conversation". Follow-up suggestions follow the language of the chat.

Status page — traffic lights

One password-protected page, checked live on every open: app, model provider, disk, RAM, last night's backup, failed logins, today's users and chats. No alerting stack on a 5-person company's VM.

Controls, not instructions. Which tools a model can call is filtered server-side, before the tool list reaches the model — for it, the removed tools don't exist. A system prompt that asks for caution is a sentence the model may ignore; a filter is a limit. The same mechanism can give everyone read access to an internal system and only some people write access.
06

Technology stack

Customer VM

EU cloud · one per company · no GPU
  • InterfaceOpen WebUI 0.12.2, pinned digest
  • HTTPSCaddy, Let's Encrypt, HSTS
  • DataSQLite + Chroma, in one volume
  • Operationsbackup, purge, audit cleanup, boot check — bash + cron
  • StatusPython stdlib service behind Caddy

Regolo.ai

Italy · zero retention · OpenAI-compatible
  • Defaultglm5.2
  • Smallqwen3.8‑27b (also compaction)
  • Task modelgemma4‑31b, non-reasoning
  • EmbeddingsQwen3‑Embedding‑8B, 4096‑dim

External / client

declared, one hop each
  • Web searchTavily via MCP
  • CodePyodide, in the browser
  • Loginlocal accounts · Microsoft SSO ready
07

Key architectural decisions

No gateway in front of the models

ProblemGateway stacks add governance a 10-person company never uses — and more parts to patch.
ChoiceGovernance inside the interface; one instance and one provider key per customer.
ResultTwo containers per customer. Spend and outages are isolated by design.

No conversation logs

Problem"Log everything" means a register of employees to justify and sensitive data to guard.
ChoiceAudit of accesses without content; admin chat reading and export off.
ResultOne sentence for the privacy notice: chats are deleted 90 days after last use and gone from backups within 14 more.

Excel computed in the browser

ProblemModels guess numbers; server-side sandboxes run model-written code on the customer's VM.
ChoicePyodide interpreter in the user's browser + a dedicated model with Excel rules.
ResultFile read, modified, returned — and reopened in Excel without repair.

A context budget the product never had

ProblemThe interface counts tool iterations, not tokens or euros — 256 by default.
Choice40 iterations, 16k output cap, compaction threshold per model on the measured window.
ResultLong chats compacted from 41.5k to 13.9k tokens, answer still correct.

Upgrades as part of the service

ProblemA moving tag pulls a different image on a new machine; database migrations don't roll back.
ChoiceDigest pinning, "new version" notice off, upgrades tested on the template first.
Result0.11.4 → 0.12.2 done with a checklist of known traps and a rollback path.

Measure the model before exposing it

ProblemDeclared context windows and real ones don't match.
ChoiceThree sources per model: engine error, provider metadata, real long prompt.
ResultOne model excluded for a 30k declared input window; thresholds set on data, not on labels.

Vector search only, no reranker

ProblemRegolo's reranker ranked Italian passages badly; a local one would need a GPU.
ChoiceSemantic search only, 20 chunks passed to large-context models.
ResultNo GPU, one provider. To be re-measured on the first customer's real documents.
08

Problems solved

Long prompts failed on the default model

SymptomHTTP 400 with only 6–12k tokens of input.
CauseThe provider silently injected a 200k output budget on that model.
FixExplicit 16k output cap — a 30k-token input now answers in 9.1s.

New models invisible to users

SymptomA model added by the admin never showed up for employees. No error.
CausePer-model grants — and the admin always sees everything.
FixVisibility decided by the connection's whitelist; every check now done as an end user.

Deleted wasn't deleted

SymptomA file removed from the app still had its text in the vector database.
CauseDeleting a file doesn't delete its embeddings.
FixThe purge removes vectors first; the technical residue is declared in the notice.

Admin "chat access off" — but not quite

SymptomWith chat access disabled, the analytics API still returned message text and reasoning.
CauseThe flag covers the interface, not the statistics endpoint.
FixKept, verified and declared — a layered wall, written up for the privacy consultant.

A sidebar full of "Empty Conversation"

SymptomChats opened with "ciao" got an English, useless title — forever.
CauseThe title is generated once, from the first exchange, in English when unsure.
FixItalian title template with a "Nuova conversazione" rule — 0.4s per title.

Modified Excel opened only after "repair"

SymptomExcel asked to repair the file the model returned.
CauseValues written into the totals row; the Excel Table not extended.
FixExplicit rules in the Excel model — file reopened clean.

Lesson: verify the content, not the status code — a 200 can carry an empty answer, or the web app's own HTML.

09

What's next

One-command deploy

Half of the configuration lives in the database, not in files. Today a new instance is a move of the template; the goal is a script that builds a clean customer instance from zero — keys, name, notice — and verifies it as an end user.

Hardening before the first customer

A real Let's Encrypt certificate on an EU VM, two-factor authentication for local accounts, off-site backups, and an alert when the service or the provider goes down.

Enhanced edition

Same workspace with a LiteLLM gateway in front of the models and full Microsoft 365 integration, for companies that outgrow the light version.

Word and PowerPoint in chat (planned)

The same seamless integration Excel has today, extended to Word and PowerPoint: create documents and decks with the company letterhead, or upload one and get it back edited. Candidate tools reviewed; code to be audited and pinned before any install.

A European web search

Replace the only US hop with a European engine when a customer asks for it — two candidates are already supported natively.

10

Some measured results

Measured on the template instance, not on a benchmark suite.

~1 GBapp RAM at rest, 8 GB VM
0.4schat title, task model
9.1s30k‑token input, default model
41.5k→13.9ktokens after compaction
256→40tool iterations per answer
4096embedding dimensions

Every change is logged with the date, the before/after value and how it was verified — which is what turns a configured app into a product that can be rebuilt for the next customer.