Conventional AI service
The provider sees everything
Prompts, documents and generated answers are processed in the clear on someone else's servers.
Exposed
Private inference for mission-critical work
Artificial intelligence, under your control.
Run generative AI with privacy on the hardware you choose, even hardware you don't fully trust. With blind processing, your prompts, responses and keys never leave your side.
- Blind processing
- Keys in your custody
- On-premise, cloud or partners
01 The difference
More protection. Less exposure.
Conventional AI services receive your data in plain form and may retain your intellectual property, secrets, financial data and what you are working on. OmnyLLM keeps your information, secrets and interests under your control with private LLM sessions.
OmnyLLM
The GPU host sees neither side
It never receives your prompt or the generated response. It computes on obfuscated weights and activations while your side keeps the text, the tokens and the keys.
Kept on your side
-
Blind processing
The heavy part of the model runs remotely without access to your prompt, the generated tokens or the keys that protect them.
-
Institutional custody
Your organization controls and holds every key. No third party can unlock your protected data.
-
Flexible infrastructure
Run fully on-premise, in OmnyLLM Cloud, or on GPUs from our partners.
02 Blind processing
The GPU does the work. Your data stays on your side.
OmnyLLM splits each model in two. The parts that touch your words, the input and output layers, run where you trust. The heavy middle of the model runs on the GPU host with obfuscated weights, on obfuscated data it cannot read back.
-
Your side · input
Prompt stays local
Tokenized, embedded and transformed with keys only you hold.
-
GPU host
Computes blind
Runs most of the layers on obfuscated weights and activations. Never receives the prompt, the response or the keys.
-
Your side · output
Response stays local
Results are reversed with your keys and turned into tokens on your side.
-
Less exposure
The machine with the most compute never sees your prompt, tokens or protected data.
-
Less implicit trust
Security does not depend on the remote host's policies or administrators. It is enforced by how the model is split and executed.
-
No quality trade-off at runtime
Session transforms are exactly reversed on your side, with no noise added during inference.
03 Security by architecture
Four pillars. One architecture.
Protection isn't a policy layered on top. It's built into how OmnyLLM splits, stores, moves and executes a model.
-
Pillar 01
Blind processing
The model is split so the remote host computes without ever receiving your prompt or the generated tokens. Each model copy also carries a secret alteration that no inverse removes, so a host cannot simply undo the protection.
-
Pillar 02
Keys in your custody
Session keys, transform parameters and storage keys are generated and held on your side. Remote hosts get read-only access to the single part of the model they run, and nothing more.
-
Pillar 03
Encrypted storage and transport
Model files live in a zero-knowledge encrypted store (AES-GCM with Argon2id key derivation), and traffic between components runs over TLS.
-
Pillar 04
Exact, reversible transforms
Protection is refreshed as a session runs, yet every runtime transform is exactly undone on your side. You get the model's own answer, not an approximation.
04 Sovereignty
Run it where you decide.
Three ways to operate, with the same architecture and the keys on your side in every one.
your servers
On-premise
Everything inside your infrastructure
The whole model runs on your hardware. Network, operations and governance stay under your control.
any GPU
Private split
Use GPUs you don't own, without trusting them
Keep the input and output layers on your side and send only the obfuscated middle of the model to rented or third-party GPUs.
early access
OmnyLLM Cloud
We, or our cloud partners, run the GPUs. You keep the keys.
A managed option built on the same split: the infrastructure computes on obfuscated data while your side holds the prompt, the response and the keys.
05 The engine
One native engine, from laptop to data center.
OmnyLLM runs on an inference engine built 100% in-house, designed from scratch for private sessions and for deployment across multiple architectures. It is accelerated with native Metal and CUDA kernels. No Python runtime, no llama.cpp, no third-party inference framework.
- Acceleration
- Metal on macOS, CUDA on Windows, WebGPU, and a portable CPU backend everywhere. The best available backend is selected automatically.
- Models
- Llama, Qwen 2.5, Qwen 3, Qwen 3.5, Gemma 4 and LFM2 families, including hybrid architectures whose recurrent layers keep memory constant as context grows.
- Formats
- About 20 quantization formats, from 16-bit down to 1-bit and below. Hugging Face and GGUF models are imported into a single optimized
.omnyllmfile. - API
- An OpenAI-compatible server with streaming and tool calling that works with the OpenAI SDKs, LangChain, LlamaIndex and Open WebUI. Also available as an embeddable API and a CLI.
- Control
- Temperature, top-p, repetition penalty, stop sequences, and seeded, reproducible output.
27B in 3.8 GB
$ omnyllm_server --models ./models --accelerator metal
$ curl http://localhost:8080/v1/chat/completions \
-d '{"model": "…", "messages": […]}'06 Applications
Where OmnyLLM works today.
For your teams
-
Document intelligence
Summaries, search and assisted reading across internal archives.
-
Internal knowledge queries
Natural-language questions over institutional knowledge.
-
Institutional copilots
Assistants for operational and administrative teams.
-
Process automation
Classification, triage and support for critical workflows.
For your infrastructure
-
Private use of rented GPUs
Run larger models on cloud or partner GPUs without handing over prompts, responses or keys.
-
Drop-in OpenAI replacement
Point existing OpenAI SDK, LangChain or Open WebUI apps at your own endpoint by changing only the URL.
-
Edge and offline
1- and 2-bit models on laptops and air-gapped machines, with no network required.
OmnyLLM
Bring private AI to mission-critical environments.
Talk to our team about your environment, deployment model and licensing.