KimtAI

Prov1.1.0

A private on-device AI chat — every token stays on your device

KimtAI is a Pro app

It needs a free KimtOS account and a Pro subscription. Everything still runs on your device — Pro unlocks the app, it does not send your data anywhere. Install it without Pro and the app opens to an upgrade wall. See what Pro includes

Overview

KimtAI is a private AI chat that runs a real language model inside your browser tab — not in the cloud. You download a small instruction-tuned LLM once, and from then on every message is answered on your own machine. Not a single token is ever sent to a server. It is part of the kimtos new-tab app suite.

KimtAI is a chat assistant. It answers questions, drafts and rewrites text, and explains things — all locally. It does not act on your other apps: it cannot add a task, save a note, or change anything for you. Ask it to, and it will tell you so rather than pretend.

Highlights

• A real LLM in your tab — a small instruction-tuned model runs locally via Transformers.js (@huggingface/transformers), preferring WebGPU for speed and falling back to CPU (WASM) when no GPU is available.
• Fully on-device — the conversation never leaves the browser. There is no backend, no account, and no network round-trip for chat.
• The model — KimtAI runs a small chat model on your own machine, fetched on demand and cached, so it downloads only once. The exact model KimtOS ships is listed in Getting started; more are planned.
• Streaming replies — tokens stream into the message bubble as they are generated, rendered as Markdown.
• Per-chat tuning — set a system prompt (persona), creativity (temperature), and response length per conversation.
• On the home screen — an optional "Ask KimtAI" card gives one-click access, and the KimtOS home search bar can send your typed question straight into a new chat when the app opens. Turn the card off in Settings ▸ Preferences.
• Persistent history — conversations are saved locally; pin, rename, search, export to Markdown, or batch-delete with undo.

What runs where

The model runs in a Web Worker (kimtai.worker.ts) so token-by-token streaming never blocks the UI. It loads the model with Transformers.js and streams tokens back to the app. Nothing else is involved — there is no server and no second process.

Privacy model

KimtAI writes only to local browser storage and never uploads your data:

• Chats (messages, title, per-chat config) are records in the shared IndexedDB (chats collection).
• Model files are cached by Transformers.js in Cache Storage (transformers-cache); which models you have downloaded is tracked in a small local key/value set (kimtai:downloaded).
• Chat itself makes no network calls. The one exception is explicit and user-triggered: downloading a model's files. That happens once, and the model runs offline from then on.

Getting Started

Open KimtAI from the dock. Before you can chat you need the model on your device, so the first thing to do is download it.

The model

KimtAI ships 3 models. Download the ones you want, see each one's size on your device, and pick which to chat with. Each is fetched on demand and cached, so it is downloaded only once.

Model	Parameters	Download	Needs a GPU	What it is for
SmolLM2 360M	0.36B	~300 MB	No	Small, fast chat model — runs even without a GPU. Great for quick questions and drafting.
TinyLlama 1.1B	1.1B	~700 MB	No	A well-known small chat model — noticeably better answers than the tiny models; a bit heavier.
SmolLM2 1.7B	1.7B	~1.4 GB	No	The largest model here — the best quality, but it wants a strong machine and ideally a GPU.

SmolLM2 360M is downloaded on install.

Click Models in the header to see it, with its size and — once you have used it — its measured speed on your machine.

• Download — fetches the model's files once and caches them; a progress bar shows the one-time download. You can close the overlay and it continues in the background.
• Cancel — sits next to the progress bar while a download is running, and is still there if you close the overlay and reopen it. It stops the transfer for real. Files that had already arrived in full stay cached, so downloading again picks up from them rather than starting over. There is no Pause: the loader fetches whole files and cannot continue one from where it stopped, so a Pause button could only throw the current file away — which is what Cancel does.
• Remove — frees its cached files (your other apps are untouched). You can download it again later.

It is only ever downloaded when you ask; nothing is prefetched behind your back.

Start chatting

Type in the composer and press Enter (Shift+Enter for a newline). The first time you send in a session, the active model loads — you will see "Downloading model…", then "Warming up…", then replies stream in token by token.

• Stop — the Send button becomes Stop while generating; click it to interrupt.
• Retry — regenerate the last reply.
• Edit & resend — edit one of your own messages to branch the conversation from there.
• Copy — copy any reply.

If you have no model yet and try to send, KimtAI opens the Model Library for you.

GPU vs CPU

If your browser supports WebGPU, models run on the GPU (fast). If not, KimtAI falls back to CPU (WASM), which is much slower — so a small model is recommended there. The model chip in the header shows which the current model is using (· GPU or · CPU), and replies show a rough tokens/second rate.

To save memory, the resident model is freed automatically when idle (after a few minutes, or sooner when the tab is hidden) and reloaded on your next message.

Tune a chat

Click the sliders icon to adjust the current conversation:

• System prompt — the persona / behaviour for this chat (the default comes from the app's settings).
• Creativity — temperature, from Precise to Wild.
• Response length — Short, Medium, Long, or Max.

These are saved per conversation, so different chats can behave differently.

Manage conversations

The left sidebar lists your chats. Every conversation is saved locally as you go.

• New chat — start a fresh conversation.
• Pin / Rename — from a chat's menu.
• Export .md — download a conversation as a Markdown file.
• Search — filter chats by title.
• Delete — remove one, or multi-select to batch delete with an undo.

What KimtAI cannot do

KimtAI chats — it does not act. It cannot add a task, save a note, edit an image, or change anything in your other apps, and it cannot search the web. Ask it to and it will tell you it cannot, rather than claim it did.

This is a deliberate limit, not an oversight. Models small enough to run on your own device — with no server and no data leaving the machine — are not yet reliable enough to be trusted with actions on your behalf. A model that confidently reports doing something it never did is worse than one that simply says no, so until an on-device model can do it dependably, KimtAI stays a chat assistant.

If you want your notes and PDFs searched and answered from, that is Thoth — a separate app that indexes your own content and answers with citations.

Architecture

KimtAI has two live parts: the app UI (chat thread, composer, Model Library) and the engine that runs the model in a Web Worker, off the main thread. The code lives in extension/src/apps/kimtai/ and extension/src/lib/.

A third part — an agent runtime that would let the model call the other apps' tools — exists in the tree but is switched off; see The dormant agent runtime below.

Modules

File	Role
apps/kimtai/meta.ts	The app's identity — id (kimtai), name (KimtAI), accent, order, plan, and marketing copy.
apps/kimtai/index.ts	The app descriptor and UI — chat sidebar, thread, streaming composer, per-chat tuning, and the Model Library overlay. Answers every message with plain chat, and persists chats via the local API (chats collection).
lib/engines/kimtai.ts	The on-device chat engine — the model catalog, the downloaded-models set, measured-speed tracking, cache add/remove, and createChat(), which spawns and talks to the worker.
lib/engines/kimtai.worker.ts	The Web Worker that loads the LLM with Transformers.js (@huggingface/transformers), prefers WebGPU and falls back to WASM, and streams tokens back to the app.
lib/agentTools.ts	The dormant agent runtime (see below) — it would collect each installed app's tools, build the system prompt, parse the tool call the model emits, run it on-device and loop. Not reachable today.

The dormant agent runtime

Every app in the suite declares its capabilities as an MCP-shaped tools array on its descriptor — 95 tools across 21 apps — and lib/agentTools.ts contains a complete runtime to drive them. None of it runs. KimtAI answers every message with plain chat.

This is deliberate. The runtime is only as good as the model driving it, and the models small enough to run on a user's own device are not good enough yet. Measured on our own prompt and parser, the default model (SmolLM2-360M) got 1 of 6 tool calls right — and its failures were the dangerous kind: asked "what tasks do I have today?" it answered "You have no tasks today" without ever calling the tool. A model that confidently reports doing something it never did is worse than one that cannot act at all.

So the tools stay declared and the runtime stays in the tree — inert, costing nothing at runtime — until an on-device model can be trusted with them. Keeping the declarations means switching the feature on later is a runtime change, not 95 rewrites.

The research into which small model (if any) can do this reliably is tracked separately, along with whether we speak MCP to anything external at all.

How a message is answered

Every message takes the plain chat path: one streamed generation, no tools. The code still contains the branch that would enter the agent loop, but no shipped model satisfies its condition, so it is never taken.

Flow

The model always runs in the worker; the app talks to it only through postMessage. Nothing in this diagram crosses the network except the one-time model download. The dashed edge to the agent runtime is the path that is never taken.

Storage

• Chats live in the shared IndexedDB chats collection, saved through the suite's local API (getApi / jsonApi) — the same router the other apps use.
• Model files are cached by Transformers.js in Cache Storage (transformers-cache); removing a model deletes only that model's cache keys.
• Which models are downloaded and their measured speeds are tracked in the shared key/value store (kimtai:downloaded, kimtai:speed), and per-chat settings persist per conversation.

Changelog

1.1.0 — 2026-07-27

• Give the extension a session model, and run the apps in one tab

1.0.5 — 2026-07-23

• Stop now works while the model is still loading: the reply you cancelled is never written, the composer goes straight back to Send, and the message you had already sent is kept. The model finishes loading in the background, so your next message does not wait for it again

1.0.4 — 2026-07-22

• Stop takes effect the moment you press it: the reply keeps the text streamed so far instead of waiting for the model to wind down
• Leaving a conversation mid-reply no longer releases the model — your next message reuses it instead of loading it again