altAI
An open-weight model split across several computers, behind one OpenAI-compatible endpoint. A machine that cannot fit a model scores zero, not slow. Several of them together can serve it.
Compute you can wait for. Capacity you either have or you do not.
Download · macOS
The engine is inside. No terminal, no Python, no Docker, no address to type — the network's address is already in the app. You will need a join token; the network is invite-only while it is this young.
xattr -dr com.apple.quarantine "/Applications/Altai Node.app"
export ALTAI_JOIN_TOKEN=<your token>
open "Altai Node.app"Unsigned, so macOS asks once — that is the xattr line.
Both builds run on macOS 12 and later, measured rather than claimed:
every library in each archive is built against that floor. The Intel engine gives up
Accelerate’s BLAS to get there, which makes prompt processing slower on an Intel
Mac — there is no Metal on those to take over. Windows and Linux are in the pipeline
and not posted yet.
What a token gets you
An ordinary token can only ever make your machine a stage: it runs a block of layers and nothing else. It never downloads a model, it is reached only by its own group's head, and nothing on the internet can dial into it. Being trusted to drive a group is a separate privilege the token has to carry, and we do not hand that out.
Architecture
The coordinator decides who plays which role. It never holds model weights and never sees a group's internal traffic.
Holds a complete model. Serves a request start to finish.
Holds the embedding and output layers plus its own block. Drives its group and emits every token.
Holds a contiguous block of middle layers. Runs them for every token. Never downloads the model file.
The argument
The binding constraint on a frontier-class model is memory, not FLOPs. A fast GPU with 24 GB cannot hold a 600 GB model at any clock speed. Twenty of them can.
And capacity is what is scarce. Datacenter servers are supply-constrained — vendors say so in their own filings and expect it to persist — while the memory in workstations, gaming machines and office hardware is already built, already powered, and mostly idle. It is not waiting on a fab.
So the aim is not to beat a hosted API on price or latency. It is to serve a model that otherwise could not be served at all, for the people who need it, on machines that already exist.
The target
Sparse mixture-of-experts models — DeepSeek-V3, Kimi-K2 — are the problem this exists to solve. They are where the memory wall is worst and where it is most tractable: only a fraction of the experts fire for any token, so the weights that must be resident are far smaller than the weights that must be available.
The route is pipeline stages across a low-latency regional group, each stage running its own block of layers with that block's experts offloaded to ordinary system RAM. Experts never cross the network. Only hidden states do — on the order of 16 KB per token per hop. That is what makes a machine with plenty of RAM and a modest GPU useful, instead of only machines with datacenter cards.
This route does not run yet
In the engine we use, expert offload and pipeline parallelism are currently mutually exclusive — switching on offload disables the pipeline, verified in the engine's own source. The upstream change that would lift it is open and unmerged. Until then the sparse frontier is the design target and not a capability, and nothing here should be read as saying otherwise.
State
Roadmap
Intentions, not dates. Each one is listed because something concrete blocks it, and the blocker is named.
The shape we are building toward
Ten machines of 64 GB, in one metro, serving a DeepSeek-V3-class model at roughly 10–20 tok/s. A target, not a result — nothing like it has been run.
Two numbers in that sentence are not taste. Per-token time across
N stages is W/(N·B) + N·L: the compute term falls
as 1/N, the latency term grows as N. Past
N* = √(W/(B·L)) each extra machine makes it
slower — about four or five at residential latency. And availability
compounds: ten nodes at 99% give 90%, forty at 95% give under 15%.
So it is ten large machines close together, not forty small ones far apart. More-and-smaller is the intuitive answer and the arithmetic says it is wrong.
Limits
It does not serve frontier models today. What has actually run is a 14B open-weight model split across two Macs; the near-term target is 32B–70B dense. The sparse-frontier ambition above is the problem being worked on, not a description of what you would get this morning.
It is not cheaper than a hosted API. At the speeds a pipeline across several machines runs, electricity alone costs more per token than the market price of a hosted model. The argument is capability, not price.
A receipt will be an attestation, not a proof. When they exist, a receipt will say that a node asserted it produced an output. A dishonest node can sign a false claim, and nothing here will imply otherwise.
Operators
One file, opened. It profiles the machine, proposes how much to contribute, opens a page in your browser, and joins. No terminal, no Python, no Docker, no address to type. You set the ceilings; the coordinator never asks for more than you allowed.
export ALTAI_JOIN_TOKEN=<the token you were given>
open "Altai Node.app"A stage downloads nothing — its head sends it the tensors for its own layers. Only a
head or a whole node needs the model file on disk.