Open source · MIT · macOS, Linux, Windows

Local models,
wired up in one command.

FLE Link sits between your inference engine and everything that talks to it. It reads your machine, installs the engine that fits, downloads a model that fits, picks the settings and serves an OpenAI-compatible API. Nothing to configure.

curl -fsSL https://flelink.run/install.sh | sh

One file. No Python, no git, no sudo. Checked against its SHA-256 before it runs.

66 s: from nothing to the first answer on Linux with an NVIDIA card
92 s: the same on an 8 GB M2 MacBook, model download included
91 tok/s: a 125B MoE on one 32 GB GPU with the FLE engine, 2.5× plain llama.cpp

first run · M2 MacBook, 8 GB

$ flelink ask llama-3.2-1b "Name the capital of France in one word."

==> Engine: llama.cpp for Apple Silicon (Metal)
==> Installing the engine (checked against the SHA-256 GitHub publishes)
    engine llama-b11540-bin-macos-arm64.tar.gz  100 %
==> Downloading Llama 3.2 1B Instruct (Q4_K_M) (0.8 GiB)
    model Llama-3.2-1B-Instruct-Q4_K_M.gguf    100 %
    starting the engine ...

Ready: llama-3.2-1b on llama.cpp for Apple Silicon (Metal)
  API: http://127.0.0.1:8080/v1   (OpenAI-compatible, no key needed)

> Name the capital of France in one word.

Paris

[2 tokens, 76.1 tok/s decode, 240 tok/s prompt]

Flagship models

Big mixture-of-experts models on one workstation

The FLE engine keeps the experts a token needs on the GPU and the rest in RAM. FLE Link picks the build and the settings for your machine.

Measured

Qwen3.8-Flash-Next · FLE

125B MoE · FLE quant, 88.9 GB · on Hugging Face

FLE91 tok/s

Strata74 tok/s

llama.cpp36 tok/s

Decode, median over six tasks, same machine and model: one 32 GB RTX PRO 4500, 120 GB RAM, 16 cores (6 Oct 2026). Smaller than Unsloth's IQ4_XS at the same KL divergence to BF16. Prefill (reading long prompts) was where FLE trailed in this run; it has been reworked since, and new numbers follow when they are measured the same way.

Measured

DeepSeek-V4.1-Flash-1017

763B MoE, 384 experts · GSQ-RCO IQ2_XXS by ISTA-DASLab, 232 GiB

23.3–23.5 tok/s, two runs

One 32 GB GPU RTX PRO 4500, 18 GB expert cache, RAM capped at 102 GB, the rest read from NVMe; decode after warm-up, 16k context, loads in 30 s
FLE engine DeepSeek-V4.1 support is merged; FLE Link installs it with the next engine release
Today it also runs through FLE Link on llama.cpp b11870 or newer

Where it sits

The layer between the engine and your tools

Inference engines are fast and bare: you pick the build, the flags, the context, the cache. Chat apps, editors and agents just want an endpoint. FLE Link does the part in between, and the engine underneath can be swapped.

Your tools

Chat page
Editor
Agent
Any OpenAI client

FLE Link

reads the machine
checks fit before downloading
installs and verifies the engine
picks the start settings
supervises and restarts
shows what the engine does

Engine

FLE expert offloading for big MoE models llama.cpp the upstream build, as fallback Your build any llama-server compatible binary

How it works

Three steps, each one you can read

It reads the machine

CPU, memory, GPU and free disk, and which engine build fits.

$ flelink system
OS        macos arm64
CPU       Apple M2 (8 cores)
Memory    8.0 GiB
GPU usable 5.3 GiB

Engines that fit (best first):
  llama.cpp for Apple Silicon (Metal)

It checks fit before downloading

No 80 GB download that ends in an out-of-memory error.

$ flelink models
llama-3.2-1b      0.8 GiB  fits
llama-3.2-1b-q8   1.2 GiB  fits
fle-iq3-m        82.8 GiB  too big

It serves, and shows the command

Every start is a plain llama-server command you can copy and run without FLE Link.

Ready: llama-3.2-1b
  API: http://127.0.0.1:8080/v1
  Command (shown, never needed):
    llama-server -m Llama-3.2-1B-Instruct-Q4_K_M.gguf
      -c 32768 --jinja --metrics -ngl 99

Ways in

Terminal, browser, desktop or API

Terminal

flelink opens a full-screen interface: pick a model, watch it load, chat. flelink ask answers one question and cleans up.

TUI recording

Studio in the browser

flelink studio on 127.0.0.1:8642: chat with live speed numbers, models, the engine log, launch profiles.

Desktop app

The same Studio in a window. Quitting stops the server and the model.

App screenshot

API

OpenAI-compatible /v1. Point any client at it.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hi"}]}'

Host mode

Run it on the GPU box, use it from the laptop

flelink studio --host 0.0.0.0 opens the Studio to your network. Other machines sign in with an access key, over https with a certificate of its own whose fingerprint is printed at start.

$ flelink studio --host 0.0.0.0
FLE Link is open to the network. Access key:
  4-yB…Tc
FLE Link 0.1.0 — https://127.0.0.1:8642/
Other machines: https://192.168.1.20:8642/
Certificate fingerprint (SHA-256): 9f3c…

Tested by breaking it

What happens when things go wrong

Every command runs in the test suite twice: once as intended, once with something broken on purpose. On macOS and on Linux.

Download killed halfway The next start resumes where it stopped and checks the hash.
No network A clear message in under a second, no hang.
Broken model file Cut off, empty or random bytes: refused with a reason.
Engine crashes The status says so, the API answers 503, a restart works.
Program killed The engine it left behind is stopped before a second one loads.
Port taken The next free port, and it tells you which.
Wrong key on the network Refused, and every wrong guess waits a second.
Bad requests Broken JSON, huge bodies, path tricks: all refused, the server keeps running.

Roadmap

Open, and built to be changed

MIT licensed. The engine sits behind an adapter, so a new engine is a new adapter, not a fork.

0.1
Now one command from nothing to an answer, any GGUF from Hugging Face with a fit check, Studio, desktop app, host mode, engine adapters for FLE, llama.cpp and your own build
next
Plugins recipes and engine adapters as files you drop in
later
Quantization make a quant that fits your machine
later
Training LoRA adapters on your own data

Questions

Before you install

Does anything leave my computer?

No. FLE Link only connects to GitHub (to fetch the engine and its checksums) and Hugging Face (when you download a model). Nothing is sent to us, there is no account and no telemetry.

Which models work?

The built-in catalog shows what fits your machine. Any GGUF file works with flelink run /path/to/model.gguf. The FLE engine is made for large mixture-of-experts models on small GPUs, see FastLocalExperts.

macOS says the app is from an unidentified developer

The builds are not signed, to keep the project free of yearly fees. Right-click the app, choose Open, then confirm once. The command-line install is checked against its SHA-256 instead.

Windows SmartScreen warns about it

Same reason, the build is unsigned. Choose More info, then Run anyway.

Can I use my own llama.cpp build?

Yes. Point FLE Link at any llama-server compatible binary; it reads --help to see what the build can do.

Local models,wired up in one command.