Needle: an 8-29 MB automation model that runs on tiny devices

Adding a little AI to a phone, a watch or a microcontroller usually means calling a cloud API and accepting the latency, the privacy exposure and the per-call bill. Needle is Cactus Compute’s open-source automation foundation model for exactly those devices: the whole model ships as a single 8-29 MB binary that does tool calls, structured extraction, text embeddings and speech recognition on the device, with 14,245 GitHub stars as of October 23, 2026, under Apache-2.0. Its most interesting trick is a ladder: the network is trained so that every depth from 2 to 20 layers is a deployable model, so a small device can run a shallow slice of the same weights, and the fine-tune-to-export tooling around that is already in the box.

Core features

• Tool calls: hand it your function signatures and Needle picks the functions and fills every argument. Ask for two things and you get two calls in order; a request no tool covers returns an empty list. Each response carries a calibrated confidence score you can route on: act, escalate to a human, or refuse.
• Structured extraction: declare a schema, hand over messy text, get typed fields back — an invoice, a booking, a form. A byte-level decode grammar constrains every token, so the output always parses; point the same machinery at an enum schema and it becomes classification.
• Whistle speech-to-text: one 16.9 MB file on the CPU, covering English, German, French, Spanish, Italian, Dutch and Polish. It shares Needle’s .cact container and C++ engine, so audio in and tool calls out is one engine call rather than two runtimes glued together.
• Text embeddings: the same model returns a vector for a sentence, so search, matching and routing stay local.
• One engine, 13 platform folders: the official devices guide lists 13 platform folders, each engine under 1 MB, covering macOS, Windows, five Linux targets (x86-64, ARM64, ARMv7, RISC-V, MIPS), Android, iOS, watchOS/tvOS, the browser (wasm) and WASI hosts; the weights are one needle3.cact file on Hugging Face (about 35 MB).
• Fine-tune at any depth: local needle finetune trains LoRA adapters at 4-bit, or you hand a key to the hosted platform for 2-bit full-model training. Per the README, fine-tuning on DroidCall lifts every subnetwork by 18-36 points, and from 4 layers up the tuned subnetwork passes DeepSeek V4 Flash starting at 29M parameters.

Typical use cases

• Smart homes and wearables: one spoken sentence toggles a light or sets a timer, with the speech and the tool call closed locally — no network, no per-call cost.
• Offline extraction inside an app: tickets, notifications and forms become typed fields, in airplane mode.
• Robots, cars and MCUs: places where memory is counted in megabytes and outputs must be deterministic; pick a shallow rung and fine-tune it.
• Driving a full desktop is a different problem: for that, see Agent-S, which works from screenshots and clicks; Needle calls the device’s own functions.

Quick start

Python 3.9+; the README’s install is one line:

pip install cactus-needle

A first tool call takes about a dozen lines:

import needle

@needle.tool
def get_weather(city: str):
    "Get the current weather for a city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?")["results"])
# [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

The engine and weights download from Hugging Face on first run and are cached; to install nothing at all, open the browser sandbox at cactuscompute.com/needle. A community-maintained cactus-needle npm package runs it in Node and browsers over WebAssembly.

Benchmarks

The numbers below are the project’s own (the README’s benchmark chart): Needle 3 runs through the shipped 2-bit binary with the confidence gate on, baselines run at f16 under vLLM, and DeepSeek V4 Flash goes through its cloud API.

Model	Size	Mobile Actions accuracy
DeepSeek V4 Flash	cloud	88.4%
Needle3-20L-121M	121M	86.0%
LFM2.5 1.2B	1.2B	82.4%
Needle3-16L-98M	98M	80.7%
Qwen3.5 0.8B	800M	76.0%

Mobile Actions is exact-match tool calling on a 961-row test split, and this is the claim behind the README’s “beats models 10x its size” line: the 121M Needle 3 lands above LFM2.5 1.2B, ten times its size. The same chart shows the price: the 52M rung scores 36.8% and the 29M rung 11.7%, so shallow rungs lean on fine-tuning. On speech, the project’s own measurements put the 16.9 MB Whistle at 8.6x smaller than Whisper base and 6.6x faster to the first token on 10 seconds of audio (Apple M4 Pro); it leads on both LibriSpeech test sets, SPGISpeech, Earnings-22 and the FLEURS average, and trails Whisper base on TED-LIUM, AMI and the MLS average.

Summary

Needle suits teams automating devices and apps at the edge, where cloud latency and per-call billing do not fit, and developers adding a local voice entry point get Whistle in the same runtime for free. It does not suit anyone looking for a general chat model; the README opens by saying chat capacity was traded away.

The trade-offs are stated plainly. Quantization and ladder slicing cost capability: the README’s own phrasing is that extraction matches models 2-3x bigger, and the gap to the cloud model on the two SNIPS extraction sets is far wider than on tool calling. The prebuilt engines ship with telemetry on by default; set NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1 to turn it off, and remember that for an embedded product. Every benchmark is the project’s own, and we could not rerun it. The repository was created on February 24, 2026, was last pushed on October 19, 2026, and is not archived; the needle3 weights repo on Hugging Face shows about 112k downloads over the past 30 days (checked October 24, 2026). We verified the README, llms.txt and repository metadata, and we did not install or run it. For local speech synthesis to pair with recognition, see our earlier entry on VoxCPM.