1-Bit Bonsai
AI
local llm models
SHOW HN2026
430points
153comments
#71of 4,366 in 2026
Top 2%of 2026 by points
How HN took it
MixedNo clear lean
18 positive, 35 neutral or asking, 5 mixed, 14 critical, out of 72 comments about the project. Thread score +0.06.
Performance: 8 for, 10 againstIdea: 5 for, 1 againstExecution: 0 for, 3 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Impressive performance for its small size
From 3 comments
I ran my custom agentic SQL debugging benchmark against it and I'm impressed. Results: 8 passed, 0 failed, 17 errored out of 25 That puts it right between Qwen3.5-4B (7/25) and…
nl on HNYou can run this model on an iPhone via the latest update to this Locally AI app: https://apps.apple.com/us/app/locally-ai-local-ai-chat/id6741426692 For its size (1.2GB…
simonw on HNreally impressive for the size. Curious to see what happens when someone trains a 100B+ model natively at 1-bit.
robonot on HN
Runs on old CPUs and phones
From 2 comments
Don't have a GPU so tried the CPU option and got 0.6t/s on my old 2018 laptop using their llama.cpp fork. Then found out they didn't implement AVX2 for their Q1_0_g128 CPU…
wild_egg on HNanecdotal experience report: They link the (free) locally.ai iPhone app, but the bonsai model doesn't present in the list. You have to get it via settings. On my ancient SE-2,…
w10-1 on HN
Objections
Comparisons made against full-precision rather than quantized models
From 3 comments
I'm very skeptical of the advantage they're claiming here. The whitepaper [0] only compares these to full precision models, when the more interesting (and probably more…
fxwin on HNI feel like it's a little disingenuous to compare against full-precision models. Anyone concerned about model size and memory usage is surely already using at least an 8 bit…
hatthew on HNA bit misleading to say they take 14x less memory, no one is doing inference with 16-bit models.
WhitneyLand on HN
Output quality is poor in some tests
From 3 comments
> User: what color is the sky? > Assistant: .es | ) for the and > 1 / up to to > ( or > > in : > the2 > it it?1
fgfarben on HNThe 8B model response to my "Harry Potter knowledge-bench" question is too funny not to share. > *Fathers of Harry and James Potter*: - Sirius Black is the *father* of *James…
druskacik on HNAre you getting anything besides gibberish out of it? I tried their recommended commandline and it's dog slow even though I built their llama.cpp fork with AVX2 enabled. This is…
UncleOxidant on HN
Memory claim of 14x less is misleading
From 2 comments
A bit misleading to say they take 14x less memory, no one is doing inference with 16-bit models.
WhitneyLand on HNThe site says 14x less memory usage. I'm a bit confused about that situation. The model file is indeed very small, but on my machine it used roughly the same RAM as 4 bit quants…
andai on HN
Requests
Ollama and MLX support
From 2 comments
Extremely cool! Can't wait to give it a spin with ollama, if ollama could list it as a model that would be helpful.
keyle on HNEagerly waiting for mlx to merge 1bit quantization pr to try this out.
vx_r on HN
Larger 1-bit models
From 2 comments
i hope someone do a 100b 1-bit parameter model. that should fit into most 16GB graphics cards. local AI democratized.
ycui1986 on HNreally impressive for the size. Curious to see what happens when someone trains a 100B+ model natively at 1-bit.
robonot on HN
Questions
How to run on CPU or Android
From 3 comments
Does anyone know how to run this on CPU? Do I need to build their llama.cpp fork from source? (I think they only offer CUDA options in the release page, which I think might…
andai on HNHow do I run this on Android?
OutOfHere on HNany tutorial on how to run this on linux cpu only?
afaik69 on HN
Trade-off versus larger models
From 1 comment
What’s the trade-off? If it’s smaller, faster and more efficient - is it worse performance? A layman here, curious to know.
_fw on HN
Alternatives named
Microsoft BitNet
From 1 comment
is this somewhat similar to Microsofot Bitnet?
est on HN
Qwen3 quantized models
From 1 comment
I'm very skeptical of the advantage they're claiming here. The whitepaper [0] only compares these to full precision models, when the more interesting (and probably more…
fxwin on HN