Tiny-vLLM
AI
inference engine
SHOW HN2026
205points
18comments
#280of 4,366 in 2026
Top 7%of 2026 by points
How HN took it
LovedMost comments that take a side are positive
10 positive, 2 neutral or asking, 1 mixed, 1 critical, out of 14 comments about the project. Thread score +0.52.
Docs: 8 for, 0 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
README is a clear, practical lesson-style walkthrough of inference internals
From 5 comments
Really practical teaching approach. I clicked in to see how safetensors are loaded and just kept reading. Thanks for sharing.
janalsncm on HNGreat README. Genuinely one of the clearest walkthrough of inference internals. The KV cache section is worth lingering one as most of the OOM and throughput issues trace back to…
paralleliq on HNThe lesson-style README is a great approach. Breaking down LLM inference into digestible steps makes the codebase approachable even for people who haven't touched CUDA before.
xuanlin314 on HNI love the documentation formatted in lessons. I can't wait to read through it.
nazgulsenpai on HNI feel like I learned twice as much in 10 minutes reading this than I did reading LLM for Dummies. Thank you
tom-wal on HN
Approachable for people new to CUDA and LLMs
From 3 comments
I am not super familiar with C and CUDA, so I read solely for the README and enjoyed it supremely. The blend of cheerful walking through instructive examples and your…
lukemerrick on HNThe lesson-style README is a great approach. Breaking down LLM inference into digestible steps makes the codebase approachable even for people who haven't touched CUDA before.
xuanlin314 on HNThanks for sharing this. As someone currently researching LLMs, I'm sure I'll be referencing this quite a bit going forward.
GoldenJade on HN
Objections
README is dense and could use schemas
From 1 comment
love the details you put into to explain different techniques. it's a bit dense though, some schemas will help i think
quanglee on HN
Author skips CUDA API return value checks, against the tiny goal
From 1 comment
It seems the author believes checking the return values of CUDA API calls is not "tiny" enough :-(
einpoklum on HN
Requests
Wants C or assembly LLM inference implementations
From 1 comment
I am looking at a plain and simple C implemented LLM inference, and/or x86_64 assembly implemented, and/or AMD GPU RDNA assembly. Anybody?
sylware on HN
Alternatives named
llama.cpp
From 1 comment
Looks interesting, it reminds me of the first llama.cpp, but better documented.
juancn on HN