Stella Nera
Hardware
ai accelerator chip
SHOW HN2023
How HN took it
5 positive, 6 neutral or asking, 0 mixed, 1 critical, out of 12 comments about the project. Thread score +0.28.
Idea: 3 for, 0 againstPerformance: 1 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Exciting energy-efficiency results and potential for LLMs on devices
From 3 comments
I am surprised that they do not mention comparing against quantized matrix multiplication because their "encoding" appears to be something like a quantization step with unevenly…
fxtentacle on HNThat's the beauty of their method: If you can replace a 8192x8192 matrix multiplication with a 8192x256 decision tree and then a 256x8192 look up table, your memory requirements…
fxtentacle on HNMind blown. Sounds almost too good to be true except the human brain runs on 20W and this brings us to the same ballpark. This was hard scifi a year ago! Can an approach like…
baq on HN
Objections
Comparison against TPUv3 is unfair since TPU is a datacenter chip
From 1 comment
Thank you for the feedback :-) We have to be careful with the comparisons we make. The TPUv3 is a training and datacenter chip and not an Edge/Inference chip. They optimise for a…
joennlae on HN
RAM access latency would limit gains on regular CPUs
From 1 comment
I'd expect that on a regular CPU, the RAM access latency will destroy any performance improvements. This work is much better suited for FPGAs or ASICs.
fxtentacle on HN
0.5-bit quantization effectiveness questioned versus 4-bit schemes
From 1 comment
Based on the first figure in the paper, it seems that this scheme effectively turns 8 input values into a 4-bit number, thus giving an effective 0.5-bit quantization. Considering…
devit on HN
Questions
Where and how the technology can be used and its overall impact
From 1 comment
Ok, I get excited by seeing the numbers, but can someone please explain in a single sentence where this can be used and how big the overall impact would be?
jansan on HN
Whether LoRA-style adaptation is possible given hardcoded matmul
From 2 comments
This is a dumb question but I guess this means that you can't make something like a LoRA in software, right? Because the network is physically hardcoded?
ilaksh on HNNo, the network is not hardcoded, you can build any network you want. Only the expensive step, the matmul, is hardcoded.
ddalex on HN
Why the name Stella Nera
From 1 comment
As you are the author: why the name Stella Nera / Black Star?
pmontra on HN
Alternatives named
Quantized matrix multiplication
From 1 comment
I am surprised that they do not mention comparing against quantized matrix multiplication because their "encoding" appears to be something like a quantization step with unevenly…
fxtentacle on HN
Binary and ternary neural networks (BNN/TNN) and pruning
From 1 comment
Author here: Let me try to give an overview as I saw some questions repeating itself. * This accelerator is for an Edge/Inference case, so there is no training on this chip. * We…
joennlae on HN