Semantic Grep
Dev tools
text search cli
SHOW HN2024
356points
57comments
#141of 3,515 in 2024
Top 5%of 2024 by points
How HN took it
CuriousMostly questions and neutral comments
5 positive, 20 neutral or asking, 2 mixed, 4 critical, out of 31 comments about the project. Thread score +0.07.
Idea: 3 for, 3 againstDesign: 1 for, 0 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Clever, useful idea
From 4 comments
That's totally clever and sound really useful. And it's one of those ideas where you go "Why didn't I think of that" when stumbling over the materials, word2vec in this case.
onli on HNThis would be really useful if it could take a descriptive phrase or a compound phrase (like SQL 'select X and Y and Z') and match against the semantic cluster(s) that the query…
randcraw on HNThis tool seems really cool, want to play with it for sure. Just in general, semantic search across text seems like a much better ux for many applications. Wish it was more…
sam_perez on HNI really like how simple the implementation is!
tgw43279w on HN
Objections
Slow compared to grep
From 2 comments
I take a Show HN to be a solicitation for feedback. It took me a few tries to get this to run. I tried passing -model_path, but it complained that it couldn't find config.json. I…
SloopJon on HNHow fast is it?
pgroves on HN
Word2vec cannot handle phrases or negation
From 2 comments
The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even…
throw156754228 on HNThis implementation does not because the query has to be a word. One way to extend it to phrases is to average the vectors of each word in the phrase. Another way is to have a…
arunsupe on HN
Setup and configuration were hard to get working
From 2 comments
This is a good idea. I'm going to offer some unsolicited feedback here: The configuration thing is unclear to me. I think that "current directory" means "same directory as the…
samatman on HNI take a Show HN to be a solicitation for feedback. It took me a few tries to get this to run. I tried passing -model_path, but it complained that it couldn't find config.json. I…
SloopJon on HN
Requests
Multi-CPU parallelism
From 1 comment
I wonder if it would be possible to easily add support for multiple CPUs? It seems to be taking at most 150% CPU, so on my workstation it could be (assuming high parallellism) 10…
_flux on HN
Performance optimizations such as BLAS or SIMD
From 1 comment
Some small tips from superficially reading the code: https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab4fc4fdc1d4ef86092a071e48455/sgrep.go#L96 You can read the vector…
microtonal on HN
Support for phrases and multiword queries
From 2 comments
This would be really useful if it could take a descriptive phrase or a compound phrase (like SQL 'select X and Y and Z') and match against the semantic cluster(s) that the query…
randcraw on HNThis implementation does not because the query has to be a word. One way to extend it to phrases is to average the vectors of each word in the phrase. Another way is to have a…
arunsupe on HN
Questions
How fast is it on large corpora and does it use a GPU
From 2 comments
I might have a work use case for which this would be perfect. Having no experience with word2vec, some reference performance numbers would be great. If I have one million PDF…
fbdab103 on HNHow fast is it?
pgroves on HN
Can it search filenames
From 1 comment
Really cool. Often just want to fuzzy search for a word, and this would be useful. Can it do filenames as well ? Or do I need to pipe something like LS first.
gunalx on HN
Alternatives named
Semgrep
From 1 comment
Fyi, there is already a widely used tool along with a company called semgrep, which stems from semantic grep: https://semgrep.dev/.
fzeindl on HN
Semantra
From 1 comment
You might be interested in Semantra: https://github.com/freedmand/semantra
ttpphd on HN
fltr
From 1 comment
Similar: https://github.com/moritztng/fltr Like grep but for natural language questions. Based on Mistral LLMs.
molli on HN
Sentence-BERT embeddings
From 1 comment
Definitely a limitation of word2vec as applied here. Something like SBERT addresses this.
djoldman on HN