Natural-SQL-7B
AI
text-to-sql model
SHOW HN2024
How HN took it
10 positive, 21 neutral or asking, 2 mixed, 15 critical, out of 48 comments about the project. Thread score -0.16.
Idea: 3 for, 7 againstExecution: 5 for, 3 againstPerformance: 1 for, 4 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful as a first draft for SQL queries
From 2 comments
So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with…
rgbrgb on HNFor anyone who says it's useless because it's only 75% correct, please consider these two points: (1) this is the first instalment, and it's already close to be a thousand times…
zeroq on HN
Objections
Only about 76.5% accurate on SQL-Eval, behind larger models
From 1 comment
So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with…
rgbrgb on HN
Does not understand business meaning of column data
From 2 comments
I don't think any of those text-to-sql models are solving the right problems. The hard part is not syntax or I don't know how to write a group by query. Most data scientists and…
lolpanda on HNHard to claim success with "complex" questions if you don't account for business context and organizational nuances. For example, "active" listings on Redfin may be a combination…
ainesh93 on HN
Custom license with use restrictions is not open source
From 2 comments
This is not open source, because you have use-based restrictions. Call it what it is, source available.
satvikpendem on HNSo I need to ask my lawyer to review a custom license before I can consider using this? No thanks.
wantsanagent on HN
Requests
Support for other SQL dialects such as BigQuery, MySQL, DuckDB
From 2 comments
I would 100% use this if you had a text -> Big Query SQL feature. Please add.
floridageorgia on HNThe card states: > This model was evaluated on SQL-Eval, a PostgreSQL-based evaluation framework developed by Defog for testing and alignment of model capabilities. But this…
moltar on HN
Support for longer context windows for large schemas
From 1 comment
Context is 4096? My app db DDL is 19877 tokens (using Llama2 tokenizer) long, so that means we need to do a RAG for handling the DDL prompt injection. A model like this with a…
xfalcox on HN
Questions
Can it be used with Vanna or other tools under its license
From 1 comment
Very cool. Would the license allow for use with Vanna? https://github.com/vanna-ai/vanna
zainhoda on HN
How was the model fine-tuned
From 1 comment
Would be very interesting if you wrote a bit about what you did to actually do the finetuning
htrp on HN
Alternatives named
GPT-4 and ChatGPT
From 2 comments
So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with…
rgbrgb on HNI've been using GPT-4 to write SQL to get a bunch of insights. Its very good at writing up queries for all kinds of metrics. These queries answer questions that would have…
aussieguy1234 on HN
sqlcoder-15b
From 1 comment
So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with…
rgbrgb on HN
Vanna
From 1 comment
Very cool. Would the license allow for use with Vanna? https://github.com/vanna-ai/vanna
zainhoda on HN