ClickBench
Data
database benchmark
SHOW HN2022
137points
57comments
#441of 2,570 in 2022
Top 18%of 2022 by points
How HN took it
DebatedReal support and real criticism
8 positive, 11 neutral or asking, 1 mixed, 10 critical, out of 30 comments about the project. Thread score -0.01.
Idea: 1 for, 4 againstExecution: 1 for, 4 againstPerformance: 1 for, 1 againstDocs: 2 for, 0 againstPricing: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Impressive effort and useful open-source benchmark
From 2 comments
This is impressive work: it's time consuming to set up and benchmark so many different systems! Impressiveness of the effort notwithstanding, I also want to encourage people to…
gianm on HNBenchmarks can be quite difficult to interpret when the test parameters vary between tests. However, I think the point in providing the open-source ClickBench benchmark…
thomoco on HN
Transparent comparison is useful
From 2 comments
Seems like a great ad for c6a.metal ! Without dollars as part of the analysis, I don't find these comparisons useful.
kthejoker2 on HNYeah, also a comparison with SparkSQL on Kubernetes would be interesting! Let's get in touch, I may help with that.
Lucasoato on HN
Objections
Single-table queries only; no joins or complex query shapes
From 3 comments
It looks like the queries are all single table queries with group-bys and aggregates over a reasonably small data set (10s of GB)? I'm sure some real workloads look like this,…
AdamProut on HN> Regarding the joined table queries that are missing in the tests looks like huge issue with this benchmark.
riku_iki on HNThere are several existing benchmarks that test query optimisers with a lot of joins. It does not show performance of query engine, but more likely how good is your optimiser was…
qoega on HN
Druid results look misconfigured
From 2 comments
In this benchmark, druid was killed and restarted after every query because druid seems to get into a degraded state otherwise. Very very likely that there is something wrong…
abhishcool on HNIt does seem odd, especially since in real world cases I'm more accustomed to seeing Druid and ClickHouse be in the same ballpark of performance. Sometimes one is somewhat faster…
gianm on HN
Redshift and ClickHouse hardware not comparable
From 1 comment
Yea, Redshift looks like it was run on an ra3.xl (4 cores) and is comparing against clickhouse on a c6a.4xl (16 cores). I suspect if this were normalized Redshift would have an…
ruw1090 on HN
Benchmarks can be misleading and should not be over-trusted
From 1 comment
This is impressive work: it's time consuming to set up and benchmark so many different systems! Impressiveness of the effort notwithstanding, I also want to encourage people to…
gianm on HN
Requests
Add cost or price dimension to comparisons
From 2 comments
A small suggestion: since the machines aren’t always comparable (eg Redshift) maybe it makes sense to allow the user to make a plot of ($/hr) vs query time? I realize this adds…
singhrac on HNSeems like a great ad for c6a.metal ! Without dollars as part of the analysis, I don't find these comparisons useful.
kthejoker2 on HN
Add BigQuery
From 1 comment
Can BigQuery be added? I'm interested to know how it compares in these sorts of scenarios. We use BQ because I do NOT want to have to deal w/management of any of the…
garciasn on HN
Add Databricks
From 1 comment
This is great! Will it be possible to include Databricks in the benchmark?
arikfr on HN
Add clustered ClickHouse setups
From 1 comment
Do you have any benchmark results with Clickhouse in a clustered configuration (and other db's)? Is Clickhouse expected to be run in only single node deployments?
ruw1090 on HN
Questions
Why ClickHouse performs so well
From 1 comment
Anyone got a succinct explanation for why ClickHouse is doing so well in this kind of benchmark? What are the key tricks they're using to get great analytical performance, and…
simonw on HN
Why Snowflake performs so well
From 1 comment
Wow Snowflake absolutely crushes these benchmarks. Anyone know why? I don't know much about this space.
throwawaylala1 on HN
Alternatives named
Snowflake
From 1 comment
Wow Snowflake absolutely crushes these benchmarks. Anyone know why? I don't know much about this space.
throwawaylala1 on HN
StarRocks
From 2 comments
Full disclosure - I work for StarRocks (starrocks.com) First of all, this is great. Transparent and healthy competition is always great for the customers! Regarding the joined…
lk888 on HN--- The replication and sharding seems pretty difficult and from reading the documentation feels like it might be pretty fragile. That's why trip.com replaced Clickhouse with…
lk888 on HN
DuckDB
From 1 comment
I don't know where you draw the line between SQL analytics and SQL data warehousing. I think your typical analytical workload definitely involves more data then this benchmark…
AdamProut on HN
Apache Druid
From 1 comment
This is impressive work: it's time consuming to set up and benchmark so many different systems! Impressiveness of the effort notwithstanding, I also want to encourage people to…
gianm on HN