Bistro
Data
data processing engine
SHOW HN2018
95points
29comments
#456of 2,155 in 2018
Top 22%of 2018 by points
How HN took it
MixedNo clear lean
4 positive, 6 neutral or asking, 1 mixed, 6 critical, out of 17 comments about the project. Thread score -0.13.
Idea: 4 for, 2 againstDocs: 0 for, 3 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Ideas around linking and column operations are interesting
From 3 comments
Ok so you're defining new operations on top of existing primitives. Makes sense! The concepts look more interesting than the library right now (it doesn't move the needle for me…
agibsonccc on HNI'm working on a columnar Python DSL right now, I think of it like SQLAlchemy for Pandas/Spark/Flink. My goal is to create a language that makes the cumbersome parts of the…
munro on HNThis is neat. Vectorspace based AI calculations will benefit from this approach. Great work!
KasianFranks on HN
Objections
Not clear what problem it solves or why use it over SQL or pandas
From 3 comments
An example would be great. Can you show how to do a given task with SQL, map/reduce, and your framework? Because right now I have no idea why I’d choose to learn this new stuff…
jitl on HNThe core looks close enough to dataframes that I'd be curious to know how you compare to tablesaw: https://github.com/jtablesaw/tablesaw This looks neat but I'm not sure why I…
agibsonccc on HNMight new a cool idea, but not nearly fleshed out enough. I think a larger example instead of just individuals lines of code would be useful. Show a toy widget sales spreadsheet.…
jnordwick on HN
Lacks documentation and examples
From 2 comments
Agreed -- a motivating example in the Readme would really help people totally new to this concept grasp the how/why of the project.
hoprocker on HNMight new a cool idea, but not nearly fleshed out enough. I think a larger example instead of just individuals lines of code would be useful. Show a toy widget sales spreadsheet.…
jnordwick on HN
Code style issues
From 1 comment
Sorry for being that guy, but I just clicked into a random file in src to read the code, and found the code style (indentation etc.) to be quite weird…
krat0sprakhar on HN
Requests
Persistence and connectors such as Arrow, Parquet or JDBC
From 2 comments
The arrow integration and persistence and connecting this to jdbc would be an MVP to me that would at least allow folks to imitate pandas. While you do have new ideas here if…
agibsonccc on HNAt the physical (storage) level it is in-memory and organized as a column store, that is, internally it stores a list of tables (with no data) and a list of columns (each being a…
asavinov on HN
Python DSL or Python API
From 2 comments
I'm working on a columnar Python DSL right now, I think of it like SQLAlchemy for Pandas/Spark/Flink. My goal is to create a language that makes the cumbersome parts of the…
munro on HNYou are right - it is an MVP, and the goal is to choose a direction. In fact, I am still not sure in which direction to go: * JavaScript data processing framework for in-browser…
asavinov on HN
Backend agnostic execution such as Spark or Flink
From 1 comment
Ok so you're defining new operations on top of existing primitives. Makes sense! The concepts look more interesting than the library right now (it doesn't move the needle for me…
agibsonccc on HN
Questions
How does it differ from views or derived columns
From 1 comment
I skimmed the readme but didn't see the answer to what I regard as a basic question. How is this different from a view? I can easily make derived columns based on functions and…
nickpeterson on HN
How does it compare to tablesaw, SAS DATA steps, or Spark
From 2 comments
The core looks close enough to dataframes that I'd be curious to know how you compare to tablesaw: https://github.com/jtablesaw/tablesaw This looks neat but I'm not sure why I…
agibsonccc on HNHow do that compare to SAS software (https://en.wikipedia.org/wiki/SAS_(software)) ? Particularly the "DATA" steps.
julienfr112 on HN
Does it support in-memory only, sharding, or time series
From 2 comments
Is it in-memory? Does it support replication or sharding? What's the main use-case? How does it differ from ORC, Parquet or Arrow? The repository doesn't have any information.
buremba on HNMight new a cool idea, but not nearly fleshed out enough. I think a larger example instead of just individuals lines of code would be useful. Show a toy widget sales spreadsheet.…
jnordwick on HN
Alternatives named
tablesaw
From 1 comment
The core looks close enough to dataframes that I'd be curious to know how you compare to tablesaw: https://github.com/jtablesaw/tablesaw This looks neat but I'm not sure why I…
agibsonccc on HN
pandas
From 1 comment
Bistro is not about a physical model and columnar (physical) representation although it relies on it. It is about logical level of representation and processing. There are at…
asavinov on HN
Spark
From 1 comment
The core looks close enough to dataframes that I'd be curious to know how you compare to tablesaw: https://github.com/jtablesaw/tablesaw This looks neat but I'm not sure why I…
agibsonccc on HN
SAS
From 1 comment
How do that compare to SAS software (https://en.wikipedia.org/wiki/SAS_(software)) ? Particularly the "DATA" steps.
julienfr112 on HN
Vertica
From 1 comment
Interesting idea. Columnar ETL can be quite efficient in some scenarios because frequently an ETL transformation (e.g. calculating a new column) effectively modifies an existing…
dgudkov on HN