XetHub
Data
data versioning
SHOW HN2022
279points
144comments
#156of 2,570 in 2022
Top 7%of 2022 by points
How HN took it
CuriousMostly questions and neutral comments
13 positive, 39 neutral or asking, 2 mixed, 19 critical, out of 73 comments about the project. Thread score -0.19.
Idea: 7 for, 13 againstExecution: 3 for, 0 againstPerformance: 1 for, 2 againstDesign: 0 for, 2 againstDocs: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Block-level deduplication is a real improvement over file-level dedup
From 2 comments
Sounds like `git annex` is file-level deduplication, whereas this tool is block-level, but with some intelligent, context-specific way of defining how to split up the data (i.e.…
kspacewalk2 on HNXetHub Co-founder here. Yes, one illustrative example of the difference is: Imagine you have a 500MB file (lastmonth.csv) where every day 1MB is changed. With file-based…
rajatarya on HN
Mount command for browsing without downloading is useful
From 2 comments
The tl;dr is that "xet" is like GitLFS (it stores pointers in Git, with the data in a remote server and uses smudge filters to make this transparent) with some additional…
Game_Ender on HNCofounder/CEO here! I think it less about "versioning", but the ability to modify with confidence knowing that you can go back in time anytime. (Minor clarification: we are not…
ylow on HN
Objections
Using Git for large binary data is a bad idea
From 3 comments
The lack of reasons for doing it IS the reason against. GIT isn't a magic 'good way' to store arbitrary data, it's a good way to collaborate on projects implemented using most…
ltbarcly3 on HNI think this is a foot-gun, it's a bad idea even if it works great, and I doubt it works very well. You should manage your build artifacts explicitly, not just jam them in git…
ltbarcly3 on HNThis feels like something that is prime for abuse. I agree with @bastardoperator, treating git for file storage is going to go nowhere good.
subnetwork on HN
Pricing unclear for individuals with mid-sized repos
From 1 comment
You say you support up to 1TB repositories, but from your pricing page all I see is the free tier for up to 20GB and one for teams. The latter doesn't have a price and only a…
V1ndaar on HN
Login page as the link is confusing
From 1 comment
The link takes me to a login page. It would be nice to see that fixed to somehow match the title.
ledauphin on HN
Requests
Support for Perforce-style locking
From 1 comment
The centralised model of Perforce is more of a natural fit for one thing, since by default it allows you to clone subsets, and just the latest version of files. File locking is…
sk0g on HN
Better progress bar during fetch
From 1 comment
Does this fix the problem that Git becomes unreasonably slow when you have large binary files in the repo? Also, why can't Git show me an accurate progress-bar while fetching?
amelius on HN
Support for Unreal Engine binary assets
From 1 comment
Have you tested this out with Unreal Engine blueprint files? If you all can do block-based diffing on those, and other binary assets used in game development it'd be huge for…
AustinDev on HN
Questions
How does it differ from Git LFS
From 1 comment
How's this differ from using git LFS?
angrais on HN
How are data files chunked
From 1 comment
How data is split in chunks ? Just curious.
amadvance on HN
Can it store Docker container images
From 1 comment
Can it be used to store container images (Docker)? As far as I remember they are just compressed tar files. Does the compression defeat Xet's own chunking? Can you sync to…
chubot on HN
Alternatives named
git-annex
From 1 comment
I also have a lot of issues with versioning data. But look at git annex - it's free, self hosted and has a very easy underlying data structure [1]. So I don't even use the magic…
JZL003 on HN
DVC
From 2 comments
I actually encountered a 4TB git repo. After pulling all the binary shit out of it the repo was actually 200MB. Anything that promotes treating git like a filesystem is a bad…
bastardoperator on HNYou're suffering from a failure of imagination, maybe because you've never been able to version data usefully before. There are already lots of interesting applications, and it's…
zachmu on HN
Perforce
From 2 comments
If I had to "version control" a 1 TB large repo - and assuming I wouldn't quit in anger - I would use a tool which is built for this kind of need and has been used in the…
Izmaki on HNThe centralised model of Perforce is more of a natural fit for one thing, since by default it allows you to clone subsets, and just the latest version of files. File locking is…
sk0g on HN
Dolt
From 1 comment
Founder of DoltHub here. One of my team pointed me at this thread. Congrats on the launch. Great to see more folks tackling the data versioning problem. Dolt hasn't come up here…
timsehn on HN
Pachyderm
From 1 comment
I actually encountered a 4TB git repo. After pulling all the binary shit out of it the repo was actually 200MB. Anything that promotes treating git like a filesystem is a bad…
bastardoperator on HN