sweep.dev
Dev tools
caching decorator
SHOW HN2024
How HN took it
1 positive, 10 neutral or asking, 0 mixed, 5 critical, out of 16 comments about the project. Thread score -0.15.
Execution: 1 for, 2 againstPerformance: 0 for, 3 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Simple and clean implementation
From 1 comment
This is a pretty good implementation. I like the simplicity of it, reminds me of SQLite backed storage decorators we used to have, where the data was persisted to a DB instead of…
skp1995 on HN
Objections
Cache key ignores dependency changes, so stale results can be returned
From 2 comments
``` def hash_code(code): return hashlib.md5(code.encode()).hexdigest() ``` Be warned. The above function is used as part of the hash. The ostensible purpose is to prevent using…
epr on HN> How do you suggest one might fix that issue? Also pin the cache to a hash of all dependency versions? Pretty much. Recursively collect dependencies by analyzing the AST of the…
epr on HN
md5 hashing is slow and a poor choice
From 1 comment
Using md5 for this seems like an odd choice. Sha1 is a better choice even for non-cryptographic use cases, it's quite a bit faster than md5. Even better would be something like…
chlorion on HN
Not thread or process safe for concurrent writes
From 1 comment
Thread safety is a big issue with ours, we'll run into issues when two different processes attempt to write to the same location, or we'll get a bad read. This is a better…
williamzeng0 on HN
Requests
Make kwargs order-independent in cache keys
From 1 comment
I recently wrote a version of this that I use in my projects, some things I do differently that you may or may not care about: - from your code it seems you're not sorting…
mpeg on HN
Add a version parameter to the decorator for deliberate invalidation
From 1 comment
I feel like adding an argument to the decorator that labels the "version" of the function would make deliberate cache invalidation more straightforward for cache users.
Karellen on HN
Questions
How it differs from diskcache and joblib Memory
From 2 comments
I have extensively used https://pypi.org/project/diskcache/. Is there a reason you decided to make an in house solution?
rmholt on HNWhat's the difference to using joblibs Memory class similar to this implementation: https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/cache_utils.py
rassibassi on HN
Whether it handles thundering herd problem
From 1 comment
This is a pretty good implementation. I like the simplicity of it, reminds me of SQLite backed storage decorators we used to have, where the data was persisted to a DB instead of…
skp1995 on HN
Alternatives named
diskcache
From 2 comments
I have extensively used https://pypi.org/project/diskcache/. Is there a reason you decided to make an in house solution?
rmholt on HNYou'll be happy to know that Diskcache is backed by SQLite (and/or spill files for large enough (size configurable) blobs).
akx on HN
joblib Memory
From 2 comments
What's the difference to using joblibs Memory class similar to this implementation: https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/cache_utils.py
rassibassi on HNThe diskcache docs state: """ Caching Libraries ``` joblib.Memory provides caching functions and works by explicitly saving the inputs and outputs to files. It is designed to…
rassibassi on HN
Hishel
From 1 comment
I write some toy Python scripts/apps for a couple APIs (the Pokeapi, Spacetraders, etc). I use the HTTPX library as a request client, and wasn't aware of the Hishel library for…
quickslowdown on HN