hypothesis-auto
Dev tools
test generation
SHOW HN2019
How HN took it
3 positive, 15 neutral or asking, 1 mixed, 5 critical, out of 24 comments about the project. Thread score -0.11.
Idea: 2 for, 3 againstExecution: 1 for, 1 againstDesign: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Good convenience layer over Hypothesis; catches edge-case runtime exceptions.
From 3 comments
By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of…
timothycrosley on HNIt's also useful to check the function doesn't crash. I use hypothesis to test a DB abstraction, and it often caught bugs in edge cases I didn't consider: timestamps/timezones…
progval on HNThis is great, thank you! Another great example of the benefits strong typing brings.
StavrosK on HN
Idea of generating tests from runtime observation is appealing.
From 2 comments
I really like this idea. So, broken out, would the development process look like: 1) Write code 2) Open interpreter (ipython) 3) Import code and call it manually 4) This magic…
noobiemcfoob on HNI'd use it!
timothycrosley on HN
Objections
Amounts to snapshot or fuzz testing that only checks types and crashes, not behavior.
From 3 comments
Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing…
ebg13 on HNThis sounds like snapshot testing. It is the responsibility of the person to browse the produced snapshots to ensure they are correct.
anonytrary on HN> By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). So it's a fuzzer?
ebg13 on HN
Generated coverage may give false confidence.
From 1 comment
On one job, I had to disable code coverage for a whole suite of tests that were simply making calls and completely ignoring the results. I know, coverage is Yet Another Metric,…
ben509 on HN
Edge cases found by generated inputs may be irrelevant and need constraints.
From 1 comment
While I had really liked the idea of hypothesis in Python I found that the edge-cases it was uncovering were the ones that were obviously gonna break but at the same time cases I…
somada141 on HN
Side-effecting functions like DB calls are hard to test without mocks.
From 1 comment
Nice concept but does it work with real-world application. I failed to understand how will it work with methods like `authenticate_user(user)` or `load_permissions_from_db(user,…
kburman on HN
Requests
Generate standalone test files that can be kept with other tests.
From 1 comment
I like the idea, but I think I would be more comfortable with it generating test files that I could keep beside my other tests. Also, `auto_pytest_magic` doesn't seem to exist.…
rcfox on HN
Use trailing underscore for parameters instead of leading underscore.
From 2 comments
I like this idea. One piece of feedback, a parameter with a leading underscore feels very odd. In python I interpret leading underscores to indicate the programmer thinks of this…
iandanforth on HNI believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to…
duckerude on HN
Questions
What advantage over Hypothesis' native inferred strategies?
From 1 comment
This is neatly packaged, but it's not immediately clear what advantages it has over Hypothesis' native offerings for accomplishing this? (…
hchasestevens on HN
Does it only check return types?
From 1 comment
Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing…
ebg13 on HN
Alternatives named
Hypothesis inferred strategies and from_type
From 1 comment
This is neatly packaged, but it's not immediately clear what advantages it has over Hypothesis' native offerings for accomplishing this? (…
hchasestevens on HN
Snapshot testing
From 1 comment
This is basically what they call snapshot testing
rcap on HN
VCR and MonkeyType
From 2 comments
Possibly vcr [0], except that's for mocking out APIs in unit tests. There's a whole bunch of ports to other languages midway down the page. [0] https://github.com/vcr/vcr
Izkata on HNInstagram created 'MonkeyType' to do something similar for type annotations. A lower barrier way of trying this out could be to run MonkeyType in combination with OP's…
OJFord on HN