Guess which government programs work
Learning and science
quiz on social policy evidence
SHOW HN2015
How HN took it
4 positive, 15 neutral or asking, 2 mixed, 11 critical, out of 32 comments about the project. Thread score -0.24.
Idea: 3 for, 3 againstDocs: 0 for, 4 againstDesign: 0 for, 2 againstExecution: 0 for, 1 againstPerformance: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Interesting and educational, reveals counterintuitive results
From 3 comments
This is great! One of the interesting things about social sciences (and engineering, although just like computer hacking, social engineering has unfortunately strong negative…
asgard1024 on HNThe Elderly care question said that there was no effect on 3 year mortality, which is fine and I'm sure that is accurate but it doesn't answer the question of "Did it have a net…
noir_lord on HNInteresting that the average user is less than 50%. I got two wrong and still felt like an idiot.
Freeboots on HN
Objections
Hard to tell no effect and negative effect apart
From 1 comment
It's really hard to tell "negative effect" and "no effect" apart, I got two of them switched. Besides, I would consider both of those failed programs, not even sure if there's a…
iopq on HN
Outcome measures or question wording are unclear or misleading
From 3 comments
Home visits for older adults is a little disingenuous. The purpose is not really to reduce death risk, but to increase quality of life. I don't think that counts as "no effect"…
superuser2 on HNI find it a bit unintuitive that "no change" errors are valued the same as "opposite" errors (e.g. answering "positive" when the change was actually negative). I find it far less…
pluma on HNI was thinking about why this website triggered my skeptic alarms and came up with a few things. 1. The web interface makes it hard to go back. The first thing I want to do when…
alexandercrohde on HN
Slow loading and poor web interface
From 2 comments
While the content is great, the tech seems bloody awful. Why is it so slow to load? It takes 5-10 seconds to load on both my laptop and my desktop (both < 1 year old) and I have…
mattmanser on HNI was thinking about why this website triggered my skeptic alarms and came up with a few things. 1. The web interface makes it hard to go back. The first thing I want to do when…
alexandercrohde on HN
Requests
Show a results summary after the quiz
From 2 comments
It would be nice to see a summary of the projects and their effects after finishing the quiz. By that time, I had already forgotten about some and wondered what the answers were.
jrnvs on HNMaybe you can link to that at the end of the quiz?
ikeboy on HN
Use a slider with error bars instead of three choices
From 1 comment
I enjoyed the quiz, but I am not a big fan of the positive_effect/no_effect/negative_effect classification, because it hides the error of the measurement. Instead, I propose to…
maho on HN
Questions
Whether the 'no effect' versus 'negative' distinction is meaningful
From 2 comments
It's really hard to tell "negative effect" and "no effect" apart, I got two of them switched. Besides, I would consider both of those failed programs, not even sure if there's a…
iopq on HNIf it helps, any time you choose "No effect", you're making a guess about how thorough the experiment was, not about the program itself. Every program will have some effect, but…
Asbostos on HN
Alternatives named
Ben Goldacre's government randomised policy trials paper
From 1 comment
Ben Goldacre (of Bad Science) wrote a paper with the UK Government on A/B tests of policies.…
frabcus on HN