Ever made a prediction market and everybody ignored it? Manifold has two ways to buy yourself attention. Manifold sponsored a little experiment to compare them.
Very cool! It sounds like you selected paired control group questions after selecting the main question that would receive the subsidy; presumably this means that the control group was systematically less interesting / deserving of the subsidy?
An approach you could try is to select pairs of well-matched questions, then within a pair randomize which is the control question and which gets the subsidy / boost. (You could also try things like boost one / subsidize the other, but at small N that seems like it would mostly just make the math a mess.)
You are correct. This is a weakness in the methodology. Well, I didn't actually use them in the analysis, so no harm done. It was tough to find ten markets, so I would have to relax the constraints.
There doesn't seem to be any interesting insight at this point. The control group definitely stagnates compared the supported ones. Duh. I didn't check it numerically, but the variation is so wide that ten markets is by far too little data to compute the effect size of the subsidies.
In general, I think the next level of analysis requires a more advanced approach involving the API and I believe there is a also database dump available somewhere.
Very cool! It sounds like you selected paired control group questions after selecting the main question that would receive the subsidy; presumably this means that the control group was systematically less interesting / deserving of the subsidy?
An approach you could try is to select pairs of well-matched questions, then within a pair randomize which is the control question and which gets the subsidy / boost. (You could also try things like boost one / subsidize the other, but at small N that seems like it would mostly just make the math a mess.)
You are correct. This is a weakness in the methodology. Well, I didn't actually use them in the analysis, so no harm done. It was tough to find ten markets, so I would have to relax the constraints.
There doesn't seem to be any interesting insight at this point. The control group definitely stagnates compared the supported ones. Duh. I didn't check it numerically, but the variation is so wide that ten markets is by far too little data to compute the effect size of the subsidies.
In general, I think the next level of analysis requires a more advanced approach involving the API and I believe there is a also database dump available somewhere.