:::: MENU ::::

Big on BITSS

Briefly: BITSS is highly necessary.

As mentioned in the new transparency series on CEGA’s blog, there are a lot of problems in the discipline today. Focusing on interaction terms or particular subgroups is one way of increasing the odds of obtaining the elusive 5% significance level, so a lot of people do it, but this overstates the results’ true significance. Something can always appear significant by accident, and tests of significance are only legitimate if they are defined beforehand.

Because of this issue, AidGrade has been forced to take a very conservative stance with regards to the values it collects from studies. In the absence of pre-analysis plans, we have avoided collecting interaction terms and have also focused on results containing few controls (another way that people can lie with statistics). This is obviously a shame, because there is a real sense in which these terms could be important. However, in the absence of pre-analysis plans, this is the best we can do in order to avoid fraud.

Here’s hoping the discipline will change with all the new attention brought to bear on this topic!

Academic proposal

This is a very old idea that I’m putting out in the public domain so that someone can use it. I’ve already shared it with several academics but so far nobody has tried to use it to my knowledge….

For dual-academic couples: one person publishes an academic article and puts in the acknowledgements “X, will you marry me?” The other person then has to either publish another article with their reply or perhaps write up their reply as a comment to the first article!

Reminded of it as I see this is making the rounds. Yes, I know, my idea takes a lot longer!

Call for feedback

Over 100 million impressions. 50,000 tweets. Twitter dump here.

This might be the last update of the twitter dump, as others have continued to scrape the tweets and have added in some of the data from the older tweets. Please let me know if you’d like the occasional update to continue here in .csv form (e.g. for some of the other data contained in the tweets beyond the scraped pdfs).

Looking back on coining a viral hashtag + behavioural economics argument about tweeting

- You get to see it mentioned on almost every major online news source (NYTimes, FT, BBC, Washington Post, CNN, NBC, CBC, Al Jazeera, Bloomberg, Businessweek, FastCompany, Huffington Post, SFGate, The Daily Beast, Slate…), almost every major tech source (TechCrunch, Gizmodo, Arstechnica, CNET, PCMagazine…) and some cool ones like Nature’s blog, the Chronicle of Higher Education and Wikipedia!
- You meet a great group of people.
- Long-lost friends and acquaintances get in touch.
- You get a better understanding of how journalism works.

- You get a better understanding of how journalism works (just joking!).
- It really screws with the results that appear if someone googles you. I have done more things than coin a hashtag, people.
- You get almost no sleep for several days.

I alluded to this before: I have a lot of things depending on me that are not this. I’m not the one to carry on the fight, though I may post from time to time about it.

Finally: tweeting can’t be the end, but it’s a very good start. Apart from the arguments previously posted, there’s a behavioural economics literature that says that when people commit to something publicly, they are more likely to do it. I hope the tens of thousands of people who tweeted about #pdftribute continue to feel a connection to the movement, keep sharing their papers and keep pressing the system to change.

(NB: Since there might also be an “I already did my part” effect, the effects of tweeting could in principle go either way, but that’s my bet.)

Some concrete things coming out of #pdftribute

#pdftribute as a hashtag is dying down, but several concrete endeavours are arising from many good people who were involved in or motivated by it.

Here are a couple of initiatives about which I am aware (merely curating; this does not count as an endorsement):

1) The Papester Collective. Need to get behind a paywall? Send a tweet.

2) Github for research. Searchable paper repository. Easy upload, perhaps with a tweet. Brought to you by a collaboration of the people behind pdftribute.net, http://edward.io/pdftribute, @tmccormick, @thejbf, and @mrgunn at Mendeley.

Some people have argued that since these kinds of ventures don’t change the underlying incentives for academics to publish in top journals, they won’t change the system. I beg to differ. You don’t need to change the academics’ incentives to publish in top journals. You need to change their incentives to share their papers regardless or to have others share their papers regardless. For example, it is usually perfectly fine to share late stage working papers, which many already do to help others gain access to their findings without being stuck at a paywall. These kinds of sharing mechanisms could in principle work, just as Napster changed the music industry.

Here are some other ideas from the comments received on twitter. What are your comments?