The HHMI/Wellcome/MPI super journal and the triumph of open access

By Michael Eisen | Published: July 2, 2011

On Monday the world’s three most prominent private funders of scientific research – the US’s Howard Hughes Medical Institute, the UK’s Wellcome Trust and Germany’s Max Planck Society – announced plans for a new “top-tier, open access journal for biomedical and life sciences research”. Basic features of the as of yet unnamed journal (I suggest they call it Science of Nature and Cells) include:

  • It aims to publish “highly significant contributions that will extend the boundaries of scientific knowledge”
  • It will be fully open access (using a Creative Commons attribution license)
  • It’s operations will be completely underwritten by the funders – there will be no author charges for at least the first few years
  • All editorial decisions will be made by practicing scientists
  • They aim to speed up the peer review process by potentially paying reviewers for timely reviews, and to get rid of the practice of reviewers demanding additional experiments
  • Reviews will be published

To be honest, when I first heard rumors about this journal, my initial reaction was “ugh – the last thing in the world we need is another high-profile journal”. However, as it’s taken shape and I’ve had more time to think about it, my opinion feelings are quite the opposite, and I think this is a great and important event, for many reasons:

It represents the triumph of open access. The most amazing thing about the announcement, and the discussions leading up to it, was how it was universally assumed that this would be a fully open access journal. As far as I can tell, at no point was any consideration given to any other possibility. What a change a decade makes.

In the early 2000’s when PLoS and BioMed Central introduced the open access publishing model, it reception from the scientific establishment ranged from skepticism to derision. One of our major goals was to prove these doubters wrong, and show that open access publishing was best for science. What better testament to our success is there than having three pillars of the biomedical research establishment announcing the most prominent journal launch in almost a decade and treating it as a given that doesn’t even warrant discussion that their journal will be open access.

Many people have asked if I think journal is a threat to PLoS. It’s possible this journal will impact the flow of papers to PLoS Biology – which is close in target in scope. But even if it does, it’s important to remember that we didn’t start PLoS to become a publisher – we started PLoS to promote open access – and any success this new journal has will be an unqualified good thing from the perspective of PLoS.

Funders of science are finally recognizing the importance of reforming scientific publishing and their potential role in doing so. This new journal also represents a significant development in the relationship between funders of science and scientific publishing. Funders  – both public and private – recognize that the publication of papers is one of the major products of the the research they support. Thus I’ve always found it a strange the extent to which they allow publishers to engage in practices that are inconsistent with the open and efficient exchange of results and ideas that science funders are trying to promote. Even above and beyond open access, I think the most newsworthy aspect of this new journal is that, in deciding to launch it, HHMI, Wellcome and MPI are saying, in essence, ” Enough is enough. The publishing system is broken, is interfering with science, and needs to fixed. You’ve had your chance, and things are getting worse and worse. So we’re just going to have to do it ourselves.” I think it’s fantastic that funding agencies are finally recognizing that publishing is an integral part of the research they fund, and that they have a vital interest in making sure that the process works in the best way possible.

Anything that shakes up peer review is a good thing. What ultimately motivated HHMI, Wellcome and MPI to get in to publishing was the feeling that peer review is broken and needs to be fixed.

One of the most bizarre things about the thousands of conversations I had with colleagues about reforming scientific publishing is how fervently they defend the existing system of peer review, even as they spend regaling you with their own personal horror stories about inexplicable delays, idiotic and meddlesome reviewers and arbitrary decisions. Never has their been a system so universally derided and defended at the same time by the same people.

This perverse attachment to a broken system is arguably the single biggest impediment to making scientific publishing maximally open and efficient, and there is great potential in this public expression from HHMI, Wellcome and MPI that they believe peer review needs to be significantly improved. There are really three targets for their ire:

  • The whole process takes too long, with papers languishing on reviewers’ desks, and papers going through multiple rounds of revision and re-review before publication
  • The review process has become too intrusive, with reviewers routinely demanding major changes to manuscripts and often additional experiments
  • Editorial decisions on what to publish are arbitrary and based too much on sexiness and aesthetics, with too much power over the process given to junior scientists who are no longer active researchers

It remains to be seen whether their solutions to these problems (paying reviewers, hiring prominent active scientists as editors, enforcing new standards for review and publishing reviewers comments) will make a big difference. My own opinion is that a system in which the outcome of review is a binary “publish or not publish decision” is fatally flawed, and should be replaced by one in which publication and assessment are decoupled. But despite feeling that the new journal doesn’t go far enough in this regard, any disruptive effect it has on peer review will be a good thing.

[I’ve been meaning for some time to write more about the future of peer review – and this has prompted me to do it. More on that front soon.]

Paying peer reviewers is not insane. People seem to generally view the idea of paying reviewers as weird and dangerous. But I don’t see why. Reviewers are providing a service to the authors and publishers, and it seems perfectly sensible to reward them for performing this service in a timely and responsible manner. There’s no reason to suspect that this would bias their decisions – only that it would lead them to give a slightly higher priority to reviews than they normally do. It could also scale reasonably well, especially if journals started to charge for submissions. And it might help recruit reviewers. The only aspect of scaling that I really worry about is whether, if this became the norm for all journal, if these payments became a kind of entitlement, with the positive effect wearing wear off.

So, while a lot of details about the new journal are still to be determined, and I’m sure there will be some things I like and some things I don’t, the entrance of HHMI, Wellcome and MPI into the open access publishing world is an unambiguously good thing.

Posted in HHMI, open access, PLoS, publishing, science | Comments closed

Why is Google keeping a list of Jewish names?

By Michael Eisen | Published: July 2, 2011

Google has a new feature – “Something different” – that, according to their description returns “queries that may be in the same category as your original search”.

Well, yesterday I entered a colleague’s name – “goldstein” – in to google, and got the following:

Hmm. So I tried another search, this time for “Friedman”:

So I tried another, “Greenbaum”:

Then I tried a fairly obscure one “Freudenthal”:

And finally I tried “Brin”:

So what’s going on? I find it hard to believe the structure of the internet really clusters Jewish names together, but maybe it does (an Israeli phonebook?). Or has someone at google compiled a list of Jewish names, and, if so, why?

Posted in misc stuff | Comments closed

Announcing The Batavia Open Genomic Data Licence

By Michael Eisen | Published: June 7, 2011

Prepublication release of genomic and other large-scale biological datasets is incredibly value to the research community. For the last decade big genome sequencing centers – backed by the NIH and other funders – have followed a set of principles outlined at a January 2003 meeting in Ft. Lauderdale sponsored by The Wellcome Trust. This so called “Ft. Lauderdale Agreement” outlined a set of “responsibilities” for funders, data producers and data users. Significantly, it reserves for data producers the right to first publication of analysis of their data, discouraging precisely the kind of prepublication use of data it is supposed to be encouraging. In practice it has also given data producers the power to create enormous consortia to analyze data they produce, effectively giving them disproportionate credit for the work of large communities. It’s a horrible policy that has significantly squelched the development of a robust genome analysis community that is independent of the big sequencing centers.

As my lab has started to produce more genome sequence data, I have pondered how best to release these data to the public. On the one hand, I don’t want to do anything like Ft. Lauderdale. I want to encourage people to use my data – without feeling any obligation to include me in their plans or publications, and placing no restrictions on when they can publish.

However, I feel strongly that people taking advantage of the free and open release of scientific data should reciprocate by ensuring that publications arising from the use of these data also be free and open. Therefore, I have created a new agreement – which I will start using to release sequencing data from my lab. To reinforce the open sharing principles which this sharing represents, the license requires users to publish any work arising from the use of the data in open access journals – that is it requires that publications arising from the use of openly shared data be themselves openly shared.

I decided to name the license after the Batavia Seamount, a subsurface feature in the Indian Ocean that is antipodal to Ft. Lauderdale, FL, to reflect failures of the “Ft. Lauderdale Agreement” for sharing of DNA sequence data that restricts users’ rights to publish public sequence data and fails to lead to the open sharing of publications arising from these data.

So here it is.

Read More »

Posted in open access, PLoS, science | Comments closed

Mystery in a children’s classic: Who is the 12th girl at dinner while Madeline is in the hospital?

By Michael Eisen | Published: June 4, 2011

Nearly everyone is familiar with Madeline, Ludwig Bemelmans’ classic 1939 children’s tale of the girls in a Parisian boarding school.

Madeline Cover

You will recall that there are 12 of them, and they go about their days in two nice little lines. Always 12 of them, whether they are out and about.

Or eating, washing up or sleeping.

But then one night, poor little Madeline gets sick :-(.

She has appendicitis, and needs surgery. She has to spend a few weeks recovering in the hospital. So now there are only 11 girls.

But in the midst of the lonely days spent pining for Madeline’s return, they have dinner. The girls are sad. All 12 of them.

Count them up.

It’s back to eleven when they brush their teeth and go to bed.

What’s going on?  I’ve studied the faces of the 11 remaining girls, and while the images are not entirely consistent, this girl never appears anywhere else.

Who is the mystery girl? Where did she come from? And why didn’t she brush her teeth after eating?

But seriously. I find this kind of thing curious. Bemelmans was clearly pretty careful about counting the number of girls in each scene. So how did this one get screwed up? And why didn’t anyone notice before it was published. Or since (as far as I can tell from searching the web, this has not been noted before).

Posted in Uncategorized | Comments closed

Amazon’s $23,698,655.93 book about flies

By Michael Eisen | Published: April 22, 2011

A few weeks ago a postdoc in my lab logged on to Amazon to buy the lab an extra copy of Peter Lawrence’s The Making of a Fly – a classic work in developmental biology that we – and most other Drosophila developmental biologists – consult regularly. The book, published in 1992, is out of print. But Amazon listed 17 copies for sale: 15 used from $35.54, and 2 new from $1,730,045.91 (+$3.99 shipping).

I sent a screen capture to the author  – who was appropriate amused and intrigued. But I doubt even he would argue the book is worth THAT much.

At first I thought it was a joke – a graduate student with too much time on their hands. But there were TWO new copies for sale, each be offered for well over a million dollars. And the two sellers seemed not only legit, but fairly big time (over 8,000 and 125,000 ratings in the last year respectively). The prices looked random – suggesting they were set by a computer. But how did they get so out of whack?

Amazingly, when I reloaded the page the next day, both priced had gone UP! Each was now nearly $2.8 million. And whereas previously the prices were $400,000 apart, they were now within $5,000 of each other. Now I was intrigued, and I started to follow the page incessantly. By the end of the day the higher priced copy had gone up again. This time to $3,536,675.57. And now a pattern was emerging.

On the day we discovered the million dollar prices, the copy offered by bordeebook was1.270589 times the price of the copy offered by profnath. And now the bordeebook copy was 1.270589 times profnath again. So clearly at least one of the sellers was setting their price algorithmically in response to changes in the other’s price. I continued to watch carefully and the full pattern emerged.

Once a day profnath set their price to be 0.9983 times bordeebook’s price. The prices would remain close for several hours, until bordeebook “noticed” profnath’s change and elevated their price to 1.270589 times profnath’s higher price. The pattern continued perfectly for the next week.

But two questions remained. Why were they doing this, and how long would it go on before they noticed? As I amusedly watched the price rise every day, I learned that Amazon retailers are increasingly using algorithmic pricing (something Amazon itself does on a large scale), with a number of companies offering pricing algorithms/services to retailers. Both profnath and bordeebook were clearly using automatic pricing – employing algorithms that didn’t have a built-in sanity check on the prices they produced. But the two retailers were clearly employing different strategies.

The behavior of profnath is easy to deconstruct. They presumably have a new copy of the book, and want to make sure theirs is the lowest priced – but only by a tiny bit ($9.98 compared to $10.00). Why though would bordeebook want to make sure theirs is always more expensive? Since the prices of all the sellers are posted, this would seem to guarantee they would get no sales. But maybe this isn’t right – they have a huge volume of positive feedback – far more than most others. And some buyers might choose to pay a few extra dollars for the level of confidence in the transaction this might impart. Nonetheless this seems like a fairly risky thing to rely on – most people probably don’t behave that way – and meanwhile you’ve got a book sitting on the shelf collecting dust. Unless, of course, you don’t actually have the book….

My preferred explanation for bordeebook’s pricing is that they do not actually possess the book. Rather, they noticed that someone else listed a copy for sale, and so they put it up as well – relying on their better feedback record to attract buyers. But, of course, if someone actually orders the book, they have to get it – so they have to set their price significantly higher – say 1.27059 times higher – than the price they’d have to pay to get the book elsewhere.

What’s fascinating about all this is both the seemingly endless possibilities for both chaos and mischief. It seems impossible that we stumbled onto the only example of this kind of upward pricing spiral – all it took were two sellers adjusting their prices in response to each other by factors whose products were greater than 1. And while it might have been more difficult to deconstruct, one can easily see how even more bizarre things could happen when more than two sellers are in the game. And as soon as it was clear what was going on here, I and the people I talked to about this couldn’t help but start thinking about ways to exploit our ability to predict how others would price their books down to the 5th significant digit – especially when they were clearly not paying careful attention to what their algorithms were doing.

But, alas, somebody ultimately noticed. The price peaked on April 18th, but on April 19th profnath’s price dropped to $106.23, and bordeebook soon followed suit to the predictable $106.23 * 1.27059 = $134.97. But Peter Lawrence can now comfortably boast that one of the biggest and most respected companies on Earth valued his great book at $23,698,655.93 (plus $3.99 shipping).

Posted in misc stuff, science | Comments closed

Interesting comments from Dennis Overbye about press releases

By Michael Eisen | Published: March 26, 2011

Marketing for scientists has an interview with NYT science reporter Dennis Overbye. In response to a question about whether science reporters can be manipulated, Overbye responded:

The easiest way to manipulate the press is to embargo some result and then send a press release about it to a thousand different news organizations.  They will cover it because they are afraid everyone else will cover it.   It’s a kind of artificial competition that’s stirred up.

It does two things.  By embargoing the information it makes it harder to get an informed opinion on the paper.  It put you at the mercy of time.   And you whip up competition between news organizations.   You have to have your story ready to go online the instant the embargo ends.

You will see that every story has a little note after it with the time that the story came out so you can see who was first, who was a few minutes late with it.  For some people this constitutes bragging rights—-in terms of business news it’s not so silly.  So there’s a deadline—-you’ve got to have something to say.  Your access to informed opinion may be limited.

Scientists complain all the time that reporters don’t take the time to understand the science in what they are writing about, and are just eager to get a story out quickly. But I think Overbye hits the nail on the head – just another reason to do away with embargoed press releases in science.

Posted in publishing, science | Comments closed

Press release? We don’t need no stinking press release?

By Michael Eisen | Published: March 24, 2011

I hate press releases – especially around scientific papers. They rarely explain the work clearly, almost always overstate its significance, and are often grossly dishonest. But scientists and their press offices, working in close collaboration with journals, continue churning them, hoping to earn popular press coverage of their latest findings. They go through this unseemly process because they believe that reporters will only cover papers if they get advance notice of publications, and are spoonfed highlights.

After the grotesque NASA press releases around their 2010 arsenic paper in Science, I vowed to do whatever I can to end this practice. So I was very happy when my brother decided not to press release a very cool paper he had coming out in PLoS One (and double props for publishing it there). He chose, instead, to write a long, detailed blog post after the paper was published explaining not only the paper, but the story behind it. This violated all of the assumptions of the embargoed press release mindset – he provided no advanced notice, and the blog post was anything but a flashy, dumbed down account of how the paper would change the world.

And, of course, he was punished mercilessly for his apostasy…. NOT!

His blog post and paper were immediately picked up by a host of prominent and thoughtful science bloggers (e.g. Carl Zimmer and PZ Myers), who appreciated (I am speculating) not being talked down to. But more impressively, the story got tons of play in the popular press who supposedly had not time to read long posts and wouldn’t report on “yesterday’s news”. Today there was a really good article about the paper in The Economist.

So “Way to go Jonathan!” – I hope this is the beginning of the end of the scientific press release.

Posted in open access, PLoS, science | Comments closed

Felisa Wolfe-Simon (of arsenic infamy) is no more convincing in person than in print

By Michael Eisen | Published: March 18, 2011

I went to an informal seminar today at Berkeley by Felisa Wolfe-Simon, the lead author on the much criticized 2010 Science paper “A bacterium that can grow by using arsenic instead of phosphorus”.

I went because, as bad as I thought her paper was, as poorly as I thought she handled concerns expressed about the work, and as cringe-worthy was her performance in the NASA press conference, I understand that sometimes people get caught up in things that are out of their control, and I hoped that in a more comfortable setting she might reveal a different side.

And I have to say, I loved how enthusiastic she is about her  subject. The ways in which microbes elaborate on the basic chemical processes of life are fascinating – although I’m not sure they tell us much about extraterrestrial life (if we find aliens, and they’re just like us except they use arsenic instead of phosphorous, I am going to be disappointed that they’re so similar). And despite the many good arguments for why arsenic in DNA shouldn’t work, I don’t think it’s crazy to remain open to the possibility that they exist.

What I and others found so frustrating about her Science paper was the sense that Wolfe-Simon and her colleagues did not take seriously the task of actually proving that their bug was really using arsenic. Nonetheless, as her talk progressed into the arsenic data, I held out hope that the criticism of the paper would have led to a more sober assessment of her data. Unfortunately, that was not the case.

The most highly-criticized aspects of the paper remained in her talk – with little change or explanation. Too much was made of how radio-labeled arsenic partitioned in a phenol-chloroform extraction. The gel supposedly of GFAJ-1 DNA still had a weirdly well-resolved single band. And the key test of arsenic incorporation was done on a highly impure sample. I asked her about this later point during the Q&A. And she gave the astonishing answer that they lacked the equipment needed to purify DNA. I find it hard to believe Wolfe-Simon thinks you need an HPLC to separate agarose from DNA – a google search for “DNA purification” reveals many simple alternatives. But even if she does think this, her failure to investigate alternatives means means she is not serious about answering the question. And the alternative she proposed – scanning the whole gel at the synchrotron – is hardly a simple alternative, and wouldn’t address the criticism at all.

The clear sense I got from listening to her talk is that she is 100% sure that GFAJ-1 has arsenic in its DNA, and since she does not feel the need to prove it to herself, she feels only mildly compelled to prove it to others. Of course she could be right – as much as her poorly controlled experiments fail to demonstrate that arsenic is incorporated into DNA, they don’t disprove it either. And the publicity that her paper received means that the right experiments will almost certainly be done at some point – if not by Wolfe-Simon then by others. And we will know. If she’s right, the sloppiness of these initial experiments will likely be forgotten and she will be praised for sticking with an unpopular idea in the face of withering criticism. But what will happen if she is wrong, as has to be considered the more likely possibility?

The acid test of a scientist is how they respond when their work is criticized. The best scientists listen and consider what is being said, defend the things they still believe and, most importantly, recognize where their work fell short and use criticism to make their work better. This is, of course, not always so simple. It’s easy to get defensive instead – to view criticism as an attack, see sinister motives in its sources, and ignore its substance.

But I think the worst response is to view criticism as a kind of virtue. And there were signs in Wolfe-Simon’s talk that she is beginning to relish the role of the iconoclast. She appears to see herself as someone who has unconventional ideas that the scientific community can’t deal with. And that criticism of her work is not an effort to get at the truth but a conspiracy to suppress it. At several points she made reference to other scientists whose ideas were not accepted when they were proposed, but which turned out in the long run to be correct. The problem is that many people get stuck this way – and being iconoclastic becomes their whole scientific identity (we can all think of people like this….).

It would be a real shame if this happened to Wolfe-Simon. She seems affable and completely sincere. I’m sure she is, in many ways, a capable scientist. And, as I said above, it’s hard not to be drawn in by her enthusiasm for her subject. Science needs people who challenge the limits of what we know and believe, and who seek weird things in the natural world. And there is no doubt that Dr. Wolfe-Simon has discovered a species that does unusual things with arsenic. As the hoopla over this work dies down, I hope Dr. Wolfe-Simon spends less time at TED and more time doing the experiments that will compellingly demonstrate what GFAJ-1 is really all about.

Posted in science | Comments closed

Science Magazine really, really, really doesn’t get it

By Michael Eisen | Published: October 22, 2010

Bruce Alberts has an editorial in this weeks science in which he proposes the idea of “Science, Technology, Engineering and Math (STEM) Merit Badges” – a set of “100 different challenges to choose from at each level of schooling” – to engage students, patents and teachers in science.

Whether you think this is a good idea or not, it’s simply astonishing that this editorial is available only to subscribers of Science – i.e. it is not available to any of the people it hopes to engage!!! What a fitting way to close out open access week…..

Doesn’t Dr. Alberts – whose work as a scientist I greatly respect – understand that one of the biggest problems in science education is that the public is not engaged in any meaningful way with the process of science? And that the imprisoning of the best work of American scientists behind the subscription walls of our most prominent scientific journal is a huge part of the problem?

Posted in Uncategorized | Comments closed

I know something that Nature does not know… DNA is not lefthanded

By Michael Eisen | Published: September 10, 2010

I’ve written before about Nature’s tendency to publish biologically inaccurate covers. But this one really caught my attention.

This is a cover about DNA sequencing, but the structure of the DNA molecule they show is wrong – in a crucial way. DNA is a chiral molecule – meaning it can occur in forms that differ like our right and left hands. They are similar – but are not interchangeable. DNA in our cells is normally righthanded, but the DNA pictured here is the non-natural lefthanded form.

Tom Schneider describes the difference well and has tirelessly chronicled the myriad times that left handed DNA has been erroneously portrayed in lieu of its righthanded cousin. But it’s particularly offensive that the journal that published the original structure of DNA would get its structure wrong. Here is a figure from the original paper:

It’s more than just an accident. They clearly care only about the aesthetics of their covers – and are happy to farm out to a graphic designer who doesn’t understand the biology. More distressingly, noone at Nature caught the mistake. I’m tempted to make the obvious analogy to the content of the journal, but I’ll leave that alone….

(For those of you who don’t get the title of the post – it’s a reference to The Princess Bride).

Posted in Uncategorized | Comments closed