27
Feb

Please save the unsolicited R01s

Tweet about this on Twitter96Share on Facebook92Share on Google+5Share on LinkedIn4Email this to someone

Editor's note: With the sequestration deadline hours away, the career of many young US scientists is on the line.  In this guest post, our colleague Steven Salzberg , an avid defender of NIH and its peer review process, tells us why now more than ever the NIH should prioritize funding R01s over other project grants .

First let's get the obvious facts out of the way: the federal budget is a mess, and Congress is completely disfunctional.  When it comes to NIH funding, this is not a good thing.

Hidden within the larger picture, though, is a serious menace to our decades-long record of incredibly successful research in the United States.  The investigator-driven, basic research grant is in even worse shape than the overall NIH budget.  A recent analysis by FASEB, shown in the figure here, reveals that the number of new R01s reached its peak in 2003 - ten years ago! - and has been steadily declining since.  In 2003, 7,430 new R01s were awarded.  In 2012, that number had dropped to 5,437, a 27% decline.

number-of-new-r01s

For those who might not be familiar with the NIH system, the R01 grant is the crown jewel of research grants.  R01s are awarded to individual scientists to pursue all varieties of biomedical research, from very basic science to clinical research.  For R01s, NIH doesn't tell the scientists what to do: we propose the ideas, we write them up, and then NIH organizes a rigorous peer review (which isn't perfect, but it's the best system anyone has).  Only the top-scoring proposals get funded.

This process has gotten much tougher over the years.  In 1995, the success rate for R01s was 25.9%.  Today it is 18.4% and falling.  This includes applications from everyone, even the most experienced and proven scientists.  Thus no matter who you are, you can expect that there is more than an 80% chance that your grant application will be turned down.  In some areas it is even worse: NIAID's website announced that it is currently funding only 6% of R01s.

Why are R01s declining?  Not for lack of interest: the number of applications last year was 29,627, an all-time high.  Besides the overall budget problem, another problem is growing: the fondness of the NIH administration for big, top-down science projects, many times with the letters "ome" or "omics" attached.

Yes, the human genome was a huge success.  Maybe the human microbiome will be too.  But now NIH is pushing gigantic, top-down projects: ENCODE, 1000 Genomes, the cancer anatomy genome project (CGAP), the cancer genome atlas (TCGA), a new "brain-ome" project, and more. The more money is allocated to these big projects, the less R01s NIH can fund. For example, NIAID, with its 6% R01 success rate, has been spending tens of millions of dollars per year on 3 large Microbial Genome Sequencing Center contracts and tens of millions more on 5 large Bioinformatics Resource Center contracts.  As far as I can tell, no one uses these bioinformatics resource centers for anything - in fact, virtually no one outside the centers even knows they exist. Furthermore, these large, top-down driven sequencing projects don't address specific scientific hypotheses, but they produce something that the NIH administration seems to love: numbers.  It's impressive to see how many genomes they've sequenced, and it makes for nice press releases.  But very often we simply don't need these huge, top-down projects to answer scientific questions.  Genome sequencing is cheap enough that we can include it in an R01 grant, if only NIH will stop pouring all its sequencing money into these huge, monolithic projects.

I'll be the first person to cheer if Congress gets its act together and fund NIH at a level that allows reasonable growth.  But whether or not that happens, the growth of big science projects, often created and run by administrators at NIH rather than scientists who have successfully competed for R01s, represents a major threat to the scientist-driven research that has served the world so well for the past 50 years.  Many scientists are afraid to speak out against this trend, because by doing so we (yes, this includes me) are criticizing those same NIH administrators who manage our R01s.   But someone has to say something.  A 27% decline in the number of R01s over the past decade is not a good thing.  Maybe it's time to stop the omics train.

  • Titus Brown

    I must admit to wondering about the BRCs myself... good to hear someone with a more panoramic view questioning them.

  • http://twitter.com/Malarky67 Stephen Henderson

    Since the price of sequencing is falling exponentially we could just mothball the humungome projects for a few years (till seq. cost << patient/sample cost) and save a packet. Everyone competing to use the most expensive and imperfect technology and publish first is a 'tragedy of the commons'. Its hardly a moon race.

    • http://twitter.com/drgitlin Jonathan Gitlin

      Um, you do realize that the only reason those costs have fallen the way they have is because NHGRI has invested heavily in this area? http://www.genome.gov/10000368

      • Shiran Pasternak

        "Only reason"? How can you prove that? Improvement in sequencing technology and reduction in reagent costs come from innovations within private companies (with academic partnerships, of course). To suggest that falling costs are due to forethought by some government agency is a stretch.

        • http://twitter.com/drgitlin Jonathan Gitlin

          Click that link, take a look at the grantees who have been supported by the tech dev program during their early years...

        • http://twitter.com/mjcgenetics Michael James Clark

          But the customers for that tech were almost completely academic for the first few years, and academia (esp. big research centers and core facilities) remains a huge customer base for it. Innovations in this space only happen if there is a demand for the tech (and if there's competition), and a huge portion of the demand comes from academia, and the bulk of the profit comes from reagents (to large centers and cores in particular).

      • http://twitter.com/Malarky67 Stephen Henderson

        Yes that is partly true. If there is a critical mass of people working in a field it will improve and costs will fall. I suspect - though I can't know- we are way past that incubation phase. In any case I'm only half serious - it won't happen.

  • http://www.facebook.com/people/Evan-Zamir/100000371881846 Evan Zamir

    The advice I would give young investigators is to get involved with the big center grants. If you can't beat 'em...

    • Titus Brown

      NO NO NO NO NO dear lord. They suck up massive amounts of time and bureaucratic effort, result in virtually no first or last author papers, and are largely unrewarded at the promotion & tenure level! Run screaming!!

  • Nick Schurch

    or... you could just climb on the genomics train and ride it to wherever it goes? Weirdly, although I work mainly on sequencing data I agree with the tone of this post. People need to sequence less and look at what the data they have a bit more carefully, but as Titus is fond of saying "Its cheaper to sequence something than it is to think".

  • http://incubator.rockefeller.edu/?cat=9 Gabrielle Rabinowitz

    Thank you for this post! I am happy to see someone reporting on the way Big Science can hurt individual researchers. I make this argument as part of my case against the 3 billion dollar investment in the Brain Activity Map here: http://incubator.rockefeller.edu/?p=730

  • Shiran Pasternak

    Well said, but I wouldn't necessarily conflate bioinformatics resources projects with the monolithic big data projects. There's still a sore need for integration, refinement, and access to public data procured by both top-down projects and smaller investigator-driven R01 projects.

    • Steven Salzberg

      Why do you think the BRC's do any of those things? As far as I can tell, they mostly download bacterial and viral genome data from GenBank/NCBI and other public resources, and re-display it with little or no added value. No one in the scientific community seems to use them other than the BRCs themselves, so I think it's fair to say they have been a flop. If they were R01s, they would have to go through peer review, and I'll wager they would have little chance of being renewed. All this while NIAID has a pay line of 6% for R01s.

      • Omar Harb

        I just would like to point out that hundreds of scientists use BRC resources. Just do a google scholar search with the term PlasmoDB (part of one of the BRCs) you'll get a few thousand hits (over 300 in 2012 alone!). I don't think you are being fair at all. http://tinyurl.com/c6z9zo4

      • Omar Harb

        I just wanted to point out that hundreds of scientists use BRC resources. Just do a google scholar search with the term PlasmoDB (part of one of the BRCs) you'll get a few thousand hits (over 300 in 2012 alone!). So I think you are not being fair at all. http://tinyurl.com/c6z9zo4

  • http://twitter.com/drgitlin Jonathan Gitlin

    You've singled out a number of NHGRI's research programs in this post (ENCODE, 1000 Genomes, TCGA). I don't know that this supports your claim that the result will be fewer R01s; NHGRI's research portfolio has always been much more heavily represented by cooperative agreements and large grants than R01s. And that's before you think about how much more research can be conducted now that sequencing is as cheap as it is, which is a direct consequence of the investment in DNA sequencing technology by, you guessed it, NHGRI.

    Big science is terrible, except for all the times when it isn't.

  • http://www.facebook.com/antistokes Allison Stelling

    We need "big science" if we're going to "cure cancer"--- but, I don't think that work should be done at the universities. Universities should be for training scientists, and doing risky "discovery work" without fears of losing your job if you don't discover exactly what shareholders think the laws of the universe ought to be. The "big science" stuff should be done at huge centers and should offer permanent positions for technical staff (the UK does this a lot). We have a few of those in the USA, but we need something on the order of the Manhattan Project right now-- or larger-- if we're going to get serious about curing many human diseases. It's not going to be just one institute or lab group that does this, it's going to require a ton of communication and cooperation across disciplines over many different labs and hospitals in different locals so results can be replicated and verified.

  • http://twitter.com/mjcgenetics Michael James Clark

    At first I thought I agreed with your post here, but on second thought, I think you're understating the usefulness of these large projects to their respective fields.

    I also think you're misrepresenting the situation by only using the number of applications and the number of funded projects as your metrics. I think you must include the amount of money actually available for funding and account for inflation.

    I agree that funding for huge projects seems to suck up funding that could go to smaller projects, but I also see a lot of utility from these huge projects that you seem to gloss over (I mean, I use results from HGP, 1kG, ESP, ENCODE, TCGA every day).

    This is why I think funding at LEAST needs to keep up with where it was at ten years ago. In fact, I think funding should be increased dramatically relative to where it was ten years ago in order to facilitate both these large projects which benefit us all and the small projects with more defined goals that people like yours truly would try to have funded.

    I also object to you using "omics" as a synonym for "big huge research project". There are "little guy" omics projects too, you know. Or at least, there would be if we could get funding! ;)

    • Jonathan Badger

      That's the problem -- people want to do "small science" themselves, but if they are honest, they acknowledge that they need the data generated by "big science" as a starting point. But guess what? That data has to come from somewhere and that means funding "big science".

  • http://twitter.com/drgitlin Jonathan Gitlin

    After spending some time poking around in REPORTER, I don't see the data backing up these claims. The trends for centers vs RPGs hasn't been that strong, whereas the average cost of an R01 has almost doubled since 1998: http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=204&catId=2

    What's more, the number of applications to NIH has doubled since 1998: http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=200&catId=2

    Meanwhile, biomedical inflation has eaten ~20% of NIH's purchasing power, and indirect costs have been increasing, along with salaries, contributions towards health insurance, and so on. To lay all of this (or even the majority, IMO) at the doors of 'big science' is misguided.

  • Elton Zeqiraj

    I think it was Freeman Dyson who said that "around year 2000 it became cheaper and easier to generate data than make sense of it". Apparently we are taking the easier, some might say lazy, path of doing science. Certainly some omics projects are hugely beneficial, but the obsession of the funding agencies with generating data and headlines is a cause for concern.

  • David Roos

    As a taxpayer who has benefited from investigator-initiated research,
    and someone whose research lab has been funded by unsolicited R01
    awards for 20+ years, I completely concur with Salzberg's support for
    this program. I believe that some of his comments on the
    Bioinformatics Resource Centers may be misguided, however.

    In
    addition to my research program, I also direct one of the BRCs -- the
    Eukaryotic Pathogen Genomics Resource (EuPathDB.org) -- a family of
    databases that supports genomics efforts in the protozoan parasite
    community. These are *not* research projects, although they depend on
    the algorithms, statistical tools, software, genomic-scale datasets, etc
    developed by researchers elsewhere (including the Salzberg lab). They
    are therefore funded as (peer-reviewed) contracts rather than grants,
    and staffed by data loaders, software engineers, DBAs, etc, rather than
    research students and post-docs.

    As service operations, BRCs
    should be *rigorously* evaluated based on whether they provide
    cost-effective value to the research community (including, but not
    limited to, R01 investigators). If they do not, they should be shut
    down.

    Does usage justify the ~$2M/year that NIH spends on
    EuPathDB? I believe so, and invite any reader of this forum to ask any
    researcher working on organisms we support about the value of these
    resources to their group. Assessing the utility of on-line resources is
    a challenge, but here is some relevant data:

    1. The parasitology
    research community is relatively small, but EuPathDB databases are
    visited by >13K unique users each month, from >100 countries, with
    the average user returning ~3.4x monthly. This family of databases has
    received a cumulative total of 5K citations to date (as assessed by
    Google Scholar), and serves up many terabytes of data to users ... not
    just "genomes downloaded from GenBank", but also comparative genomic
    alignments, SNP calls based on consistent criteria applied to data from
    multiple sources, extensive functional genomics data, etc.

    2.
    One component of the project, the TriTrypDB.org database supporting
    research on the kinetoplastid parasites responsible for Sleeping
    Sickness, Chagas disease, Kala-azar, etc has been supported by the Bill
    & Melinda Gates Fdn & The Wellcome Trust, rather than
    NIH,
    leveraging infrastructure developed through the BRC program. In an
    effort to determine value added, an independent assessment conducted by
    the Wellcome Trust determined that the direct value of this resource to
    WT-funded R01-type researchers in the UK alone is >10x the cost of
    the project. The UK accounts for ~17% of TriTrypDB usage, vs 19% in
    Latin America and 26% in the US.

    3. EuPathDB runs annual
    workshops in the US, UK (funded by the WT), and Germany (funded by the
    EU), and we have offered many additional workshops in Africa, India,
    Latin America, and elsewhere. Applications typically exceed available
    slots by 3-fold, despite the fact that participants are generally
    required to cover their own travel expenses.
    These on-line
    resources have also helped to expand global engagement in genomics
    research: India has recently surpassed the UK as the second largest user
    of the malaria parasite database PlasmoDB.org (10%, vs 34% US), and few
    resources have been as effective at engaging African researchers (3%).

    In sum, I believe that these resources are worth the substantial
    investment that NIH has made in the BRC program, but would be happy to
    hear evidence-based arguments to the contrary. I would also be happy to
    to hear suggestions from others on better ways to assess value.

    • Steven Salzberg

      Full disclosure: I was a co-PI on one of the original BRCs, funded at The Institute for Genomic Research. We certainly benefitted from having it, but it quickly became clear to me that the community didn't really need any of the BRCs, not really. It also became clear that the contract mechanism was a terrible way to fund a research database. The BRCs are run by NIAID, so they are answerable only to a contract manager, not to the scientific community.
      David Roos's EuPathDB is, in my opinion, the best of the 4 current ones. But this means it would stand up to the "rigorous" evaluation that David suggests should be required. Unfortunately, these BRCs are not evaluated by the community (and I note that David R. doesn't claim this).
      To pick on another example, the virus-specific BRC, ViPR, contains almost nothing other than data and functions (BLAST searching, oh gee thanks) that I can get at NCBI already. If it were peer-reviewed, it would never survive.
      The amount of $$$ going into these contracts is huge. Let's put that into R01s instead.

      • David Roos

        What kind of community evaluation would you suggest, beyond:
        - the peer review of contract proposals
        - the metrics I listed under points 1-3, above

        - daily feedback from users via the on-line resources and social media

        • Steven Salzberg

          Put the BRCs through the same peer review that the rest of us have to go through. Just reviewing them isn't enough if the contracts get renewed anyway (as they have, for >10 years now). They should be in the same competitive pool as R01s.

  • homolog,us

    Thank you for the very sensible suggestion. The train wreck of funding more and more centrally directed mega-projects should stop.

    http://www.homolog.us/blogs/2013/03/02/please-save-the-unsolicited-r01s-salzberg/

  • nana

    Um, you do realize that the only reason those costs have fallen the way
    they have is because NHGRI has invested heavily in this area?
    obat kuat
    obat kuat alami
    obat kuat pria
    obat kuat sex
    obat pembesar penis
    obat perangsang wanita
    alat bantu sex
    obat penambah sperma
    pembesar penis