Pausing NIH study sections is a good thing

 by Vinay Prasad

Panic unfolded yesterday as the NIH announced a pause in study sections. Study sections are groups of mediocre scientists who decide which grants are funded. You may bristle at my word choice of ‘mediocre’ but data support that claim. Here is research by Ioannidis in Nature:

He looked at authors of papers with more than 1000 citations. These are highly influential studies. I have published 530+ papers, but only one of mine fits this bucket. As such, I would make this group, but I would not have 2 years ago.

He compares this to study sections members and you can see the poor overlap. Ioannidis conclusion: conform and be funded. NIH seeks mediocre ideas that tread along established lines and not highly novel views. It does a bad job of funding people who do truly transformational work.

Trump has paused study sections to allow future NIH director Jay Bhattacharya to revisit the priorities. This is completely normal and reasonable. Jay might decide to run a randomized trial testing the current study section structure against proposed alternatives, such as the modified lottery, and other ideas.

If you randomized grant giving to several strategies, and follow the portfolio of funded projects 5 or 10 years into the future, you could analyze measures of citation, publication, impact, patent and other downstream proxies. If, there are no differences between the current process, and less bureaucratic processes, you should choose the method that has the least overhead price. This might be something Jay explores.

Alternatively, Jay might decide that having NIH funding for things like CT screening for homeless people is a misplaced used to priorities. There are many examples of woke science funding that will not survive in a Trump administration.

Scientists who quit X may protest but we live in a democracy. If you prefer more funding for diversity supplements, try to win the next election. For the time being, prepare for cuts in diversity supplements.

Interestingly, the one type of diversity that NIH is not interested in funding is intellectual diversity. That's probably a reason why they've had so much stagnation on intractable problems such as cancer and neurological conditions. They went all in brute force sequencing of tumors, and the amyloid hypothesis, for e.g.

Some people say that if the pause, which is completely reasonable, continues, people will lose their jobs in research. Of course this is true. I suspect the pause will not continue for a great period of time, but, at the same time, some people in research need to lose their jobs.

The government cannot be a welfare program for everybody doing low quality, low credibility, irreproducible, low value of information research. It has to use public dollars in a wise way. That has absolutely not occurred in the past. A pause is necessary to tackle this intractable problem.

In many ways, Jay is the perfect person to tackle this problem. He's not a laboratory scientist. He's an economist. The difference between laboratory scientists and economist is that the latter are much better at thinking brutally and clearly about the trade-offs and expected payoffs of research. Jay has already been on record as saying he thinks the NIH is not willing to push the envelope. It doesn't fund truly transformative work. I completely agree with him. And he should direct funding in that way.

The NIH call some of its pathways early innovation or something similar, but typically this is derivative work as well. The NIH does not actually take risks on truly different ideas. The envelope is not being pushed.

And the American people that want the envelope pushed. They don't want continued marginal drugs. They want new ideas. We have made no progress in Alzheimer's disease in part because of the NIH's dogmatism. I look forward to a renewed focus on a diversity of ideas in science research.

Finally, if academics want to take a sky is falling approach to every single thing Donald Trump does, they're only going to exhaust themselves.

https://www.drvinayprasad.com/p/pausing-nih-study-sections-is-going