Preproposal news is starting to leak out of the BIO directorate, and with it a renewed sense of anguish among those who were not invited to submit a full proposal. In particular, the invite rates were painfully low this year, coming in around 22%. Considering that only 25-30% of THOSE will ever see funding, it appears that success rates in the NSF core programs continues to decline.
This is bad. Worse than we've seen in this generation.
Naturally people see these numbers and turn on the process, calling it random and a lottery, then focusing on the expertise of reviewers as a problem.
I get it, I really do. But before we decide to tear down the machine, let me make two points:
1) There is a certain amount of randomness to this process, but it is not as great a negative force as people seem to like to believe. Everyone who has been doing this long enough has a story about a proposal that did well once or twice and then never crossed the line. Everyone. Of course, if you have survived long enough to tell that story, you probably have a story about a proposal that got denied a couple of times, but through your will and brilliance managed to break down the doors to funding!
If you have served on enough panels, you may or may not recognize these as two sides to the same coin. The only difference is on one hand we were screwed by variance in the system and on the other we won by grit and determination. Both these narratives fit nicely in our comfort zone, while ignoring the reality of the enterprise. For every proposal you managed to "get through despite the odds", some other PI "got screwed by the system". And thus the mythology of funding goes....
2) Funding is no lottery. That is not to say there is no luck involved (there is), but a lottery treats every entry the same. I can assure you, having reviewed well over 100 proposals to two different panels, all proposals are NOT the same. At least half are not even in the running for funding. At the preproposal stage IOS has taken to triaging probably about 1/4 right off the bat, and another 1/4 might as well be. In that upper half though, things get more interesting.
To make the jump from upper 50% to invited for full proposal, you really need to hit all the right marks and have people on panel who you get excited over your ideas. Easier said than done, I know. Once you get into the full proposal stage, well, now is when separating signal from noise gets pretty tough at 6% funding rates. I would argue that probably 70-80% of the science that makes it this far is fundable and would produce quality publications. We even have data on funded projects that demonstrates that reviewers can not predict the success of the project and it's hardly a stretch to say that our ability to draw a line between what should and should not get money at that level is suspect.
And here's where we identify the real problem.
All of the issues people identify in the system are symptoms of there just not being near enough money to fund all the good science. Of course there is luck involved when so many proposals that make the initial cut are worthy of support! The variance we see is mainly because of the constraints we are operating under and not a fault of the system, per se. If the 22% invite rate was the funding rate, none of this becomes a problem. But when you need to fit 100 hippos in a Chevy Nova, shit breaks down.
We need to stop complaining about the process and continue to apply pressure in the only place that will fix any of this. Without additional funds, this continual flatlining of the NSF budget will continue to make consistent funding a pipe dream.
