The problem for the plotters is that, apart from the usual suspects at either end of the political spectrum, there is no real stomach for a Labour leadership battle. More importantly, there is no one with any credibility who would want the job.
As I have often argued, modern politics is very different from even a decade ago. We live in the Big Brother, I'm a Celebrity political age.
The old rules simply don't apply. Gordon Brown can go from hero to zero in a few weeks, the good folk of Hartlepool can elect a monkey as mayor and Londoners vote in a panellist from Have I Got News for You.
So can Brown survive? Well, if you believe the political pundits, no. But doing the rounds of the Westminster bars, there is a slightly more sane view.
I overheard a discussion between two Labour MPs and one said that the Prime Minister wouldn't survive if he lost the Crewe by-election. The other MP was horrified that the loss of this seat would mean the end for the PM - and he was no Brownite.
Most sensible MPs believe Labour can win the next election, but two things need to happen. First, Brown and his team must focus on the issues that matter most to voters. The most glaringly obvious is petrol prices.
Why, they ask, can't the oil companies that make huge profits be hit with a windfall tax? Labour did this with privatised utilities in 1997 and it was a popular policy. Even Republican presidential candidate John McCain is proposing to hit US oil companies with such a tax.
The second thing that has to happen is for the economy to turn around. Here, Labour needs luck. It might just get it too, because, despite all the talk of gloom and doom, so far the British economy has been remarkably resilient. One well-respected economist told me last week he thought the economy would do much better than most people expected.
The final piece of the jigsaw is the Conservative Party. The old adage that 'oppositions don't win elections, governments lose them' is probably still true. The danger for the Tories is that they become overconfident or are seen to be triumphalist too early.
At The Spectator party last week, the biggest cheers weren't for Boris Johnson but for Veronica Wadley, editor of the Evening Standard, who fed Londoners a daily diet of bile against Ken Livingstone in the run-up to the mayoral election. They might not get such a sycophantic service from the media at a general election.