Sir Peter, the former chief executive of the Bank of Scotland, said he would not have taken the job if it involved getting rid of ITV's chief executive.
"I would not have accepted the job if my first task had been to get rid of the chief executive. That would have been absolutely daft," Sir Peter said.
There had been speculation that figures within the City who approved Sir Peter's appointment wanted to see Allen replaced as ITV chief executive.
It has been reported that Fidelity fund manager Anthony Bolton, dubbed the "quiet assassin", had approached Greg Dyke for the role of ITV chief executive.
Bolton led the charge to get rid of Carlton chairman Michael Green and was reported to want to see Allen removed as well.
Allen is blamed for failing to cut costs and losing investors millions through the £1.2bn collapse of ITV Digital.
However, Sir Peter said yesterday that he was confident that he could work with Allen on the key task of merging the operations of Carlton and Granada and cutting costs.
Sir Peter emerged as the front runner in the final days before his appointment to the £200,000-a-year non-executive job. He beat out the Tesco chairman John Gardiner to win the post.
While his appointment has gone down well in the City on the back of his long banking experience, it has been greeted less warmly elsewhere.
One senior media executive told The Times that it was "quite extraordinary that ITV was now being run by a banker and an accountant".
Not only do Allen and Sir Peter both have financial backgrounds, they are both Scots as well.
If you have an opinion on this or any other issue raised on Brand Republic, join the debate in the Forum here.
This article was first published on brandrepublic.com