The 84-year-old was earlier declared the landslide winner of an internationally-condemned poll, marred by violent intimidation, in which he was the sole candidate.
As African Union ministers gathered in Egypt for a summit, Gordon Brown pledged "substantial" help rebuilding the country if democracy was restored.
"I think what we are now looking for is that combination of African countries in the African Union, working I hope with the United Nations, sending envoys to Zimbabwe to see what progress can be made, to see what the way forward is.
"And let me also say that we and a group of countries are working together and prepared to contribute substantially financially to the reconstruction of Zimbabwe.
"Countries round the world will do so as long as it is restored as a democracy."
African observers in Zimbabwe called for a re-run, saying the election was not free and fair and supporters of Mugabe were beating people who couldn't prove they voted.
Movement for Democratic Change (MDC) leader Morgan Tsvangirai pulled out last week in protest at a brutal Government-organised campaign against his supporters.
But a defiant Mr Mugabe went ahead with the swearing-in ceremony. State media in Harare, however, did report that he was prepared to hold talks with the MDC.
Foreign Office Minister Lord Malloch-Brown, who is attending the AU gathering, said: "If law and order breaks down in the country or if Mugabe is utterly resistant to change and continues to oppress violently people's human rights, then I hope the African neighbours will do whatever it takes to secure his departure.
Status quo
"What you cannot accept is the status quo continuing - President Mugabe has to go."
Some form of national unity government would probably be required, he said, something Mr Tsvangirai has said could involve Mr Mugabe.
Archbishop Desmond Tutu said there was "a very good argument" for sending in an international force "to restore peace".
Mr Brown said the move by African leaders over recent months to condemn the regime had already proved pivotal.
"I believe that there will come a time when he (Mugabe) will realise that the rest of Africa, leaders that he has worked with over the years, are no longer ready to support the brutality and the violence and oppression of his regime.
"I believe that those leaders will make it absolutely clear that for Africa's reputation and for the sake of the people of Zimbabwe a great deal has got to be done to change things in Zimbabwe."
Mr Brown defended South African president Thabo Mbeki, who has been under increasing pressure to publicly condemn Mr Mugabe - not least from Archbishop Tutu.
"I have worked very closely with Thabo Mbeki over the years and I have got a very high regard for him," he said.
"I think he has felt that the private diplomacy that he has been engaged in has had an effect, and certainly did have an effect in forcing Mugabe initially to call the election."
Tweet

ace, manchester (29/06/2008 at 19:53)