Their album I Am A Bird Now beat a shortlist of 12 including hot favourites Kaiser Chiefs, Coldplay and The Magic Numbers.
But their victory immediately provoked controversy.
The band are American but qualify for the Brits-only £20,000 prize because singer Antony Hegarty was born in the UK.
Hegarty spent his early years in Chichester, west Sussex, and settled in the US aged 12 after a spell living in Holland.
Standing 6ft 4in and with a shock of black hair, the 34-year-old has a voice as extraordinary as his looks.
It has been likened to Nina Simone and gay magazine Attitude described his album as "the gayest thing ever recorded".
The prize was presented by Jools Holland in a ceremony at Grosvenor House Hotel in London.
Hegarty said: "They must have made a mistake. I'm completely overwhelmed, I think it's insane.
"I love so many of the acts tonight and I think it's a bit bonkers to give the prize to one person. It's a bit nutty.
"It's kind of like a crazy contest between an orange and a spaceship and a potted plant and a spoon - which one do you like better?"
Some of the other nominees appeared less than pleased with the results.
M.I.A.
British-Sri Lankan rapper M.I.A. walked out of the ceremony within minutes of the announcement.
Favourites Kaiser Chiefs did not pass comment.
But when the nominations were announced a month ago, their drummer Nick Hodgeson complained: "He's an American, really. It's a good album, but it's daft he's got in on a technicality."
Hegarty was a chorister as a schoolboy in Chichester and his musical heroes are British - Boy George was his childhood idol and repaid the honour by singing on I am a Bird Now.
The band is based in New York and has been a fixture on the arts scene there for over a decade.
Hegarty often appears on stage with a Japanese hermaphrodite mathematician called Julia.
Speaking after his win, the singer and pianist refused to be drawn into the argument about his nationality.
"It's interesting you would call me an American. I just feel really pleased to be a part of this," he said.
He said he was "utterly shocked" to win, adding: "It's a really prestigious award and I feel honoured to have won it. I've no idea what it signifies, I'm just taking it one step at a time."
Asked what he would do with the £20,000 prize money, he shrugged: "Maybe I should buy some clothes. I know I look a bit rotten."
Winning the Mercury Prize will guarantee a huge boost in sales for Antony and the Johnsons, a hitherto little-known act.
When Ms Dynamite won in 2002, her sales shot up by 700 per cent.
Chairman of the judges Simon Frith said: "It is the most extraordinary album. It doesn't sound like anything else around. I don't think anybody could be unmoved by it, which is a what a Mercury record should be all about.
"There must be a lot of people out there who have never heard it and I hope this win will encourage them to listen to it."
Of the controversy, Frith said: "We just listen to the records we are given. If he wasn't eligible then we wouldn't have been listening to it." Tweet

Comments
Login or Register to comment
What a joke this groups is completely...well rubbish doesn't even begin to describe them!! How can they be considered above acts like Coldplay. The winners of this prize get worse every year its almost like the jusges like being controversial it is unbelievable!!!
i first heard some antony & the johnsons tracks about 3 months ago and thought it was complete rubbish ! however in recent weeks i have listened again and i just can,t get their tunes out of my head (that would make a good song title ?) i would recommend to anyone with any feel at all for original & innovative music to give A&TJ a try . I am so glad that artists like coldplay & kaiser chiefs did not win the "mercury" - their albums(good as they may well be) get hyped enough in the press and the original idea of these awards was to recognise artists who may not get the tv or radio airplay they deserve .
I have never heard of this band, however I think it is a good thing that they won as surely this award helps unknown bands gain recognition. Bands like coldplay and the kaiser cheifs already have plenty of airtime and are nominated for plenty of other awards so it would be nice if they could show a bit of grace in defeat rather that make a fuss about where this person lives.
Antony and the Johnsons is a very intelligent band and won't be liked by most Coldplay fans or fans of pop in general. Also, how selfish is M.I.A.?
A few facts about Antony and the Johnsons:
Antony and the Johnsons is not a band - it is an individual.
Little known? Antony has sold pre -Mercury win- 100 thousand records in the UK alone. He is also signed to Rough Trade.
Forget what MIA and the Kaiser Chiefs say - they are only bitter.
Antony deserved to win this award. 'I Am A Bird Now' is an amazing, emotional, honest record. I would like to see MIA or the Kaiser Chiefs top that.