Carter, former chief executive of Ofcom, will be the Prime Minister's principal special advisor at 10 Downing Street.
He will be in charge of political strategy communications and research and the policy unit at Number 10.
Brown said: "I am delighted that Stephen Carter has accepted my invitation to join the Downing Street team as we build on our programme of reform and pursue an ambitious legislative programme.
"As both a chief executive and industry regulator, he has an enviable track record in the private and public sectors, and will bring his considerable qualities and experience to the work of government."
Carter will stand down as chief executive of Brunswick Group with immediate effect.
He said: "I will do my utmost to make a difference to effective government."
Carter a former chief executive of JWT, stood down as Ofcom chief executive in July 2006