Neville-Rolfe, who served as a civil servant in the Ministry of Agriculture between 1973 and 1992, has been appointed under-secretary of state for intellectual property.
She will be responsible for the department’s business in the House of Lords.
Commenting on Neville-Rolfe’s four years as a non-executive director, Archie Norman, chairman of ITV, said: "The board would like to thank Lucy for her help in steering ITV through its transformation over the past four years. The fact that ITV is as strong as it is today owes much to her guidance and support."
Neville-Rolfe was made a life peer last year, when she was also recognised with a Lifetime Achievement gong at the PRWeek Awards.
She said: "It has been a privilege to serve on the board of ITV through such a compelling period, and I wish ITV further success as it grows into a great global entertainment company."