The move reflects growing impatience in Germany at what is perceived as U.S. nonchalance about being caught spying on a close ally.
"The representative of the U.S. intelligence services at the United States embassy has been asked to leave Germany," government spokesman Steffen Seibert said in a statement.
"The request occurred against the backdrop of the ongoing investigation by federal prosecutors as well as the questions that were posed months ago about the activities of U.S. intelligence agencies in Germany," he added. "The government takes the matter very seriously."
Seibert said Germany continues to seek "close and trusting" cooperation with its Western partners, "especially the United States."
The U.S. government declined to comment directly on the decision. But White House spokeswoman Caitlin Hayden said the security and intelligence relationship with Germany was very important to the United States.
"It keeps Germans and Americans safe. It is essential that cooperation continue in all areas and we will continue to be in touch with the German government in appropriate channels," she said.
Shortly before the decision was announced, Chancellor Angela Merkel told reporters that Germany and the United States had "very different approaches" to the role of intelligence agencies.
She stressed the need for greater trust between allies, a position she has repeatedly voiced since reports last year that the NSA eavesdropped on her cellphone.
In separate cases over the past 10 days, one man has been arrested and an investigation against another has been launched on suspicion that they worked for foreign intelligence. German media have reported that the men are suspected of passing secrets to the U.S.
Interior Minister Thomas de Maiziere said the scope of the cases and who was involved are not yet clear, but talks are taking place with the United States at various levels.
"If the situation remains what we know now, the information reaped by this suspected espionage is laughable," de Maiziere said in a statement. "However, the political damage is already disproportionate and serious."
Geir Moulson in Berlin and Nedra Pickler in Washington contributed to this report.