"We don't do that in the United States," Hagel told reporters at a NATO defense ministers meeting. "We rely on facts."
Hagel said the Army will review the circumstances surrounding how Bergdahl left his unit and was captured by the Taliban, and added, "It's not my place as a former sergeant in the Army to decide who's worthy of being a sergeant and who isn't."
Bergdahl was released Saturday from five years' captivity by the Taliban in exchange for five top Taliban officials.
Asked whether men had died in the efforts to rescue Bergdahl, Hagel said, "I don't know of any circumstances or details of U.S. soldiers dying as a result of efforts to get Bergdahl."
Men in Bergdahl's unit have said that some did die in the efforts, even if indirectly.
"Those soldiers who died on those missions, they would not have been where they were ... if Bergdahl had never walked away," said Evan Buetow, a sergeant in Bergdahl's unit at the time. "At the same time I do believe it is somewhat unfair for people to say, 'It is Bergdahl's fault that these people are dead.' I think that's a little harsh."
Gen. Martin Dempsey, chairman of the Joint Chiefs of Staff, said Tuesday that the Army might still pursue an investigation, and those results could conceivably lead to desertion or other charges.
Congressional hearings and briefings are looking into the deal that swapped Bergdahl for Taliban officials who had been held at Guantanamo Bay, Cuba, and several lawmakers said President Barack Obama didn't notify them, as required by a law governing the release of Guantanamo detainees. White House staff members called key members of Congress to apologize.
The Taliban on Wednesday released a video showing Bergdahl's handover to U.S. forces in eastern Afghanistan, touting the swap as a significant achievement.
The 17-minute video emailed to media shows Bergdahl in traditional white Afghan clothing, clean-shaven and sitting in a pickup truck. More than a dozen Taliban fighters with machine guns, their faces mostly covered by headscarves, stand nearby. Bergdahl blinks frequently and appears to be listening as one of his captors speaks to him.
A Black Hawk helicopter lands and two Taliban fighters, one carrying a white cloth tied to a stick, lead Bergdahl halfway to the helicopter. Three Western-looking men in civilian clothes take Bergdahl to the helicopter, where soldiers in Army uniforms are waiting.
After his release, the 28-year-old from Hailey, Idaho, was reported to be in stable condition at a military hospital in Germany.
The freed Taliban Guantanamo detainees were flown to Qatar, the Persian Gulf Arab country that served as a mediator in the negotiations for the swap.