"The Tonight Show" host agreed to make Sunday morning's so-called "Polar Plunge" with Mayor Rahm Emanuel as a condition for the mayor appearing on Fallon's show in New York, following an exchange of tweets and challenges.
The annual event draws several thousand hearty plungers to raise money for Special Olympics Chicago.
Fallon fans mingled with participants, carrying signs with pictures of Fallon's head. Many were dressed in costumes - including, appropriately, polar bears. It was 10 degrees an hour before the plunge, and the water temperature was 32.
Seventeen-year-old high school senior Marilyn Lamanna and a friend got up at 5 a.m. to snag a spot where they hoped to watch the feat unfold.
"Between Jimmy Fallon and the Special Olympics, it doesn't get much better than that, even though it's super cold," Lamanna said.
Emanuel - dressed in gray Nike sweatpants, a red sweatshirt and an orange hat - talked to reporters about an hour before the plunge. He was flanked by Detroit Lions defensive lineman and former Bears player Israel Idonije and Chicago Public Library Commissioner Brian Bannon.
He got emotional as he talked about meeting the mother of a special needs child in a Chicago grocery store, saying she was grateful for his participation in the event. He said they cried and hugged.
"There are parents out there, there are children out there who have something they can give," he said.
Special Olympics Chicago President Jen Kramer credits Fallon and Emanuel with attracting more than 3,000 people for the run into the lake from the city's North Avenue beach. That's more than ever. Last year 2,300 people did it.
Emanuel said last summer that if city's children read 2 million books as part of a Chicago Public Library program called "Rahm's Readers," he'd jump in the lake. When it appeared that they had met the goal, Emanuel jokingly called for a recount.
When Emanuel heard that Fallon wanted him to appear on the late-night show that he took over in February from longtime host Jay Leno, the mayor called the comedian and challenged him to join in the plunge.
Fallon joked on his show Wednesday that he agreed to make the plunge before realizing what the plunge actually entailed. He called it a "nightmare" but said he was a "man of my words." He said Emanuel told him he had to "toughen up" before Emanuel would appear on his show. The comedian's followers egged him to do it with a #JimmyPlungeWithUs campaign on Twitter.