"I have decided to enter the government as foreign minister," he told reporters hours after clinching a hard-fought coalition deal with Merkel's conservatives.
Schulz tapped the party's parliamentary group leader Andrea Nahles to replace him as SPD chief, a role he himself held for just over a year.
Nahles, 47, would be the first woman to run Germany's oldest party.
"The party has to become younger and more female. After 150 years, the party finally belongs to a woman," Schulz told reporters at a Berlin press conference, flanked by Nahles.
"I believe that this step will appeal to large groups in our party," the 62-year-old said.
The SPD will hold a special party congress to propose Nahles for the leadership role.
That congress will take place after the SPD's 460,000 rank-and-file members vote on whether to approve the coalition deal hammered out with Merkel.
The result of the yes-or-no referendum is expected on March 4.
If they vote it down, Germany could be headed for fresh elections.