A heavy police presence of about 1,500 officers, with helicopters overhead and water-cannon trucks at the ready, blocked off the area around the Hofburg palace where the ceremony was to begin at 1000 GMT.
Several separate marches by leftwing and anti-fascist groups converged at the central Heldenplatz square. Placards included "refugees welcome" and "Nazis out" and "No Nazi pigs".
Police fired a smoke grenade when some protestors tried to break through a barricade, an AFP photographer said.
The new coalition was agreed on Friday by the conservative People's Party (OeVP) of Sebastian Kurz and the Freedom Party (FPOe), pledging to stop illegal immigration, cut taxes and resist EU centralisation.
FPOe chief Heinz-Christian Strache, 48, will be vice-chancellor. He has said that Islam "has no place in Europe" and last year called German Chancellor Angela Merkel "the most dangerous woman in Europe" for her open-door refugee policy.