The offensive came after soldiers shelled the mosque and home of Mohammed Yusuf in Maiduguri, the capital of northern Borno state, to end an uprising by the self-styled Nigerian "Taliban" which has said it wants to lead an armed insurrection and rid society of "immorality" and "infidelity."
President Umaru Yar'Adua said late on Tuesday the group would be hunted and punished.
The military operation under way will "contain them once and for all", he said, adding: "They will be dealt with squarely and forthwith."
Fresh fighting broke out here late Tuesday after the assault. An AFP correspondent witnessed soldiers shooting three young men dead at point blank range close to the city's police headquarters.
The men, who had just been arrested, were seen kneeling and pleading for their lives before being shot.
During the assault, security forces overran the home of the elusive sect leader Yusuf and a nearby mosque used by his followers.
"We are not sure whether he has been killed in the shelling or has managed to escape," a police officer said of Yusuf.
"It is the first time in my life that I hear this kind of mortar shelling," exclaimed one man in a hushed tone, as he huddled in Maiduguri's police station to seek shelter along with dozens of others.
"I thought they targeted my house," he said, bringing along his wife and three daughters.