Prosecutor Michel Valet says the 30-year-old man and his 3-year-old and 6-year-old sons were killed in the shooting just before school started.
He said another child, between 8 and 10 years old, was also killed, and a 17-year-old seriously injured.
The shooter drove away on a scooter, a national police official in Paris said. The official did not wish to be named in line with departmental rules.
Police cordoned off the private Ozar Hathora school in the northeast of Toulouse. Some two hours after the shooting children were still in the school.
President Nicolas Sarkozy called the shootings an "abominable drama and a "frightening tragedy."
The shooting occurred about 8:10 a.m. — just ahead of the start of classes in most schools.
Gil Taieb, a vice president of the CRIF, France's Jewish umbrella group, told The Jerusalem Post he had no doubt the attack was a hate crime.Real time updates here, in French.
"For someone to locate this school in a place like Toulouse means he knew what he was doing," Taieb said. "He went there to kill Jews."
Taieb said the community was in a state of shock.
"There are occasional anti-Semitic attacks but they are small, nothing like this," he said. "We haven't had something like this in at least ten years."
The attack on the Jewish school may be linked to two other mysterious shootings that have taken place in southern France over the past week .
Last week an off duty French soldier was shot dead by a motorcyclist in Toulouse.
On Thursday three French soldiers were shot by an unknown man at a shopping mall in Montauban, 50 kilometers north of Toulouse. Two of them later died of their wounds.
French police said similar ammunition was used in both shootings.
In 2003, a poll found that over a quarter of French Jews were considering making aliyah because of increased anti-semitism there.
UPDATE: From Israel HaYom:
Victims identified as Yonatan Sandler, 30, from the Kiryat Yovel neighborhood of Jerusalem, along with his two sons Aryeh, 3, and Gavriel Yissacher, 6, as well as the 8-year-old daughter of the school’s principal, Miriam Monsenengo.A father who witnessed the shooting was interviewed.