In the late 1930s, the site was abandoned. Documents show the British authorities, which then ruled Palestine, evacuated the Jewish residents, fearing they were vulnerable to an Arab insurrection. After the British left and Jordan occupied the West Bank in 1948, Palestinian families moved onto the uninhabited plot.That is a sanitized way of saying that the Jews were constantly attacked by Arabs in Silwan.
Here is the Palestine Post article about their evacuation, from August 15, 1938:
The British authorities were supposed to protect the homes. They didn't - Arabs broke into a synagogue and defiled the Torah there several months later:
UPDATE: Here is the letter from British officials about the evacuation - and the expectation that the Jews would return shortly. (h/t Stephen)