It turns out that even that argument was used by the classic antisemites of history.
This op-ed in The Wisconsin Jewish Chronicle, October 03, 1941 shows that this argument was used by the antisemitic "America First Committee."
This op-ed shows a methodology of antisemitism that remains popular today.
"That's the trick, the technique and the plan of the anti-Semite. Blame the Jews for everything bad, then deny that it is anti-Semitism, and then blame the Jews themselves for bringing up the subject at all."