Tuesday, November 17, 2015

  • Tuesday, November 17, 2015
  • Elder of Ziyon
Last week, the leftist British group Yachad released a poll of British Jews which received wide coverage. Its main findings are summarized by The Guardian:
75% of British Jews agree that “the expansion of settlements on the West Bank is a major obstacle to peace”, and 68% have a “sense of despair” whenever new expansion is approved.
73% believe Israel’s approach to peace is damaging its standing in the world.
71% see the two-state solution as the only way Israel can achieve peace.
72% reject the statement that “the Palestinians have no legitimate claim to a land of their own”.
62% support ceding territory to achieve peace, but that falls to 50% if withdrawal is seen as posing a risk to Israel’s security.
47% see the Israeli government as “constantly creating obstacles to avoid engaging in the peace process” (32% disagreed).

UK Media Watch noted how the questions in the poll themselves were biased, but there is a far more fundamental problem with the sampling methodology itself.

Over 50% of the respondents were chosen this way (page 50):

A common method for recruiting a sample from an inaccessible population is to identify a group of individuals from the population and invite them to recruit other members who will both complete the questionnaire and invite still others to do the same. ‘Snowballing’ methods of this kind are capable of achieving relatively high response rates because of the element of personal contact, but they run the risk of (i) recruiting an uncontrolled and potentially unrepresentative sample, and (ii) allowing abuse by vested interest groups who may submit multiple responses or circulate links to large numbers of people within that interest group.

In order to mitigate these risks, we developed a discriminative snowballing methodology with the following features:

(i) a group of 72 initial contacts (seeds) was selected by the research team and advisory group such that the group was roughly representative of the Jewish community as a whole with respect to synagogue affiliation, age and geographical location.

(ii) each seed was then asked to send invitations by email to between 10 and 40 of their Jewish contacts asking them to participate; each contact received a personal and unique code that could only be used once (phase 1).

(iii) the phase 1 recipients, in addition to being asked to complete the survey themselves, were provided with three additional unique codes and asked to send those to Jewish contacts of their own (phase 2). We limited the number to three to prevent blanket responding.

(iv) the phase 2 recipients were also asked to send links to up to three contacts using the unique links that were displayed on screen on completion of the survey (phase 3). This methodology generated 568 responses, of which 444 were generated at phase 1 and 124 at phases 2 and 3

So the advisory group chose the initial seed which then recruited the remainder of the 568.

Who was on the advisory group?

10 of them are listed in the paper, and Jonathan Hoffman identified nine of the ten as leftists.



It is of course likely that the leftist group would recruit like-minded friends, who would again do the same in their next pass. Meaning that fully 50% of the respondents to this poll are from a likely biased set.

Like J-Street in the US, Yachad commissioned this poll to confirm their own attitudes towards Israel and to find - or create the illusion of - widespread criticism of the Jewish state that mirrors their attitudes. Which is exactly what happened.

How many supporters of Likud do you think would be in the initial 72 people chosen by this leftist "advisory panel?"  Yet two thirds  of British Jews who expressed an opinion said before the Israeli elections,  that they would vote for Netanyahu if they were Israeli!

This is not just shoddy - this is deceptive.

UPDATE: As I mention in the comments, there is a very easy way to discover whether the people selected by the advisory group and then snowballed from there have the same opinions with the other half of those polled, whose methodology of selection seemed reasonably sound (at least in comparison.) Publish the results. If I am wrong, I will retract; if I am right and the results are significantly different, Yachad will issue an apology. Deal?

UPDATE 2: The report itself shows that the bias existed;
As predicted, the panel sample [145 people - EoZ] was the least close match; it under-represented younger Jews and to a lesser extent those with postgraduate degrees. These deviations would have shifted the findings in a hawkish direction. The snowball sample [568 people - EoZ] over-represented Jews with a left-leaning political stance and those with post-graduate qualifications; this would have produced a dovish bias. The DJN sample, as expected, was the closest match to the population profile; it slightly underrepresented members of central Orthodox synagogues and slightly over-represented older respondents; these two biases would have tended to neutralize one another in terms of any overall bias towards hawkish or dovish views.
So the pollsters admit that half the respondents held over-represented left leaning views!

The report then claims that it used weighting to eliminate the bias, but the weighting as to make the respondents fit better into the synagogue affiliation and age groups of the overall population.. But the left-leaning bias that they admit was not compensated for; they make the assumption that post-graduates will bring more post-graduates into the poll but not that people who are politically to the left would be far more likely to have friends who think the way they do.

And this footnote is interesting:
 Since it is impossible to determine which sample is the most representative, and to what extent each one departs from representativeness, there is no rational basis for weighting the samples relative to one another
Yet they made the stated assumption throughout that the DJN sample was the closest, demographically, to the Jewish population as a whole. That means that they are effectively giving four times the weight to the left-leaning snowball sample than to the (supposedly) right-leaning panel sample. 

Again, the only way to figure this out is to see the actual numbers and the specific formulas they used to supposedly reduce bias.

UPDATE 3: I am hearing that I am wrong - the data is similar between the snowballed sample and the rest. Will update when I get the details.

UPDATE 4/RETRACTION: Jonathan Hoffman contacted the pollsters and they took my challenge. The snowball and non-snowball halves had similar views on Israel, so in this case snowballing didn't majorly affect the responses except in a couple of specific questions.


I thank Stephen Miller for doing this analysis.

If I may interpret the differences between the groups, the "snowball" group seems to be more actively engaged in the issue of Israel than the other groups. They aren't more "hawkish" by any means but they are more supportive of Israel's right to exist as a Jewish country than the population at large and proud of Israel's achievements just as they are more supportive of a Palestinian state.

This blog may be a labor of love for me, but it takes a lot of effort, time and money. For over 11 years and over 22,000 articles I have been providing accurate, original news that would have remained unnoticed. I've written hundreds of scoops and sometimes my reporting ends up making a real difference. I appreciate any donations you can give to keep this blog going.



EoZTV Podcast

Podcast URL

Subscribe in podnovaSubscribe with FeedlyAdd to netvibes
addtomyyahoo4Subscribe with SubToMe

search eoz

Loading...

comments

Speaking

follow me

Follow by Email

translate

Share on Whatsapp


E-Book

For $18 donation








Sample Text

EoZ's Most Popular Posts Ever

Hasbys!

Elder of Ziyon - حـكـيـم صـهـيـون

This blog may be a labor of love for me, but it takes a lot of effort, time and money. For over 12 years and over 25,000 articles I have been providing accurate, original news that would have remained unnoticed. I've written hundreds of scoops and sometimes my reporting ends up making a real difference. I appreciate any donations you can give to keep this blog going.

Donate!

Donate to fight for Israel!

Monthly subscription:
Payment options


One time donation:

subscribe via email

Follow EoZ on Twitter!

Tweets

Compliments

The Jerusalem Report:"A seemingly indefatigable one-man operation, armed only with a computer, chutzpa and stamina."

Algemeiner: "Fiercely intelligent and erudite"

Omri: "Elder is one of the best established and most respected members of the jblogosphere..."
Atheist Jew:"Elder of Ziyon probably had the greatest impression on me..."
Soccer Dad: "He undertakes the important task of making sure that his readers learn from history."
AbbaGav: "A truly exceptional blog..."
Judeopundit: "[A] venerable blog-pioneer and beloved patriarchal figure...his blog is indispensable."
Oleh Musings: "The most comprehensive Zionist blog I have seen."
Carl in Jerusalem: "...probably the most under-recognized blog in the JBlogsphere as far as I am concerned."
Aussie Dave: "King of the auto-translation."
The Israel Situation:The Elder manages to write so many great, investigative posts that I am often looking to him for important news on the PalArab (his term for Palestinian Arab) side of things."
Tikun Olam: "Either you are carelessly ignorant or a willful liar and distorter of the truth. Either way, it makes you one mean SOB."
Mondoweiss commenter: "For virulent pro-Zionism (and plain straightforward lies of course) there is nothing much to beat it."
Didi Remez: "Leading wingnut"

Interesting Blogs

Categories

Abbas liar Academic fraud administrivia al-Qaeda algeria American Jews Amnesty analysis anti-semitism apartheid arab refugees Arafat archaeology art ASHREI B'tselem bahrain bbc BDS BDSFail Bedouin Beitunia beoz book review breaking the silence Cardozo Chanukah Christians conspiracy theories Cyprus Daphne Anson Davis report DCI-P double standards Egypt Elder gets results ElderToons Electronic Intifada EoZNews eoztv Erekat EU Euro-Mid Observer Fake Civilians 2014 Fatah featured Features fisking flotilla Forest Rain free gaza freedom of press palestinian style future martyr Gary Spedding gaza Gaza Platform George Galloway George Soros gideon levy gilad shalit gisha Goldstone Report Good news Grapel Guardian gunness Haaretz hamas Hamas war crimes hasbara Hasby 2014 Hasby 2016 Hebron helen thomas hezbollah history Hizballah Holocaust denial honor killing HRW Human Rights Humanitarian crisis humor Hypocrisy ICRC Ilan Pappe impossible peace incitement Indonesia international law intransigence iran Iraq Islamic Judeophobia Islamism Israel Loves America Israeli culture Israeli high-tech J Street jabalya jeremy bowen Jerusalem jewish fiction Jewish Voice for Peace jihad jimmy carter John Kerry jokes jonathan cook Jordan Juan Cole Judaism Judea-Samaria Kairos Karl Vick ken roth khalid amayreh Khaybar Lebanon leftists Linkdump lumish mahmoud zahar Malaysia max blumenthal McGraw-Hill media bias Methodist Miftah Mohammed Assaf Mondoweiss moonbats Morocco music Muslim Brotherhood Nakba Natural gas Nazi News nftp NGO NIF norpac NYT Occupation offbeat Omar Barghouti Opinion oxfam PA corruption PalArab lies Palestine Papers pallywood pchr PCUSA Peter Beinart Petra MB poll Poster Preoccupied Prisoners propaganda Proud to be Zionist purimshpiel Qaradawi Qassam calendar Rafah Ray Hanania real liberals reference Richard Falk rogel alpher roger cohen roger waters Saudi Arabia saudi vice self-death self-death palestinians sex crimes SFSU shechita sheikh tamimi Shujaiyeh Simona Sharoni SodaStream South Africa Speech stamps Syria Tarabin Temple Mount Terrorism This is Zionism Thomas Friedman Tunisia Turkey UCI UK UN UNDP unesco unhrc United Arab Emirates Unity unrwa UNRWA hate unrwa reports UNRWA-USA Varda Vic Rosenthal Washington wikileaks work accident X-washing Yemen zahran zionist attack zoo Zvi

Blog Archive