Navigate to News section

The Problem With the Pew Study

We can learn a lot from the recent study on American Judaism, but the research is not without its flaws

Marvin Schick
October 18, 2013

Before there were numerologists, there were thinkers and writers who described with insight societal trends. They did so without overwhelming us with statistics. In this country, the greatest was Alexis de Tocqueville, whose Democracy in America bordered on prophecy. American Jewish life was enriched by the outstanding writings of Marshall Sklare, Charles Liebman, Seymour Martin Lipset, and others.

Then came the 1990 National Jewish Population Survey with its doleful statistic of the 50 percent intermarriage rate for recently married American Jews. The floodgates of quantitative analysis were opened wide and we have been inundated ever since. There is a constant stream of statistics telling us everything we may want to know about American Jewish life and much that we need not know. It may not be off the mark to say that there are more quantitative surveys of American Jews than there are of African Americans and Hispanics combined, although these two groups alone have more than ten times the population of American Jewry. We are addicted to the numbers game.

There is an old adage, “Thinkers don’t count and counters don’t think,” and it is not meant as a compliment to those who count. Although I am tempted to apply the saying to our hyperactive demographers, I won’t because it was once applied in a book review by Judge Henry J. Friendly, who did not like my use of statistics in Learned Hand’s Court, my study of the U.S. Court of Appeals for the Second Circuit. I mention this to underscore my belief that there is a place for quantitative analysis without going overboard.

It’s time for us to rein in our obsession with demography and it’s time for us to recognize that for all their claims of fidelity to science, the number counters invariably are skating on thin ice because unlike the U.S. census, the statistics they present are predicated on extrapolations and much else that can be errant.

The Pew study of American Jewry has already generated what seems to be more than a million words in print, and the ink on the report hasn’t been dry even a month. I have contributed a bit and even more in personal communications with the principal Pew researcher. Although I am critical of certain aspects of the research, there is much of value in the study, especially in the analysis. There is balanced writing, as well as compelling graphics.

Where Pew falls especially short, though, is in its analysis of the Orthodox, a group that constitutes but 10 percent or so of the American Jewish population. However, Pew pays significant attention to the Orthodox, dividing this segment into three subcategories and focusing in other ways on this relatively small portion of American Jewry.

The problems begin with the subdivision. A case can be made that inasmuch as there are no subcategories for Reform and Conservatives, there shouldn’t be for the Orthodox. The Reform are said to constitute three and a half times the number of Orthodox, on its face an absurd statistic. If the Orthodox have Modern and Haredi subcategories, why not subcategories for Reform, such as “weak Reform” or “non-participating Reform?”

Pew provides three alternatives for the Orthodox: Modern, Yeshivish, and Hasidic. The middle category is sophomoric usage that I think has been borrowed from the recent Federation study of New York Jews. If a person with university degrees and television sets in his home, who goes to the movies and dresses in modern fashion but does not want to be identified as Modern, by default he is put into one of the two categories designated by Pew as “ultra.” What makes this even less palatable is that in the questionnaire Pew does not have any “ultra” option. The designation is by the researchers and not by the respondents, which is remarkable because in defense of its statistics claiming that a great number of Orthodox Jews handle money on Shabbat, Pew claims that all it does is rely on self-identity. It should not be able to have things both ways.

The reliance on self-identity is problematic because inevitably it means that persons included in the report as Orthodox may be thus identified because that’s the only synagogue they are members of or go to, but who in fact are far from Orthodox in practice. Half a century ago in the American Jewish Year Book, Charles Liebman published a seminal study of the nominal Orthodox, people who in practice deviate significantly from Orthodox norms.

Over a period of more than two decades, I have been involved in what is called the Guttman research in Israel, focusing on the practices and beliefs of Israeli Jews. Guttman relies primarily on scaling, so that behavior and beliefs—and not self-identity—are the critical determinants of Orthodoxy. A case can be made for Pew’s preference for self-identity, but it needs to be accompanied by an acknowledgement that there is an alternative approach and that by including persons who by behavior and attitude are not Orthodox, the statistics for attrition away from Orthodox are badly skewed.

I recognize that if those self-identifying as Orthodox but who are not practicing Orthodox are removed from the statistics, the number of Orthodox Jews would be decreased. However, there is another defect in Pew’s research, namely the failure to understand much of the Orthodox community, a failure that arises in large measure from the circumstance that although Pew engaged a small army of consultants, it did not have the presence of mind to involve a single person with knowledge of the Haredim in the research.

I will not go into detail as to why Haredim are difficult to adequately include in demographic research, even when efforts are made to reach out to them, as has been true of Guttman and is true, in fact, of the Israeli government agency that conducts demographic research. Briefly, these factors include a language barrier, distrust of outsiders, and large family size. It’s astonishing that Pew went the extra mile to have Russian-speaking interviewers reach the small number of American Jews who speak only Russian yet could not recognize the language barrier in reaching Hasidim. It’s also astonishing that Pew did not recognize that when questions about identity include Christianity, to put it mildly, there is a powerful turnoff.

Again, if Pew did not harp so intensely on Orthodox life, there would be less justification for the criticism expressed here. Despite flaws in the research, there is much of value in the Pew report. This said, let’s pray for a return to good sociological analysis that is not obsessed with numbers.

Marvin Schick is a former political science and constitutional law professor at Hunter College and the New School for Social Research.

Marvin Schick is a former political science and constitutional law professor at Hunter College and the New School for Social Research.