An excerpt from a new history of Commentary shows how the fiction published in the magazine’s early years shook not just the world of Jewish literature but the very foundations of American letters