By Sheldon Greaves
A recent Op Ed by Jack Hitt in the New York Times("Science and Truth: We're All in it Together")recalled the excitement of a few years ago in 2004 when what was believed to be the first ivory-billed woodpecker (Campephilus principalis)Â in over fifty years was spotted in a an Arkansas swamp. There was the inevitable circus of media and congratulations, followed by a few skeptical voices who expressed some reservations.
As the media hype faded, however, the skeptical voices persisted. Bird experts such as David Sibley carefully examined the video take of the bird and concluded that it was a more common species, not the long-lost ivory-billed woodpecker. One venue where the discussion and debate twisted and turned for years over the identity of the bird was a blog (tomnelson.blogspot.com) run by Tom Nelson, who at the time was an amateur birder. Gradually, the tide turned and the evidence grew sufficiently compelling that only a few die-hards remained convinced that the original identification was the correct one.
Hitt points out that this kind of "story arc" seems to accompany many of the sensational articles in today's news regarding science and technology:
Almost any article worth reading these days generates some version of this long tail of commentary. Depending on whether they are moderated, these comments can range from blistering flameouts to smart factual corrections to full-on challenges to the very heart of an articleâ€™s argument.
Look at the online version of this piece and youâ€™ll already see (I hope) a long string of comments. These days, the comments section of any engaging article is almost as necessary a read as the piece itself â€” if you want to know how insider experts received the article and how those outsiders processed the news (and maybe to enjoy some nasty snark from the trolls).
Should this part of every contemporary article be curated and edited, almost like the piece itself? Should it have a name? Should it be formally linked to the original article or summarized at the top? By now, readers understand that the definitive â€œcopyâ€ of any article is no longer the one on paper but the online copy, precisely because itâ€™s the version thatâ€™s been read and mauled and annotated by readers. (If a book isnâ€™t read until itâ€™s written in â€” as I was always told â€” then maybe an article is not published until itâ€™s been commented upon.) Writers know this already. The print edition of any article is little more than a trophy version, the equivalent of a diploma or certificate of merit â€” suitable for framing, not much else.
The article continues by raising some interesting questions about the changing nature of scientific review. In addition to the stately and still very prestigious peer review we now have crowd review (and sometimes crowd correction or even crowd overturning); Hiit points out that these days everyone realizes that the most up-to-date version of an article is the one online, not in the published print journal, because that is where the latest discussion happens.
Some scientists are already experimenting with variations of this idea within the stately world of peer review. New ways to encourage wider collaboration before an article is published â€” through sites like ResearchGate â€” are attempts to bring the modern world of crowd-improvement to empirical research.
Already, among scientists, there is pushback, fear that incorporating critiques outside of professional peer review will open the floodgates to cranks. Not necessarily. The popular rejection last year of the discovery of a microbe that can live on arsenic was mercifully swift precisely because it was executed by online outsiders. Not acknowledging that crowd-checking and amateur commentary have created a different world poses its own dangers.
I have to wonder if this presages a sea change in the role of the amateur scientist, not merely as an experimenter or theorist, but as a reviewer of "established" scientists and their work? It certainly seems that way, which makes the difficulties amateurs still face in getting their work published by professional journals just that much more ironic.