Saturday, April 23, 2005
The Peer-Reviewer's New Clothes
Bless your geeky hearts, MIT pranksters! A Reuters piece reported on CNN last Thursday explains:
Jeremy Stribling said Thursday that he and two fellow MIT graduate students questioned the standards of some academic conferences, so they wrote a computer program to generate research papers complete with "context-free grammar," charts and diagrams. The trio submitted two of the randomly assembled papers to the World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI), scheduled to be held July 10-13 in Orlando, Florida. To their surprise, one of the papers -- "Rooter: A Methodology for the Typical Unification of Access Points and Redundancy" -- was accepted for presentation.
Stribling said the trio targeted WMSCI because it is notorious within the field of computer science for sending copious e-mails that solicit admissions to the conference..... Nagib Callaos, a conference organizer, said the paper was one of a small number accepted on a "non-reviewed" basis -- meaning that reviewers had not yet given their feedback by the acceptance deadline.
Oh. So that's all right, then.
The pranksters were, New Scientist reports, "Sick of receiving spam emails requesting submissions to the 2005 World Multi-Conference on Systemics, Cybernetics and Informatics - which charges $390 for each attendee..." Other news outlets have slowly been picking up the story, many of them simply rehashing the Reuters report without doing any actual, um, reporting of their own. But Thursday's Arizona Republic carries an AP piece which further updates this delightful little farrago:
"E-mails to a conference address and to organizer Nagib Callaos were not immediately returned Wednesday, and there was no answer at the Orlando telephone number listed under Callaos' name.... in addition to mocking academic jargon, the pranks shed light on what Stribling sees as a problem: conferences with low standards that pander to academics looking to pad their resumes, but which harm the reputations of more reputable gatherings.
"We certainly exposed this conference as being willing to publish any paper regardless of whether it's been peer-reviewed, which is kind of a dangerous precedent to set," Stribling said. "It's kind of dangerous to be able to pass anything off as scientifically valid." "
Want to try generating some learned-sounding gibberish of your own? Try head prankster Jeremy's website, which includes the SGIgen "random CS paper generator." It's the best wheeze since Alan Sokal pulled one over on the chumps (sorry, editors) of Social Text.
No doubt academy-bashers will try to make something out of all this, much as they did with Sokal. That probably shows that they, like the deserving victims of both hoaxes, can't be bothered to read carefully. Stribling himself points out that the Conference was clearly a "pandering" sort of affair: the problem lies not in scholarship itself, but when charlatans both inside and outside the academy adopt its rhetoric.
Jeremy Stribling said Thursday that he and two fellow MIT graduate students questioned the standards of some academic conferences, so they wrote a computer program to generate research papers complete with "context-free grammar," charts and diagrams. The trio submitted two of the randomly assembled papers to the World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI), scheduled to be held July 10-13 in Orlando, Florida. To their surprise, one of the papers -- "Rooter: A Methodology for the Typical Unification of Access Points and Redundancy" -- was accepted for presentation.
Stribling said the trio targeted WMSCI because it is notorious within the field of computer science for sending copious e-mails that solicit admissions to the conference..... Nagib Callaos, a conference organizer, said the paper was one of a small number accepted on a "non-reviewed" basis -- meaning that reviewers had not yet given their feedback by the acceptance deadline.
Oh. So that's all right, then.
The pranksters were, New Scientist reports, "Sick of receiving spam emails requesting submissions to the 2005 World Multi-Conference on Systemics, Cybernetics and Informatics - which charges $390 for each attendee..." Other news outlets have slowly been picking up the story, many of them simply rehashing the Reuters report without doing any actual, um, reporting of their own. But Thursday's Arizona Republic carries an AP piece which further updates this delightful little farrago:
"E-mails to a conference address and to organizer Nagib Callaos were not immediately returned Wednesday, and there was no answer at the Orlando telephone number listed under Callaos' name.... in addition to mocking academic jargon, the pranks shed light on what Stribling sees as a problem: conferences with low standards that pander to academics looking to pad their resumes, but which harm the reputations of more reputable gatherings.
"We certainly exposed this conference as being willing to publish any paper regardless of whether it's been peer-reviewed, which is kind of a dangerous precedent to set," Stribling said. "It's kind of dangerous to be able to pass anything off as scientifically valid." "
Want to try generating some learned-sounding gibberish of your own? Try head prankster Jeremy's website, which includes the SGIgen "random CS paper generator." It's the best wheeze since Alan Sokal pulled one over on the chumps (sorry, editors) of Social Text.
No doubt academy-bashers will try to make something out of all this, much as they did with Sokal. That probably shows that they, like the deserving victims of both hoaxes, can't be bothered to read carefully. Stribling himself points out that the Conference was clearly a "pandering" sort of affair: the problem lies not in scholarship itself, but when charlatans both inside and outside the academy adopt its rhetoric.
