Saturday, March 05, 2005
Lies, Damned Lies, and Book Recommendations
Behold the latest idiotic misuse of statistics by the Independent:
A survey published today to coincide with World Book Day confirms what authors from Louis de Bernierès to Alexander McCall Smith can attest - nothing sells better than the recommendation of a friend or relative. One in four of those polled said the last book they read was on the basis of what a colleague or family member had told them, with almost a third of under-35s citing it as the most important factor. Only loyalty to a favoured author counted as much, with 26 per cent of readers saying their last choice of a book for pleasure was because they had read others by the same author.
In a disappointing result for the promotional teams who spend up to £100m on book advertising every year, only 6 per cent said they chose a book because they saw it advertised, with 7 per cent citing the cover design as the deciding factor.
This was issued amid book recommendations and the like for World Book Day. My suggestion for what journalists should read? How To Lie With Statistics. Then they should call their paper's ad department and see if publishers have been rushing to cancel ad buys. I suspect they'll find nothing of the sort. Think publishers know something the study's authors didn't?
There are two fundamental problems with this survey: the first is a methodological one so severe that it renders the results worthless, and the other is a matter of flawed interpretation of this useless data. This survey measures what people say made them buy a book -- indeed, saying what they recall made them buy a book -- and not what demonstrably didmake them buy it. Consumers are notoriously unreliable reporters of their own purchasing motives: nobody, particularly anyone who considers themselves at all intelligent, wants to admit or even realize that their purchasing decisions are influenced by advertising or by product design.
But if survey numbers can lie, sales figures do not. If design and advertising only actually accounted for 13% of sales, every book designer and copy writer in the country would have bruised asses from the doors hitting on them on the way out. But go to a midtown Starbucks, and I think you'll find any number of them seated at their iBooks without any apparent discomfort. Go to a bookstore and watch what consumers do rather than what they say, and it quickly becomes apparent that book design matters a great deal more than anyone cares to admit.
At the very least, a survey relying on the quasi-delusional fantasies of consumer responses should have at least gathered data at the point of purchase: i.e. not about "The last book you bought" but about "This book that you're buying right now." To use consumer memory of purchasing motives just makes the already suspect result altogether useless.
So: enough about the raw data. What about the interpretation?
There is the curious assumption in The Independent's article -- and perhaps in the study itself, which they do not bother to identify or provide a link to -- that purchase motives do not overlap each other. This is a basic misunderstanding of advertising. Your friends may have recommended a book, but is it not possible that a purchase is made more likely by continued prompting from advertising? Or, indeed, that a recommendation is more likely to be remembered or heeded because it was preceded by advertising or by a memorable book cover?
More importantly, the Independent's notion of causality is utterly simple-minded. Let us say, for the sake of argument, that the recommendation of friends is indeed the panacea that is being claimed. Fine. So how did they hear about the book?
There may be influential social nodes whose word-of-mouth influences later sales, but to reach them in the first crucial months after a book's release, and before books start making their long journey back to the pulp grinders... that requires promotion. In other words, the efforts of marketing departments necessarily precede word-of-mouth sales. To set the two against each other in survey results creates a false dichotomy of the sort that no publisher would be fooled by... though I suppose an arts reporter might be.
In any case, I'm a little dubious about just how widespread the word-of-mouth phenomenon is, though it is certainly very visible in the case of some bestsellers. But when the article cites "what authors from Louis de Bernieres to Alexander McCall Smith can attest," they are running the gamut from A to... uh, B. The vast majority of books do not have long sales runs: they have a sharp spike in their first few months, with the nearly all sales occuring within six to nine months, and then a race to the bottom afterwards.
Friends recommending books to each other over the course of years and buoying up worthy authors with a steady stream of sales and praise may make for feel-good headlines, but a hell of a lot more titles are destined for the remainder table than for afternoons at the book club over chamomile tea.
A survey published today to coincide with World Book Day confirms what authors from Louis de Bernierès to Alexander McCall Smith can attest - nothing sells better than the recommendation of a friend or relative. One in four of those polled said the last book they read was on the basis of what a colleague or family member had told them, with almost a third of under-35s citing it as the most important factor. Only loyalty to a favoured author counted as much, with 26 per cent of readers saying their last choice of a book for pleasure was because they had read others by the same author.
In a disappointing result for the promotional teams who spend up to £100m on book advertising every year, only 6 per cent said they chose a book because they saw it advertised, with 7 per cent citing the cover design as the deciding factor.
This was issued amid book recommendations and the like for World Book Day. My suggestion for what journalists should read? How To Lie With Statistics. Then they should call their paper's ad department and see if publishers have been rushing to cancel ad buys. I suspect they'll find nothing of the sort. Think publishers know something the study's authors didn't?
There are two fundamental problems with this survey: the first is a methodological one so severe that it renders the results worthless, and the other is a matter of flawed interpretation of this useless data. This survey measures what people say made them buy a book -- indeed, saying what they recall made them buy a book -- and not what demonstrably didmake them buy it. Consumers are notoriously unreliable reporters of their own purchasing motives: nobody, particularly anyone who considers themselves at all intelligent, wants to admit or even realize that their purchasing decisions are influenced by advertising or by product design.
But if survey numbers can lie, sales figures do not. If design and advertising only actually accounted for 13% of sales, every book designer and copy writer in the country would have bruised asses from the doors hitting on them on the way out. But go to a midtown Starbucks, and I think you'll find any number of them seated at their iBooks without any apparent discomfort. Go to a bookstore and watch what consumers do rather than what they say, and it quickly becomes apparent that book design matters a great deal more than anyone cares to admit.
At the very least, a survey relying on the quasi-delusional fantasies of consumer responses should have at least gathered data at the point of purchase: i.e. not about "The last book you bought" but about "This book that you're buying right now." To use consumer memory of purchasing motives just makes the already suspect result altogether useless.
So: enough about the raw data. What about the interpretation?
There is the curious assumption in The Independent's article -- and perhaps in the study itself, which they do not bother to identify or provide a link to -- that purchase motives do not overlap each other. This is a basic misunderstanding of advertising. Your friends may have recommended a book, but is it not possible that a purchase is made more likely by continued prompting from advertising? Or, indeed, that a recommendation is more likely to be remembered or heeded because it was preceded by advertising or by a memorable book cover?
More importantly, the Independent's notion of causality is utterly simple-minded. Let us say, for the sake of argument, that the recommendation of friends is indeed the panacea that is being claimed. Fine. So how did they hear about the book?
There may be influential social nodes whose word-of-mouth influences later sales, but to reach them in the first crucial months after a book's release, and before books start making their long journey back to the pulp grinders... that requires promotion. In other words, the efforts of marketing departments necessarily precede word-of-mouth sales. To set the two against each other in survey results creates a false dichotomy of the sort that no publisher would be fooled by... though I suppose an arts reporter might be.
In any case, I'm a little dubious about just how widespread the word-of-mouth phenomenon is, though it is certainly very visible in the case of some bestsellers. But when the article cites "what authors from Louis de Bernieres to Alexander McCall Smith can attest," they are running the gamut from A to... uh, B. The vast majority of books do not have long sales runs: they have a sharp spike in their first few months, with the nearly all sales occuring within six to nine months, and then a race to the bottom afterwards.
Friends recommending books to each other over the course of years and buoying up worthy authors with a steady stream of sales and praise may make for feel-good headlines, but a hell of a lot more titles are destined for the remainder table than for afternoons at the book club over chamomile tea.
