Bestsellers as trend indicators
Someone again asked me about trends and picking them from bestsellers, and talked about the statistical approach to this. Yes, I know. Lies, damned lies, and statistics. Stats can get used to confuse and obscure, and the world is full people who would choose to have their pizza cut in 8 slices because if it were cut in 12 pieces (the same pizza) would be too much for them to eat. They have children, drive and vote ‘no award’ – which makes worrying about global politics or cataclysmic disaster seem a waste of time. What is a mere megaton meteorite strike compared to their talent for world-wrecking?
However, if applied properly statistics can be nice be useful friendly beasts who hardly ever soil the house, and keep the pests down. And if done right stats lets one compare apples with apples – even if we’re faced with mixed basket of apples, Ugli-fruit grapes, watermelons and pomegranates.
This of course is the real problem with ‘Bestsellers’ and ‘trends’, particularly in traditional publishing, but also increasingly, Indy. Without correcting for fruit, to make them all the equivalent of apples: Joe sold 10 grapes, and only one watermelon… Grapes are 10 times as popular as watermelon. Put in more grape-vines…
And this, literally, appears to be the level of understanding that NYC traditional publishing operated at. It worked, sort of, for values of ‘worked’ which included losing reader-customers slowly instead of very fast.
Now, let’s be clear correctly weighting the different fruit – or books – to make them comparable is really hard. Part of this is ‘why do you need to know?’ A trucking firm, operating by weight sold has vastly different needs to a farmer working out what is most profitable to grow, or a marketer working for a supermarket working out what is most attractive to customers. Some of these link to each other, and in reality a store that only sells watermelons or only has grapes is going to be as successful as a store that only carries one author – it can work, but customers need to have a good reason to come there, rather than a place where they can buy watermelons, grapes, vegemite, turkey and tampons and whatever else they happen to want or need.
Yes, yes. But what you, the author – or publisher – need is the bestseller, and the trend. So I should stop wasting your time. You’ll just look at the number of books sold or the sales ranking, and take it from there. Be my guest. It worked for traditional publishing… oh, wait.
You see part of the problem is we’re talking name recognition, marketing, cover, distribution etc… and we’re talking HISTORY. For an imaginary example: Traditional Publisher A suddenly releases… a new Robert Jordan. It’s a historical vampire story. The company throws the kitchen sink at it, great cover art, great marketing spend, 50-1000 copies in every bookstore between here and Far Floodle… and it sells a lot. Fantastic sales numbers, fantastic amazon rankings. Historical vampires are the new trend!
So using that sales history, every trad publisher in NYC rushes about bringing out Historical vampire books…
There’s just one problem. The book is dreadful, the only thing it had going for it was the fact that a very well-known author wrote it, and couldn’t sell it while he was alive. The sales are really to Wheel of Time fans and they are bitterly disappointed, and will never touch another Historical vampire book. The Name, push it had and expenditure it had made for the sales.
Using ‘Bestseller’ as an indicator of trends without weighting is a sure road to the weeping cross for an author or publisher… but it is common practice. Why? Because spotting a trend early is very lucrative. Most publishers and authors get on the bandwagon far too late. They’re like the people who buy in last to a pyramid scheme…
So how does this weighting thing work? The answer is: done right, very complicatedly. And if you make a mistake disastrously (the pollsters in the latest US election, and in Brexit were all using weighted sampling.) At best however, there are simple ‘weights’ which have such a large effect that they swamp out the others. In my old trade, Fisheries Science, catch per amount of diesel (as an indicator of size of vessel and time at sea) was sometimes used – which IIRC worked quite well comparing demersal trawlers with demersal trawlers, especially across years. If about the same tonnage of fish was landed in two years… but a lot more diesel was used, it meant the fishermen were working a lot longer and harder for their fish, and was a warning of trouble that could not be seen just looking at the tonnage. You could use money spent instead of diesel. There are other factors, it should never be taken as simplistic, but how much it cost to achieve a result is a valuable measure of trend to be looked at. I’m going to use data from the last US election to illustrate the point (not to take sides or bring politics into it, – not my country or business – but because we have the data and it illustrates a second point I want to talk about. We’re predicting trends not playing ‘who won’.)
http://www.fec.gov/disclosurep/pnational.do here is money.
And here is final tally. http://uselectionatlas.org/RESULTS/national.php
While the Democrat and Republican candidates would have had extra money from the party coffers once the primaries were over, I have simply taken it as what was donated to them. The Republicans had less in total, and their candidate had less in total so it’s probably not going to change things. (had the party coffers made the one candidate ‘richer’ while their personal funds were lower it would be different). I rounded the figures up for ease and with as much generosity as possible.
HC 66 million votes
DT 63 million votes
GJ 4.5 million votes
JS 1.5 million votes.
EMcM 0.73 million votes.
So as NYC publishing would put it the bestseller is HC and that’s the trend to watch.
Now let’s look at the money they had for their campaigns (there are other factors, sure.)
HC 66 million votes 563.9 million dollars
DT 63 million votes 328.4 million dollars
GJ 4.5 million votes 11.6 million dollars
JS 1.5 million votes 3.5 million dollars
EMcM 0.73 million votes 1.6 million dollars
So let’s ask how much each of these had to spend to get those votes. How much per vote?
HC 66 million votes at $8.50 per vote
DT 63 million votes at $5.21 per vote
GJ 4.5 million votes at $2.58 per vote
JS 1.5 million votes at $2.33 per vote
EMcM 0.73 million votes at $2.19 per vote.
How much money you have to spend on that campaign to get that vote is probably not a bad measure of effort – the less you spend per vote, the less effort it has been to get people to turn out and vote for you. It raises the question: had each of the candidates had precisely the same money how would they have fared? If money was the only factor, who would be the ‘bestseller’ then? Let’s imagine they each had a thousand dollars. This probably doesn’t translate into reality because it’s not the only factor and it’s questionable whether the smaller candidates (or lesser known books) could expand far beyond their base – but just for fun…
HC 117 votes
DT 192 votes
GJ 387 votes
JS 429 votes.
EMcM 456 votes.
So who is the bestselling trend-setter now?
Now to even begin to ‘weight’ this properly you’d have to include name recognition, positive and negative media coverage, as well as factors like spite and punishment voting, rusted-on voters, and the maximum voters a candidate could persuade. It’s complicated, and would be for books too. One could possibly take this as a measure of the major candidates against each other – measuring Correia against Scalzi, with the minor candidates against each other. It does paint a very different picture of what the trend – as in subject or type of book that requires little hard sell – is in line with the Zeitgeist, and could do very well given the right push.
Broadly speaking, as an author (or a publisher) wishing to get onto that bandwagon and follow the trend – the ONLY figure he has is Amazon sales ranks (which are already weighted in strange ways). Maybe if he is fortunate he has bookscan numbers (although these aren’t terribly accurate – over represented in specific brick and mortar, missing others and online) – but spend, promotion etc, will be at best a guess. My own take aways are: unless it is runaway success, orders of magnitude better selling than their previous work, well-known front-list names are terrible trend indicators. You’re as likely to be seeing history and marketing as popularity.
When a newb, or even minor midlister bursts into the top 100 and stays there for a few weeks – I’d take notice. The difficulty here is separating readers enjoying the quality of writing rather than the genre trend per se (some authors – Dean Koontz for instance have readers follow them regardless. I have a mixed bag with some readers who will read it if I wrote it, and others liking a specific series and not enjoying the rest of my books. Eric sells vastly more 163… than anything else. )
But seriously, bestsellers and trends… are a kind of gambling. Every publisher is doing so, and trying to skew the odds. So are most authors. But the truth is you can put so much into this, that one loses sight of the reason for this: writing books you love, that at least some readers will love.