Someone again asked me about trends and picking them from bestsellers, and talked about the statistical approach to this. Yes, I know. Lies, damned lies, and statistics. Stats can get used to confuse and obscure, and the world is full people who would choose to have their pizza cut in 8 slices because if it were cut in 12 pieces (the same pizza) would be too much for them to eat. They have children, drive and vote ‘no award’ – which makes worrying about global politics or cataclysmic disaster seem a waste of time. What is a mere megaton meteorite strike compared to their talent for world-wrecking?
However, if applied properly statistics can be nice be useful friendly beasts who hardly ever soil the house, and keep the pests down. And if done right stats lets one compare apples with apples – even if we’re faced with mixed basket of apples, Ugli-fruit grapes, watermelons and pomegranates.
This of course is the real problem with ‘Bestsellers’ and ‘trends’, particularly in traditional publishing, but also increasingly, Indy. Without correcting for fruit, to make them all the equivalent of apples: Joe sold 10 grapes, and only one watermelon… Grapes are 10 times as popular as watermelon. Put in more grape-vines…
And this, literally, appears to be the level of understanding that NYC traditional publishing operated at. It worked, sort of, for values of ‘worked’ which included losing reader-customers slowly instead of very fast.
Now, let’s be clear correctly weighting the different fruit – or books – to make them comparable is really hard. Part of this is ‘why do you need to know?’ A trucking firm, operating by weight sold has vastly different needs to a farmer working out what is most profitable to grow, or a marketer working for a supermarket working out what is most attractive to customers. Some of these link to each other, and in reality a store that only sells watermelons or only has grapes is going to be as successful as a store that only carries one author – it can work, but customers need to have a good reason to come there, rather than a place where they can buy watermelons, grapes, vegemite, turkey and tampons and whatever else they happen to want or need.
Yes, yes. But what you, the author – or publisher – need is the bestseller, and the trend. So I should stop wasting your time. You’ll just look at the number of books sold or the sales ranking, and take it from there. Be my guest. It worked for traditional publishing… oh, wait.
You see part of the problem is we’re talking name recognition, marketing, cover, distribution etc… and we’re talking HISTORY. For an imaginary example: Traditional Publisher A suddenly releases… a new Robert Jordan. It’s a historical vampire story. The company throws the kitchen sink at it, great cover art, great marketing spend, 50-1000 copies in every bookstore between here and Far Floodle… and it sells a lot. Fantastic sales numbers, fantastic amazon rankings. Historical vampires are the new trend!
So using that sales history, every trad publisher in NYC rushes about bringing out Historical vampire books…
There’s just one problem. The book is dreadful, the only thing it had going for it was the fact that a very well-known author wrote it, and couldn’t sell it while he was alive. The sales are really to Wheel of Time fans and they are bitterly disappointed, and will never touch another Historical vampire book. The Name, push it had and expenditure it had made for the sales.
Using ‘Bestseller’ as an indicator of trends without weighting is a sure road to the weeping cross for an author or publisher… but it is common practice. Why? Because spotting a trend early is very lucrative. Most publishers and authors get on the bandwagon far too late. They’re like the people who buy in last to a pyramid scheme…
So how does this weighting thing work? The answer is: done right, very complicatedly. And if you make a mistake disastrously (the pollsters in the latest US election, and in Brexit were all using weighted sampling.) At best however, there are simple ‘weights’ which have such a large effect that they swamp out the others. In my old trade, Fisheries Science, catch per amount of diesel (as an indicator of size of vessel and time at sea) was sometimes used – which IIRC worked quite well comparing demersal trawlers with demersal trawlers, especially across years. If about the same tonnage of fish was landed in two years… but a lot more diesel was used, it meant the fishermen were working a lot longer and harder for their fish, and was a warning of trouble that could not be seen just looking at the tonnage. You could use money spent instead of diesel. There are other factors, it should never be taken as simplistic, but how much it cost to achieve a result is a valuable measure of trend to be looked at. I’m going to use data from the last US election to illustrate the point (not to take sides or bring politics into it, – not my country or business – but because we have the data and it illustrates a second point I want to talk about. We’re predicting trends not playing ‘who won’.)
http://www.fec.gov/disclosurep/pnational.do here is money.
And here is final tally. http://uselectionatlas.org/RESULTS/national.php
While the Democrat and Republican candidates would have had extra money from the party coffers once the primaries were over, I have simply taken it as what was donated to them. The Republicans had less in total, and their candidate had less in total so it’s probably not going to change things. (had the party coffers made the one candidate ‘richer’ while their personal funds were lower it would be different). I rounded the figures up for ease and with as much generosity as possible.
So:
HC 66 million votes
DT 63 million votes
GJ 4.5 million votes
JS 1.5 million votes.
EMcM 0.73 million votes.
So as NYC publishing would put it the bestseller is HC and that’s the trend to watch.
Now let’s look at the money they had for their campaigns (there are other factors, sure.)
HC 66 million votes 563.9 million dollars
DT 63 million votes 328.4 million dollars
GJ 4.5 million votes 11.6 million dollars
JS 1.5 million votes 3.5 million dollars
EMcM 0.73 million votes 1.6 million dollars
So let’s ask how much each of these had to spend to get those votes. How much per vote?
HC 66 million votes at $8.50 per vote
DT 63 million votes at $5.21 per vote
GJ 4.5 million votes at $2.58 per vote
JS 1.5 million votes at $2.33 per vote
EMcM 0.73 million votes at $2.19 per vote.
How much money you have to spend on that campaign to get that vote is probably not a bad measure of effort – the less you spend per vote, the less effort it has been to get people to turn out and vote for you. It raises the question: had each of the candidates had precisely the same money how would they have fared? If money was the only factor, who would be the ‘bestseller’ then? Let’s imagine they each had a thousand dollars. This probably doesn’t translate into reality because it’s not the only factor and it’s questionable whether the smaller candidates (or lesser known books) could expand far beyond their base – but just for fun…
HC 117 votes
DT 192 votes
GJ 387 votes
JS 429 votes.
EMcM 456 votes.
So who is the bestselling trend-setter now?
Now to even begin to ‘weight’ this properly you’d have to include name recognition, positive and negative media coverage, as well as factors like spite and punishment voting, rusted-on voters, and the maximum voters a candidate could persuade. It’s complicated, and would be for books too. One could possibly take this as a measure of the major candidates against each other – measuring Correia against Scalzi, with the minor candidates against each other. It does paint a very different picture of what the trend – as in subject or type of book that requires little hard sell – is in line with the Zeitgeist, and could do very well given the right push.
Broadly speaking, as an author (or a publisher) wishing to get onto that bandwagon and follow the trend – the ONLY figure he has is Amazon sales ranks (which are already weighted in strange ways). Maybe if he is fortunate he has bookscan numbers (although these aren’t terribly accurate – over represented in specific brick and mortar, missing others and online) – but spend, promotion etc, will be at best a guess. My own take aways are: unless it is runaway success, orders of magnitude better selling than their previous work, well-known front-list names are terrible trend indicators. You’re as likely to be seeing history and marketing as popularity.
When a newb, or even minor midlister bursts into the top 100 and stays there for a few weeks – I’d take notice. The difficulty here is separating readers enjoying the quality of writing rather than the genre trend per se (some authors – Dean Koontz for instance have readers follow them regardless. I have a mixed bag with some readers who will read it if I wrote it, and others liking a specific series and not enjoying the rest of my books. Eric sells vastly more 163… than anything else. )
But seriously, bestsellers and trends… are a kind of gambling. Every publisher is doing so, and trying to skew the odds. So are most authors. But the truth is you can put so much into this, that one loses sight of the reason for this: writing books you love, that at least some readers will love.
Mad Genius Club 
25 responses to “Bestsellers as trend indicators”
I just had occasion to notice this. The Non-US Amazon sites don’t appear to be displaying US reviews any more. (And of course, foreign sales don’t affect your US rank.)
But hey, at least I finally had my first Canadian sale!
Congratulations! That’s a milestone. Hope you get more.
To make matters more interesting, others have access to the same information you do. If you try to follow the trend, you are likely to end up a big but over served market.
That may be the only way to get a best seller (and if you are an editor evaluated on words rather than figures that is important), but the risk adjusted reward might be better elsewhere.
In fiction there is also the question of what it is about the book that makes it a bestseller. Often the elements that appeal to readers aren’t evident from an elevator pitch synopsis.
One example is John Grisham. John Grisham’s books sold far above their original expectations, and he wrote about lawyers, so the Publishing Powers That Be decreed that courtroom dramas about lawyers must be The Next Big Thing.
However, the appeal of Grisham’s characters wasn’t that they were in the legal profession–he just wrote that because that’s the business that he happened to know. “A Time To Kill” was a classic morality play that delved into a thorny moral issue. “The Rainmaker” was the story of a plucky outsider making good by luck and daring. “The Firm” was about an innocent stumbling into a conspiracy and getting in over his head. And so on.
Every new best seller spawns a host of imitators that copy some of the superficial elements but fail to capture the essential nature of the imitated work.
And so Publisher A will release a gripping novel about love triumphing in the face of overwhelming odds in which one of the characters is a mermaid, and the book will be huge.
Publisher B will then release a second rate formulaic romance and make one of the characters a mermaid, and it will bomb.
Then all of the publishing mavens will nod their heads gravely and announce, “Mermaids have peaked.”
No, mermaids haven’t “peaked”, mermaids were never the issue. But people who never read more than a paragraph synopsis of a book before deciding to hype or kill it won’t ever understand that.
This. Definitely this.
For example: Pride, Prejudice, and Zombies . . . . and the burst of mash-ups that followed leading up to (or down to?) Abraham Lincoln: Vampire-Slayer.
FWIW I read Android Karanina. It worked, in a really Odd way, but I’m not going to read any other mash-up just because it is a mash-up.
I should have taken heed of the warning (foreword) in Pride, Prejudice, & Zombies where the author said he took the original novel and added zombies to make it “interesting.” Sorry, if you’re doing a mashup where you don’t respect one of the original source, that’s not going to end well. It’s a shame, because in the hands of someone who loved Austen, the book could have been really interesting, but there was enough contempt and lack of understanding of the time period in there that it couldn’t survive the internal inconsistencies.
Like the fad for all things Japanese ending with the housekeeper at Pemberley having bound feet. Five second fact-check there, sheesh.
Psssst, not a mash up but try A Touch of Night. I and a fellow austeniac are responsible.
At some point I will have to go through and get all of your works. (I know, bad fan, haven’t read most of them yet. I’m in a low-brain period, so I haven’t been introducing new books much.) I’m particularly amused by the concept of “furniture refinishing mysteries.”
Those are low-brain books. You might enjoy them.
The house DEMANDED a craft mystery. Like I have time for crafts in the traditional sense, with shows and everything. So…
C4c
2×4
Yes, chasing trends is much like trying to beat the house at a casino – everyone else is doing it and the odds are stacked against you, not least because, as you said, the factors that make a best-seller are often not clearly understood. Even when a newbie hits it big, like in The Martian, is it because hard-sf is the thing to write, or because Weir’s character and style appealed to a large number of people, rather than its setting?
For new writers, I think that a better use of one’s time is to look at sub-genres that are friendly to unknowns/indies and that have small but steady followings – making it easier for a newcomer to stand out. You can determine the former by seeing how many Kindle Unlimited books are on the top-seller charts for the genre. The rankings of the top 20-40 books in the subgenre will provide a good idea for the size of the market.
If everyone in the first 40 spots have ranks in the 200-1,000 range, that is a highly competitive subject; the odds of a newbie making a splash are fairly low. A genre where after the first few spots ranks start leveling at 5,000-10,000 (and which stays in that range for several books) provides more opportunities for a breakout – if you can move 20-30 copies/borrows in one day, you can start showing up on the first page of the Hot New Releases list, which adds visibility.
Granted, that tactic is far less likely to produce a huge best-seller, but it can start growing an audience, a percentage of which will follow you when you venture into other subjects. One you are more confident of getting a few dozen/hundred sales of anything out of the gate, tackling more competitive genres becomes a lot easier.
‘I think that a better use of one’s time is to look at sub-genres that are friendly to unknowns/indies and that have small but steady followings’ – I’m going off to look at the category where I seem to have the highest numbers (psychological), and which I’ve never pushed in advertising.
Maybe you have something there!
It is one of my keywords, and a good descriptor in that if you read Pride’s Children, you are going to live a lot of time in the heads of the three main (and pov) characters, and the reviewers seem to like that.
I always thought that was the entire point of fiction – living other lives vicariously. So you didn’t make the same mistakes, and you got to cheer for their victories.
When I worked at Evil Entertainment Empire, one of my jobs was to acquire films for foreign licensing. We had a guy who’s sole job was to tell us in advance how popular a film would be. He’d been doing this for more than two decades and was known to be completely reliable. One Friday we went out for drinks and after more than a couple I asked him how he did it. He grinned at me and responded, “I guess.”
There’s an art to making guesses.
yes, its an extremely educated guess… bur its a guess.
Especially after 20 years of doing so.
Bujold used that trick to good account in one of her Vorkosigan books. The weather guy makes better predictions than the computer, but when confronted with the question of “how?” he literally has no idea, much to the horror of his successor.
I went to a fairly good panel on cover design where one of the main takeaways is that you do not want to look at the bestsellers in your genre, because the rules are different. The example given was A Song of Ice and Fire, where the covers have gone from elaborate painted fantasy (when the series was new) to very simple, almost generic icons on single-color backgrounds, almost totally obscured by the author name. Instead, the suggestion was to look at what we might call decent midlist authors, where the sales are decent and steady (or increasing slightly). Those covers are likely to be more indicative of the current trends in covers.
ISTM that the same theory would work for figuring out genre trends. Look to the midlist, because they represent the general demand trend.
Orrrrrrrrr you could just write what you want to and just look to genre trends for marketing purposes.
Possibly a combination of both. Look at what you have ‘under the bed’ or idea wise and use some of that midlist stuff to figure out what order to queue up projects. /thinking out loud.
Passion is the key.
Some writers can identify trends and write to that trend, heck some of them can even fake enough enthusiasm and sincerity that it looks like the author enjoyed writing that space vampire vs. zombie kittens book. Some can’t. It doesn’t matter how hard they push themselves, it doesn’t matter how much skill they have, it doesn’t matter how much they need to put food on the table, they can’t write that Trend with conviction.
A reader can feel the difference, a story told sincerely (even if badly written) is almost always superior to a story told by a writer chasing a Trend they don’t enjoy (no matter how well written). I’m absolutely positive that of the dozens upon hundreds of Twilight imitators there are alot that exist only to chase that Trend and I’m sure many are better written than Twilight, but if the writer doesn’t believe in what they’re saying why should you as the reader?
A book written with passion has the chance to sing in someone’s soul, and one without tends to at best fill a hole until the real thing comes along (time filler Vampire books put out in between Twilight books for example).
A long time professional writer told me that the key to being a pro was to find a way to manufacture enthusiasm for work that follows those Trends and I’m sure she’s right. For the kind of career and work she liked to do. It didn’t appeal to me and the thought of cranking out books to a publisher’s whim doesn’t appeal to me. The new hot thing five years ago is Zombies? Okay, do I have a zombie book in me? Nope. Can I manufacture one? Sure, with work. Do I want to spend six months forcing myself to write one? Good God, no.
I admire those who can write to those constraints and manufacture enthusiasm and get the work out there into the world but as a reader I’m leery of the books that get written like that.
As a writer I’d sooner go draw.
It would be nice if the book my characters are writing was in a popular genre/vein/trend, but they do whatever the hell they want. I just write it down.
I feel like a janitor sometimes, coming along behind and sweeping up the loose ends.
*raised glass to fellow sufferer* Preach it!
I mean, if you start writing for a trend, more than likely (unless you’re obnoxiously fast), you’ll finish writing once the trend is over. I feel like ending up with a book in the trending genre is more a matter of luck than anything else.