By Porter Anderson, Editor-in-Chief | @Porter_Anderson

Alternative Industry Statistics at DBW

The person who calls him- or herself Data Guy—and who is sporting an embroidered image of a spider these days—is surely Arachnida ambitia, a specialist in avid calculation.

In what promises to be one of the more unusual moments at FW’s Digital Book World conference, March 7 to 9 in New York, this mystery analyst will present a keynote oration with easily one of the longest titles of the conference: Outside the Data Box: Taking a Fresh Look at ebook Sales, the Indie-publishing Market, and a Fast-changing Publishing Business.

If there’s any time left after #DBW16 impresario Mike Shatzkin has read out that title to us, the unidentified Mr. or Ms. Guy will then offer his or her “fresh look” at industry sales, based on the Author Earnings project begun two years ago with the author Hugh Howey.

In the new February Author Earnings report, released Monday (February 8), things continue to look rosy for self-publishing authors and dire for the trade. But there are also announcements of changes in the approach—not entirely clear changes, mind you but in some ways promising. We’ll look at a bit of that later in this article.

To grasp the industry-political context here, we must remember that many who are skeptical of the efficacy of the Author Earnings effort point to the fact that it is an agenda-driven exercise.

Frustrated with what they said was a skewed and pro-industry picture presented at the Digital Book World in 2014’s “What Authors Want” survey, Mssrs. Howey and Guy set out to find a way to demonstrate that self-publishing is a viable route to earnings potential for authors. The result is the Author Earnings assessments, which many in the self-publishing community have defended as proof that their pathway to publication can be as good or better, financially, as the standard trade publishing route. Those fans, again, will be chuffed.

Highlights of the New Report

There are many assertions and conclusions drawn in the latest report, as in the earlier ones. It runs to 21 pages in a PDF edition. I urge you to read the entire piece to get the full picture that Data Guy wants to present. Here, I’ll just offer a very few key points before looking at what we’re being told about how all this is now derived.

Here’s the news: Author Earnings asserts that on Amazon’s bestseller lists, indie self-published titles account for more than twice the number of Big Five titles.

“What has changed,” the report tells us, “is the degree to which Amazon’s overall Top 20 bestsellers, and even the overall Top 10, have come to be dominated by self-published titles from indie authors—nearly half of which were not priced at $0.99 but rather ‘full-priced’ sales at prices between $2.99 and $5.99.”

From the date on which “our spider ran,” in Guy-talk:

• Four of Amazon’s overall Top 10 bestselling ebooks were self-published indie titles
• Ten of Amazon’s overall Top 20 bestselling ebooks were self-published indie titles
• Fifty-six of Amazon’s overall Top 100 bestselling ebooks—more than half—were self-published indie titles
• Twenty  of Amazon’s overall Top 100 bestselling ebooks were indie titles priced between $2.99 and$5.99

The most interesting question for us at this juncture is just what the trade publishing management attending DBW will make of this. Can it be that the “legacy” industry is being outclassed so substantially by “indie published” authors—the self-published sector?

Indeed, of interest to everyone, the report goes on to submit that it can tell us “how many ebooks a day Amazon.com is actually selling.” I quote the report:

As of mid-January 2016, Amazon’s US ebook sales were running at a rate of 1,064,000 paid downloads a day…

Total: 1,064,000

• Indie Self-Published ebook KU full-read equivalents    155,000
• Indie Self-Published regular retail ebook sales    293,000
• Small/Medium Publisher ebook sales     204,000
• Amazon-Publishing Imprint ebook sales     115,000
• Big Five Publisher ebook sales     244,000
• Uncategorized Single-Author Publisher ebook sales   53,000

Where the partisan nature of the Author Earnings effort always surfaces most clearly is, logically, in its claims about indie-author earning power, that original two-year-old agenda, “the pie chart that interests us the most,” as the report puts it. Here it is:

Ebook sales on Amazon.com, Data Guy tells us “are generating $1,756,000 a day in author earnings. But less than 40 percent of those author-earnings dollars—from the largest bookstore in the world—is now going to traditionally-published authors. And less than a quarter is going to authors published with the Big Five.”

What follows immediately is the bone that Author Earnings always wants to pick with the industry, with the Authors Guild, and with standard reporting methods:

“Is it any wonder that the traditional publishing media and historic author advocacy groups are reporting declining ebook sales and shrinking author incomes for their members? We humbly submit that, for author earnings, these organizations are looking in all the wrong places. $140 million a year in Kindle Unlimited payouts is going directly to authors, and yet that enormous sum of income is somehow uncounted by traditional author surveys. And as we are now able to measure, that sum is only the tip of the iceberg. There is also a vast swath of the market not being reported on at all, along with a whole host of authors not paying dues to author advocacy groups and simply going about the business of earning an income with their art.”

This is the language of self-publishing as what some of its champions call the “shadow industry,” a creative corps that cares nothing for the customs and concerns of the industry, and yet seems never to tire of carping at the establishment. It’s always worth noting that even some of the most-honored self-publishing bestsellers have taken contracts when offered.

And as anyone familiar with negotiating basics knows, by framing its results in ways that call out “the other side”—in this case, traditional publishing—Author Earnings repeatedly has hobbled its own efforts to widen the discussion. Rather than simply present an interpretation of the market and let that interpretation speak for itself, the material is served on a bed of right and wrong. Eyes glaze over, chips remain on shoulders, collegial exchange seems hard to come by.

For the first time, Author Earnings expands to print sales, and there, the Big Five are allowed a 47-percent dominance of daily revenue to authors from print bestsellers. The chart:

The commentary that goes with this one:

“It’s interesting to note here that the Big Five hold less than a quarter of print bestseller slots, and their unit sales, dollars, and author royalties are less than half of Amazon’s print business. This is a greater percentage than any other publishing type, but it again stresses the need for balance and perspective when the top publishers’ numbers are taken to represent the whole of the industry; they don’t even represent half of online sales in the format they are supposed to dominate. And self-published indie authors, who are already taking home 14 percent of online print author earnings, have captured a significant share of the author dollars from online print sales.”

On the way out the door, Data Guy stops to pop publishing with a towel for agency pricing, of course, which gives us those out-sized prices on trade ebooks and Amazon’s “This price set by the publisher” notes on the pertinent sales pages. It may well be true that the publishers are shooting themselves in the feet with agency pricing on ebooks, perhaps contributing to the slowing of growth in digital reading on some level.

The tone with which Data Gay tells us this is typical:

“By reinstituting agency ebook pricing and forcing their own consumer ebook prices high while preventing Amazon from discounting those ebooks, the Big Five publishers…willingly did financial harm to their own bottom lines and in the process also seriously damaged the sales and earnings of their own authors, in an attempt to wrest market share and control away from their largest and most profitable retailer.”

This may not be the kind of language with which Data Guy might endear himself to the industry seated in New York for DBW.

Data Guy goes on to say—and may tell the audience at the Hilton on Sixth Avenue—that:

“Despite the Big Five’s return to agency ebook pricing, Amazon’s overall US ebook sales have continued to grow throughout 2015 in both unit terms and dollar terms. On the other hand, the Big Five’s share of those ebook sales has plunged precipitously in both dollars terms, and even more precipitously in unit terms.”

If that keynote on March 9 from Data Guy includes the new report’s drive-by shot at coloring books—five of the 11 bestselling ones were self-published by indie authors, yea, even the coloring books!—an uninformed audience member may decide that the whole industry is going to hell in a conference tote bag.

That’s why the real context here involves the methodology. The battle of publishing’s statistics is not just over the numbers themselves but over how they’re generated.

The Methodology! The Methodology!

Since the first AuthorEarnings.com reports, Howey has pushed off on his gorgeous catamaran Wayfinder—of which several of us are venomously jealous. And in the latest report, in fact, we see a new thing: Data Guy refers to him or herself in the first person when mentioning the coming DBW event. This may indicate that Guy is pretty much sole captain of the good ship Author Earnings, a question we might be able to put forward during DBW’s Data Guy Q&A, a separate session with Publishers Marketplace’s always entertaining Michael Cader.

One of the most hopeful things you can find to read on the DBW Conference site, in fact, is in the brief description of that Q&A session: “In his main stage talk,” it says, “Data Guy is going to lay out his methodology.”

As the late comedian Henny (“Take my wife, please!”) Youngman would have put it, “Now, you take this data. Please!”

Some of the most highly placed operatives in the trade will privately tell you (they’ve told me) that they’ve spent two years scratching their heads over Author Earnings’ digital derring-do. It’s reputed to have several parts:

1. Sales reported by authors are compared to rankings of various titles on Amazon.com;
2. A “crawling” of Amazonian sales pages is accomplished (our spider man or woman at work) to “scrape data” on a single day for each report (January 10 in the newest one)—spider goes out, spider comes back with the scrapings;
3. An extrapolation of the results is made from that date; and
4. Inevitably, a web of upbeat news for fans of self-publishing is spun from the results.

It might all be searingly accurate, spot on, perfectly right. But we can’t tell that.

And, as I mentioned in some commentary on the last report of 2015 at The FutureBook, what’s probably needed is a full analysis by a completely independent, reputable, capable firm, a unit of Nielsen or Forrester or PWC, KPMG, Deloitte, somebody, anybody. Please!

In the new report, Data Guy tells us that things have changed. In an email to me, M. Guy describes it thusly:

“For 2016, we’ve revamped our Author Earnings calculation methodology from the ground up, basing our latest calculations upon reams of hard, irrefutable Amazon sales data. What data, you ask? Actual raw, unfiltered Amazon-reported daily sales figures for hundreds of books from many different authors—each day’s sales for each title individually verified line-by-line against their royalty reports. Hundreds of thousands of data points in all.”

I’ve asked for clarification on this and, as ever, Data Guy has come right back to me, unfailingly friendly, affable, good to work with. I’ve never met a spider I liked so much.

What I’m being told is that a group of more than a dozen authors now makes available to Author Earnings, in some form, its actual royalty statements. The approach thus takes out the second-hand nature of reported sales. This takes us, Data Guy says, past the realm of “volunteered data points” collected and on to more precise observations made “during the precise same time period.” Significantly, Data Guy says that rather than using single-day sales, “this model incorporates sales history and matches Amazon’s true algorithm far better.”

And that’s the crux of the change: it involves the formula Data Guy uses to infer what a sales ranking means, based on what he tells us is now better-sourced actual data on sales provided by authors.

In fact, the new report says that the previous methodology for two years “overestimated overall Amazon Kindle sales by roughly 18 percent.”

While the new approach, we’re told, takes care of that, one has to wonder what might even Data Guy not know at this point is flawed in this edition of the process? Time may tell us more. But it’s hard not to think that an independent review could tell us best.

An update here, about eight hours after initial publication. Data Guy has tweeted to us: “For two years, industry stats underestimated ebooks by 50 percent. All stats are imperfect. Some more than others.” This is, of course, the logical response from the Author Earnings perspective and it’s respected as such. Of course, its assertion of a “50-percent” underestimation is opinion. As is so much of what we’re dealing with: an industry that cannot count its own output accurately because of a major problem of data control by retailers. Data Guy also says that the 18-percent factor leaves the Author Earnings “market share pie charts unchanged, [by the] old or new method. That 18 percent only affects [the] whole size of [the] pie.”

At the least, it’s going to be interesting when Data Guy invites the business into his or her parlor at DBW in March.