skip to main |
skip to sidebar
Last month I went to a talk at the Computer History Museum by Ward Cunningham, the father of the wiki (not of Wikipedia, but of the general idea of a wiki).
Wikis are a pretty controversial subject in the blogging community. Some people view them as one of the crowning glories of Web 2.0 software design. Others view them as a prime example of the moral bankruptcy of the whole Web 2.0 crowd. Generally Wikipedia is used as the example to prove either viewpoint.
I'm somewhere in the middle – I view the wiki as a tool, something that's neither good nor bad in and of itself. To me, the most interesting thing about the wiki concept is what it says about software engineers.
Cunningham described his thinking as he developed the wiki concept. He was looking to encode in software the sort of discussions that engineers have among themselves. If you're not an engineer, this will require a little bit of explanation. Engineering is closely allied with science, and shares many of its values. There's a strong belief in the existence of objective truth, something that everyone can agree on if presented with the right evidence. Vigorous debate is deeply respected as a way to discover the truth, a kind of Darwinian process in which the best ideas are the ones that survive. And everyone who participates in a debate is assumed to have the same energy level and commitment to seeking the truth. (I suspect this alignment with the values of scientists is one reason why Nature magazine came out in support of Wikipedia.)
A wiki is designed to facilitate the sort of debates that engineers have among themselves. When it works right, it can dramatically increase the speed with which a group reaches agreement, and can quickly integrate the ideas of many contributors.
A challenge for wikis is that many (actually, most) people don't share the engineering culture. Many people are deeply attached to their beliefs and aren't willing to revisit them no matter how much evidence is presented. In many subjects one person's idea of objective truth may be very different from another's, and in some (religion, for example), it's arguable whether there can be any truly objective truth at all. Energy levels and willingness to participate in an extended discussion also differ dramatically from person to person. Often the most energized people are the fanatics, the people who are least likely to engage in an unbiased debate.
All of these mismatches between wiki ideals and human behavior can cause a wiki to misfire.
So is that a fatal flaw in the wiki concept? I don't think so. Every human institution that I can think of encodes the values and assumptions of the people who created it. Laws are generally created by lawyers (at least in the US), who have a very strong culture based on adversarial balances of power. They tend to focus on creating idealized processes with strong checks and balances in them. Economists have their own view of human nature, based on the assumption that people are self-interested and driven by risks and rewards. They tend to focus on creating systems of incentives and punishments that will encourage the desired behavior. Religious leaders focus on faith and ethics, seeking to adjust what people think and want.
All of these systems have strengths to the extent that they embody truths about human beings, and weaknesses because no single set of ideas can explain all of human behavior. None of them are perfect, but I think we're stronger when we have several different ways of looking at the world. So I'm glad that the engineers are joining the lawyers and economists and priests in designing human institutions. It gives us another set of tools in the tool belt. Rather than arguing about whether wikis are intrinsically good or bad, I think we should be asking when that particular tool is most useful, and what safety precautions we should use with it.
(A side note: it turns out that Cunningham pronounces wiki as "weekee," not "wicky." Who knew?)
______________
By the way, I'm sorry I've been slow to post lately. I've been working with my neighbors on a dispute with a property developer, and that used up most of my blog-writing time in the last two weeks. We're now past our big city hearing, and I should be able to get back to posting at least once a week.
"If we execute appropriately we will be the engagement platform, or the layer, on top of anything that has an LCD display, any computing device -- everything from a refrigerator to an automobile to a video game to a computer to a mobile phone."
—Adobe CEO Bruce Chizen
Amid all the fuss and excitement about Web 2.0 and the latest speculation about Google's plans, it's easy to lose sight of the rest of the industry. But for most of the 1980s and 1990s, the marquee conflict in high tech was the battle for control over the PC operating system. That battle faded when Apple stumbled and Netscape failed to use the browser to break the hold of Windows. There have been efforts from the Linux and Java crowds, but in general it's been so long since there was an effective assault on desktop Windows that most people have given up on the idea.
But now the new Adobe/Macromedia is trying to break Windows, using Flash. Most people in the tech community are viewing this as a battle over web graphics, but it's really about next-generation applications in general, which cuts to the heart of the Windows franchise. Adobe's plan is extremely ambitions, and I think it might even work.
How to displace an operating system: Make it irrelevant
What is an operating system? To a computer scientist, an operating system is the software that enables computing hardware to work. It manages the basic operations of the system (thus the name), so applications can perform the tasks desired by a designer or user.
But to a user, the underlying plumbing of the operating system is irrelevant. It's just part of the hardware. What matters is the user interface, because that's what the user has to learn; and the application programming interfaces (APIs), because they determine which software programs you can run on the device.
Interfaces and APIs together are often referred to as a "platform," because they are the thing on top of which application software is built. So you'll hear industry people refer to the Windows platform or the Palm OS platform. This term platform is sometimes confusing to people outside Silicon Valley, because they think a platform is a wooden thing you stand on to give a speech.
The distinction between platform and OS is important because the underlying operating system plumbing doesn't generate much value. It's the platform that users and developers are loyal to. If you separate the platform from the underlying OS, the user interface and applications can run on any OS. The OS itself becomes just commodity technology.
There have been examples in the past of platforms separated from OS's. For example, Java is a platform of sorts (although it lacks a standardized user interface and it's not really consistent in the mobile world).
Flash: Adobe's new platform. The new software products that Adobe and Microsoft are working on are very complex, so I'm going to simplify aggressively here. You can find endlessly detailed commentary elsewhere on the web if you want it.
Basically, what Adobe's doing is merging the Flash animation environment with the Acrobat document environment, and converting them into a full-fledged application development platform. Last week Adobe announced Apollo, a software program that will let a Flash program run outside of the browser, even if the user is not connected to the Internet. This converts Flash programs from little presentations that run inside a web page into full-fledged applications that the user can store on a PC and run anytime by clicking an icon on the desktop. The Apollo program takes care of managing the underlying operating system, so a single Apollo application could run on a Windows PC, a Mac, or a Linux box.
This is very convenient for application developers who want to create a program that can run on any computer that downloads it. But since the vast majority of PCs run Windows, it's not that big a deal – we all know of websites that just don't work well on anything other than a Windows computer, and that's not a world-shaking crisis.
But Apollo becomes vastly more important because a version of it will also run on mobile devices. Unlike the PC world, there there's no OS standard for mobile phones and smartphones, and as I've written before, we're not likely to get one anytime in the foreseeable future. But if you could put Apollo on all those phones and handhelds, the OS wouldn't matter any more. Developers could just write their applications in Flash, and they would run anywhere.
This was how mobile Java was supposed to work, but it was allowed to fragment into dozens of different incompatible versions. Developers of Java mobile applications often complain that they spend more time rewriting their software to run on different Java versions than they do actually creating the programs in the first place. This is an intolerable burden for small developers, and it stifles mobile software innovation overall.
Today Adobe charges phone manufacturers to include Flash in their phones. This is a nice revenue stream, but it prevents Flash from becoming a standard the way it did in the PC world, where Flash is given away.
In a very detailed and interesting interview, Adobe CEO Bruce Chizen talked about the overall strategy and said he intends to make the mobile version of Flash free in the future. Some quotes:
"The changes to Flash "enables us to create an 'engagement platform.' Think of it as a layer or a vehicle in which anybody can present information that could be engaged with in an interactive, compelling, reliable, relatively secure way -- across all kinds of devices, all kinds of operating systems."
....
Q: Are we going to see head-on competition [between Adobe and Microsoft] to establish the platform for the next generation of web and application development?
Chizen: "Perhaps.... The fact that we are ubiquitous today makes us believe that we will end up being the appropriate platform in which to have these rich Internet applications."
....
"Our goal is to get our engagement platform, or the 'light' version of it, on these devices (mobile phones)....Even though I like the revenue, I suspect over time you will probably see us back away a bit from charging for the client and look to make our money through servers, desktop software, advertising revenue models, and so on.... If I have to make a trade-off between ubiquity and revenue -- as it relates to the mobile client business -- I'll go for ubiquity. Because if I have my client everywhere, then I can make money doing other things."
Thanks to my colleague Nilofer Merchant for pointing out this interview to me.
So here's the picture of what will happen: Adobe will offer Flash/Apollo as a standalone platform on PCs, Macs, and Linux boxes; and a light version on mobile phones. The third element in the strategy is development tools. Adobe inherited the Flex app development tool from Macromedia. Flex will be positioned as the easiest way for a developer to create a single application that runs anywhere, on both desktops and mobiles. This pitch will be appealing to many commercial developers, and especially attractive to in-house developers within corporations, who want to span the range of range of devices in the marketplace with as little investment and support burden as possible.
Adobe's strategy potentially gives a very seductive message to IT managers – deploy the new Flash on your PCs and mobile devices and you won't have to worry about anything else.
Apollo and Flash don't kill Windows directly, but they are an attack on its financial model. People pay a lot of money today for Windows because you must have it to run Windows applications. If the applications of the future are written for Flash, Windows turns into just a bunch of plumbing (and pretty insecure plumbing at that). Its intrinsic value drops tremendously, and hardware vendors will feel free to substitute Linux or something else for it.
This won't happen overnight, and no platform layer will be able to run all types of applications (for example, highly graphical games will still need to be written to the "native" OS, in order to get the best performance). But Adobe is aiming at the next generation of productivity applications, the things that will matter most five or ten years from now.
Microsoft's response
Microsoft isn't sitting still for all of this, of course. It's developing Windows Presentation Foundation, which will include graphics, text, video playback, animation, and a lot more. It's the graphics engine built into the next version of Windows, Vista. A derivative version of WPF called WPF Everywhere will be ported to Macs and mobile devices in addition to older versions of Windows. So like Flash, WPF/E will be a platform that runs on top of other operating systems. Like Apollo, it will also enable applications to run independently of the browser. It's basically a new platform.
But there are important differences. To protect the Windows business, Microsoft is holding some features out of WPF Everywhere (in particular, 3D and graphics acceleration). That means applications written for the full WPF may not run on other devices. This makes sense for Microsoft – it wants the Windows version of its software to be the most powerful, so people have an incentive to buy Windows. But that makes WPF/E a second-class citizen. If Adobe does a better job of adding features to Flash, and keeps its implementations consistent, developers and IT managers may prefer its fully cross-platform software over what may be perceived as an "intentionally crippled" WPF/E.
The other logical move for Microsoft would be to bleed Adobe everywhere else, so it won't have the money to fund the Flash initiative. Sure enough, Microsoft is working on products targeting PDF and possibly Photoshop. But Microsoft's motives aren't clear – it is attacking almost every successful franchise in the industry, in a drive to get more revenue. It's not clear yet if Adobe is being singled out for special attention.
What to watch for
Adobe is by no means a lock to prevail in this competition, and indeed you can find a lot of people online who feel WPF/E is more promising. Here are some of the factors that will help to determine who wins:
Product quality. Flash's legacy as an animation tool is both an asset and a burden for Adobe. Flash has a large installed base and lots of developers, which will help Adobe. But turning an animation tool into a full-fledged application development platform can be awkward. In contrast, Microsoft is architecting WPF from the start to be an application development tool. So the architecture of WPF may be cleaner and more flexible than Adobe's.
Developer base. The developers who use Flash tend to be designers and artistic types. The users of Acrobat tend to be more oriented toward corporate processes and print publishing. These groups don't necessarily mix well (in fact, among parts of the online community Acrobat is seen as the spawn of the devil; check out the comments posted here). Can these groups merge and segue into full-scale applications development, and if not can Adobe attract other sorts of developers? In contrast, Microsoft's base of developers is generally corporate and commercial app creators. Can these people step up to new, more web-like development paradigms, and can Microsoft win over the creative types?
Feature set. Microsoft has a history of burying competitors in a blizzard of added features. Although Adobe is by no means a tiny company, Microsoft can definitely outspend Adobe if it chooses to. I think the main thing restraining Microsoft will be competition with itself. Will Microsoft be willing to add features to WPF/E that hurt the differentiation of native Windows?
Pricing. Microsoft hasn't announced licensing terms for WPF/E. Flash is already free on PCs, and eventually will be on mobile devices as well. It's fairly easy for Adobe to do this since it doesn't have an OS business to defend. But Microsoft doesn't want to destroy the value of Windows, and so it will feel a lot of internal pressure to extract some sort of payment for use of WPF/E. When I was at Palm, Microsoft approached us about licensing some Windows CE software elements and development tools to work with Palm OS. It was a great opportunity for Microsoft to establish a position as a standard-setter in mobile devices, but it would have hurt the differentiation of Pocket PC. We seemed to be having good discussions until we came to the topic of price. Microsoft demanded a per-copy price that would have eaten up most of our revenue for the OS. When we made it clear that we couldn't afford to pay that much, the talks ended abruptly.
Price also matters on the Adobe side. Much of Adobe's opportunity depends on giving away the mobile phone version of Flash, so it can make the software ubiquitous and set a standard. If Adobe continues to charge for mobile Flash, the whole strategy will implode.
Potential outcomes
At this point it's impossible to predict who will win. Adobe might cripple Windows, or Microsoft might crush Flash. It's also possible that the different developer bases of Microsoft and Adobe will continue to be different, with the creative types choosing Flash and the traditional app types choosing Microsoft.
There's also a pyrrhic victory scenario in which Microsoft defeats Adobe, but in the process gives away the differentiation of Windows and permanently weakens itself. In this scenario, Microsoft would have to fall back on Office and Outlook/Exchange as its main moneymakers. I think there's a good possibility that something like this will happen – not because Microsoft wants it to, but because the competitive situation will force it to choose between an immediate loss to Flash and a gradual decline in Windows sales. If forced to choose, most companies pick the option that delays pain.
The winner in the pyrrhic victory scenario would probably be Google – a Microsoft without the Windows franchise would have much less cash available to fund its assault on search. Speaking of Google, it has been making moves toward assembling bits and pieces of a development environment that could be a competitor to both Flash and WPF. Fully implementing this would require some unusual (for Google) attention to development tools, and to creating thicker clients that can execute apps when a device is offline. But it's within Google's capabilities. Google doesn't benefit if either Microsoft or Adobe controls the app platform of the future.
Would breaking the Windows standard be a good thing or a bad thing?
It depends. It would be a good thing financially for computer users. Microsoft extracts a huge tax on every computer sold; if there were effective competition to Windows, most of that tax would disappear and PC prices would come down a bit.
It's less clear to me what the effect would be on innovation. Considering that I spent ten years of my career at Apple, I'm surprised that I feel that way. At one time I believed Windows was one of the most evil software creations ever. Now I'm more muddled. I disapprove very strongly of the business tactics Microsoft used to establish and hold the Windows standard, but I've seen the effect of OS fragmentation on the mobile market. Software development is stagnating in the mobile world, and the lack of an open and uniform platform standard is a big reason why.
We could end up with a confused situation in PCs where there is no dominant platform standard in the future. This could hurt innovation and reinforce the proprietary instincts of hardware manufacturers. Imagine Sony's obsession with Memory Stick, but applied to operating systems and multiplied across a dozen vendors. It could be an awful mess.
On the other hand, it could be that the market will sort things out, pick a single winner, and we'll all rally around it. I think that probably would have happened in mobile devices already if the operators weren't standing in the way. There isn't an equivalent roadblock in PC-land.
What rough beast slouches toward Redmond?
I wonder what it feels like to be a senior executive at Microsoft right now. The company's own success has extended it into so many different markets that it's hard to even keep track of them all, let alone figure out how best to win. As soon as you focus your attention in one area, a fire breaks out in another.
This challenge from Flash is largely Microsoft's own fault. If it had been able to innovate rapidly in Windows, and had established an OS standard for mobile devices, there wouldn't be much of a market opening for Adobe. Instead Vista has been delayed repeatedly, and although Windows Mobile has made some progress, it is nowhere near setting a standard in the mobile world.
A situation like this cries out for autonomous business units, in which you could hire CEO-types to run their businesses independently and make quick decisions. But Microsoft's greatest competitive asset is its ability to tie together its businesses, to leverage strength in one area to give it an edge in another (for example, leveraging the Exchange standard to dominate mobile e-mail). So Microsoft has recently consolidated business units, making the company more centralized and designating Ray Ozzie the Pied Piper of technical direction for the whole company.
But that means less responsiveness in the individual businesses, and it's easier for new competitive assaults to slip through the cracks. At some point your vigilance will slip and something bad will happen. While Microsoft struggles to find ways to bite into Google's advertising revenue, Apple has already sprinted to early dominance in mobile music, a new world of component software is emerging on the Web, and now Adobe of all people is attacking in probably the last place where Microsoft expected to be assaulted, the core Windows OS franchise.
Although the situation must feel overwhelming for someone in the middle of it, to an outside observer like me it's exhilarating. So many things in the industry are changing all at once that I don't think anybody can see where it will take us. You get the feeling that the old order in high tech is melting away, and the old rules and assumptions will all be called into question.
Historically, this sort of transition in high tech has killed off most of the leading companies and cleared the ground for a new generation of companies and standards to take root. Whether you see that as a good thing or a bad thing probably depends on whether you work in one of the big established leaders in the tech industry today.* I don't know if Adobe's strategy will succeed or fail, but I admire it for trying to take advantage of the transition rather than just clinging to its current franchises.
____________
*Remember the old story that the Chinese character for "crisis" is composed of the symbols for "threat" and "opportunity?" Unfortunately, that turns out to be a crock. But if it were true, it would apply here.
_____________
Thanks to The 3G Portal for including my post on the info pad in this week's Carnival of the Mobilists.
Or, What I Did on my Summer 'Vacation'
I want to tell you about a dream.
Last summer, after the cutbacks at PalmSource, I spent some of my "vacation" (ie, job search) thinking about products I wanted the tech industry to build. That's what you're supposed to do in Silicon Valley, right? You leave a company, come up with the Next Great Thing, get it funded, and create a wildly successful startup.
I got the idea part. Unfortunately, it involves hardware. Most VCs don't like to fund hardware companies. I mean, really don't like. As in, you talk about the idea, and their eyes get glassy and they become overpolite and they start to inch away as if you had a highly communicable disease.
It is just barely possible to get a hardware company funded, but the odds against success are enormously higher than doing yet another Web 2.0 company – so high that I wasn't willing to take that sort of risk when I need to be saving for my kids' college. Besides, I also wanted to try consulting, and that's working out nicely.
But that leaves me with this product idea that I really believe in, probably more strongly than any new product in the last decade. I know there's a big market for it, and I think the business model is solid. It's an opportunity not just to create a cool device business, but to establish an online service franchise that's at least as big as the franchise Apple has with iTunes. But you have to start with hardware, and that makes it very uncomfortable to the investment community here.
So I decided I'd just share the idea. Somebody steal it. Please. I'm interested in your comments and suggestions , but most of all I want someone to build one of these darned things so I can use it!
Let's start with the customer
A lot of product pitches in Silicon Valley start with the product, but I think it's best to start with the customer. If you understand their needs well enough, the product opportunity becomes almost obvious.
When I was at Palm, we did a lot of market research in the US, France, Germany, and the UK on what people would be willing to pay for (if anything) from a mobile information device like a handheld or smartphone. We found three groups of customer with very different needs. Each is about 11-12% of the population, and each has wildly different needs from the others. I've written about them before, but a quick refresher would he useful:
Entertainment lovers. This group is relatively young, generally college age or young professionals. They're very lifestyle and entertainment-aware, and they want their mobile device to serve that lifestyle. All kinds of entertainment appeal to them: music, video, games, etc. The iPod is their canonical device today, although the Danger Hiptop is also interesting, as is the Sony PSP.
Communication lovers. This group is professionals who are also extroverts, people who live to communicate with others. Think sales reps and marketing professionals. They're hungry to be in constant contact with their friends and business associates, and they'll use any communication technology they can get. A RIM Blackberry or a Treo are the best mobile devices for them today, because they combine phone calls with e-mail.
Information lovers. The third group is professionals who are more introverted, people who deal a lot with information rather than communications. The information could be documents, it could be databases, it could even be artwork. But the emphasis is on the individual's interaction with that information. Think doctors, artists, educators, and researchers. The ideal mobile device for these people is...well, it doesn't exist.
What they want is something that can go with them all the time, and that will function as an extended memory and as a way to capture their ideas. Specifically, they need to capture notes, sketches, and documents; work with databases; and look up information instantly. They need a brain extender, a true information appliance.
I call it an info pad. That's the product I want someone to build.
It's larger than a handheld and smaller than a tablet PC. About the size and thickness of a steno pad. It has a touch-sensitive screen on the front, and very few buttons.
I thought about trying to draw a picture of the info pad, but that would have been risky – both to my credibility and to your sense of aesthetics. Then I remembered a drawing that graphic artist Mike Rohde did of his ideal device, a digital sketchpad. It looks very much like what I'm picturing for the info pad:
Thanks to Mike for letting me reproduce this. I'll say a little more about sketching below.
Ideas for mini-tablet devices like this have been kicking around for years in various forms. But in our research we found that the customers are very sensitive to even small variations in price, weight, size, and functionality. If it's too big or too complex or too expensive, the market doesn't happen at all. This has caused a lot of tablet computers to fail over the years, and most people in Silicon Valley have written off the market as a result.
That's a mistake. Remember, a lot of people wrote off the mobile music market until Apple created the right solution. I think the same thing's going to happen with info pads.
What the product does
First and foremost, ink. You write on the screen and it captures your notes and drawings. It's as much like writing on a pad of paper as possible, because the thing you're replacing is the paper notepad or journal that students and knowledge workers carry with them all the time.
When I say "ink" I mean literally ink – put pixels exactly where the user touches the pen. Tablet PC converts pen strokes to quadratic b-splines, which is mathspeak for curved lines. That process subtly changes the letter forms, smoothing and altering them. It uses a lot of computing power (meaning it needs a faster processor and bigger batteries), and it seems to introduce a slight delay to the interface. You feel like you're using the stylus to push lines around on the screen rather than just writing and forgetting about the computer. I know some people like it, but I found it maddening.
One of the most important features of the info pad is something it doesn't do: handwriting recognition. Most of the note-taking devices that companies have tried to make over the years, from Newton to Tablet PC, make on-screen handwriting recognition a marquee feature. Your handwriting turns into printed text. That's a logical feature to pursue if you're an engineer; character recognition is an elegant way to bridge the gap between human and computer. It frees us from the tyranny of the keyboard.
The only problem is, it doesn't work.
Or maybe a better way to put it would be, it almost works. It's just good enough to get people to try it, creating the expectation that it'll be as foolproof as using a keyboard. But then a few words get garbled, you start going back and trying to correct things, and suddenly you're spending more time managing the device than doing your work.
This is deadly. It's also unnecessary. The purpose of our device is to let you capture your own ideas and information, so you can refer back to it later. In this context, character recognition is useless. You can read your own handwriting. Just capture ink, do a great job of making that effortless, and punt the rest. You may not get written up in Scientific American, but you'll sell a heck of a lot more product.
Okay, so now we're writing on the screen. We've replaced two incredibly useful and inexpensive tools, pen and paper, with something more expensive and less flexible. What's the benefit? A couple of things.
First, our device is an endless notebook. You can keep all your notes in it. Forever. For your entire life. This won't seem like a big deal to a 20 year old, but after you've been in business for a while, there comes a time when you remember a meeting you had a year ago when you heard something brilliant and relevant to the issue at hand. You know you wrote it down, but you also know it's in an old notebook that you filled up and stored in the garage somewhere. Forget about finding it – you might as well have never taken the notes in the first place.
If you have an info pad, that need never happen again. We'll compress and store all your notes, permanently.
What's more, we're going to sync the device to your calendar and address book. So it'll know when and where you took the notes, and if the meeting had an attendee list you'll know who was there as well. You can then use all this information to look up old notes.
This mimics the way people remember things, through associations. You'll remember that the meeting was at a particular conference, or that someone specific was in the room, or that it was the same month as your trip to Mexico. With notes that are cross-referenced with your calendar and contacts, you can browse just the ones that you took at that time, or with that person, or in that location. You may have to look through a few pages, but we should be able to narrow the search enough that it'll be pretty easy to find what you need.
I said earlier that we won't use handwriting recognition in the device, but I exaggerated. There is one useful task for handwriting recognition in an info pad: indexing. In the background, without pointing it out to the user, the info pad will attempt to recognize the user's notes, in order to build an index to them. The recognized text will never be shown to the user, so we don't have to worry about how many words are misspelled. Recognition that's only 80% or 90% effective is useless for writing a memo, but good enough to create a fantastic index.
The killer app in an info pad isn't the note-taking, it's the lookup and indexing functions. This produces one simple benefit for a user: If you write something down in an info pad, you'll never forget it again.
I don't know about you, but in my information-overloaded life, that would be golden.
The personal archive. The other primary task of an info pad is storing and displaying documents and databases. People in information-heavy jobs typically have documents, files, or reports that they may need to refer to during the day. We'll make it easy for the user to identify those documents, whether they're on the user's PC or on the Internet, and then we'll keep them synced so the user always has the latest version.
This archive of documents can be quite rich if the user wants it to be. Storage capacity on mobile devices has been growing explosively. We're kind of blase about that, maybe because storage capacity is even higher on PCs. But even a few gigs of storage can hold an amazing amount of information. For example, one gigabyte could hold the uncompressed text from about 2,800 novels. With compression, you could easily double that, if not a lot more. So we're talking the text of at least 5,000 novels, which is one a week for every week of your life if you live to be 96. That's more text than most people will read in their lives.
What would our information-hungry, memory-extending user do with all that storage? I'm not sure, but one thing I'd do is carry an archive of all my e-mails. Every e-mail I've ever sent. Incoming and outgoing, personal and business. Not the enclosures (they're too large), but the text. It would be great to be able to also capture snapshots of Web articles that I want to refer back to in the future. Make all of this indexed and searchable just like the notes. So this is another part of my life where I'll never forget anything.
Sketching. Mike Rohde's idea was to produce a digital version of a Moleskine, the small notebooks that artists and creative people carry to sketch and take notes. Moleskine owners are obsessive about them, as you can see here. Mike has some very ambitious ideas for the device, including exchangeable nibs for the pen, which would produce different sketching effects on the screen. I think that's an intensely cool idea, but I'm not proposing it in version one of the info pad because of cost and complexity.
I do think we should put basic sketching functionality in the info pad, though. It may not a Moleskine replacement in version one, but it should let users create simple sketches and drawings easily. That's a part of note-taking.
What would your lawyer say? I don't know if I have any lawyers in my reading audience, but if I do their hair is probably standing on end right about now. The idea of a perfect archive of all your notes and documents, with nothing forgotten, is terrifying to a corporate attorney. Companies rely on old information being destroyed, so it can't be subpoenaed in a lawsuit. For this reason, many companies have adopted document destruction policies.
IT managers might also be uncomfortable with an info pad. If an employee can walk away with a device holding every e-mail they sent or received in the company, that's a potential security hole. The reality is that employees can do that with a USB drive today, but I'd expect resistance anyway.
So, to assuage their fears, we'll give corporations the ability to tag company-confidential business documents with expiration dates. After they expire, they're purged from memory. I hate the idea of deliberately destroying a document, but I think we need to be sensitive to these things before they become an issue. Besides, the reality is that many companies will insist on the function and then never bother to use it.
The features
Before I get into the details, let's talk for a second about philosophy on features. I think a lot more mobile products fail because they have too many features than because they have too few. The more features you add, the more cost you add – and the more software you have to write. I'd rather have the engineers focused on making five features work wonderfully than making ten features work okay. So if something's not essential to the core functionality of the device, my inclination is to throw it off the boat.
Size. 9" high, 6" wide, 1" thick (23cm x 15cm x 2.5cm). If you can make it thinner, all the better. The info pad does not fit in your pocket; it goes in your bag or on your desk. Basically, it lives wherever your paper notebook lives today.
Weight. The weight of a thick paperback book – 16 ounces or less (450 grams). This would be far too much for a phone or handheld, but this is a different device. You won't be holding it up to your face.
Screen. High resolution grayscale, very high contrast ratio, touch sensitive. Color is optional – color screens generally have larger pixels and lower contrast ratios, making them harder to read. I think some people are going to disagree about color, but it's not essential to note-taking or document reading. (Think about it – how many of us carry colored pens or pencils so we can take notes in color?)
The ideal screen technology would be the e-ink displays being used in Sony's and Philips' new e-book products. I've seen e-ink technology in person, and it's stunning – the whites are very white, and the blacks are pretty darned black. It looks like a photocopied sheet of paper. It's very hard to see in photographs how much better the screens look; you have to see them in person. To me, as an old-time printing guy, they were breathtaking.
Unfortunately, e-ink screens have a huge drawback – latency. They work by physically driving tiny black particles to the front or back of a white liquid. This takes a lot more time than flipping on and off a liquid crystal pixel.
This is very visible in Sony's e-book reader – when you flip the page, it visibly turns all black, then all white, then draws the new page. It's like the flicker you get from a bad video edit, and just as annoying. This is acceptable in an e-book, where the pages don't change often. But it eliminates the possibility of doing anything interactive, like drawing or writing.
When I was investigating the info pad idea last summer, I talked to someone deeply involved in e-ink technology. The sad message was that it'll be at least two more product generations before they can flip pixels fast enough for effective note-taking – and that will happen only if some potential customer pushes them to do it. Right now the push is for other features -- the biggest demand for e-ink displays right now is for advertising signs that can be changed when needed, and high latency is acceptable there.
So for version one of the info pad, I think our first choice is a very high-resolution, high-contrast grayscale LCD screen. I've seen some beautiful ones in Japan, so I know they exist.
The other constraint on the screen technology will be cost. There's not a lot of demand for grayscale LCDs, so I'm very worried that we may not be able to get the right size at the right price. There are some very nice grayscale screens produced for medical imaging (viewing x-rays and such). But they're generally 19 or 20-inch diagonals, far too large for us. We need about a nine-inch diagonal grayscale LCD, and I couldn't find one in a quick web search. That implies we'd likely have to order a custom-made part, which creates huge inventory risks. We might be safer settling for a color LCD screen.
Battery life. It needs to run all day with heavy usage (assume eight hours of meetings or classes). That's one of the reasons I specified the thickness at one inch. I think the customer would accept a little more thickness to get a device that can run all day long.
Slots. One SD, one PCMCIA. The SD slot is for adding extra memory. This lets our base device be less expensive. The PCMCIA slot is for a cellular wireless card, if the user wants it. It would add a lot of cost to build a cellular radio into the info pad, and more to the point we'd then have to create separate devices for separate network standards, and sell through the carriers. Been there, done that, want none of it.
Built-in wireless. Mandatory: Bluetooth. Not so much to talk to other devices, but for syncing with the user's PC. Cradles are a pain in the butt for a manufacturer. They're inevitably expensive, and the connector is subject to all sorts of breakage and other problems. Instead we ship the device with Bluetooth built in, and a small USB Bluetooth dongle that the user can attach to his or her PC. Then we can buy a nice cheap standardized power supply that plugs into the info pad.
Optional: WiFi. If we can afford it, we should have WiFi built into the device, just so no one complains about it being missing. But keep in mind that this is a note-taking appliance, not a PC. WiFi isn't essential to the core operation of the device.
Camera. Built-in one megapixel camera. The lens is on the back or front edge of the info pad. Why build in a camera? Because it helps with note-taking – you can take pictures of notes on a whiteboard, and you can take pictures of pages in a book or magazine. No more time wasted jotting down things from a whiteboard, or copying quotes out of a book for a research paper.
Operating system. I think there are four candidates – Windows Mobile, Windows CE, Palm OS, and Linux. I would have killed to get a licensee working on a product like this while I was at PalmSource, and Palm OS would be my sentimental choice. But both Microsoft and Access/PalmSource are too focused on phones to take care of our needs. By using a Linux distribution, we can tap into the Linux developer community for supporters and engineers.
Built in applications. Note-taker, document viewer, calendar, contacts, to-do, calculator, search. That's it. It would have been nice to use Palm OS or Windows Mobile, because they have many of these apps ready to go. But they're not formatted for our size of screen, so better to do the apps ourselves. Palm managed to create the original Palm OS apps with pretty small teams, so I think we can handle it.
The other app we're going to build into the device is a third party software store. We'll open up the APIs, and give an on-device store where people can discover new apps and install them directly. We'll take a reasonable cut of the developer's revenue – 20%. They get the other 80%.
Having this software store built in gives us two benefits. First, it encourages a nice third party ecology of developers who can fill in the applications that I have deliberately left out of the device: browser, e-mail program, etc (remember our philosophy – focus our engineering team on making the most essential features work great). The other benefit of the store is that as the developer ecology grows, we get an ongoing revenue stream. We don't have to choose which applications win, but we do have an incentive to make our developers successful. We actually make more money when we leave opportunities for developers to add software. This should create a nice win-win dynamic that is absent on most mobile platforms today.
Price. In addition to the market research I mentioned above, PalmSource did some pretty extensive customer research on specific product concepts, including note-taking and information archive devices. The feedback was very clear – the maximum price for an info pad is about $350, and $299 is much better. By the time you subtract dealer margins, warranty and support costs, and your own profits, the hardware cost has to be in the $100-$150 range. I think that's doable, just barely. But we'll have to be very careful not to add in any unnecessary cost. That's why I want to avoid things like cradles.
The business model
So now we have our product designed, and we know what software we need to write. It looks like a moderately expensive startup – figure about $5-$10 million to fund the actual design and software, and another $3 million to build inventory before our launch. (Normally inventory costs would be a lot higher than that, but we're going to launch our sales direct on the Web rather than through computer stores, which allows us to launch with much less inventory on hand.) But we'll still need to do some advertising, and that eats another $7 million. So we should figure we're going to chew through at least $20 million to put the product on the market.
That's enough to make a venture capitalist think twice, but it's not out of the question at all. The market for an info pad is plenty big. I said above that the information-centric users are about 11% of the US and European populations. That's about 60 million potential buyers. But the market is actually larger – when we tested the note-taker and information archive concepts separately, we found that up to about 20% of the adult population was interested, centered around college students and middle-aged executives. That's over a hundred million people in the US and Europe – a very nice market indeed.
What will scare the VCs is our ability to hold onto this market. I'm sure we'll find some things we can patent, but mini-tablets have been played with for a long time, so I doubt we can patent anything that's so fundamental it'll prevent competition. So if we're successful we'll definitely attract competitors. Those competitors, even if they are inferior, will force down our prices somewhat. They'll also force us to add more features in order to stay competitive. Competing on price and features against large consumer electronics companies is deadly. At best, you hope one of the big players will buy you. At worst, your company dies a slow ugly death. You have the emotional satisfaction of launching a new product category, but none of the profits.
So you need an angle, some feature or approach that will create a lasting franchise rather than a grand gesture.
This is where ebooks come in
In my post last week on the ebook market, I said that ebooks are caught in a nasty chicken and egg situation. You can't get a really good base of ebooks until ebook readers are owned by about 20% of the core book-buying market. But those people won't buy ebook readers until there are a ton of ebooks available to read on them.
I think selling an info pad is the way to break this logjam. People will buy it to take notes and archive documents, but the hardware I've described above is also ideal for reading ebooks – it's light, it's small (but with a screen that's almost as large as a hardcover page), and we've gone for the highest-resolution screen we can find. We've even built in a software store that could also easily be used to sell books.
So we'll make the same offer to publishers and authors that we make to software developers – sell through us, and we'll send you 80% of the revenue. I don't know how the publishers will feel about this (it's a bigger cut than they get from distributors and bookstores, who keep about 50%). But even if the publishers won't go for it, think about what this does for authors. If you're getting an 8% royalty for a paperback, our royalty is ten times better. As soon as we can sell 1/10 what you would sell through traditional publishing, it will pay you more to focus on ebooks.
We'll also create a market for short stories and other literary forms. We'll accept anything you want to publish. I think electronic publishing might blur the lines between a short story and a novel. Today the length of literary forms is fixed by media -- short stories have to fit in magazines, and novels have to fit into about 300-800 glued pages. But once we're distributing electronically, a story can be as long or as short as it wants to be. The author sets the price and get 80% of the revenue.
One benefit I'm looking forward to is that authors might feel less of an urge to stretch a great short story idea into a novel (or even a series of novels).
In the long term, the content store becomes the core of our business. Once we get it to critical mass, it will be very hard for anyone else to copy, because they'd have to replicate our whole ecosystem – the devices, the store, and the base of content. What's more, because we've encouraged the creation of new content products, there are a lot of creative people who will be selling only through us.
We have duplicated the iTunes model, but this one is focused on words rather than music.
At this point, who cares if the consumer electronics companies clone our hardware? We can use our superior solution to compete with them. Or, the approach I like best is to license our software to them. Actually, we'll give them the software, on the condition that we run the store and keep all 20% of the revenue from it.
What would that revenue look like? Well, let's assume we that we captured 10% of book sales. Book sales are about $43 billion a year, so 10% is $4.3 billion. That's our revenue; our profit is $860 million. And if we have licensed the software, we make that profit with miniscule overhead.
Now, to be realistic, book prices will come down when ebooks are available, because we've cut out so much high-cost infrastructure. But our revenue wouldn't be capped at 10% of the book market either. For example, iTunes has about 80% of the downloadable music market in the US. At some point the numbers become arbitrary -- just pick the share figure you think you could get. No matter how you construct the model, there's potentially hundreds of millions of dollars in profit, with very low overhead. Eventually you can toss aside the hardware business like a butterfly shedding a chrysalis.
But to get this you have to be the first one to make the ebook market take off. Which means you have to start by building hardware.
Market entry
In reply to my previous post on the ebook market, John Mayer commented that the education market is a great starting point for ebooks. I think he's right, and it's an especially great starting market for the info pad. Students are logical targets from two perspectives: they take a lot of notes, and they have to carry a lot of books. If a small and inexpensive device could help them with both problems, I think it would be very well received. And although many students don't have a few hundred dollars to spend on yet another device (especially after they blew all their cash on an iPod), I think we could convince mom and dad that an info pad would be the ideal gift for a college freshman.
I think the publishers might not welcome etextbooks, since they could seriously disrupt the very lucrative market for printed textbooks. I'd expect resistance from college bookstores as well, which could be a major hurdle on campuses where the bookstore is a fundraiser for the student union or other activities. But we might be able to work directly with the professors who today create thick readers of photocopied articles for their classes. It would be much easier for everyone involved if those readers could be delivered electronically. We'd need to create or license some business structure to purchase reprint rights to articles, but that service could become a revenue stream for us.
One major asset for a collegiate info pad is Project Gutenberg, a nonprofit volunteer organization dedicated to converting every non-copyrighted book into electronic format. Pretty much everything written before the 1920s is now out of copyright, so over a period of years the project has amassed a huge library of classics from authors like Mark Twain, Shakespeare, and Charles Dickens, plus enormous numbers of more obscure works. There are about 18,000 titles in all. Unfortunately, there isn't enough demand for classic books to push a reader device to critical mass, but it'll help with college students, who are after all forced to read the darned things (and sometimes actually enjoy them).
Since we have an application programming interface for our device, we could also offer a simple software tool to let professors create quizzes and interactive content for their students. Something like Hypercard or its modern descendants.
Competitors
There are a lot of companies working on pieces of this marketplace, but no one has put together the whole solution. Here are the most prominent examples.
Microsoft. Tablet PC is the most obvious example of an effort to make a tablet computer for note-taking, and for me it's also the most disappointing. There's an old joke that if you own a hammer, every problem looks like a nail. The Microsoft corollary is that if you own a PC operating system, every device looks like a PC. What customers need is a note-taking appliance, small and sleek and focused. What Microsoft gave them was a PC without a keyboard, awkward and expensive and (in my opinion) bloated. I'm not going to do a full Tablet PC review here, or this post would go on forever. But price and weight alone disqualifies Tablet PC from serious consideration for the sort of role an info pad would play.
I had high hopes that Microsoft's new Origami tablet computer design (now called the Ultra-Mobile PC) might address the problems. But it's looking more and more like a warmed-over OQO, demonstrating that Microsoft has graduated from ripping off its licensees' most successful products to ripping off their marginal ones. With a price of around $800 and four hours battery life, the product's a non-starter as an info pad.
Now Microsoft is touting a future tablet that would be very small and light, but it's a couple of years away and would cost $800. Everything that Microsoft does in tablets seems to converge up close to $1,000, whereas what the market needs is something that converges down below $299. I would love to see Microsoft do a true info pad product. But it would have to be radically simplified and run Windows CE rather than full Windows, and I fear Microsoft isn't capable of working that low in the market.
Please, Redmond, prove me wrong.
Sony Reader. This tablet device is about the right size and weight, and it has an e-ink screen. The key to its success is getting lots and lots of books available for it at paperback prices. The sample screen shot for Sony's bookstore shows close to hardcover pricing for books, which gives me the willies.
I also worry when I see the Sony website brag that it'll have "thousands of titles." We had thousands of titles at SoftBook too, and that turned out to be a couple of hundred thousand too few. There's no note-taking -- the screen is not touch-sensitive, and besides the screen response will be too slow to enable that. Sony might be able to get enough books for the device if it bought a major publisher. That's not out of the question – Sony bought a movie studio to make sure it had content for its media devices. But I think Sony's new management isn't as adventurous as the old team.
I want the Sony Reader to succeed, but unless something creative happens on the book supply side I suspect that in a couple of years we'll be reading about the quiet discontinuation of the product line.
Iliad. The Iliad, from a Philips spinoff called Irex, looks very similar to the Sony Reader. Irex appears to be positioning it for use in vertical professional markets. That's a little disturbing – business verticals are the place where failed consumer tech products go to die. When I was at SoftBook, we started focusing on business verticals after the consumer business failed to take off. There are definitely some verticals for a small tablet, but in that space you'll be competing with Tablet PC. Tablet PC will be more expensive, but it's easier for a company to develop vertical apps for it (because it's Windows), and generally companies are less price-sensitive than consumers. Best of luck, Irex.
Anoto. This is an interesting attempt at note-taking. A special pen incorporates an optical sensor that can detect a faint pattern of dots on specially-printed paper. The pen captures your handwriting and transfers it to a PC, so you can look at your notes later. The drawbacks are that you have to buy special paper, and the notes are stored on your PC, so you can't refer to them in a meeting. Also, all the electronics crammed into the pen make it very fat. You feel like you're writing with the business end of a cow thermometer.
The Anoto technology is included in LeapFrog's Fly pen and Logitech's IO Pen. I couldn't find sales data for either one. Interesting side note: the last time I was at Logitech's headquarters, they had you sign in using an IO Pen. But wasn't connected to a computer system, and you signed in on a plain paper form. They were using it just as a pen.
Note-taking software. There are several companies creating note-taking software for tablet devices. Two prominent examples are EverNote and GoBinder. EverNote is a generalized note-taking and archive product, while GoBinder is optimized for students. My take on them is that they're both interesting products that have been hamstrung by the lack of appropriate hardware.
Who could build an info pad?
Microsoft's an obvious candidate, but I think they're too focused on miniaturizing a PC. Palm could do it, but they need to spend most of their energy on smartphones. There's an outside chance that Jeff Hawkins' secret project at Palm is an info pad-like device, but people who were with Palm in the early days tell me that they were so badly burned by note-taking on the Casio Zoomer (for which Palm provided software) that they'll never do it again.
Amazon could do it, as could Google. Amazon's running a secret hardware development project in Silicon Valley. For a while I thought it might be an ebook reader, but the Wall Street Journal reported recently that it's a music player. Google is working on all sorts of mobile technology, including a project called Android. No one outside Google seems to know what Android is, but there have been rumors that it's a mobile device OS. All I know is that project Android has hired some Palm OS engineers, but that doesn't mean they're necessarily working on the same sort of things they did at PalmSource. Yahoo could do it as well, but given their ties to Hollywood I'd expect them to be more focused on a video and music player.
I think LeapFrog would be an interesting candidate. They make learning tools for children, and also the Fly pen for middle school/junior high kids. I could see them branching into an info pad for college students. Unfortunately, LeapFrog is having some financial troubles, and I don't know if they can afford to launch a new product line. They say they intend to spend more on R&D, though, so maybe there's hope.
Apple is the other obvious guess. They have the money, the motivation, and the skills. But Steve Jobs supposedly has a phobia against anything that even vaguely resembles a Newton, and I wonder if that might keep them away from the market. I have a feeling he could overcome his fear if he wanted to.
It would be amusing if Apple ended up controlling both emusic distribution and ebook distribution, wouldn't it?
Conclusion
There you have it: I think the best way to build the ebook market is to sell a note-taking and document archive device that also just happens to read ebooks. I did enough research that I'm confident you could build the product, and the business, today. No magical breakthroughs needed, just careful execution.
If anyone out there has $20 million to spare, let's chat. I'll give you some more details on how to build an e-publishing empire. You just have to promise to make me a beta tester.
_____________
By the way, I wanted to say thanks to the folks at WAP Review for including my post on ebooks in this week's Carnival of the Mobilists.
My friend and longtime coworker Dave Garr just won a Webby People's Voice award for best personal website, for his online marriage proposal "Dave Loves Elizabeth."
As far as I can tell, Dave's site was the only self-produced nominee in this year's Webbys, which really says something for his skills and creativity.
Until recently, Dave was director of web marketing at PalmSource. He's now starting out as a consultant. If you want to inquire about his award-winning services, his e-mail address is in the weblog at the end of Dave Loves Elizabeth.
"The sums involved are so minuscule, they're not even insulting: they're *quaint* and *historical*, like the WHISKEY 5 CENTS sign over the bar at a pioneer village. Some writers do make it big, but they're *rounding errors* as compared to the total population of sf writers earning some of their living at the trade."
--Author Cory Doctorow on the financial realities of being a writer
Most people agree that the Internet and technology changes will make it possible to replace the pipe companies – the carriers, publishers, and networks that deliver the world's information, entertainment, and communication. But that change has been predicted for years. Will it really happen? If so, when? And what business and technology infrastructure needs to be created first?
In Part One, I gave an overview of the situation. Part Two discussed emusic. This time I'll cover ebooks.
Summary
This is a long post, so here's a quick summary of the highlights:
1. The traditional publishing business is incredibly inefficient. It absorbs most of the dollars paid for a book before they get to its creator. That reduces our favorite authors' incentive to write more books. Consolidation and other changes in publishing are limiting the availability of books and short stories. Ebooks could theoretically solve all of these problems.
2. People have been predicting the imminent takeoff of a mass market for ebooks since the late 1970s. It didn't happen then, and it's not happening now.
3. Unlike emusic, there are major structural barriers to the broad adoption of ebooks. The problem is not just technological. The most important barrier is a chicken and egg situation: People won't buy an ebook reader until there are millions of ebooks available for it, but there won't be lots of ebooks available for an ebook reader until it has millions of users.
4. Until we break that logjam, the mass market for ebooks will never develop.
My life as a publishing enthusiast
I should start with a little background. I'm passionate about publishing, and have been since I was a kid. In high school I hung out in the print shop, doing everything from hand-setting type to running a clanking old Linotype machine that cast lines of type from molten lead. I think the Linotype machine was one of the most impressive purely mechanical devices ever created. It had to deliver molds for every letter as quickly as the operator typed them, space them properly, send the finished line of text over to the caster, inject the lead, eject the finished line of text, and automatically sort all of the molds back into their proper positions for re-use. None of this was computerized – the operator typed on a mechanical keyboard and yanked a lever to start the caster, whose actions were then automated by a fantastic array of gears, pulleys, and levers. The molds for each letter had a unique set of notches on them, which allowed them to be sorted automatically by a series of gravity-fed chutes and cogs.
A Linotype machine in full operation was a symphony of noise – the clacking of the keyboard (the best operators typed fast), the clank and whir of the caster, and a fantastic tinkling sound as the little brass molds ran through the sorter.
You can see video of a Linotype machine in operation here. Man, I wish I had one of these things in my garage.
The most dramatic thing about running a Linotype machine was that if you sent over a line of text that didn't have enough letters or spaces in it, the machine might shoot hot lead straight out between the molds, where it would splash all over the place, including the operator's left arm. This was called a "squirt." I remember diving for the floor more than once when I realized I had accidentally sent out a line of text with the wrong spacing.
It was a pretty cool school district that let a teenager run a complicated and dangerous machine like this.
This boy isn't smiling, but I guarantee he's having fun with his Linotype machine.
In college I spent five years on the campus paper as it transitioned from typewriters to CRTs, and after graduation I started a desktop publishing company that made the first downloadable PostScript fonts for the Macintosh. In 1999 I spent time as the acting VP of marketing for SoftBook, a company that was making electronic books. And now here I am publishing my own writing online.
The most exciting thing about all of this technological change (other than the part about flying molten lead) has been the promise that publishing would become more democratized and more accessible to everyone. In some ways it has happened – desktop publishing paired with Kinko's and the photocopier made it much easier and cheaper for people to print on paper. And the Internet has spawned an unimaginable explosion of commercial and self-published media.
Yet in many important ways the revolution is incomplete. Books are still generally published through the same business processes used a hundred years ago. The distribution system is so inefficient that it absorbs almost all of the revenue paid for a book, and it's still a struggle for first-time authors to get paid for their work.
For decades it has been obvious that there's a better way to publish – deliver books electronically to a screen where the reader can enjoy them. Almost 30 years ago, Alan Kay and Adele Goldberg talked about using a computer to read digitally-stored books in their visionary paper, Personal Dynamic Media. Management guru Peter Drucker made a similar prediction at about the same time. Ever since, the computer industry has produced periodic waves of exuberance for ebooks. Donald Hawkins reviewed the forecasts of major industry analysis firms and found:
"IDC said that the U.S. e-book market would grow from $9 million in 2000 to $414 million in 2004, and Forrester Research predicted a revenue increase from $383 million in 2000 to $7.8 billion (!) in 2005. These are large numbers indeed, and the ... market research firms companies making them are well known in their field... To be fair, these numbers may apply to print-on-demand as well as e-books. (But) it is now apparent that the e-book shot missed its mark, and the e-book revolution has fizzled. Indeed, it never really got off the ground. The marketplace did not develop as originally predicted."
He wrote that in 2002, but the situation isn't much different today. Despite decades of effort, and numerous failed startups like SoftBook, we're still generally using an inefficient old publishing model that would feel familiar to Ben Franklin.
How could so many smart people all be so spectacularly wrong?
My belief is that the takeoff of ebooks is not just a matter of time. Unlike the e-music situation, where the march to electronic distribution is now inevitable, e-books are blocked by technology and market barriers. Until they're overcome, I think the revolution will be delayed indefinitely. Here's why...
The economics of book publishing today: "You don't write a book to make money"
That's the consensus from everyone I know who's involved in the publishing business. When you look at the economics of book publishing, it's easy to understand why they say that. For hardcover books, an author typically gets 10% to 15% of the retail price, depending on how many copies are sold. For paperbacks, the royalty rate is 4-8%. An agent will take about 15% of those royalties off the top. There are good summaries here and here.
That means if you write a hardcover book that sells for $30, you'll probably get about $3 to $4.50 per copy sold. If you write a paperback that sells for $7, you may get about 42 cents per book sold.
This ought to make it very attractive for an author to sell e-books direct rather than going through a print publisher. Let's assume the cut taken by an e-book reseller would be similar to Apple's iTunes cut – 30%. If everything else is equal, you'll make the same amount of cash selling 1,700 e-books as you will selling 10,000 print books. So the tipping point in book publishing comes when about 17% of the book-buying public purchases e-books rather than paper.
But that's just a first pass. The real situation's a lot more complex, for several reasons:
Advances. In most cases, an author gets a non-refundable advance when the book is accepted or published, and no more royalties are paid until the advance has been covered by sales. The advance can range from a few thousand dollars for a first-time author to much more for an established author whose book sales are more predictable.
This means that in many cases the royalty percentage is merely an academic exercise – what really matters is the size of the advance.
Publishers sometimes use the advance as an investment mechanism. They may deliberately pay an advance greater than the expected royalties on the first couple of books by an author they believe has great potential, in the hope that they'll eventually get a very profitable bestseller (there's a very good discussion of the process here). On the other hand, if a book is expected to do well in sales and fails, the disappointment over that might get the author blacklisted throughout the industry.
Price of the book. This one works against e-books. Most people buying a book, especially a hardcover, feel they're getting an object of tangible value. A hardcover book has heft to it, the pages have a nice texture, it has a pretty color, and it makes you look educated when you put it on a shelf (unless it's a book by Yann Martel, in which case you just look mildly befuddled).
An e-book is ephemeral. It's just a bundle of bits. You might erase it accidentally, and you can't display it on a shelf. Although it can be more convenient to carry an e-book than a paper book (you could hold thousands of books on a single USB drive), that isn't enough compensation for most people. Based on what I saw at SoftBook, no one other than an enthusiast will be willing to pay hardcover pricing for an e-book. Paperback pricing, maybe – but that's the upper limit.
This changes the tipping point for some authors. If you're successful enough to be published in hardcover, chances are you're getting the high end of the royalty range, and your book might sell for $30 a copy. You probably get about $4.50 per book, or about the same as your total revenue per book if you self-distribute an e-book at paperback prices. So for these authors, the tipping point is harder to map. I start to wonder about weird hybrid distribution models, in which successful authors publish first to hardcover paper books, and then the ebook takes the place of the paperback.
Investment by the publisher. This one's less of a factor than you might expect. The public image of publishers is that they carefully select and nurture authors, editing and marketing them like a jeweler carving a diamond and setting it in gold. That does happen sometimes, but the reality is that the big publishers deal in volume. They publish thousands of books a year (ie, tens of books every single day). How much time is available to coach and edit all of those authors? And how much marketing budget is available?
The answer to both questions is, "not much." Publishers don't actually market books, for the most part. They look for books that will market themselves.
The reality of modern book publishing is almost like digging for gold – you toss a lot of dirt and gravel into the sluice, hoping to find an occasional nugget. Since it's hard to predict where the gold will be, it doesn't pay to be picky about what you shovel. Just throw in as much dirt as you can and trust that the law of averages will eventually reward you.
Translating that metaphor back to publishing, the bookstores are the sluices, and the dirt is the flood of new books that publishers shovel through the stores. If a book starts selling well on its own, that's gold and the publisher makes money. If not, there's another ten books coming out tomorrow.
A publisher who believes in a new book may pay a little extra to give it a fancy cover, or place a small ad for it somewhere. But that's about the limit; otherwise, the market rules. The book and author have to sink or swim on their own.
There's a fantastic discussion of book publishing economics here (thanks to Lee Fyock for pointing it out to me). The most important takeaway from that discussion is that the publishers aren't getting wildly rich off this system. Most of the money is absorbed by various inefficiencies and expenses unique to printing: writing off the production cost for returned or unsold books, editing and typesetting, shipping, and so on.
So most of the value added by publishers is logistical – they manage the complicated production process, get the book placed in stores, handle returns, and so on. If the printing and bookselling process is bypassed, the value-add of most publishers goes away.
And so, perhaps, does the publisher.
Where's the tipping point? Netting out all the pluses and minuses, my guess is that the tipping point comes when about 20 percent of the book-buying public switches to e-books. Once we hit that point, ebooks will take over the majority of new book sales quickly. I want to emphasize the 20% figure is just a guess, though. There are so many variables that we won't know exactly when we've reached the point until we're already past it.
E-book publishing today: Slow growth
When will we hit magic 20% adoption rate for e-books? It could be a long wait.
The fragmentary statistics available on ebooks indicate that the market's still extremely small and is not growing explosively. A trade organization called the International Digital Publishing Forum (formerly the Open eBook Forum) compiles statistics from about 15-20 ebook publishers, and issues occasional press releases about the results.
I created the chart below based on IDPF's press releases. I had to extrapolate the totals for 2002 and 2003 because the forum released only first half statistics. The other drawback of these statistics is that the base of publishers reporting sales has changed over time, but the numbers were not revised to account for that.
The numbers show publishers' ebook unit sales and revenue growing about 30% each year. About 1.6 million units were sold in 2005, and revenue was about $12 million for the year. Unit sales were flat from 2004 to 2005, but revenue increased because the average selling price per ebook rose from about $5.70 to $7 (remember, that's publisher's revenue, not necessarily what the user paid for the book). Curiously, the number of new ebooks published has remained steady since 2002 at about 5,000 per year.
Units and dollars in thousands.
That sounds impressive in the abstract, but compare it to the printed book business. There were about 175,000 print books published in 2004, according to a trade group called the Book Industry Study Group. About 2.3 billion books were sold. Publisher revenue from book sales was about $43 billion, ($28.6 billion from major publishers, and $14.2 from small and specialty presses). You can add to that about $800m to $2.2 billion in used book sales, depending on which authority you believe.
Don't get me wrong, I know some people are having a great time reading ebooks, and that's wonderful. I was a big promoter of ebooks for Palm OS when I worked at Palm. But the question I'm asking right now is when they'll displace paper books and transform the publishing industry. From that perspective, the reality is that ebooks are about three one hundredths of one percent of the book market today. They're a hair on a wart on an elephant. At the current growth rates for both markets, we'll hit the tipping point for ebooks in spring of the year 2032. That's well past the date when Ray Kurzweil says we'll all transcend to cybernetic superhuman form. At current course and speed, we'll all be reading books by direct absorption before ebooks hit the big time.
[*Note that total ebook sales are undoubtedly higher than what the IDPF reports, because not every publisher submits its numbers to them, and because a lot of ebooks sold are old titles that are now out of copyright (and therefore wouldn't be counted in the publisher statistics). I wasn't able to find any firm figures on those sales. Nevertheless, the basic situation for ebooks is very clear.]
Other troubles in the book industry
Before I get into a discussion of why ebooks aren't growing quickly, I want to mention several other troubling trends in the book industry that electronic publishing might help to address.
Consolidation: Reduced diversity. Like the music industry, book retailing is being hit by consolidation as discount retailers like Wal-Mart, and chains like Barnes & Noble, skim off the most profitable books and sell them at discount prices. This drains the profits out of independent bookstores, which many publishing observers believe play an important role in discovering and promoting promising new authors.
Publishers have also been consolidating, reducing the diversity in the industry and therefore the opportunity for new authors to be discovered.
Pat Holt, former book reviewer for the San Francisco Chronicle, wrote passionately about the situation. I've excerpted part of the essay below, but it's worthwhile to read the whole thing here:
"From the 50 or so independent publishers in the mainstream industry about 30 years ago, when I started work in publishing, today about 7 conglomerates publish the bulk of mainstream books. This awesome consolidation of power is having a disastrous effect on the kinds of choices readers are offered... Not only are gimmicky, high-turnover, "no-risk" books replacing serious literature, an invisible censorship is taking hold of the industry as more and more decisions about what Americans can read fall into fewer and fewer hands....
"At the same time, the appearance of chain bookstores, beginning in the 1970s, has been devastating to independent booksellers.... I'm also concerned about the significance of independent booksellers in the larger scheme of things - how they discover new books that the chains often miss; how they support unknown writers; how they cultivate audiences of readers and writers, how they aren't comfortable with the kind of "paid placements" that bring Amazon.com, Barnes & Noble and Borders hundreds of thousands of dollars; how they continue to play a key role in the preservation of literature, even though they are routinely dismissed by mainstream publishers as powerless.
"Well, how powerless is this: Taken together, independent bookstores offer a wider range and variety of titles than do all chain stores put together."
The decline of the short story market. There has been a lot of angst in literary circles over the declining market for short stories. Short stories were once the training ground for many new authors, and some authors specialized in them. But since at least the 1980s, magazines have been reducing the space they devote to short stories, and short story collections don't sell as well as novels (according to a study conducted in the UK, short story collections sell about 1/3 to 1/4 the volume of novels).
Author Quinn Dalton summarized the situation nicely here:
"The conventional wisdom is that magazines are bowing to the realities of necessary advertising revenues, which put pressure on sections that don't perform based on reader feedback. As a result, the inclusion of fiction in commercial magazines has become an exception to standard procedure. The number of available outlets for short fiction has been steadily shrinking, from The Saturday Evening Post to Redbook to Cosmopolitan, which still publishes excerpts occasionally, to Mademoiselle, which ceased publication in 2001. More recently, Seventeen dropped regular stories in 2003, and Jane discontinued fiction during its redesign in 2004, though both run an annual fiction contest."
Editor and commentator Kelly Jane Torrance adds:
"Today short fiction is confined to an elegant ghetto: the New Yorker, the Atlantic Monthly, and Harper's. There still exist many literary magazines, but these publish seldom and have few subscribers. Moreover, outlets like the New Yorker publish mostly established names—John Updike, Alice Munro, Joyce Carol Oates, etc.—regardless of whether they are past their sell-by dates. Some joke the New Yorker would publish a grocery list if it had Updike's name on it."
I've watched the effect of this market change in science fiction (hey, this is a technology blog, of course I'm going to talk about science fiction). There were once a large number of relatively successful science fiction magazines that specialized only in short stories. A couple of them still exist, but they're not as numerous and vibrant as they once were. There are online efforts to take up the slack, but most of them pay authors nothing or a very close equivalent. One exception was Sci Fiction, an online forum of short stories funded by cable television's SciFi Channel. Unfortunately, the parent channel pulled the plug on it at the end of 2005.
In the period before Sci Fiction was shut down, Ellen Datlow, the site's editor, commented on the market for short story magazines:
"Until a workable business model is created for webzines independent of large corporations funding them as part of a larger entity (like mine), it's going to be a tough sell. But what I see in print magazines isn't any better. They start very small as semi-prozines, the best get bigger, more ambitious, come out more often and then the publisher/editor (usually the same person) burns out and/or runs out of money. Flourish? I don't know. Limp on over the years, yes definitely."
We're losing a lot of great fiction as a result. Science fiction authors like Arthur C. Clarke (2001: A Space Odyssey) and Robert Heinlein (Starship Troopers), are known for their novels, but in my opinion their best work was their short stories. And then there's Ray Bradbury (although he prefers to be classified as a fantasy author). I think we're missing out on much of the next generation of short story creators because they can't get paid for their work.
Out of print: The amazing disappearing author. Although books from the most popular authors often remain available long after their deaths, many others fall out of print rapidly. The figure I found quoted most often on the Web is that 90,000 books go out of print every year. Barnes & Noble reports that there are about 5.6 million books in its database, but only 1.5 million are readily available from publishers.
The result of this is that even well-known, award-winning authors are disappearing from view. I'll use science fiction as an example again. The late Clifford Simak is considered one of the grand masters of science fiction (he was the third author officially recognized as such by the Science Fiction and Fantasy Writers of America). He published a total of 27 novels and 141 short stories, and several of his stories won the industry's highest awards, and are considered classics of the genre. Yet only two of his novels are currently in print, plus one partial collection of his short stories. And those are available only because a small publisher has recently done a limited run of them.
To be fair, I should acknowledge that there isn't a huge market for some of Simak's books (his quality was pretty uneven). But a lot of excellent stuff is available only through used bookstores, if you can find it at all. Similar situations exist for other famous authors. Galen Strickland, who has written a lot of online commentaries about famous science fiction authors and their best-respected work, searched for copies of all the books he had referenced in articles on the website Templeton Gate. He found that of 996 books mentioned, almost half were out of print. Strickland pointed out that even living authors have problems with books going out of print – for example Arthur Clarke's The Fountains of Paradise, which won the field's two highest awards in 1979, is not currently in print.
Some people in the publishing industry complain that used book sales, especially those facilitated by Amazon.com, are undercutting the sales of new books. It's hard to quantify the real impact of used books (estimates of used book sales range from $800m to $2.2 billion a year), and some studies say the cannibalization of new book sales by used books is low. But even if used books do affect the demand for new books, until the industry does something about books going out of print I don't think it'll be in a position to say anything negative about the used book economy.
Generally, authors go out of print because there isn't enough demand for their work to justify a print run. It's uneconomical to do a print run for a small number of books, and besides most bookstores won't stock older books, especially if their authors are dead and no longer generating new fans (remember, the point of the system is to shovel fresh dirt into the gold sluice).
One way to address the print run problem is print-on-demand technologies, which produce short print runs of books that have been encoded into electronic files. The printing cost per book is about double that of traditional printing, but the shorter print runs mean less inventory risk. One of the leaders in this space is Lightning Source, an on-demand printing service that claims to have more than 100,000 titles in its library. Authors can self-publish via print on demand through services like iUniverse.
Although print on demand helps with availability, there's still the problem of getting books onto bookstore shelves. And the economics of publishing continue to eat up most book revenue – for example, an author selling through iUniverse gets 20% of the net proceeds for the book (after the costs of printing, shipping, and bookstore discounts are taken out). That's not much different from the payback from traditional publishers, which at least pay royalties based on the gross revenue from the book, not the net. There's a good discussion of the whole print on demand situation here.
Summary of the situation: This airplane won't fly
Add it all together, and here's how the publishing situation looks. Book publishers are not feeling as much financial pressure as music publishers (major publishers' revenue was up about 2.8% year over year in 2004, although book unit sales were down fractionally). But the diversity of books is suffering as channels and publishers consolidate. There's a serious problem with books going out of print, and the short story marketplace is moribund. And hanging over it all, the gross inefficiencies of print publishing prevent most of the money that you and I pay for books from ever reaching the people who write them. Our favorite authors don't have as much incentive, and freedom, to write as we'd like them to.
The situation seems ripe for ebooks. Not only would they allow a much higher percentage of book revenue to reach authors, but once ebooks are a standard no book need ever go out of print again. The only restraint on diversity will be our ability to sort through all the ebooks that are available. And I personally believe that a properly functioning ebook marketplace could revive the short story market, by making it possible for people to purchase short stories directly at low cost. (In fact, Amazon is experimenting with paid short story downloads).
However, most of these problems have existed for decades. People have been talking about ebook-like products for at least 30 years. And yet while sales of emusic are exploding, sales of ebooks poke along at a rate of growth far too low to change the industry. The ebook market simply isn't taking off.
Why the heck not?
Why ebooks haven't taken off
There are two main reasons, one technological and one structural.
The technical problem is that, for most people, reading long documents on a computer is unpleasant.
I know, you read weblogs and news online. So do I. But most of us do so in relatively short sessions, taking breaks and doing other things in between. Unlike those short reading sessions, the ergonomics of reading a book on a computer are uncomfortable for a lot of reasons.
The first problem is the screen. A piece of paper is reflective – it's only as bright as the room around it. A computer screen is transmissive – you read it by the light that shines through it. To be read clearly, a computer screen has to be substantially brighter than the ambient light around it (that's why most computer screens are hard to read in direct sunlight). This excessive brightness produces eyestrain when you're reading text for a long period of time. If you're like me, you can't read a computer screen nearly as long as you can a printed book.
Contrast ratio is also a problem for computer screens. If you look closely at the on-screen text in a word processing document, you'll see that it's not really black and white. The text is fairly dark gray, and the background is a bluish-slate color that you'd probably call light gray if you saw it on a piece of paper. Speaking as an old printing fanatic, it's been well established that people read most easily when they have the highest contrast ratio between white paper and black ink. Dark gray text on a light gray background is not at all ideal for reading. And the computer's readability gets even worse if there's any reflection off the screen, which is pretty common unless your lighting conditions are perfect.
Screen resolution is another important issue. Speaking from my printing background again, most people can read English (and other Romance languages) most efficiently when text is written in a serif font like Times Roman or Palatino, as opposed to a sans-serif font like Helvetica or Arial. The pattern of thick and thin lines in a serif font, and the little hooks at the edges of letters (called serifs) help our eyes quickly recognize letters and words.
To accurately reproduce the serifs and line thicknesses, you need about 300 dots per inch, or the resolution of an early laser printer (600 dpi is even better, and the printing standard is 1,200 or 2,400 dpi). Most computer screens are about 80 dots per inch. Text is readable on them, but not nearly as readable as it would be on a piece of paper. The result, once again, is more eyestrain and slower reading speeds.
Which one do you think is easier to read?
Despite all the drawbacks of computer screens, I think they're not the biggest barrier to ebook adoption. When I was at SoftBook, people sometimes complained about the screens, but other issues were a much bigger problem.
You can't use a computer the way you use a book. There are general ergonomic problems with reading on a computer. If you're using a desktop computer, you have to hunch over a desk in an office chair, rather than sitting back in an easy chair or couch the way most people do when they read a book.
You can read from a laptop, of course, which makes it easier to sit in a comfortable spot. But laptops have their own problems. The batteries don't last all that long, so your laptop may hibernate just as a book comes to a thrilling climax. Laptops are much heavier than books, so they are hard to hold comfortably. And they get disturbingly hot when used for hours (like the one that's trying to scorch my thighs right now).
Goldilocks and the two tablet devices. Then there are the two leading alternate mobile devices – tablet PCs and handhelds. Neither of them make great e-book readers for the average person. Tablet PC is basically a PC with the keyboard sawed off. It's still quite heavy compared to a book, so much so that you can't hold it by the base the way you would a book (your wrists will wear out rapidly if you try). Instead, you have to prop it against something, which puts you back into the wrong ergonomics for most readers. And it has the same battery life and screen problems as a portable computer.
I had hopes that Microsoft's new shrunken tablet design, code-named Origami, would give a better experience. But press reports say it's going to cost $800, weigh two pounds, and will have short battery life. To me that's just another tablet PC.
At the lighter end of the hardware scale, ebook reading software has been available for years for handhelds, and it's now moving to smartphones. There are people who read ebooks on handhelds (I'm one of them), but unfortunately it's not the right platform for most people. The screens are just too small for a comfortable reading experience. It's great if you're trapped on an airplane or bus and have nothing else to do, but there's no way it substitutes for a printed book for most people.
There's a nice discussion here on the problems of reading on a small screen.
So I find myself feeling like Goldilocks – the Tablet PC is too big, and the handheld is too small. I need something with the thickness and cost of a handheld, but the screen size of small tablet.
Chicken and egg. But even if that hardware were available, it wouldn't be enough to make ebooks succeed. After all, the Rocket ebook reader came pretty close to the right size in the year 2000, and it went out of business.
The Rocket ebook reader
The biggest problem facing ebook adoption is market structure. It's a classic chicken and egg dilemma, and it works like this:
--1. Most dedicated book consumers, the core customers for ebooks, won't buy an ebook reader device until there are millions of affordable books available for it, including a lot of current titles. I saw this one first-hand when I worked at SoftBook.
--2. Most publishers won't make their books available for an ebook reader until it has an installed base of millions of units, and they won't price them at paperback price points unless forced to.
This dilemma broke SoftBook and Rocket Book and every other attempted ebook reader launched in the last decade. And I think the new ebook readers being developed by Sony and Philips are at grave risk for the same reason.
Even a world full of PCs and handhelds and smartphones equipped with ebook software is not going to change the situation. If they were acceptable reading devices for most people, the revolution would have happened already.
Emusic vs. ebooks. Looking at all the barriers to ebooks, it's interesting to ask why emusic succeeded. A lot of people try to draw parallels between the two markets, but in fact some critical factors are very different.
1. Content was available for mobile music devices from the start. There were two sources of emusic: pirated MP3s, and CDs owned by the customer. Remember Apple's old "Rip, Mix, Burn" campaign? Even before the iTunes store was fully stocked with songs, you could use an iPod to do something interesting and useful with your music, just by ripping your CDs and transferring the music to the iPod. That helped to jump-start the iPod market.
2. Fear of piracy made the record companies deal. The record companies were so fearful of piracy that Apple was able to talk them into selling singles for 99 cents each – something they never would have agreed to if customers hadn't been stealing the songs for free. This allowed Apple to assemble a very large library of affordable songs.
Because of the special conditions in the music market, Apple managed to give birth to both a chicken and an egg at the same time. Unfortunately for fans of ebooks, the conditions Apple took advantage of don't apply in books. There are no book CDs that can be ripped and burned to an ebook reader, and there is no large-scale book piracy problem that would convince the publishers to make their books available at paperback prices. In fact, they have a strong incentive to keep the ebook market just as weak as it is today.
I'm not trying to paint the publishers as villains. Some of them, such as Baen Books, have been experimenting with ebooks as a promotional tool. And the O'Reilly publishing organization has been doing some very interesting work with hosting technical books online. They're also very active as promoters and communicators of new technology ideas. I think they realize that their core business is the marketing of ideas, not slabs of paper. But Baen and O'Reilly are both smaller, more flexible publishers willing to take risks. I think it's unrealistic to expect the major publishers to convert their catalogs en masse into ebooks, and sell them at paperback prices, when doing so will undercut their current channels and marginalize their companies. I think we'll have to work around the publishers, not through them.**
Our challenge. If we want to see ebooks really take off, we have to find some way to get a well-designed ebook reader to into the hands of about 20% of book buyers, before it has a critical mass of first-run ebooks available for it.
I have some ideas on how to do it. That'll be the subject of my next post, in about a week.
Next time: Desperately Seeking the InfoPad
________________
** This is why I think Google's Book Search program, formerly called Google Print, was one of the most strategically bone-headed moves made by a Silicon Valley company in the last five years. By asserting the right to scan books and post excerpts from them without prior authorization, Google managed to do the seemingly impossible – it got the authors to make common cause with the publishers. That's like getting the National Rifle Association to partner with People for the Ethical Treatment of Animals. Way to go, Google – in pursuit of the niche of searching scanned library books, you screwed up your ability to move the entire publishing industry to electronic format.
_________________
More reading
Although it's several years old, this is a splendid essay by Berkeley professor Clifford Lynch on ebooks and all their implications.
An interesting ebook commentary by author Cory Doctorow
Mobile Read (a discussion forum on general mobility topics, but with a special focus on ebooks).
Weblog of Bill McCoy, who works ebook issues for Adobe. He sounds more optimistic about the short-term prospects than I am.
_________________
Thanks to the latest Carnival of the Mobilists for linking to my post on technology predictions. This week's Mobilist edition had a number of interesting articles in it; it's worth checking out.
