Wak Oentoenk

It's two days later and I'm still confused.  When I saw the headline yesterday, my jaw literally dropped.  "Google bought who?  That's got to be a misprint.  They must have bought a mobile operator, like Sprint or something.  But Motorola?  Really?" 

Usually when a big tech merger happens you can see the logic behind it.  Even if you don't agree with the logic, you understand why they made the deal.  But in this case the more I think about it the more confused I get. 

Did Google buy Motorola for the patents?  If so, why isn't it spinning out the hardware business?  Or did Google buy Motorola because it wants to be in the hardware business?  If so, does it understand what a world of other problems that will create for Android and the rest of Google?  Seriously, if Google tries to integrate Motorola into its business we could end up citing this as the deal that permanently broke Google.

Why roll the dice like that?  Maybe I'm missing something, maybe Google has a screw loose, maybe both of the above.  Or maybe I'm wrong to look for airtight logic.  Companies sometimes make decisions on impulse, especially when they are under stress, and it's a sure thing that Google is under stress these days on IP issues.

So I have a lot more questions than answers.  My questions are about Google's intent, its next steps, and how other companies will react...


Why did Google do it, really?  The conventional answer is that Google wanted Motorola Mobility for its patents.  That's what Google itself implied, and Marguerite Reardon over at CNET agreed (link).  That might well be the explanation.  Om Malik had a really intriguing take: Google bought Motorola as a defensive move to prevent Microsoft from getting the Motorola patents (link).  And Richard Windsor of Nomura, who I respect deeply, said in an e-mail that this is all about the patents.  He predicts that Google's new patent portfolio will create a balance of power enabling Google to quickly force a settlement to the patent lawsuits against its licensees.

But if you wanted only the patents, I think you'd buy Motorola, keep the patents and then spin out the hardware company to avoid antagonizing your licensees.  Google says it intends to keep Motorola and run it.

Besides, as Andrew Sorkin pointed out in the New York Times, Google could have bought a different but also important mobile patent portfolio from InterDigital for about $10 billion less than Motorola (link).  Maybe there's some magic patent at Motorola that Google feels is worth $10 billion more, or maybe there are some terms in Motorola's patent cross-license agreements that Google desperately needs.  But again, if that's the case, why not keep the patents and resell the hardware business?

Unless Google is lying about keeping Motorola intact, I think Google intends to be in the mobile hardware business.  Which raises the next question...


Does Google know how to run a hardware business?  No, of course not.  The processes, disciplines, and skills are utterly different.  The same business practices that made Google good in software will be a liability in hardware.  Google's engineers-first, research driven product management philosophy is effective in the development of web software, because you can run experiments and revise your web app every day in response to user feedback.  But in hardware, you have to make feature decisions 18 months before you ship, and you have to live with those decisions for another 18 months while your product sells through.  You can't afford to wait for science.  Instead, you need dictatorial product managers who operate on artistry and intuition.  All of those concepts (dictatorship, artistry, intuition) are anathema to Google's culture.  Either Google's worldview will dominate and ruin Motorola, or worse yet the Motorola worldview will infect Google.  Google with Motorola inside it is like a python that swallowed a minivan.

To put it another way, I think Google has about as much chance of successfully managing a device business as Nokia had of running an OS business.

But the real question is, does Google realize that it doesn't know how to make hardware?  I doubt it.  Speaking as someone who worked at PalmSource for its whole independent history, an OS company always believes that it could do a better job of making hardware than its licensees.  It's incredibly frustrating to have a vision for what people should do with your software, and then see them screw it up over and over.  The temptation is to build some hardware yourself, just to show those idiots how to do it right.

I think maybe Google just gave in to that temptation.

But if Google really wants to sell hardware, that raises questions for the other Android licensees...


How will Google really manage Motorola?  Google says it's going to treat Motorola as an independent company without any special access to the Android team.  But what's the point in that?  Motorola hasn't exactly been dominating the mobile device world lately, so I find it very, very hard to believe that Google would buy it and leave it intact.  Wouldn't you want to have Motorola create special products that take advantage of the latest Android features?  Kind of like a flagship operation?  Then when you announce a new initiative at Google IO, you can have some nice new Motorola hardware ready to ship with it on day one.  Of course, the other Android licensees will be allowed to participate too.  They're welcome to run flat out to keep up with every Google software initiative, disregarding expense and business risk, just like Google's Motorola subsidiary will.

Which makes you wonder...


How will the Android licensees react?  I think we can safely disregard the positive quotes from the other Android licensees.  What would you do if your company depended utterly on Android, and Google called you up twelve hours before the announcement and asked for a quote?  Would you risk Google's anger by refusing to give a nice quote?  Of course not.

But would you honestly be happy?  Of course not.  In the last year, you gained share at the expense of Motorola.  Now instead of being a weak and failing vendor you can snack on, Motorola has infinite financial resources and cannot physically go broke.  Sure, I am happy to compete with that.

The other issue is the one everyone else has already pointed out -- even though Google says there will be a firewall between Motorola and Android, you suspect it'll be semi-permeable, meaning you'll always be at a bit of a disadvantage.

So what do you do?  A lot of people are predicting that Android could be in danger of losing licensees.  For example, Horace Dediu at Asymco drew a parallel to the Symbian consortium, whose members were uncomfortable because Nokia held the largest share of the ownership (link).  But when Symbian was launched, those companies were happy to sign up, despite the asymmetric ownership, because they thought Symbian was going to dominate the mobile OS market, and they were scared of Microsoft.  They dropped out only after it was proven conclusively that only Nokia was capable of making a Symbian phone that sold well in Europe. 

I can tell you from personal experience at Palm that licensees don't care about governance issues when they think your OS will help them sell a lot of units.  It's only after growth slows down that they get twitchy.  As long as Android continues to grow explosively, the licensees will be right there with it because they're terrified not to be.

Google probably knows the licensees can't go anywhere.  In fact, it has a history of treating them very roughly in private (check out the nasty tone in the private memos between Google and Samsung exposed by the Skyhook lawsuit here).  So in some ways the Motorola deal is just more of the same.

But there is still a risk to Google.  Android licensees will probably be more willing to talk to Microsoft now, and they might do a few more Windows Phone products, if only to get leverage against Google.  So Google has just thrown a lifeline to Windows Phone, which otherwise might have been headed for extinction if the first round of Nokia products failed.

This might also be an opportunity for other mobile platforms.  If there were any...


Is there a third path?  The Android licensees are probably pretty wary of both Google and Microsoft at this point, and may be wishing forlornly that there was a third alternative for mobile operating systems.

Unfortunately, I don't think there is.  The handset vendors' embrace of "royalty-free" Android strangled the other Linux mobile platforms.  TrollTech was bought by Nokia and then killed, while Access's evolution of Palm OS died for lack of customers.

There's speculation that HP might broadly license Web OS (link).  But HP has its own hardware conflict of interest (a much stronger one than either Google or Microsoft).  Far more importantly, keep in mind that mobile phone companies license an OS because they believe it's going to sell millions of units for them.  If HP, with all of its resources and channel presence and strong brand, can't sell significant numbers of Web OS phones, why would HTC or Samsung believe they could do it?

[Edit: In the original version of this post, I failed to mention MeeGo.  A couple of people have told me that was unfair, and I think they are right.  Based on past experience, I have a lot of skepticism about OS consortia, especially ones involving Intel.  But if MeeGo's ever going to get serious consideration from hardware companies, now is the time, and I should have acknowledged that.]

Hint to Android licensees: If you build up HTML 5 as a platform, you won't have to depend on anyone else's platform.  But in the meantime, your realistic choices are Android and Microsoft.

Speaking of Microsoft...


What will Microsoft do now?  Steve Ballmer faces a very interesting decision.  Windows Phone just got a boost because it's now seen as a more vendor-neutral platform than Android.  The door is probably open for Microsoft to build deeper relationships with Android licensees.  If Microsoft sill believes in its licensing model, it will focus on walking through that door.

But as others have pointed out, Microsoft's position is now a bit lonely in some ways.  The other major smartphone platforms (iOS and Android) now have captive hardware arms.  Even RIM has both hardware and OS, although it's been a while since RIM was held up as a model for others to emulate.  Will Microsoft feel exposed without its own hardware business?  And if it does feel exposed, will it buy Nokia?

I'd be very surprised if it did.  Buying Nokia would decisively end the Windows Mobile licensing business.  You'd be betting Microsoft's mobile future even more completely on the ability of Nokia to execute in hardware.  Besides, why buy the cow when you're already milking it?

I'd also like to think that Microsoft learned from the Zune debacle that it's not great at creating mobile hardware.

And then there's the fruit company...


What will Apple do?  Apple's history since Steve returned is that it doesn't react to competitors; it forces competitors to react to it.  Apple is brilliant at setting the terms of the competition so other companies are forced to compete on Apple's turf.  Everyone else is focused on building licensed commodity hardware, so Apple creates integrated systems.  Everyone else has optimized their supply chains to sell through third party retailers, so Apple creates its own stores.  Everyone else stopped making touchscreen smartphones, so what does Apple make?

You get the picture.  So I don't expect Apple to make any changes in response to the Motorola deal, but I would be shocked if Apple didn't have plans for changing the terms of the competition again now that Google is trying to build more integrated hardware and software.  There are all sorts of game-changing moves Apple could make -- do a much larger push in web services, create an iPhone Nano (fewer features and lower price), even create its own search engine or social network (potentially valuable just to make Google crazy).


What's next?

To sum it all up, it's impossible to predict what will happen.  Hopefully the new balance of power in patents will make the big lawsuits go away, although I doubt we'd see a resolution before the deal closes, and that could take many months.  If Google bought Motorola for the patents, it'll either sell the company or let it gracefully rot, and we'll go back to business as usual. 

On the other hand, if Google tries to integrate Motorola into its business, that's a noble mission, and I hope they'll succeed because the mobile industry needs more competition to Apple in systems design.  I dearly hope Google will take the challenge seriously and recognize that it'll need to make fundamental changes to its culture.  But those changes would be daunting even for a company experienced in mergers, and Google's never done a deal this big before.  I think the most likely outcome of the Google-Motorola merger is some flavor of train wreck.

I hope I'm wrong.

(Part 3 of "Who Will Pay for Mobile Data?")

There's a big nasty dilemma hidden at the heart of mobile computing:  No one knows how we'll pay for all that mobile data we're supposed to use in the next few years.  The question doesn't get much publicity, but it drives some of the most intense debates in mobile, including net neutrality and the wireless bandwidth "crisis."

This is the conclusion of a three-part series on the issue. In Part 1 (link), I talked about the tech industry's unlimited vision for the growth of mobile data, and why I think it won't come true because we'll run out of people willing to pay for data service

In Part 2 (link), I discussed the alternate scenario, in which everyone is willing to pay for mobile data and adoption of it continues to accelerate.  In this case, the mobile operators will need to invest urgently in increased capacity, and even with that investment we'll eventually run out of wireless bandwidth. 

The two scenarios leave mobile operators trapped between the need to expand their networks and the fear that they won't be able to pay for the expansion.  So the operators are trying to get other parties to help pay for the network.  I believe that's the real driver behind the net neutrality debate and the rhetoric about a wireless bandwidth "crisis."  Ultimately, government regulators will decide who will pay and how the mobile data network is structured, which will have a huge effect on which companies win and what we can do with the network.

In this part I'll give my take on what we should do about the situation, and I'll talk about the opportunities all of this change creates for operators, handset companies, and developers.


The look of mobile data in the future

If you only took away two messages from the first two posts in this series, these are the ones I'd want you to remember:

1. The only thing we can predict for sure about the future of mobile data is that it's unpredictable.  Maybe I'm right that it'll saturate soon; maybe Cisco's right that it'll go on growing explosively for years; maybe we'll average out to something in the middle.  The variables in play are so numerous, and so complicated, that absolutely no one can predict for sure what will happen.

In that sort of uncertain situation, I think our top priority should be to keep the mobile market as flexible as possible, so it can respond quickly and efficiently to whatever the customers decide to do.  That means we should ensure that market signals -- things like pricing and customer demand -- are as clear and unambiguous as possible, so we'll all know what the real level of demand is, and we can all respond to the same base of information.  The word "transparency" gets overused these days, but goodness gracious we need as much transparency as possible in mobile data.

2. We should plan wired and wireless data together.  We need to deal with the reality of the mobile network and market, not what we might want it to be.  And the reality is that we're not creating a separate wireless data network, we're creating a single integrated wired and wireless network.  A lot of the political rhetoric about mobile data talks about a completely cellular data future as some sort of public goal.  It's more like a public fantasy.  Every forecast I've seen from the wireless operators requires that they be able to offload a lot of traffic to the wired network.  Forget about wireless replacing wired; what we need to do is make sure they both work together well, with each focusing on what they do best.  That means wired is used whenever possible because in most cases it's cheaper and higher capacity, while wireless fills in the gaps.

We should set up a level playing field between wired and wireless so the market can sort out which traffic should go where.  Artificial political goals for the penetration of wireless, or favoring one network technology over another, are incredibly dangerous because they may lock in a market structure that turns out to be unaffordable.  In fact, because the market is so unpredictable, those sorts of goals are almost certain to be wrong.

So I get queasy when the US Federal Communications Commission, and even big companies like Google, argue that wireless data should have different regulations than wired data.  I think that increases the risk that we'll accidently bias the overall network in the wrong direction.


What we should do

As I've said before, I am not a big fan of government regulation in business, because it's usually inefficient and slow.  However, there are some situations in which you can't get the government out of the market, and I think cellular wireless is one of those cases because the public ultimately owns the airwaves in most countries.

So if we're going to have government regulation, let's do it right. 

The grand bargain.  The operators are asking for some mammoth benefits.  In the US, some of the biggest operators want to merge.  Okay, let's let them do it.  I don't think TMobile US is large enough to be viable in the long term anyway, so we need to merge it with either AT&T or Sprint.  If TMobile joins AT&T, which is the current proposal, the next merger in the US will be Verizon-Sprint; I think we have to accept that as well, for the same reason. 

The operators in the US and Europe want more spectrum allocated to them.  Again, I'd go ahead with it.  In the US, the television networks aren't using the extra spectrum, so it ought to go somewhere useful.

But in return, we should demand serious changes in the cellular data market.  I'm not talking about tweaks at the edges, I mean permanent changes in the rules of the game, designed to ensure lasting competition and a more flexible market that responds better to customer needs.

Here's what I propose:


Stop whining about the wireless "crisis" 

The first step is to change our rhetoric.  The bandwidth "crisis" is the tech industry's equivalent of the War on Terror: it's based on a genuine problem, it can never be completely solved, and it can be used to justify many actions that people might not otherwise consider.

The idea of a wireless crisis is an incredibly convenient tool for motivating government regulators.  Elected officials assume they are responsible for solving a wireless spectrum crisis, since they allocate wireless spectrum.  If it were called a "Verizon and AT&T don't want to pay for a bunch more cell towers crisis," I don't think President Obama would propose spending $50 billion on it.

This isn't just a US issue.  Anything one government does in mobile data is played back in other countries as a justification for equivalent actions there.  On a recent trip to Australia, I was surprised to hear a radio commentator complaining at length about the government's plan to supply broadband service to many Australians through landlines rather than wireless.  You can make a good argument for using landlines, since (as we discussed in part 2) they can carry a lot more data than wireless.  But the commentator was upset that Australia was failing to do "what Barack Obama is doing in the United States."

It's reasonable to ask what's so wrong with a little crisis hype and international competition.  After all, governments move far too slowly in most cases, so if a bit of alarming rhetoric makes them respond faster, isn't that a good thing?  The trouble is that we'll all have to live with the results after the "crisis" is "solved."  In that world, no matter how much spectrum we allocate to wireless data, service will continue to have slowdowns, outages and service gaps, especially in the United States, because it's more profitable for the operators to run their networks right at the edge of overload (in this sense they have the same financial incentives as airlines). 

We're lying when we tell people that the whole wireless data network could collapse.  Although service problems are a certainty, there is virtually zero risk of a full network collapse, unless the operators cause it themselves by underpricing data plans and selling more smartphones than they can support.  And we're misleading people when we say that prices will go up unless we allocate more spectrum.  Prices will eventually go up no matter how much spectrum we allocate to data, because demand for cellular data is growing faster than supply.

By overstating the risks and talking about the "crisis" as a temporary, fixable thing, we create an unrealistic public expectation for the quality and price of cellular data in the future.  That may well be advantageous for a couple of quarters or even a year, but in the long run it will erode public trust when we don't deliver the benefits we promised.  The wireless operators, especially AT&T in the US, already have big image problems.  Overpromising will make the problems worse.  To the extent that government agencies, and mobile tech companies like Apple and Google, participate in the crisis rhetoric, they risk their credibility as well.

We need to ask ourselves as an industry if we want to have the same sort of public image in five years as the airlines have today.  If not, we should be honest with people now.  For example, I think there is a convincing, legitimate case for reallocating old TV spectrum for data services.  Without it, mobile data prices will go up faster, and a lot of the features many of us want from mobile data may not be affordable.  But we should also be honest with people that cellular bandwidth overload is a chronic disease rather than a crisis, the network is not going to collapse unless we're incompetent, cellular service will not be as fast or cheap per bit as a wired, and cellular data will generally be a supplement to our wired broadband, not a replacement.


Make the cellular data market transparent

The problem with the cellular data market as it's structured today is that it often hides from users the real cost of the network they use, so they can't make well informed choices, and it's hard for us to tell which buying patterns are genuine and which ones have been created artificially.  For example, the cost of your smart phone is subsidized, so you don't realize what an expensive piece of hardware you're carrying in your pocket.  You're told that you have unlimited data, but actually if you use it too much your operator will probably reduce your data speed without telling you. 

By making cellular data seem cheaper than it is, we encourage people to use the network more, increasing the very overload that we're supposed to be fixing.  Some of the proposals for the future of mobile data would further increase the overuse of cellular data by making it seem even cheaper to users.

The structure of the mobile market also limits competition among mobile operators (especially in the US), and reduces competition between mobile phone manufacturers.

I think this systematic distortion of the market must stop.  If people could see the real cost of cellular data, they would make better-informed decisions about when and how to use it, and we wouldn't need secret back-end controls on traffic.  Meanwhile, more competition in services and phones would mean faster innovation, more consumer choice, and more efficient prices.

Here are some specific steps I think we should take:


1.  Ban covert traffic limits.  Today some wireless operators (and some wired ones as well) are quietly reducing the quality of service they deliver to some users, without telling them.  This is done through various techniques including "traffic shaping" (prioritizing or delaying certain types of data packets) and "throttling" (reducing the throughput of the network, or the speed of certain transactions).  In effect it usually means reducing the connection speed of people or apps that use the network the most.  For example, Dean Bubley recently wrote about an ISP who consistently reduced data throughput at particular times of the day (link).

There are some types of traffic management that make sense.  E-mail spam can be reduced through throttling that limits the number of e-mails that can be sent by a single account per second.  Throttling can also be used to limit malware attacks, by reducing the ability of a rogue app to flood the network with traffic.  And I think it's fine to enforce the speed you paid for in your Internet connection.  For instance, if you've paid for a 10 MBPS connection and the operator limits your throughput to 10 MBPS, I do not have a problem with that.

But in some cases the operators are limiting network performance to covertly restrict users, either by interfering with certain types of traffic, or by limiting the speeds of some users without telling them.  For example, the current Verizon Wireless terms of service give them the right to reduce the throughput in your "unlimited" data plan if you're in the top 5% of data users (link).  They can do this without notifying you.

This sort of hidden restriction is damaging to the market because people may sign up for a wireless plan believing they will get more service than they actually will.  They can't make a fully informed decision between wired and wireless service because they don't know how much wireless data they're really going to get.  This may misallocate resources and make the wireless network even more overloaded than it would be otherwise.

The answer to this is simple: Require operators to notify a customer when they have throttled or shaped his or her service (other than enforcing the promised speed of the connection).  I am not against throttling in general, but it should not be done without notification.  A text message would be fine.  The Internet speedometer, which I discuss below, will also help with this problem.


2. Require a data gas gauge and speedometer in smartphones.  Can you imagine buying a car that didn't have a gas gauge and speedometer?  That's essentially what we do today with smartphones.  For most smartphone users today, there is no easy way to tell how much data throughput you're getting from the network, and how close you are to any limits on your data usage.  Some operators bundle apps to do this, some have more arcane ways to check, and some send you a text if you get close to the limit.  But I think it's fair to say that most people are in the dark about their usage until they get their monthly bill, and if they do go over a limit they will have trouble figuring out why.

This is an easy problem to fix.  We should require that every smartphone have an app, accessible at the same level as the Settings app, that tells the user how close he or she is to hitting any data caps in the service plan (for example, if you are a Verizon user, how close are you to getting throttled?).  The app should also show how much data you're using at any particular time, so you can see how much throughput the network is really giving you. 

We also should modify the signal strength bars to change color depending on how much data you're consuming at any moment.  This would show you when you're using a website or app that uses huge chunks of data.  When customers see that video or Flash makes their signal bars turn red, they'll be much more cautious about using those sites on the wireless network.


3.  Decouple the phone purchase from the network.  Currently in the US and much of Europe, if you sign a contract for a data plan, you get a discount of several hundred dollars on a new phone purchased at the same time. But you have to buy the phone through the mobile operator, giving them huge control over the selection and features of the phones they sell.  Basically, users are not free to pick the phones they want; they have to take the phones their operator chooses to sell.

This operator lock-in is subject to all sorts of backroom manipulation.  Weak phone vendors are forced to comply with a huge list of tests and requirements, while for stronger vendors the rules are often waived.  I've also been told privately by some operators that they deliberately discriminate against some handset vendors because they just don't like them.

The handset vendors aren't completely clean either.  A vendor with a hot handset may restrict its availability to a single operator in order to extract concessions from them.  Can you say iPhone?

It's a wonder that some operator or handset company hasn't been sued already for restraint of trade.  With the amount of operator shelf space shrinking in the US due to mergers, I think it's only a matter of time before there's a legal detonation. 

In addition to the legal risk, these restrictions have the effect of restricting customer choice and competition, so they are bad for transparency.  It's time to open up the handset market.  To make that happen, subsidies should be separated from the purchase of a particular phone.  When someone signs up for a plan, they should get a voucher for a discount on any phone.  The voucher can be used at that time to buy a phone in the operator's store, or it can be used later to buy a phone in any other store. 

This would encourage more selection and competition in mobile phones.  It would create more direct competition between operator service plans.  And it would put the wireless and wired networks on an even footing (can you imagine a wired data provider limiting the brands of PC that you can use with your cable data connection?).

In the US, I think we should consider one other step to open up the handset market.  In most of Europe, and many other parts of the world, there is a vigorous retail market in mobile phones sold separately from an operator.  Because everyone is on the same network standard, and because all the phones use SIM cards, it is easy to buy a new phone at retail and pop your card into it.  You do lose the subsidy, but virtually all customers know they can at least switch phones if they really want to.  This leads to a much larger selection of phones, and to higher competition between operators because it's easier to choose separately the phone and service plan you want.

The US market is much less open.  Most mobile phones are sold only through operator stores, and it can be very hard to switch from one operator to another because they have different network technologies, and some of them don't even use SIM cards.  Because it's so hard to switch phones, I think most US mobile users are barely even aware of what a SIM card is, and how to find it in their phone (most of them would probably confuse it with the SD card).

To open up the handset market, the US should require that all mobile phones use SIM cards, and that they be switchable between the major operator networks.  That way someone could go into a consumer electronics store, buy the phone they want, and use it with any network.  This will have to be phased in over time, but we're already moving toward it anyway.  Verizon and AT&T are both moving to LTE, and there are very strong rumors that Sprint will do so as well.  So some day we'll have one standard cellular technology base in the US.  In the meantime, we'll have to buy dual-mode phones that use both LTE and either GSM or CDMA, depending on which operator you use.  But the chipsets for smartphones are increasingly capable of handling several different networks, so they can switch between LTE, GSM and CDMA.  I think it would be reasonable to require that future smartphones sold in the US be SIM-based and capable of operating on all three standards.  I think the real question is how quickly we could phase in that requirement; if you have thoughts on that please post a comment.


4. Enable toll-free apps and websites.  As I discussed in Part 1, we need the data equivalent of a toll-free phone call, in which a website or mobile app company would pay for the data traffic generated by a particular app or site.  This requires changes to the operators' billing infrastructure, but I think it will be essential for enabling the growth of mobile data.  It should be an extremely high priority for the operators, and it's in the interest of web and app companies to get together with the operators to define standards for these charges, so they'll be easy for developers to work with.  I suspect there's an important role government regulators can play in helping to encourage these negotiations.


5. Do not allow the operators, or the web companies, to discriminate against one-another.  I agonized over this one a lot.  The operators would like to be able to charge web companies extra if they want reliable delivery of data (for example, in a time-sensitive app like video streaming), or if they want a guarantee of a certain level of throughput.  I understand why they want to do this, because it would help pay for their infrastructure, and I do not think it is inherently evil.  But I think it would cause too much collateral damage to the mobile market.  In fact, I think it would put us on a road toward wrecking mobile data.

The first problem is that hidden back-end charges like this are essentially an invisible subsidy for cellular data.  A user won't know the real cost of the data he or she is using, and this could end up increasing traffic on the cellular network artificially, contributing to data overload.

There are also big practical problems with implementing charges for quality of service.  As Dean Bubley has pointed out repeatedly (link), there are huge drawbacks to this sort of approach.  To give one example, there is no way to guarantee quality of service when you don't know how overloaded a particular cell site will be.  If one high-priority video session comes in, does the operator shut down five other "regular" data sessions to make way for the high-priority one?  In that case, the "regular" customers are not getting the service they paid for, and they won't even know it.  They'll just think something is wrong with the web app they're using.

I agree with Dean that there's no way to make a system like this work predictably and fairly.  Better to just charge users for the data they consume, let them know how much that costs, and allow them to adjust their own usage patterns.

The other reason we should ban quality of service fees is because in some cases they could produce in a destructive power struggle between operators and websites, with users caught in the middle.  US cable television is a nightmare example of what not to do. 

In cable television, it's common for network operators and content companies (the cable channels) to pay each other for services.  For example, Home Shopping Network reportedly pays cable TV companies to be included in your service package, because they know they'll make more money if they're seen in more homes.  They are, effectively, subsidizing your cable television service. 

On the other hand, many of the most popular channels charge the cable companies a fee for the privilege of carrying them.  For example, ESPN (the leading US sports network) reportedly charges cable companies about $4 per month per household; other popular channels are in the 5-20 cent per month range. 

The same sorts of things could happen in the mobile web if the operators could charge websites for service.  For instance, what if Facebook started offering video streaming as part of its services?  If the mobile operators tried to charge Facebook for its network usage, what is to stop Facebook from turning around and demanding a fee from the operators for allowing them to carry Facebook? 

Unless we're very careful, we could end up with a situation in mobile similar to the one in cable TV, where users get caught in disputes between the network operators and the content creators.  Some of those arguments in the US have been incredibly ugly, with users tied into long-term contracts for cable service but unable to access the channels they thought they paid for.  And remember, in cable we get these messes even though we have only have about a hundred channels to negotiate.  On the web, you have literally millions of them.

The operators should not kid themselves that they would win in this sort of showdown.  If Facebook cut off its traffic to Sprint's servers, what would happen?  Would users abandon Facebook because it's not on the Sprint network -- or would they switch off of Sprint because it doesn't have Facebook?  I think we all know the answer to that: there would be crowds holding pitchforks and torches outside the Sprint stores.  The websites have far stronger brands and far more user loyalty than the operators.  So it's unlikely that the operators will really be able to coerce money out of the most successful websites.

In practice, I think the operators would be able to get fees only from small startups that don't have brand awareness with users.  That becomes a barrier to entry for those companies, which historically have been the source of most online innovation.  To give a real-world example of what that could do to the web, look again at cable television programming: A small number of networks dominate the selection of channels, resulting in slow innovation and reduced choice.  There is very low turnover in these channels. 

If the web worked like cable TV does, we'd all still be using AOL for e-mail.

I've talked with people at small startup cable channels, and they are incredibly bitter about the barriers they face getting placement on cable systems.  They're actually counting on the web to let them bypass the cable operators.

I think the only way to make the mobile market work efficiently is to make the payment mechanisms as clear and visible as possible.  Make users pay for the data they use, and allow web and app companies to make their sites and apps toll-free if they want to, but don't start creating hidden layers of fees and subsidies.  That will just distort the market and expose operators to retaliation.  My operator friends, this is a war you cannot win -- so don't start the battle.

To formalize this settlement, government regulators should ban both operators discriminating against websites or types of traffic, and websites withholding their content from a particular operator or network.


6.  Encourage open WiFi.  As I mentioned above, we're not creating a standalone cellular network, we're creating an integrated wired and wireless network.  WiFi has a critical role to play in that network, and we should make it even more central.  Here's a question for you:  How often have you tried to find an available WiFi network, and seen no networks at all in range?  I can't speak for other countries, but it almost never happens to me in any populated part of the US.  But how many times have you tried to sign onto WiFi and found only locked access points?  That happens to me all the time. 

We already have a very dense, well-populated wireless front end to the data network in most places that matter, but we can't use it fully because most of the access points are locked down.

There are good reasons for the lockdown.  If you leave your WiFi router open, it can be hacked (actually, it can also be hacked if you keep it locked, but that's a topic for a different post).  Also, in the US if someone downloads child pornography or does something else illegal on the Internet, the law often goes after the router owner because that's the only person they can find.  You can read some horror stories here.

But getting those connections opened up would have huge benefits for the public, because it would take some of the pressure off cellular wireless.  Rather than telling people to close off their connections, we should be encouraging them to leave them open.  Regulators could help this in a couple of ways:

--First, we should require that the next generation of WiFi routers have a pass-through feature enabling public access to the Internet without giving access to the user's home network.  Traffic from the user's private connection should have priority over the public one, and if public usage is excessive the user should be able to throttle it.

--Second, the law should be changed to protect people whose open wireless connections are abused without their permission.


Opportunities

So that's how I think the future of mobile data will look: unpredictable growth, always skating the line between overloaded and overpriced, and with a huge variety of users, almost all of them with some sort of limits on their data service, and many with budget plans that encourage very careful use of data.  For the health of everyone involved in the market, I hope we'll also get regulations that make the market more transparent, and more open to new players.

It's a different mobile data world than many analysts have been predicting, but that's not necessarily a bad thing.  Often the best business opportunities happen when conditions change unpredictably.  I think this is one of those times.  So I'd like to conclude by recapping the big opportunities as I see them...

For handset vendors, I think the most interesting new opportunity will be the smartphone designed for people with limited data budgets.  How do you entice people into gradually using more data?  This is an opportunity to do a fundamental rethinking of the smartphone user experience.  Since different people will probably respond to different data features, I think it will also be an opportunity for smartphone vendors to stake out their own market segments, helping to insulate them from the intense commodity price pressure we're likely to see in generic smartphones as the market fills up.
   
Try to think like an automobile vendor in 1950.  Do you want to compete with everyone else in midsize sedans, or would you like to dominate a smaller segment like station wagons or sports cars?

To target a segment, you'll need to hire people who know how to design integrated hardware-software systems rather than just devices, and you'll need to learn to partner closely with app and web companies as peers (rather than the serf-overlord relationships you're used to having).

In the last couple of days I've been contacted privately by some people who predict even more revolutionary moves by the handset companies, most notably the idea of selling a phone at retail bundled with airtime that you've bought from an operator.  In other words, the phone comes with its own network service.  That's what Amazon did with Kindle, and there's nothing in principle to prevent a handset company from doing the same thing. 

I think there would be a lot of implementation challenges, most notably keeping access to that third party network if it starts to run out of capacity.  But it would be intriguing to see what someone like Apple would do with this.

For operators, I think it's important to pick your battles.  Although covert traffic-shaping and charging websites for service is very seductive, in the long term that will lead you into intense conflicts that you're not likely to win.  It would also create more incentives for the handset companies to set up their own virtual networks, which really would transform your networks into dumb pipes.

I think it's better to focus on new business models that are a win for both you and your business partners.  The most appealing of these to me is toll-free data.  That would be intriguing to a lot of web and mobile app companies, allowing you to build cooperative alliances with them.  And it's a whole new revenue stream that might become very large over time.

For web and app developers, the emerging segmentation of mobile data makes the idea of "enticement" even more important than it is today.  How do you give people some software for free and then entice them into paying for add-ons or other apps?  Already most of the mobile app developers I talk to are thinking along those lines, and obviously that business model is very well established on the web.  But as smartphones reach down to more price-sensitive people who are less enthusiastic about data, there will be intense demand for apps and websites that can entice them into starting to pay for bits of mobile data. 

These "data on-ramp" apps are not always intuitively obvious, and will probably differ by country (for example, mobile horoscopes were a major driver of beginning data use in parts of Asia).  The companies that can find the on-ramps will be incredibly valuable to investors, handset companies, and operators.


What do you think?

That's my take on the situation. What do you agree and disagree with?  What else would you add to the picture?  How does it differ in your country?  And most importantly, what do you think the opportunities are?  Please post a comment and share your ideas.

(Part 1: The End is Nearer Than You Think)

The most important question in mobile computing is who's going to pay for all that mobile data we're supposed to use in the next few years.  The question doesn't get much discussion online, but it's at the heart of the most intense debates in mobile, including net neutrality and the wireless bandwidth "crisis."

How many users will pay for their own mobile data service?  Will web companies also pay?  Will the government step in?  And most important, how much money are any of them willing to pay?  The answers will shape the future of every company involved in mobile, and will have a profound effect on everyone who uses a mobile phone.

Here's a quick summary of my answers:
     --Because of economics and user psychology, I think we're headed for a slowdown in the growth of mobile data.  The unlimited, exponential growth forecasts are wrong.  I believe the ultimate mobile data market will be smaller, and much more segmented, than most people expect.
     --Even if I'm wrong about user demand, we're still headed for a slowdown in growth because the cellular networks can't grow fast enough to handle all the traffic being forecasted.  This is due to physics and can't be changed; you could just as easily change the phases of the moon. 
     --Many of the proposals to "fix" the problem would probably make it far, far worse.  We could end up with a cellular data network that resembles American cable TV: slow to innovate, dominated by a few players, and subject to intense politics, with users caught in the middle.
     --In the future, cellular data for the majority of users will likely be metered, and the majority of people will need to be enticed into using it.  That creates some excellent unaddressed opportunities for everyone from handset companies to app developers. 

This is a very complicated issue, so I'm going to cover it in three posts this week:

Today's post talks about the forecasted growth of wireless data, and why I think growth won't continue the way most people are expecting.  That creates some big challenges for mobile data companies, but also some fantastic opportunities.

Tomorrow I'll talk about the alternate scenario, in which mobile data growth continues at the same rate, eventually colliding with natural limits on the amount of data that can travel over a cellular network.  I'll discuss how that collision drives the rhetoric about a bandwidth "crisis" and the debate about net neutrality.

In the third part, I'll discuss what it all means for the industry, and give my take on what we should do about it.

To start today's post, let's look at the predicted growth of mobile data...


The forecasts for mobile data growth are so sunny they could burn your skin

Every mobile phone will be a smartphone.  Smartphones are already used by about a third of the US mobile population (link, link), and ownership rates are similar in parts of Europe (link).  Horace Dediu says half of US phone users will be on smartphones by the end of 2011 (link), and non-smarthones will be virtually extinct a year later (link). 

Mobile data traffic will explode.  Cisco says global mobile data traffic will increase 26X from 2010 to 2015 (link).  The growth will be driven by increased use of smartphones, and also a rise in the number of notebook PCs connected to the cellular networks.  Notebook computers generate an order of magnitude more data traffic than smartphones, so even a small number of cellular notebooks drives a huge increase in traffic.

Mobile app shipments are off the chart.  Apple says iOS users download 62 apps per device on average, for a total download rate of 206 apps every second (link). 

The big tech companies are focused on mobile.  Many of the hottest tech companies, most recently Facebook, say their biggest focus for this year and beyond is nailing the mobile opportunity (link). 

PCs will be replaced by smartphones.  A Google vice president says smartphones will render PCs irrelevant by 2013 (link).

The growing consensus is that our current cabled, PC-centric computing world will soon be replaced by an untethered world in which everyone uses smartphones and tablets to do their computing on the go.  The mobile network we all envision will be just as flexible and carefree to use as today's wired Internet, but with the added benefit that you can use it anywhere, anytime, with a wide variety of different devices. 

It sounds cool, but unfortunately no one has ever asked users if we're all willing to pay for that mobile data service.  I think most of us aren't.


We won't pay for all the mobile data we want

We'll all carry smartphones, but...  The forecast for mobile data growth is based in part on the assumption that in the near future most or all mobile phones will be smartphones.  You can make a good case for that assumption.  The price of smartphone components is continually decreasing, so at some point the parts cost a smartphone will be the same as a feature phone is today.  Even if prices aren't completely equal, once they get close, it's more cost efficient for a mobile phone company to base its phones on a smartphone OS because it requires less rewriting of apps and support software for each new phone.  The handset companies have an incentive to switch to smartphone hardware.

So I have no doubt that in the next couple of years most phones sold in the developed world will technically be smartphones.  However, I think it's not reasonable to assume that they'll all be used as smartphones, because many users won't be willing to pay the data charges.

As I've written before (link), when I was at Palm we did a lot of research on mobile phone users in the US and Europe, and we found that about a third of them were willing to pay extra for new mobile data features in addition to voice and texting.  Some of them were more interested in entertainment, some in business communication, and some in information management.  These are the people who have been buying iPhones and BlackBerries. 

The other two thirds of phone users were not willing to pay extra for any new sort of mobile data.  Some of them didn't have enough income, some of them just weren't interested, but they all flat-out refused to consider spending extra.  The Palm surveys were conducted several years ago, but since then I have seen no evidence to suggest the basic situation has changed.  On the contrary, the most recent research I've seen was done by Forrester in 2009, and it suggested the unwilling-to-pay share of the population may have dropped from 66% to about 60%.  So there is movement toward more willingness to pay, but it's very gradual.

It's hard for me to believe that most of those 60% will be willing to add about $400 a year to their mobile phone bills just for the privilege of checking their e-mail on the bus or streaming songs from Pandora onto their phones.  And remember, that research data is based on people in some of the richest countries in the world.  It may map fairly well to other rich countries like Japan and South Korea.  But in the developing world, average personal incomes can't possibly support big mobile data bills.  Most people there will need to sip data through a straw rather than gulping it from a mug.

So I have a fundamental disagreement with many industry analysts about how the mobile data market will develop.  A graphic would help explain...



This has a huge impact on what will happen next.  The consensus view says that with only a third of the population in the US and Europe owning smartphones today, the prospects for growth are fantastic -- we can still sell to the other two thirds!  And after that we'll move on to the rest of the world.  The segmented view says that with a third of the population using smartphones in the US and Europe, we've already sold to most of the world's population willing to pay for big data plans.  In this view, data plan growth will start to slow in the US and Europe by sometime in 2012.

The best way to check which scenario is right would be to conduct some market research on user willingness to pay for data plans.  If you work in a mobile tech company, you should be doing that, urgently.  For those of us without six-figure market research budgets, there are some warning signs to look for.  If the segmented view is correct, we should start seeing more price sensitivity as we use up the late adopters of data plans.  One sign would be price promotions on smartphones...


AT&T's most recent iPhone advertising (link).


Another sign would be a shift in the mix toward lower-cost data plans...


Growth in data plans, 2010 vs. 2009.  In all five countries, growth is higher in mid to low-tier plans (under 50 euros / 35 pounds a month).  Source: Comscore (link)

This isn't conclusive evidence, but you don't get conclusive evidence until something has already happened.  There's enough evidence that we should be talking very seriously about possible saturation of the user segment willing to pay for mobile data.


What it means

So to recap, in a few years I think the majority of phone users will have smartphones but won't necessarily pay for today's data plans.  Some of the phones will connect to the web by WiFi only, while others will be on pay-as-you-go plans and won't be used for much data at all.  The situation is analogous to what happened with cameraphones.  Almost all of us have cameras in our phones, but most of us don't send picture messages because of the cost. 

This stratification of data use will have some pretty profound impacts on the mobile market:

A change in the crisis.  The first effect will be that we'd hear a lot less about the wireless bandwidth "crisis."  Operators will all of a sudden feel a lot less pressure to expand their networks and get more spectrum.  However, they will not be happy.  Slowing data growth will probably make them miss their revenue forecasts, hurting their stock prices.  Some operators may end up with excess capacity, resulting in renewed price competition in data plans, and putting more pressure on earnings.  So instead of a bandwidth crisis we'll suddenly have an overcapacity crisis.

Pressure on mobile startups.  Right now mobile apps are seen as a hot investment area because there's so much growth.  There's a lot of venture capital available.  If growth of mobile data slows, the rate of investment will slow also, as investors look for the next hot thing.  This won't be a disaster for today's mobile app companies, but it would make life harder for new entrants.  Also, companies that are investing on the assumption of endless growth might find themselves overextended.

Data will go a la carte.  But the biggest change is that to make mobile data grow further, we'll need to entice people into using it.  The challenge will be getting them to pay for little bits of data service, one app or one occasion at a time.  This requires a different sort of data plan, different apps, and a different user experience on the phone...


Enticement becomes job one

A mobile data slowdown will create an enormous opportunity for smartphone companies and app developers to create a different sort of relationship with phone users.  Most users will be perfectly willing to use data; they just won't want to pay for the plans.  The single most important task for driving mobile data growth will be to gradually entice these people into using data a bit at a time.  This creates several big business opportunities:

"Toll free" applications.  Just as we enable toll-free phone numbers in which the recipient of the call pays, we should enable toll-free apps and websites in which the app or site vendor pays for the cellular data charge.  I can picture several uses for this:
     --Some sites or apps might be willing to pay the data charge because they earn enough from ads to cover the cost.  For example, I am willing to bet that Google and Bing would both pay the data charges for a mobile search on their sites. 
     --Some sites or apps might be mobile supplements to paid PC web apps whose monthly service fee is large enough to cover the mobile data cost.  This might apply to a music streaming service or a file storage service.
     --Some third parties might be willing to cover the service fees for an app or website.  For example, the movie Rio sponsored a version of Angry Birds.  Picture them doing the same thing with a web app that transfers data.  They don't want the users hesitating to use their app, so they will pay the data charges.

Although it's easy to talk in the abstract about toll-free data apps and websites, it will be hard to implement them.  We'll need a payment clearinghouse that standardizes and manages the transfer payments between developers and mobile operators.  That was done for toll-free numbers, so I assume it should be straightforward, but there's still a lot of work to do. 

We'll also need a way to let the users know about toll-free apps and websites.  I think this is a task for the operating system -- it should identify the toll-free apps and sites automatically and enable them on phones that don't have data plans.  The operators also have work to do, because they'll need to track the data used by the toll-free apps and make sure it's not charged to the user.

It might also be good to have a top-level web domain for toll-free mobile sites.  I think .up (for "unpaid") is available.

There is also an important role for government here: Don't screw this up.  We need to be sure that any net neutrality regulations don't accidentally ban toll-free sites and apps.  It's possible that toll-free apps and sites will end up being the main way most people access mobile data, and it's critical not to cut off that possibility.  (I'll discuss net neutrality in a lot more detail in the second and third parts of this post.)

After-sales billing is critical.  Mobile and web developers have already figured this out: In many cases, your best chance of making money is to give away your base product and charge for upgrades and add-ons.  That business model becomes even more important in a world where most users don't have a mobile data plan.  How do you gradually get people hooked on your product when they're not willing to even pay for the cost of connecting to your website?

For some developers the answer will be that you just ignore those customers (and in that case you'd better base your forecast on selling to only a third of the population).  But for other developers, there will be an art in figuring out how to write a very data-efficient app or website that delivers enough value to hook a user with a data charge so low that you can pay it, at least during a trial period.  That sort of art is a great opportunity for differentiation.

Micropayment is critical as well.  Because developers need to experiment in incremental billing, it's critically important that they be able to easily bill customers in very small amounts.  The best system for doing that looks to be Google's recently-announced In-App Payments system, which is supposed to launch this summer.  Google will charge a flat 5% of your revenue no matter how small the transaction.  This is a huge improvement over PayPal and Amazon FPS, both of which charge 5% plus 5 cents per transaction (in other words, they take 40% of a 25-cent transaction). 

If the operators want to facilitate this sort of billing through their own infrastructure, they'll need to match Google's terms.  Operators that are wise enough to enable this may be able to build tight alliances with the most innovative websites and apps, but my guess is that most operators won't be able to get comfortable with a cut as small as 5%.  In that case, they should just get out of the way and let Google (and its competitors) operate.

Smartphones must entice.  This is a huge opportunity for companies that make handsets and mobile operating systems.  Smartphones today are designed for unlimited data plans -- here's the browser, click away; here's the app store, download something.  Those apps will be ignored by a user who has a limited data plan.  Instead, the phone itself will need to show the user individual functions and apps they can use for small bits of money.  Want directions?  That'll cost you 25 cents.  Want to download an ebook?  That's a buck.  Folks in Europe already understand this sort of world well, because so many users there are on pay-as-you-go plans.  But to most Americans it's a new concept.  Get used to it.  Think of mobile data like an a la carte menu in a restaurant, except that for data the options are almost infinite -- so the phone will need to learn about the user and customize the offers to his or her particular interests.

This model of infinite customization and a la carte ordering requires a fundamental redesign of the user experience of the smartphone.  That means it is a huge opportunity for differentiation, maybe the biggest single opportunity in mobile computing.  Apple is the leading vendor in smartphones for people with large data plans.  Although Android is catching up on many countries, often it seems to be selling to the more price-sensitive end of the market (note the lower sales of paid apps on Android compared to iPhone).   So it makes sense that the "enticement phone" would be built on Android.  I'd like to think Google would do it, but intuitive and well-integrated user experience is not its strong suit.  So maybe it'll be an Android vendor.  Or maybe Nokia will do it.  Or even Microsoft.  Whoever gets it right first has a very good chance to be the other dominant smartphone vendor.

Or maybe Apple will do it first, and end up the leader in all smartphone price bands.  It wouldn't surprise me.


What if I'm wrong?

So that's what I think is going to happen: there will be a natural slowdown in the growth of mobile data as we use up the customers willing to pay for it, and the most critical task for mobile data companies will be enticing people to use their services a bit at a time.  But what if I'm wrong?  What if the whole population is so excited about smartphones that everyone is willing to pay for big mobile data plans?  How does the world look then?

I'll cover that tomorrow, in part 2 (link).  In the meantime, please post comments and questions.  This is a huge, complex issue, and I don't pretend to have it all figured out.

Just a quick note to let you know about a couple of informational resources for mobile developers.

--Motorola is starting the online publicity for its upcoming Android-based smartphones. They did a brief interview with me, asking how mobile app developers can distribute their software (link).

--Elia Freedman of Infinity Softworks did a great presentation on his experiences selling through the iPhone App Store, and the lessons he has learned. It’s well worth watching the video here.

It's best to watch both of these, and think about them, before you develop your mobile app.

Intuit and Stanford recently asked me to give talks on computer platforms and what makes them successful. (By platforms I mean software with APIs that third party developers can write apps on top of; Windows and Macintosh are both platforms, as is Java.) Platforms are a hot topic in Silicon Valley these days. The success of the iPhone app store in mobile, and Facebook on the web, have forcefully reminded people that you can grow a tech business more quickly if you get third party developers to help you. Almost every tech company I work with is trying to expose some sort of API or platform offering in its products.

To explain how software platforms work today, I thought it'd be good to start with their history. But I wasn't sure about many of the details myself, so I ended up doing some research. The information was surprisingly hard to find, and also pretty controversial -- for every person who claims to be the first to have done something in computing, there's someone else who begs to differ. I did my best to sort through all the claims. The picture that developed makes an interesting story, but also has some very important lessons about where the industry might go next.

Fair warning: this is a long post. But I hope you'll feel that the destination is worth the trip.

Here's what I found:


Hardware memory, software amnesia

The computer industry is often criticized for its failure to remember its own history. Supposedly we're so focused on the new thing that we forget what's come before.

In reality, though, we're actually fairly good at remembering a lot of our hardware history (for example, Apple fans are celebrating the 25th anniversary of the Macintosh this year). There's passionate controversy over what was the first computer -- was it Konrad Zuse's Z1 (link), Tommy Flowers' Colossus (link), etc. The answer depends in part on your definition of the word "computer." But it's a well-documented disagreement, and you can find a lot of information about it online, including a cool timeline at the Computer History Museum (link).

The machine most commonly cited as the first fully programmable general-purpose electronic computer was ENIAC, the Electronic Numerical Integrator And Computer. It was completed in 1946 (link).


Here's ENIAC (well, part of it, anyway)

You can find lots of histories of ENIAC online (link). There are multiple simulators of it on the web (link), and the engineering school at the University of Pennsylvania even has an ENIAC museum online (link).

But when it comes to software, our memories are much hazier. For example, I doubt there will be a 25th anniversary celebration in 2010 for Aldus PageMaker, the program that did more than any other to make Macintosh successful. And about a day after I post this article -- May 12, 2009 -- will be the 30th anniversary of the introduction of Visicalc, the first spreadsheet program. Anyone planning a parade?

Today we take it for granted that you can use a computer for a variety of business or personal tasks, but it didn't always work that way. ENIAC and Colossus were government-funded tools for solving military and scientific problems. The US Army funded ENIAC, and in addition to calculating artillery tables, it was also used for tasks like weather prediction, wind tunnel design, and atomic energy calculations.


These nice ladies are programming ENIAC, by moving cables around.

How did we end up using computers for other purposes? The UPenn site says only, "it is recalled that no electronic computers were being applied to commercial problems until about 1951."

Yeah, "it is recalled." This is where I had to start digging. Once again there are disputes (link), but you can make a very good case that business computing started in the UK, and it involved something called a Swiss roll.


The first business computer

I had never heard of Joseph Lyons & Company, but in the 1950s they ran a chain of tea shops in the UK. I have to pause here for a second and explain what the term "tea shop" means. It's not a shop where you can buy bags of tea (which is what I assumed). Instead, it is what Americans call a coffee shop -- a fixed-menu restaurant that people would come to when they wanted to have a quick meal, snack, or meeting. The closest equivalent in the US these days is probably Denny's.

In the 1950s, Lyons had the biggest network of tea shops in the UK. It employed 30,000 people and served 150 million meals a year. The company sold 36 miles of Swiss roll a day (link).

(In case you're wondering, Swiss roll is a flat sponge cake rolled around a filling. Americans call it jelly roll. In India, it's called jam roll. In Sweden, rulltårta. In Japan, "roll cake." But in Spain, for some reason it's called brazo de gitano (gypsy's arm). Don't ask me why. [link] )


A Swiss roll made and photographed by Musical Linguist on 25 June 2006

Like every other company of its day, everything at Lyons was run on paper -- tallying 150 million receipts, calculating payroll, managing taxes, and even figuring out how many miles of Swiss roll you need to make for tomorrow's customers. All of that by hand with adding machines. It was an incredibly expensive and error-prone way of running a business, but it was the best anyone could do at the time.

When the people at Lyons first heard about these new computer thingies, they wanted one immediately to help run the business. But there wasn't any way to buy one. So they donated $5,000 (about $50k today) to Cambridge University to create a modified version of a computer that Cambridge had been working on.

The result was called LEO (Lyons Electronic Computer), and when it started regular operations on November 17, 1951, it was the world's first business computer. It occupied 5,000 square feet of floor space (about 500 square meters), and its 4k memory unit weighed half a ton because it was full of mercury. LEO's lead programmer was David Caminer, who is generally credited as either the world's first business software programmer or the first systems analyst. LEO's software let it handle -- guess what -- the same sorts of tasks we handle on business computers today: payroll, inventory, financials, and so on. It cut the time to calculate one employee's wages from eight minutes to 1.5 seconds (link).


David Caminer

Pause for a moment and think about the courage and vision it took for Lyons -- a catering company -- to build its own computer. There was no guarantee the process would succeed, and indeed the process took two years, with plenty of setbacks along the way.

But LEO was eventually a big success, and Lyons eventually spun it out as a separate computing subsidiary. Caminer went on to have a distinguished career in computing. He died in 2008, unfortunately, so we just missed our opportunity to say thanks to him. If you want to read more about LEO, Caminer co-wrote a book about it (link). Naturally, it's out of print, and the cheapest used copy when I looked it up was $75.


What is software, anyway?

One interesting aspect of LEO is that although Caminer and his team wrote software for it, that software was not available separately from the computer. That's the way the computing industry worked throughout the 1950s. For example, if you bought an IBM computer there was a set of standard IBM programs that ran on it.

In fact, the term "software" didn't even exist until it was popularized by John Tukey in 1958, more than ten years after ENIAC began operation (link). He wrote:

Today the "software" comprising the carefully planned interpretive routines, compilers, and other aspects of automative programming are at least as important to the modern electronic calculator as its "hardware" of tubes, transistors, wires, tapes and the like.

So the whole idea of software as a separate entity, a concept that we take for granted today, did not exist at the beginning of computing. The concept of making computers reprogrammable came along quite early, but it took a couple of decades for software to fully separate itself from hardware as its own distinct discipline.


John Tukey

(Naturally, there's some dispute about whether Tukey was the first to use the term "software." You can read about it here.)

Tukey was an interesting guy. He also created the term "bit," helped design the U-2 spy plane, and did a lot of other fascinating things (link).

If you want to read more about the history of software technologies, there's an essay here. And the best (and just about only) book on the history of the software industry is here.


Software as a business

Once we got the idea of software into our heads as a separate discipline, the next milestone in platform history was the creation of the first independent computer program, the first one you could buy separately from the hardware. As far as I can tell, that idea didn't just spring into being all at once; it emerged as a slow-motion avalanche over a period of 15 years.

Computer Usage Corporation, founded in 1955, is often cited as the first computer software company. It focused on custom programming services (link). Another very early custom programming company was CEIR, founded in 1954 (link). After them, a number of other custom programming firms sprang up. Sometime between 1962 and 1965, California Analysis Center, Inc. started selling a proprietary version of the Simscript programming language as a standalone product (the Computer History Museum says it was 1962 here, but CACI's own website says 1965 here). The 1962 date is the earliest I can find for any sort of independent software product. To my amazement, CACI is still selling Simscript today (link).

Several other programming languages and compilers came to market in the early 1960s, but there's disagreement over how much they actually sold, or whether they were really managed as independent products (link). A file management program called Mark IV, by Informatics, is credited as the first independent software product to generate more than a million dollars revenue. It was published in 1967 (link). That year also saw the first publication of the International Computer Programs Quarterly, the first commercial software catalog, which helped small software companies get to market at low cost (link). Think of it as a paper version of the iPhone App Store.

But if you want to find the first snowball that started the commercial software avalanche, I think it was tossed in 1964 when a contract programming company called Advanced Data Research was jerked around on a business deal by RCA.


The first commercial software product

In the mid-1960s, a cottage industry of contract programming firms did custom software development. When a new mainframe was in the works, its manufacturer would sometimes hire these firms to create software to offer with it. Computer owners could also hire those development houses to write create custom software for them. The idea of off-the-shelf software didn't exist; you got it for free with your computer, had it written for you, or developed it yourself.

RCA, which at the time was a promising mainframe company, approached ADR asking them to create a program to draw flow charts of computer programs (the flow charts were used for documentation and debugging). That may not sound like a big deal today, but in the early days of computing the industry didn't have the sort of automated debugging tools it has today. A flowchart was very useful to help maintain and document a custom software program after the project was finished.

So ADR created a proposal and submitted it to RCA. Fortunately for the computer industry, RCA turned it down, as did every other mainframe company. But ADR believed in its concept, so it decided on its own to develop the product anyway. It spent over $5,000 (about $35k in today's money) and half a man-year on the project.

But RCA was not impressed. Once again they said no.

Now ADR had a sunk cost. In business school they teach you to walk away from those, but in real life companies hate to admit they made a mistake. So ADR decided to try marketing the software on its own. They named it Autoflow, and wrote a letter to all 100 RCA mainframe owners offering them the program for $2,400 on a three year lease. It was three milestones in one: the first commercial software program, the first subscription software, and the first junk mail urging you to buy a software program.

ADR sold two licenses.

That may not sound like much, but somebody at ADR did the math -- if we sold two copies to 100 RCA customers, what would happen if we offered our software to IBM's much larger installed base? So ADR ported Autoflow to IBM mainframes. In the second half of the1960s it sold more than a thousand licenses of Autoflow, and created a portfolio of other independent software programs for IBM systems.

IBM was not pleased. Nobody was supposed to mess with the IBM customer base; that might weaken IBM's control over its customers. The company created its own flow charting software, which it gave away for free to its customers, and started to copy ADR's other programs as well. This became a huge competitive problem for ADR -- even if its software worked better than IBM's, it was hard to compete with free. IBM was also able to freeze the market for ADR by promising that it would in the future offer a free version of something ADR was currently selling. Customers would delay ADR purchases until they could evaluate the IBM product.

ADR and other fledgling software companies complained to the US government. In 1969, the Justice Department, ADR, and several others filed antitrust suits against IBM. ADR collected $2 million in penalties, and IBM agreed to stop bundling free software with its computers.

And thus the independent software industry was born.



Martin Goetz (above) was the product manager of Autoflow. I wrote to him and asked for his take on which was the first software product. Here's his reply:

Autoflow was recognized as the first software product to be commercially marketed. Starting in 1964, ADR licensed its products nationally and through ads in all the major computer publications, started investing in the development of other products and became known as a software products company.

I think that's the right way to look at it: Autoflow was the first software product to be commercially marketed, which is why I call it the snowball that started the avalanche. Informatics' Mark IV also played an important role because its financial success validated the market -- reportedly it was the top-selling software product for the next 15 years (link).

Goetz says Mike Guzik was the lead programmer on Autoflow (link), and he cites ADR President Dick Jones as a strong supporter of the idea (link). I think we should credit Goetz and Guzik as the creators of the first commercial software application, although neither of them has an entry in Wikipedia.

Incidentally, Goetz also holds the first software patent:


Computerworld, June 1968

That has to be one of the most visionary headlines in the history of the computer press: "Full Implications Are Not Yet Known." Here we are 41 years later, and it's still accurate.

Goetz was named the "Father of Third-Party Software" by mainframezone.com (link) and there's a very interesting interview with him here. You can find a much longer interview here and his memoirs are here.

Advocates of open source software will probably view Goetz as a bad guy, since he helped make software a for-profit industry. But he has some pretty strong opinions about the poor quality and slow innovations that happened in software when it was only free. In particular, he says that a completely free software industry was not responsive to the needs of users (link).

An amusing anecdote complaining about Goetz, apparently written by a former ADR employee, is here. I can't verify the anecdote, but if nothing else it shows that ADR was also a pioneer in the practice of engineers making catty comments about product managers.

(I should add that there are some different interpretations of the effect of IBM's unbundling decision. One is in a very interesting interview with the creator of the ICP catalog here.)


The rise of the third party application platform

The next evolutionary step was for computer companies to see their products as development platforms -- for them to actively encourage software developers rather than viewing them as a nuisance. I haven't been able to figure out when in the 1970s this change in perspective happened (please post a comment if you know the history). It may have happened in the era of minicomputers, or it may have been a PC thing. Definitely Dan Bricklin and Bob Frankston's VisiCalc, the world's first spreadsheet program, played a role when it came to market for the Apple II in 1979. It was so revolutionary that reviewers at the time didn't know how to describe it. They just said it was a way to make the computer do things you want it to do, without writing your own program. VisiCalc established the idea of the "killer app," a software program so popular that it drove demand for the underlying hardware.

"Visicalc could some day become the software tail that wags (and sells) the personal computer dog."
--Ben Rosen, co-founder of Compaq, reviewing VisiCalc when we was still an analyst with Morgan Stanley. Nice call, Ben. (Link)

By the early 1980s, software developers were being actively courted by computer manufacturers. Apple had a developer recruitment team for the Macintosh, and apparently coined the term "software evangelism." That's where Guy Kawasaki cut his eyeteeth, although he wasn't the first evangelist. As he puts it:

"Mike Boich started evangelism and hired me, and Alain Rossman worked with me as a software evangelist. Essentially, Mike started evangelism, Alain did the work, and I took the credit." (Link)

I happen to know that Guy did a bit of the work too.

The other critical change in the 1980s was the separation of the OS from the underlying hardware. Most of the new PC software platforms had been tied to hardware, just like traditional computers. For example, you had to buy a Macintosh in order to run Mac software, or an Amiga in order to use Amiga apps. But then IBM created the PC, and through a series of business blunders allowed Microsoft to separately sell the DOS operating system used on its hardware. IBM's brand and marketing power established the PC as a standard, but the company enabled Microsoft and Intel to create a "clone" hardware market, and eventually drive IBM out of the PC business.

So now there were three layers in the industry -- the application was independent of the OS, and the leading OS was independent of the hardware.


The network strikes back

That's where the situation sat until the late 1990s, when Java and web browsers threatened to create another layer in the architecture by separating software applications from the OS. The theory was that instead of writing programs that depended on Windows, programmers could create code that worked on Java, or on the Netscape browser.

Microsoft fought back very aggressively, killing Netscape by giving away Internet Explorer, and crippling Java on the PC. Looking back, it was an impressive use of business muscle, worthy of Microsoft's tutor IBM.

But it was also a pyrrhic victory. Microsoft's actions in the 1990s forced software innovation completely off the PC platform, because investors were afraid that new software apps would just get cannibalized by Microsoft. Instead software innovation moved onto the web, where Microsoft had virtually no control. That's one of several reasons why the next generation of software is being written as web apps.

And that's where we are today.


Where we go next

As I said at the start of the post, I think all of this history is fun in its own right. I also wanted to take this opportunity to thank some of the people who built the tech industry into the fun place it is today.

But understanding computing history is also very important because, if you look across the sweep of it from the 1940s to today, it's much easier to see where we might go next.

Here's what I think that long perspective shows us: The history of software is a history of disaggregation. First the application software gets separated from the hardware, then the OS gets separated from the hardware, and so on.

I think disaggregation is a natural outcome of the maturation of the industry, because multiple companies can move faster than a single one. At the start you need everything coordinated together to make sure the whole thing will work. But over time, no single company can pursue all of the innovation possibilities, so you get a backlog of potential creativity that can happen only if control over the architecture is broken into pieces.

For example, most of the interesting innovation in applications happened only after they were separated from the hardware.

But as the industry continues to grow, each of the pieces becomes its own stodgy monolith, and eventually another subdivision happens.

The fastest growth and the easiest innovation has generally happened at the leading edge of disaggregation, because each change creates new business opportunities.



That doesn't mean that old school companies are dead. IBM still sells mainframes, and Apple still makes PCs bundled with an OS. But to succeed in an old paradigm you have to execute extremely well, and it's much harder to grow explosively. The easiest progress is made at the leading edge.

A common thread among the people working at the leading edge of disaggregation is their excitement as they recognize the opportunities created by the change:

"There was a tremendous euphoria of success. You couldn't lose. All you needed was a group of highly technical people who could create a software product and that was it. And to some degree there was some truth to that. Because you didn't have to be good sales people. You didn't have to worry about the competition. For years I used the aphorism that we were like little boys on the beach each with our sand piles. There was plenty of sand to put in our buckets. We didn't have to edge out the other little boy to get all the sand we needed. We were limited by the size of our pail and our little shovels but not by the amount of the beach that was there or the fact that there was another little boy there with his pail."

That's Walter Bauer, cofounder of Informatics, talking about the birth of the independent software industry in the 1960s (link). But you could find similar sentiments from the people who built the first computers, or the first Mac programmers, or the first web app developers. The leading edge of disaggregation is where the action is; it's where the fun happens.

So, if you're looking to succeed in the software industry, it's extremely important to figure out what's going to get disaggregated next. Which brings us to the point of this article.


Say hello to the metaplatform

Sun's rallying cry in the 1990s was, "the network is the computer" (link). It was an excellent insight that pointed to the emerging importance of the Internet, but most of the industry misread what it meant. We looked at the architecture of the thing we knew best, the PC, and tried to map it directly to the network. So servers would replace the PC hardware, and software on those servers would replace Windows. The PC itself would be reduced to light client, a screen connected to a wire.


What we expected

But instead of a new OS on the network replacing the OS on the PC, what we're seeing is the breakdown of the OS into component parts that live everywhere, on both the client and the server.

In other words, the OS is the next thing that gets disaggregated.


What's actually happening

People have been talking about elements of this change for years, but like the proverbial blind men feeling bits of the elephant, we've talked about individual pieces of it, with each of us assuming that the piece in front of us was the most important. So people producing software layers like Java and Flash say that they are separating the APIs on the device from the underlying OS. And the advocates of cloud computing say they're creating a software services architecture that runs on servers. But in reality we're doing both of those things, and a lot more. The OS is dissolving into a soup of resources distributed across both the network and the local device, with the application in the middle calling on both as appropriate. We need to get off the idea that the network or the client will be dominant; they're both supporting elements in something larger.

You can see this process operating in the evolution of web applications. The first web app companies tried to make applications that were entirely light client, but they didn't work particularly well -- they were slow, and their user interfaces were too limited. Web apps took off only when they adopted an approach in which the platform was split between the PC and the network -- the user interface ran locally through the browser, while back-end calculation and data storage was done on the network.

Mobile computing reinforces the need for this sort of hybrid architecture. Wireless broadband has important limitations that make pure light client computing extremely problematic. Wireless networks are relatively slow compared to wired networks, there's high latency on them, coverage is inconsistent, heavy communication drains device batteries rapidly, bandwidth is expensive, and most importantly, total wireless bandwidth is limited. The most effective mobile application are and will continue to be hybrids of local and network resources, like RIM's e-mail solution.

Companies entering the mobile market often ask me which mobile operating systems are going to win long term. I think that's the wrong question. What we're seeing is the gradual evolution of a super-OS that includes both the network and the device.

Like software developers before the word "software" was invented, we don't have a name for this new thing, and so we have trouble talking about it. It's not just the Network or the Cloud, because those terms are usually understood not to include the software on the client computer. And it's certainly not just the local APIs on the client device.

I'm calling it the "metaplatform" because it subsumes all other platforms. No single company controls the metaplatform. Google obviously contributes a lot to it, as does Amazon Web Services, as does Microsoft. But they're only fragments of the picture. There are thousands of other contributors to the metaplatform, in areas ranging from mapping to graphics to identity.

There's still a lot of work that needs to be done on the metaplatform, especially in the mobile space. But already it's evolving faster than any single company could move it, because the work is divided across so many companies, and because there's competition driving innovation at almost every point in the architecture. Although the metaplatform isn't necessarily elegant (because it's poorly coordinated), what it lacks in beauty it more than makes up for in rate of change and versatility.


New opportunities

The metaplatform helps to solve some computing problems, but creates others. For example, a recurring problem for software in the OS era has been compatibility. Old data files, even when perfectly preserved, can become unreadable if the hardware and software that created them is no longer available. A lot of software is very dependent not just on the hardware, but on the particular version of the OS it's running on. (If you want to see that effect in action, try running a ten-year-old Windows game on a new PC. It may work, it may refuse to run at all -- or it may freeze right when you're about to defeat the boss bad guy.)

The metaplatform is helping to resolve some compatibility problems, through emulators available online. But more importantly, web apps on a PC are less vulnerable to PC-style compatibility breakdowns because PC browsers are relatively standardized, and much of the OS code the web app relies on lives on the same server as the app itself, so they are less likely to get out of sync.

But metaplatform-based software is uniquely vulnerable to a new set of problems. When a user's data is stored on a web app company's server 3,000 miles away, what happens if that company goes out of business or just decides to stop maintaining the product?

Another problem experienced by any website using plug-ins is component breakage. If you've incorporated external web services into your site, the site will break if any of those services stops working. This can happen without warning. On my own weblog, the load time for the site suddenly became ridiculously long. It took me weeks to realize that a user-tracking service I'd once signed up for had gone out of business without telling anyone. My site stopped loading while it tried helplessly to connect to a tracking site that no longer existed.

An old software application from the OS era has some hope of revival if you have a copy of the CD, because all the code that made up the app is together in one place. But an old, broken web app will be almost irretrievably dead, because huge chunks of its code will be missing.

Problems like these are just starting to emerge, but as the metaplatform grows and ages they'll become much more prominent. We don't have any systematic ways to deal with problems like these today -- which means they're a business opportunity for the next crop of software entrepreneurs.


What the metaplatform means to you

Much of the discussion in this post is pretty theoretical. But I think it has important practical implications. Here are a few specifics to think about:

If you're a computer user (and if you're reading this, you must be), keep in mind that the most interesting new software innovations are likely to come from companies that consciously work the metaplatform. If you want to be at the leading edge of software innovation, you should keep yourself open to experimenting with new web applications and plug-ins, and make sure your browser doesn't artificially cut you off from some technologies. This is especially true for mobile devices. The iPhone today gives (in my opinion) the best overall mobile browsing and app discovery experience, but you pay a price for it -- you're cut off from some web technologies (Flash, Java) and your choice of applications is limited by the Apple app police. You pay a serious price for the superior user experience of the iPhone. That price is worth paying today, but in the future I hope there will be mobile devices that are as satisfying as the iPhone but less controlled. Actually, I'm sure that will happen over time. But "over time" can sometimes mean a long time in the future. You can help the process along with what you buy and by the feedback you give to device manufacturers.

Are you working at an OS company? If so, you probably measure success by the number of devices your software controls. You need to rethink that viewpoint. The OS is going to be less and less of a technology control point in the future. It will become commodity plumbing underneath the metaplatform, limiting your ability to charge a lot of money for it. So at a minimum, you need to plan for cost control.

But you should also be asking if plumbing is the right place for your company's creativity in the long term. There will be much more profit opportunity in contributing to the metaplatform by creating APIs and developer functionality that can be used across different operating systems. OS companies have many of the assets needed to build those components of the metaplatform. A successful OS can be a great launching point for technologies that run across platforms, because you already have a big installed base that you can use to jump-start the technology's adoption.

Are you at an application company? Many successful app vendors are trying to create APIs that will enable other developers to extend their products. This is the right idea, but the implementation is often off-target. Many of the app companies I talk to are trying to make their APIs into the business equivalent of an operating system, with developers coming to them and living entirely within their private ecosystem. A warning sign is when a company uses a phrase like, "(insert company name) developer network" to describe its offering.

The wave of the future is not turning an application inward into its own little walled garden; it's opening the application outward so it can be mixed and matched with other functionality in the metaplatform. If you have the best drawing program in the industry, you should be asking how you can also become the best drawing module in the metaplatform. Get used to being a component in addition to a standalone product. You lose some identity in the process, but gain greater opportunities to grow.

And besides, if you don't do it, you'll be vulnerable to someone else doing it and taking your place.

If you're a computing student, or a computing veteran looking to create a new product, think about what role you can play in the metaplatform, and what customer problems you can solve with this new tool. There will be big market openings in both products for users and companies, and infrastructure for other developers in the ecosystem (billing, rights management, security, etc).

As in previous generations of software, the answers are not immediately obvious, and the people who figure them out first will have huge opportunities to do something impactful. Like Caminer, Goetz, Bauer, Bricklin, and Frankston, you're on an enormous beach with a trowel and bucket, and you have a chance to shape the next generation of computing.

Have fun.

====

I'd like to thank Eugene Miya of NASA Ames and Martin Goetz for helping with the research that contributed to this article. They're not responsible for any errors I made, but they definitely corrected some.

I'm sure there are folks out there who have additional information on the history I wrote about here. If you have anything to add (or correct) please post a comment.