What Comes Next

The Internet is central to the flattening of the world as described in Thomas Friedman’s book; I’d like to explore the idea that the Internet is itself being flattened in an entirely different way.

There is no question that the advancements in search have profoundly changed how users interact with the web. It can be argued that Google is the start page for most users, and some are extending that argument by stating that Google is on its way to taking over the Internet not just in search market share, but in a fundamental economic sense.

Looking at things from a slightly different angle, I wonder about the impact of these changes. If the web (the html/http application layer) is itself a network, it is interesting to look at that network in some detail. I suppose the nodes are individual web pages (hypertext documents, actually), and the connections between those nodes are the links between pages. As with the synaptic connections between neurons in the brain, the links between web pages are critically important to the web’s status as a network, rather than simply a repository of information. I think it could be argued that those links play a similar role to the synapses of the brain; the set of pathways between chunks of information and specialized processing capabilities contains the “intelligence” of the network.

Even if you don’t take it quite that far, those links are certainly important when you look at the web from the perspective of Metcalfe’s law. Metcalfe states that the value of a network is proportional to the square of the number of nodes in the network; this value is based on the exponential (or at least superlinear if you side with recent skeptics) increase in the potential number of interconnections between those nodes, and has been used to justify the business plans of countless dot-com and web 2.0 startups.

Do the changes in the use of the web, namely the use of Google as a start page and basic navigational component, change the structure of the web as a network? For the majority of users, the links between pages are decreasing in importance as the navigational pattern involves an increasing number of returns to Google as the start page. Even with this change, however, the links between pages are still important in the ranking of pages by the search engine.

The diminishing role of links for the average user and the increasing role of Google leads to a hub and spoke experience; each use starts from that central start page. More importantly, each navigation is enabled not by the inherent capabilities of the network, but of the capabilities of the central application. I think that the net result of this change may be a shift in value, in a Mecalfe sense, from the network to Google.

Google’s OneBox is a good example of how this shift in value might be happening. Where a generic search might return a variety of results, both related and unrelated to the searcher’s intent, OneBox attempts to simplify the results by guessing at intent and presenting information based on a set of rules and supported structured data sources. Josh Kopelman points out that this effectively makes OneBox a vertical search engine.

I think this is the case, but it creates a different kind of search engine from what we have become used to; the OneBox capability is essentially a query engine onto structured data, rather than a search engine onto a distributed network of connected nodes each containing largely unstructured data. The value of a set of structured data does not seem to increase superlinearly with the number of data entries available, in contrast to the network value of the web as discussed above.

These changes in the importance of links between web pages, and the shift to querying of structured data in place of exploration of linked data, represent what I’m thinking of as a flattening of the web. I’m not sure this is a bad thing; perhaps the value of the underlying web is not diminished by these changes, and sections of that web just become valuable enough that they warrant being made available in more direct ways. It is also possible that there is merely an apparent shift in value, as the AdWords/AdSense advertising economy brings the value of certain of the underlying nodes to market. On the other hand, perhaps this flattening of the web represents a real, and not just apparent, shift in value from the web of linked nodes to the central engine. That central engine, by decreasing the importance of the connections between nodes, may be decreasing the total value of the network in order to control a larger part of that value.

The blogosphere, and all of the various forms of user-generated content in other media, is certainly a rich source of new nodes and links, and new network value added to the web. The importance of the link today is clear as a storm runs its course over the topic of linking between blog posts. Even so, I think it makes sense to be aware of how changes in technology and the use patterns of the majority of users impact the value of the network itself.