Today we use the Internet in ways that require real-time performance, whether that is watching streaming video or making phone calls. At the same time, we’re generating much more data, so having networks that just look at bits and bytes is no longer sufficient. The network has to become more aware of the information it’s carrying so it can better prioritize delivery and operate more efficiently.
How do you make a network more aware of the information it’s carrying?
There are different approaches. Today, if you want to know more about the data crossing a network—for example to intercept computer viruses—then you use software to peek into the data packet, something called deep-packet inspection. Think of a physical letter you send through the normal postal service wrapped in an envelope with an address on it. The postal service doesn’t care what the letter says, it’s only interested in the address. This is how the Internet functions today with regard to data. With deep-packet inspection, software tells the network to open the data envelope and read at least part of what’s inside. [If the data contains a virus, the inspection tool may route that data to a quarantine area to keep it from infecting computers connecting to that network.] However, you can get only a limited amount of information about the data this way, and it requires a lot of processing power. Plus, if the data inside the packet is encrypted, deep-packet inspection won’t work.
A better option would be to tag data and give the network instructions for handling different types of data. There might be a policy that states a video stream should get priority over an e-mail, although you don’t have to reveal exactly what’s in that video stream or e-mail. The network simply takes these data tags into account when making routing decisions.
Even if a smarter Net can move data around more intelligently, content is growing exponentially. How do you reduce the amount of traffic a network needs to handle?
Our smartphones, computers and other gadgets generate a lot of raw data that we then send to data centers for processing and storage. This will not scale in the future. Rather, we might move to a model where decisions are made about data before it is placed on the network. For example, if you have a security camera at an airport, you would program the camera or a small computer server controlling multiple cameras to perform facial recognition locally, based on a database stored in a camera or server. [Instead of bottlenecking the network with a stream of images, the camera would communicate with the network only when it finds a suspect. That way it sends an alert message or maybe a single digital image when needed.]
Would this decentralization mean the end of “the cloud”?
No, it’s a different way of organizing the cloud. Today the cloud is made up of big, centralized data centers. That’s fine for certain functions, such as when you need to aggregate data on a global scale. In the future our devices, whether it’s a smartphone or a television set-top box, will play a larger role in the cloud. [In the case of a set-top box, the box would gather data about a viewer’s preferences, analyze that data right there in the living room and the send specific content recommendations back to the cable provider, rather than a stream of raw data.]
How does information networking address privacy concerns?
At the moment privacy is binary—you either keep your privacy or you have to give it up almost entirely to obtain certain personalized services, such as music recommendations or online coupons. There has to be something in-between that puts the user in control of their information.
The biggest problem is that it has to be simple for the user. Look at how complicated it is to manage your privacy on social networks. You end up having your photos in the photo stream of people you don’t even know. There should be the digital equivalent of a knob that lets you trade off privacy with personalization. The more I reveal about myself, the more personalized the services I receive. But I can also dial it back—if I’m willing to provide less detailed information, I can still receive some personalized, albeit less-targeted, offers.
Subscription Center
See what we're tweeting about
21 Comments
Add CommentApparently, we need to build Skynet.
Reply | Report Abuse | Link to thisWell, not too long ago people went to the local video store to rent a copy of a recent movie. This required no internet bandwidth whatsoever. However, now, whenever someone wants to watch the most popular movie release, they transmit it over the internet to their smart wristwatch or whatever, along with many millions of other customers downloading that same popular movie.
Reply | Report Abuse | Link to thisWhy are the communications companies that encouraged all this silly business wondering why the internet is running out of capacity? What's wrong with this picture?
I'm sorry to tell you, but we reached the end of the Internet some hours ago: http://bit.ly/Yo3sR1
Reply | Report Abuse | Link to thisThe communications companies are interested in short term profit. They get profit from supplying what all the young techno-nerds want or think is "kewl". Whether it is sustainable is irrelevant.
Reply | Report Abuse | Link to thisI have a "smart" phone that I use as a phone. On occasion, I check the weather or use the map app. I may even send a text once in a while. But I do not stream movies to that tiny screen. I prefer to watch that movie on my 55" LED flat screen at home.
The internet is approaching it's limits because of the users, not the providers. If the users make demands on the infrastructure that the infrastructure cannot handle, the infrastructure will fail. This applies to the electrical systems, sanitary systems or telecommunications systems equally.
There are three principles involved in this. The equipment designers and manufacturers exacerbated by competition with each other, the infrastructure providers who need to keep up with the technology and the users who keep demanding more. The law of supply and demand applies and the feasibility (or cost) will come down as supply meets demand. When the infrastructure hits it's limit, the costs of using that infrastructure will go up... and up... and up until the demand drops off. In this case, demand will drop due to perceptions of poor service and the infrastructure will be blamed.
Just saying...
Actually the problem is the pricing model for telecom is based on the price the market will bear not the cost of providing the service. Time Warner brags about achieving a 3000% profit on broadband.
Reply | Report Abuse | Link to thisThe cost of long distance fibre is zilch tiny fractions of cent a gigabyte, with the cost of adding a 120GBs fiber to a proposed fiber build from NY to LA less than $100K.
The cost of 1 Gb/s fiber to the block schemes like Google's NYC Chelsea network is less than a buck a month per subscriber.
The only road block is once again our corrupt politicians all bought and paid for by Big telecom.
The ITU just put out a paper saying the same.
www.itbusiness.ca/it/client/en/home/News.asp?id=69906
Difference algorithms are a simple way to build smartness into the data to be transmitted. For example if I am sending a stream of audio data, which is actually a stream of digitally converted analogue information, instead of sending the full value of a conversion send only the difference between the last conversion and this conversion. Considering that this kind of information only changes at a maximum by a certain amount, it may be possible to reduce the number of bits by 50% or even more therefore cutting the data in half. Video imaging is already doing this sometimes by sending only parts of the images that have changed from the last frame. Also you can load up the client, making it smarter so that runs programs that perform the function instead of sending the image of the screen. A good example of this is a graphics program that has the graphics engine on the client side and simply sends commands to the client to draw the various shapes required as opposed to sending the bit image. Many systems are starting to do this.
Reply | Report Abuse | Link to thisPersistence on servers is a good way to handle stuff like video data. For example if you are watching a Netflix movie that movie may have passed through several internet nodes to get to you. If it stayed on the various backbone stations then if others are watching it then the data could be drawn from the local node instead of coming all the way from the Netflix server again.
Reply | Report Abuse | Link to thisIt is crucial that bandwidth providers and content providers be separated. A consequence of the author's proposal would be a co-mingling of the two - with the resulting decrease in content diversity.
Reply | Report Abuse | Link to thisThe Shannon links are reversed (the first link should go to the second URL and vice versa).
Reply | Report Abuse | Link to thisA rather simple way of reducing the load on the network would be to minimize the size of some existing traffic by making it more bandwidth efficient. A relatively simple change to HTML and its brethren by using a simple binary substitution for tags which are now transferred in full ASCII representation. In many cases that I have observed, the tags use more of the packet space than the actual information transferred. The packet space could be further reduced by applying an existing compression process, e.g. LZF, to the packet contents prior to transmission.
Reply | Report Abuse | Link to thisThe article seems to suggest that we are in doomsday scenario and we should all build bunkers to protect ourselves from its coming end. I don't think things are as bad as the interview suggests.
Reply | Report Abuse | Link to thisThe first question on reaching "the limit", yes, it is expensive to add more fiber or pack more waves of data across a single fiber. The cost to do this has been coming down along with the ability to add more and more waves across a single fiber. We are also seeing commoditization enter this space and drive down prices even more. At the high end of bandwidth density, we will always see a premium price. At the low and midrange, the price drops quickly over a few years. Major service providers will pay the premium to pack data across their pipes as dense as possible. Meanwhile, at the local ISP or enterprise level, 10 gig ethernet is ridiculously cheap now.
The next question about a "dumb" vs "smart" network, I have to completely disagree with that. There is no good way to scale up prioritization on Internet scale. That is absolutely ridiculous. You can't trust the users or the apps to cooperate. Even within an Enterprise network, it is a challenge to start classifying data into various buckets and come up with a sane policy on how to handle each traffic type. It is one thing to have a priority queue for voice traffic with a constrained bit rate (ensuring you will forward the voice traffic first, but only so long as the bit rate is below some threshold), but once you start doing that for other apps, you open yourself up to a policy war on whose traffic is worth more. On an Internet scale, you will never get a consistent policy to classify traffic by some common method.
The theoretical discussion on data tags is naive. Where do you set the trust boundary on who is allowed to set the tags? What keeps a user from setting tags themselves. As reddit will attest, there are a ton of people out there who are looking for ways to improve their network performance and are willing to share that information with others. If the system can be abused, it will. This means that you end up locking down the majority of traffic as untrusted from users. Then you have an Internet that is no longer treating all traffic the same.
I'm sure the people in the interview were all smart people and subject experts. I just think the article could mislead people into thinking that the problem is more simple than it really is.
I think this is missing the #1 cause, which is businesses getting us to move as much of our information as possible to "the cloud". In other words, any time you want to access some of your own information it has to go over the internet instead of staying locally on your own computer. This has the obvious advantage to the companies to be able to pry more into our private lives, but has no meaningful advantage to us that I see. So one of the better solutions is for us to boycott anything to do with storing our own information in the cloud.
Reply | Report Abuse | Link to thisA much deeper analysis of the situation is found published in 2007 by Ray Kurzweil in his book " The Singularity Is Near " wherein he has suggested technological solutions also to keep pace with the accelerated exponential growth of Internet traffic and usage over the years . The only scary part of his analysis was that by the year 2045 ( just in another 32 years ) Non- Bio- Intelligence ( NBI ) would totally dominate Bio-Intelligence ( BI) in all major human activities .
Reply | Report Abuse | Link to thisWiki gives an overview of the predictions , at
http://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil.
The way things are progressing , I can very well believe that Kurzweil won't be far off the track !
A much deeper analysis of the situation is found published in 2007 by Ray Kurzweil in his book " The Singularity Is Near " wherein he has suggested technological solutions also to keep pace with the accelerated exponential growth of Internet traffic and usage over the years . The only scary part of his analysis was that by the year 2045 ( just in another 32 years ) Non- Bio- Intelligence ( NBI ) would totally dominate Bio-Intelligence ( BI) in all major human activities .
Reply | Report Abuse | Link to thisWiki gives an overview of the predictions , at
http://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil.
The way things are progressing , I can very well believe that Kurzweil won't be far off the track !
This shows a serious misunderstanding in confusing the use of the word "information" in the technical sense with it's use in the everyday sense. It's like using ergs to measure the amount of work people do. Thinking solely in terms of channels is confusing open systems and closed systems.
Reply | Report Abuse | Link to thisMore at http://rmf.vc/RefactoringCE, http://rmf.vc/InternetLostInTranslation, http://rmf.vc/PurposeVsDiscovery and other essays.
Need for more bandwidth in the Internet is not something that never be satisfied.
Reply | Report Abuse | Link to thisThe need comes from rising usage of video.
When eventually everyone has optical fiber connections so that they all can watch real-time hi-def video at the same time, I seriously doubt there will be any more need for more cables.
I've read this kind of article 3 times before and the predecessors were as wrong then as this guy is now. Telecom companies will add hard bandwidth (fiber optic cables, etc) and it will all be roses and penguins again.
Reply | Report Abuse | Link to thisSmart networks should be relegated to the dust bin of history. Every knowledgable telecom person knows that the internet as a dumb network is the very cheapest of the current alternatives. This bogus claim that the internet is full is just stupid. When a link is overloaded, then fix it by increasing the throughput on that link or on alternative links. This is incredibly cheaper than making the network "smart". But the obsolete telecom networks are just trying to maintain income from their old way of doing business, collecting fees for every transaction.
Reply | Report Abuse | Link to thisStill, bits are just bits, so in the future we will eventually provide digital communication as a public good, like sidewalks and streets, so that ease and convenience will be increased and costs decreased. See Bob Frankston's arguments for "ambient connectivity" at frankston.com when you want the details.
If the Internet is going to reach a fundamental capacity limit next year, why is this not being discussed vigorously in the usual Internet technical venues? Throughout the article, the reader is treated to misplaced and vague generalizations.
Reply | Report Abuse | Link to thisThe article cites theoretical limits to individual links as if they dictate limits to the overall system. The example of Skype session "freezes" ignores the possibility that it is Skype servers, and not Internet capacity, that is the source of the delays; such a cause would mean that making more demands on local processing would exacerbate the problem, not fix it.
The article further appears to be unaware that the Internet has gone through capacity challenges a number of times, over its 40+ year history, and managed to continue growing. Worse, the article invokes old telephone-like concepts, apparently localized capacity issues, and classic optimization concepts as if they were innovative. It even touts "spatial division multiplexing" as a potential solution, apparently without realizing that that is exactly the nature of packet-switching, which is already the Internet's core technology, applied at different levels of the existing Internet architecture.
The article's primary focus, on placing computation closer to the consuming system is a classic part of distributed system design and is balanced against cost and performance of "remote" exchanges.
The example of placing facial data bases in cameras ignores the cost and complexity of having such databases in every monitoring camera in the world. The example of set-top boxes appears to claim that an undifferentiated firehose of content is blasted at everyone's home; it isn't. Only the very small program guide is sent that way. Actual content is sent on demand.
There are real and serious capacity and performance limits for the Internet. And there are serious efforts to address them. This article does not do justice to any of that work.
A doubling of network traffic in 5 years, come on, your own article said it increased by 70% in 2012, I believe that the pace will quicken. I am expecting an Ultra Definition tablet at Google I/O in May, we know tablets and smartphones are going fHD 1080p this year. I bought a fHD 3D screen months ago for $270, no network traffic, is going to double, nearly every year. If we assume that by 2018 there will be 4 billion UD 3D tabs, if not by 2016, as around Xmas buying season 2014, they'll be cheap, with economies of scale. Mobile chips can already handle UD, games machines are coming Xmas 13 with 10 times that capacity, 8 GB GDDR5, SSDs now cost less than a dollar a GB. But I don't think Shannons limit will stop multiple frequency optical fiber, optical switching, nodes distributing to high capacity copper/WiFi ac systems. The Germans have pumped 23 TB over a single optic fiber, twisted pair can support 1GB/s, WiFi ac 1GB/s, over a million core super computers are already here. Those games machines I mentioned, can pump a Terra flop of calculations a second. We've just got started!!!
Reply | Report Abuse | Link to thisBe sure to get some magnifying glasses for viewing your UD tablet, not to mention watching movies on a UD iWatch! <%)
Reply | Report Abuse | Link to this