What's New Inside IBM's Cognitive Computing Chip?
Image:
By Geoff Brumfiel of Nature magazine
Today IBM unveiled a new "cognitive computing" microchip that, according to the company, emulates some of the brain's abilities. The chip is the latest development in an ongoing program by the Defense Advanced Research Projects Agency (DARPA), based in Arlington, Va., to develop systems that can analyze complex information. Nature takes a peek under the hood of the new chip.
Does this use some new kind of technology?
No. The chip unveiled today uses the workhorse of everyday electronics, known as complementary metal-oxide semiconductors (CMOSs). Each core contains 256 clusters of transistors and thousands of random access memory (RAM) elements. Compared to modern electronics, the total computing power is tiny. "It's a worm-scale chip," says Dharmendra Modha, the manager of IBM's cognitive computing program.
So what's new then?
The difference is the way the transistors and memory are wired together. In a conventional computer, the computational elements are mostly in the central processing unit, while the RAM sits off to one side. In the cognitive chips, the computational elements and RAM are wired together.
How is this like a brain?
The theory is that the computational components act as "neurons," while the RAM units act as the "synapses," which connect the neurons together. In a real brain, neurons receive electrical pulses from synapses until a sufficient voltage builds up across their membrane. The neuron then discharges, sending signals to other neurons via the synapses.
In the cognitive chip, a pattern of signals from the RAM can cause a computational element to carry out a simple operation. The result goes to another RAM synapse, which can send signals to other computational neurons. In this way, the chip is "inspired" by the brain's architecture, Modha says.
What's the advantage of building chips like these?
The main benefit is decreasing power consumption. Because the memory and computation are intermingled, less energy is wasted shuffling electrons back and forth. The new chips have the potential to be orders of magnitude more efficient than a conventional computer, according to Rajit Manohar, an electrical and computer engineer at Cornell University in Ithaca, N.Y., and member of the DARPA collaboration.
In terms of speed, it's believed that the chips will be particularly good at crunching certain kinds of problems, such as pattern recognition, but they may not be as good as a regular computer at handling other tasks.
What comes next?
The success of these first chips has led DARPA to award $21 million for further development. Meanwhile, IBM researchers are constructing algorithms that are adept at speech recognition and problem solving. Their Watson supercomputer, for example, which was victorious over human competitors on the American quiz show Jeopardy! (see "Quiz-playing computer system could revolutionize research"), currently requires a room full of power-hungry processors. Manohar hopes that low-power cognitive chips could do the job just as well. "Our goal is basically to meet in the middle," he says.
With additional reporting by Mitch Waldrop.
This article is reproduced with permission from the magazine Nature. The article was first published on August 18, 2011.
See what we're tweeting about
22 Comments
Add CommentSounds like a fancy word for a chip with a bunch of processors each with its own cache.
Reply | Report Abuse | Link to this"What's inside?"
Reply | Report Abuse | Link to thisRetired Keebler Elves. Natch.
That might be true if the processors and cache were built using von Neumann architecture. I suspect not, and the "RAM" is not storing data or programs, but is acting simply as a buffer which adds "voltages" from the "pre-synaptic" processors. When that "voltage" reaches a threshold, then the "post-synaptic" processor is activated in some way. That would be a radically different architecture than current stored program computers utilize.
Reply | Report Abuse | Link to thisI agree. In I think clearer (not too technical) terms, 'computational' processor instruction elements and their large on-chip dedicated local cache memories are connected to larger, shared off-chip memories through a 'high speed' bus connection.
Reply | Report Abuse | Link to thisAccesses to that shared large bus connected memory are much slower than accesses to the dedicated cache memory because access to the bus is shared by not only potentially multiple primary processors but by dedicated graphics processors, peripheral devices (disk drives, etc.) producing access delays. However, software running on multiple processors can all access common data in the shared memory.
I think what's being described here is that the faster on-chip memories (caches) are being interconnected to multiple processors so that their data shared. The disadvantage of this approach is that the interconnections enable processor 'traffic' delays as multiple processors may be attempting to simultaneously access the shared memory. As a result, I expect that access to the new higher speed shared memory to be slower than access to dedicated processor cache memory.
At any rate, there is no free lunch and I didn't read of any system software that enables the programming of the shared memory interconnections to provide any real 'cognitive' ability.
The IBM states that each chip contains a ""neurosynaptic core" with integrated memory ("replicated synapses"), computation ("replicated neurons") and communication ("replicated axons")."
The IBM press release upon which this article is based is saturated with time tested IBM marketing hyperbole.
This is an experimental architecture with, I think, very little software support. The IBM press release states:
"IBM has two working prototype designs. Both cores were fabricated in 45 nm SOI-CMOS and contain 256 neurons. One core contains 262,144 programmable synapses and the other contains 65,536 learning synapses. The IBM team has successfully demonstrated simple applications like navigation, machine vision, pattern recognition, associative memory and classification."
I think this is an interesting line of research, but not much has been achieved beyond the paltry $21M grant from DARPA and a lot of public clamor.
IBM Research's project leader is quoted as saying, "Imagine traffic lights that can integrate sights, sounds and smells and flag unsafe intersections before disaster happens or imagine cognitive co-processors that turn servers, laptops, tablets, and phones into machines that can interact better with their environments."
He's inspired...
I probably have this wrong, but I thought of it as one file with (A-Z) instead of a cabinet with 26 files (A B C... Z).
Reply | Report Abuse | Link to thisIf I was looking for something, it's faster when a file is more direct.
A step down and slower chip.
Always enjoy reading your comments.
Reply | Report Abuse | Link to thisI like the way you think.
That sounds right to me. Just because the memory is located on-chip does not mean that it is a cache. The term "cache" is not used to describe where a memory is located, but what it does. As much as I like to make sweeping generalities, I think it is safe to say that caches perform three functions transparently to software (caveats apply): 1. reduce read latency; 2. reduce write latency; 3. increase bandwidth. The memories on these chips are not for performing any of the above functions, or any cache function for that matter, since it looks like that they are for storing synapse state (the strength of the synapse is stored so that the neuron can decide what strength it should have the next cycle round based on its previous state and the new inputs).
Reply | Report Abuse | Link to thisThanks for your kind remarks! Years ago I did work to decipher IBM's technological developments for several decades. That's not to say that my assessment is absolutely correct - I usually had much more information to work from than a press announcement, but I found no real technical information about this research project.
Reply | Report Abuse | Link to thisUsing your file cabinet analogy, I think you could describe PC architecture (for example) to be somewhat like a file cabinet with a file clerk for each drawer (A-F, G-K...), whereas this new chip has a number of file clerks who can access any file, as long they're not trying to access the same file at once. However, the clerks will also no longer be required to work from a serial list of file requests: they will be able to scan the list to find a request that's not yet being serviced - when the operating system software is ready...
IBM's new chips are neural networks, so the cabinet and file analogy isn't quite right. My (probably inaccurate) understanding of these networks is that they mimic the properties of real neurons and how they are connected. These differ from the traditional von Neumann architecture in that there is no program to execute or data to retrieve and store (or so I am led to believe from reading various articles). The function of a neural network depends entirely on what weights the neurons assigns to its inputs, what the inputs are, what the neurons do, and how the outputs connected to other neurons. Such a circuit is unparalleled in capability and performance when it comes to pattern matching since the computation is different from that of a conventional computer (weights determine outputs, not a series of instructions evaluating conditions in a serial manner), is distributed amongst the neurons, and the communication between them is parallel.
Reply | Report Abuse | Link to thisIBM's press release contains several contradictory statements that lead to confusion and encourage speculation. It states:
Reply | Report Abuse | Link to this"While they contain no biological elements, IBM’s first cognitive computing prototype chips use digital silicon circuits inspired by neurobiology to make up what is referred to as a "neurosynaptic core" with integrated memory (replicated synapses), computation (replicated neurons) and communication (replicated axons)."
"IBM has two working prototype designs. Both cores were fabricated in 45 nm SOI-CMOS and contain 256 neurons. One core contains 262,144 programmable synapses and the other contains 65,536 learning synapses. The IBM team has successfully demonstrated simple applications like navigation, machine vision, pattern recognition, associative memory and classification."
"IBM’s overarching cognitive computing architecture is an on-chip network of light-weight cores, creating a single integrated system of hardware and software. This architecture represents a critical shift away from traditional von Neumann computing to a potentially more power-efficient architecture that has no set programming, integrates memory with processor, and mimics the brain’s event-driven, distributed and parallel processing."
I interpret the reference to "a critical shift away from traditional von Neumann computing" does not indicate that there is no programming, but that processing is not necessarily serialized.
They also state that one prototype "contains 262,144 programmable synapses" which definitely indicates that some form of programming is employed. However, the term "programmable synapses" seems to contradict the earlier description of synapses as "integrated memory". In von Neumann computing architectures programs may be loaded into memory but they are executed by instruction processing units. Perhaps some announcement writer was confused...
The other prototype is said to contain "65,536 learning synapses" does not explain how those synapses or "integrated memory" "learn" anything. Again, I suspect some confusion or at least lack of clarity. I'd conclude that it does contain some programs that perform some form of learning, probably intended to be not user programmable - but I'm only guessing here.
Readers of IBM's announcement are encouraged to guess, imagine and speculate by the intermixing of poorly defined neurological and computational terminology. Your guesses may be just as good as mine, or you may have been intentionally misled by the press release.
Not to discount your explanations entirely, but it is very interesting to me that nowhere does IBM use the term 'neural networks' to describe these chips. I suspect that they think they are different in some fundamental way, although they are also modeling their processing architecture on neurons.
Reply | Report Abuse | Link to thisOn thing though, the press announcement states:
"IBM’s overarching cognitive computing architecture is an on-chip network of light-weight cores, creating a single integrated system of hardware and software."
Clearly, some (I think significant) programming is involved, even if no traditional programming may be required of the user (I'm guessing here).
IBM's press release is indeed a mess. You are right that there are confusing inconsistencies in IBM's claims.
Reply | Report Abuse | Link to thisBut I don't think that the mention of programmable synapses necessarily implies that the chips are programed (as in provided with a set of instructions). Rather, as I hinted to in my previous comment, it is probable that the function of the neural network is defined by controlling which synapses can occur, and if they can, the strength of the synapses. I'd guess that IBM's use of "programmable" in this context is similar to that of programmable logic devices. In the latter context, the logic device is not given a set of instruction to execute, but a series of memory elements have their bits set to define what the generic logic device becomes.
The mention of learning synapses in the press release definitely needs clarification. What's the relation to programmable? My guess is (and I am way out of my depth here) that these synapses enable the neural network to learn independently without human intervention (without initially defining its function) by pattern matching and then reconfiguring itself according to stimulus. How this differs from a programmable synapse is unclear. I suspect it is the wiring rather than the basic RAM element.
Regarding whether IBM's chips implement neural networks, I won't say that I'm certain that they are, but they look very much like neural networks to me. However, taking a closer look at the press release and several other articles about them, there does seem to be hints that they are not, namely the fact that these chips seem to mimic synapses; something that some of the neural networks I have looked at in the past (albeit not very closely) lack, or at least implement differently. (Which may be due to their particular application -- branch prediction for pipelined CPUs).
Reply | Report Abuse | Link to thisLastly, there is another possible explanation for IBM's avoidance of term "neural network" in their press release -- marketing. I've seen many companies that call their research or products, which are clearly developments of existing technology, by different names to stand out from the rest.
I guess the only way to get to the bottom of this is when the IBM researchers publish their results or when a detailed follow up article is written.
I agree - we may have to wait for those 'cognitive traffic lights' to separate the architecture from the hyperbole.
Reply | Report Abuse | Link to thisI think a lot of the confusion stems from the vagueness of the word "learning" and from our long-held paradigm of von Neumann architecture. In that paradigm, it is difficult to grasp what a neural network or this cognitive chip really is and does, and what is possible with it.
Reply | Report Abuse | Link to thisIt is just the beginning though, and I'm very confident that IBM and others will take leaps and bounds in just the next couple years. There will be applications of these ideas in 5 to 10 years that we cannot even begin to imagine right now.
Sounds like DARPA is trying to build Skynet ...
Reply | Report Abuse | Link to thisI found a more detailed article on these chips in EE Times (http://www.eetimes.com/electronics-news/4218883/IBM-demos-cognitive-computer-chips) that reveals some of the high-level workings of these chips.
Reply | Report Abuse | Link to thisSeveral things that I suspected these chips have and do were close to the mark. For instance, it really looks like these chips are neural networks:
"The neuron integrates over these inputs until a threshold is exceeded, at which point it fires a pulse down its output axon, which is weighted by the synapses connected to other neurons. Pattern recognition is accomplished by the synapses "learning" which connections are used most often, which causes them to grow stronger, while seldom used connections wither away. In this way, the neural network closes the sensory-motor feedback loop, since once a pattern is recognized from the sensor inputs, the output motor neurons mobilize a response."
...And that the difference between programmable synapses and learning synapses is indeed in how they are wired:
"The only difference between the two test cores was in their use of the interconnecting crossbar array, either as 256k pre-programmable synapses, or as 64k learning synapses."
The most interesting thing is how simple these chips seem to be. The complicated axon and dendrites in a biological brain has been replaced by a 2D crossbar, and it looks like that there is just one type of neuron (using the Hebb rule) integrated with some state (the memory) that controls the crossbar that implements the synapse. The biological behavior of neurons is analog, but IBM mimicked them using a purely digital design. The genius is probably in designing the custom hardware and in getting the chips to real-life functions.
The architecture looks like it can be hosted on a programmable logic device. I wonder how it compares to existing PLD-based neural networks.
Excellent find – a news article with some technical substance!
Reply | Report Abuse | Link to thisI'd be most happy to award you my rarely used library of IEEE Transactions on Pattern Analysis and Machine Intelligence issues, 1981-1985, in fair condition – FOB.
I must maintain that the primary difference between the two prototype chips is in their "pre-programmability", despite the referenced EETimes article's statement:
"The only difference between the two test cores was in their use of the interconnecting crossbar array, either as 256k pre-programmable synapses, or as 64k learning synapses."
That statement actually indicates three distinguishing features - the number of "synapses" configured to the crossbar, their memory capacity and whether they are "pre-programmed" or "learning".
From an electrical engineer's standpoint, the most obvious physical distinction is the number of nodes attached to the crossbar switch. Functionally, I suggest the primary distinction is their pre-programmability vs learning mode of operation.
Again, IBM's press announcement states:
"IBM's overarching cognitive computing architecture is an on-chip network of light-weight cores, creating a single integrated system of hardware and software."
The EETimes article states:
"IBM envisions its cognitive computers solving a wide variety of applications in navigation, machine vision, pattern recognition, associative memory and classification. So far it has taught one to recognize a cursive letter "7" regardless of in whose handwriting. The other has learned to play (and win against humans) at the game "Pong.""
(continued)
Reply | Report Abuse | Link to thisThe EETimes article addresses the von Neumann references with a statement by IBM’s project leader, Dharmendra Modha:
"Each neuron fires in order to communicate with the other neurons which fully integrates memory with processor, instead of separating them like von Neumann."
While von Neumann’s computer architecture specifies that programs and data be stored in a bus attached memory, separate from the instruction execution unit, inferring an apparent fundamental bottleneck imposing contention between data and instruct fetch, he did not imagine virtual storage, processor cache memories or instruction prefetch, all of which mitigate against the contention between processor instruction execution and data access in separate memory in von Neumann’s original architectural specification. IMO, all this complaining about von Neumann’s architecture is moot – the actual performance of various modern processor architectures will determine their usefulness and applicability, not academic aesthetic issues. I expect that large scale ‘cognitive’ ‘neural network’ computer designs may be advantageous for some applications like automated car parking and even driving, and airplane fly-by-wire autopilots with automated landing ability. Whether they can soon be economically produced for any large volume applications is a different question.
The EETimes article states:
"The eventual goal is to create a brain-like 10 billion neuron, 100 trillion synapse cognitive computer with comparable size and power consumption to the human brain."
I expect a lot of 'learning' will be required to make it functionally comparable to a human brain...
A closely related precursor article is referenced by the EETimes report, which also provides some very interesting background:
http://www.eetimes.com/electronics-news/4086220/IBM-achieves-accurate-brain-simulation
Thank you for offering an award. However I'm going to have to decline the offer, since I don't think I can use them to their full potential.
Reply | Report Abuse | Link to thisMoving on to your points, in regard to the differences between the two chips, I don't think that having fewer learning synapses suggests that there must be differences in the sub-circuits. The wires connecting the circuits could be connected differently to give additional memory to a synapse without adding more memory, much like the gate arrays of yore. It would be interesting to see a detailed account of these differences.
Regarding the pursuit non-von Neumann architectures, I think there is more than to it than what has been said in the news. Computer organization developments can only hide a finite amount of the problems that von Neumann architectures inherently introduce. For instance, adding caches does not completely prevent the need to load or store data from and to memory (ignoring compulsory cache misses caused by the cache needing to have encountered the requested data first), unless the running program and its data are small enough to fit, which isn't realistic.
Whether the von Neumann architecture is an issue or not in regards to bandwidth and latency, I think there is another issue that neural network researchers have with the von Neumann architecture: Since the memory is separated from the computation, there needs to be a way of structuring the data in the memory; which requires that accesses to the data need housekeeping overhead even if we were to assume an unrealistic one-cycle penalty for memory accesses. Once again computer organization can minimize this penalty (by overlapping computation with memory access), but only at the cost of complexity and power. This is probably why the IBM researchers place so much importance on associative memory: Biological neural networks do this (to my understanding) and combines the storage, data organization, and computation in the same structure. It's more capable, dense, and less power-hungry, which is what IBM would need to build a shoe box-sized cognitive processor that rivals the human brain.
I've also found more background on IBM's efforts. The IBM researchers have written an insightful article recently published in Communications of the ACM that details much of their general approach to cognitive computing and related earlier work. The magazine offers free access to the article: http://cacm.acm.org/magazines/2011/8/114944-cognitive-computing/fulltext
As I understand, in a modern, multitasking von Neumann computer systems a processor instruction is executed for most machine cycles - there is effectively no idle time spent waiting for instruction fetch or memory access when active tasks are ready. The only idle time that occurs is when there are no tasks ready - most often due to requests to peripheral storage devices or people.
Reply | Report Abuse | Link to thisEven modern large scale disk subsystems are configured with high speed optical channels and many processors and and extremely large bus attached 'cache' memories to minimize processor delays when accessing large databases.
Both processors and disk subsystems, under the control of operating system software, prefetch both data and instructions. Moreover, they most often swap in very large previous 'referenced memory sets' which can include large pools of unreferenced previously prefetched data.
As I understand, neural network computers have not been constructed or configured to support access to large databases or internet searches, for example, but dedicated to very specific signal processing applications. The disadvantages you mention for von Neumann computers are all specified in relation to there use in neural network systems. In my opinion, for all general purpose computing applications this is a moot point, since neural network architectures are not generally suitable.
The inherent disadvantage of von Neumann's processor bus attached memory, in early computers requiring idle processor time for memory access, offers the advantage of overlapping instruction execution and prefetch of data and programs in today's sophisticated processor architectures. That modern general purpose computer processor designs are not not optimized for neural network applications has no bearing on their performance in general purpose computers.
I doubt that neural network designs will be implemented for general purposes in this century, although they may become widely used for very specific, mostly signal processing, applications, such as cognitive traffic control systems.
IMO, neurological science has only understood biological neuronal systems at the very lowest levels. While humans can process information very generally, employing an enormously varied and adaptable range of methods, I suggest that that capability is provided by an enormously complex system of 'software' constructed from the simplest functions of neurons. In fact I doubt that the methods of data encoding, whether binary or analog thresholds, make any difference in the final product.
This new chip has a very interesting architecture. It's about time that someone has finally started making these (could have been thought of a long time ago). Can lead to very interesting and imaginative systems.
Reply | Report Abuse | Link to thisBut it's not really like the way the brain works. Like all ordinary computers, it still has a clear distinction between CPU and memory. The brain does not. It doesn't have any cells or other devices that just store information. It's workings, and what we think of as information, are all in the connectivity. You get (what seems to us like) information from networks of interconnected neurons, usually very large networks -- with, ultimately, connections to sensory inputs.