Dharmendra S. Modha
Manager, Cognitive Computing, IBM Almaden Research Center
dmodha AT almaden DOT ibm DOT com
"...for their contributions to creating Adaptive Replacement Cache and incorporating it in IBM's Enterprise Storage Server products.
Caching is used widely in storage systems, databases, web servers, middleware, processors, file systems, disk drives, RAID controllers, operating systems and in numerous other applications. For nearly four decades, the Least Recently Used (LRU) algorithm and its variants have remained the popular class of cache replacement policies. A long-standing question in cache management has been: Is it possible to improve on LRU across a wide range of workloads and cache sizes without incurring excess overhead or requiring workload-specific pre-tuning? Hundreds of attempts have been made, most significantly, FBR, LRU-2, 2Q, LRFU and MQ. However, until now, none has been universally successful. Adaptive Replacement Cache (ARC) dynamically adapts between recency (LRU) and frequency (Least Frequenty Used (LFU)) to achieve higher cache hit rates, which imply better performance for a server or application. ARC was successfully transferred to IBM's Enterprise Storage Server (ESS) products (DS 6000 and DS 8000) and was included in the product announcements as a significant innovation. On mixed random and sequential workloads, ARC was found to notably increase the cache hit rate of ESS on random workload (almost 2x in some cases) without impairing the hit rate on sequential workloads."
"While many algorithms have been proposed for page replacement in caches over the years, Least Recently Used (LRU), one of the oldest algorithms, has remained largely popular given its simplicity, ease of implementation and low overhead. The Adaptive Replacement Cache (ARC) algorithm proposed in this paper is an elegant scheme that dynamically adapts itself to the changing characteristics of workloads without requiring any user-defined tuning parameters as many other attempts to improve on LRU have. The paper clearly covers the work done in cache replacement algorithms in the past, builds up the motivation for ARC and clearly describes the self-tuning nature. The results brought out in the paper convincingly show ARC consistently outperforming LRU. Like LRU, the algorithm has constant time complexity with regard to cache size and is simple to implement. It is expected that ARC will be used in storage systems as well as in several other applications that utilize caches such as virtualizers, databases and file systems. The ARC paper was presented in FAST '03, a key conference for Storage Systems and subsequently in Computer. A follow-on to ARC appeared in FAST '04."
"... for their contributions to the ideas and algorithms used in the eClassifier text-analysis tool and for their roles in its implementation into a variety of IBM Global Services (IGS) applications.
The eClassifier system integrates technologies for classification, taxonomy management, trend detection, document feature understanding and visualization into an innovative, interactive tool and runtime for exploring large collections (e.g. millions of documents) of text. It has been used within several important IGS HelpDesk systems; several IGS strategic outsourcing engagements; several IGS productivity-improving systems, and, most recently, the Lotus Discovery Server, Release 2, Knowledge Management product."
"The prize paper provides both a theoretical underpinning and practical instances of codes which deal with the problem of quasi-catastrophic error propagation in Maximum Transition Run (MTR) codes. The unifying theory presented led to an exhaustive characterization of these codes and revealed a connection between the conventional modulation codes used in disk drives and the recently discovered MTR codes. Instances of the specific codes described are already used in the digital recording industry. Finally, the theoretical framework provided led to the design of the combined MTR/parity codes that are being used in all current IBM hard-disk drives, including IBM’s Microdrive."
Dharmendra Modha, head of the cognitive computing group at IBM's Almaden research lab, is leading a "quest" to "understand and build a brain as cheaply and quickly as possible". Last year, his group succeeded in simulating a rat-scale cortical model - 55m neurons, 442bn synapses - in 8TB memory of a 32,768-processor IBM Blue Gene supercomputer. The key, he says, is not the neurons but the synapses, the electrical-chemical-electrical connections between those neurons. Biological microcircuits are roughly essentially the same in all mammals. "An individual human being is stored in the strength of the synapses."
Modha doesn't suggest that the team has made a rat brain. "Philosophically," he writes on the subject, "any simulation is always an approximation (a kind of 'cartoon') based on certain assumptions. A biophysically realistic simulation is not the focus of our work." His team is using the simulation to try to understand the brain's high-level computational principles.
Another leading speaker, Dharmendra Modha, manager of cognitive computing at IBM's (NYSE: IBM) Almaden Research Center, reminded the audience that "there is no computer today that can even remotely mimic the abilities of the human mind." Yet his team at IBM has set out to do just that.
"Our group has a modest goal -- that is, to understand and build a brain as quickly and cheaply as possible," he joked.
For three reasons, according to Modha, the time is now right to reverse-engineer the brain: Neuroscience has matured, and it is possible to trace neurons in the brain; Supercomputing is ready to take it on (Modha has already simulated a rat brain); and Nanotechnology is moving quickly.
By 2018, someone using an IBM supercomputer may be able to simulate a human brain in real time, Modha estimated. That's not exactly the singularity, but surely a step on the way.
IBM was a pioneer in the field and today continues to invest heavily in AI research. Dharmendra Modha, a scientist in the company's California research laboratory is working on cognitive computing, which he defines as a computer model that simultaneously exhibits characteristics seated in the human brain, including perception and emotion.
His aim is to discover how the brain works, not how the mind works, he is quick to emphasise. Last year, his group achieved a milestone by managing to simulate the operation of a mouse brain on an IBM Blue Gene supercomputer. He notes: "We deployed the simulator on a 4096 processor Blue Gene/L supercomputer with 256 megabytes of memory per processor. We were able to represent 8m neurons and 6,300 synapses (connections) per neuron in the one terabyte main memory of the system." There will be, of course, a considerable time lag before the benefits of this research are seen in actual products.
Mr Modha thinks it could be 10 years before cognitive computing of the kind he is working on makes its debut in productivity and security systems. It is, however, a giant leap from 1956 when an IBM supercomputer of the day simulated the firing of a mere 512 neurons.
As Mr Modha of IBM says of his work in cognitive computing, the technology will manifest itself in ways which today we cannot even begin to imagine.
"The researchers at Almaden are technological poets, thinkers inspired by a muse or a distraction, the sort of visionary people who ask "what if" about possibilities that might leave the rest of us asking "huh?" ... They are a different breed — the kind of people who make me wonder, "Why?"
Why are they trying to reverse-engineer the human brain and build a computer that will function the way our minds do? What is it about someone that compels them to set out into the unknown to search for an answer that might not exist?
Dharmendra Modha's work on building a massive computer that will simulate the workings of a human brain might never add to IBM's bottom line. He happens to think it will, but that's not why he does what he does. "I see a possibility," he says of his project combining neuroscience and computer science. "And I feel that if I don't manifest that possibility into reality, maybe nobody will." A human brain-like computer is years away and Modha doesn't know whether he will ever get there.
But at the top of the hill in Almaden, it's not about the destination. It's about what you discover along the way."
In a letter published a few weeks ago in the journal Science, 10 scientists said that a Decade of the Mind would help us understand mental disorders that affect 50 million Americans and cost more than $400 billion a year. It might also aid in the development of intelligent machines and new computing techniques. A breakthrough in mind research, the scientists wrote, could have "broad and dramatic impacts on the economy, national security, and our social well-being."
It could be the most ambitious computer science project of all time. At IBM's Almaden Research Center, just south of South Francisco, Dharmendra Modha and his team are chasing the holy grail of artificial intelligence. They aren't looking for ways of mimicking the human brain, they're looking to build one-neuron by neuron, synapse by synapse.
"We're trying to take the entire range of qualitative neuroscientific data and integrate it into a single unified computing platform," says Modha. "The idea is to re-create the 'wetware' brain using hardware and software."
The project is particularly daunting when you consider that modern neurology has yet to explain how the brain actually works. Yes, we know the fundamentals. But we can't be sure of every biological transaction, all the way down to the cellular level. Three years into this Cognitive Computing project, Modha's team isn't just building a brain from an existing blueprint. They're helping to create the blueprint as they build. It's reverse engineering of the highest order.
Their first goal is to build a "massively parallel cortical simulator" that re-creates the brain of a mouse, an organ 3,500 times less complex than a human brain (if you count each individual neuron and synapse). But even this is an undertaking of epic proportions. A mouse brain houses over 16 million neurons, with more than 128 billion synapses running between them. Even a partial simulation stretches the boundaries of modern hardware. No, we don't mean desktop hardware. We're talkin' supercomputers.
So far, the team has been able to fashion a kind of digital mouse brain that needs about 6 seconds to simulate 1 second of real thinking time. That's still a long way from a true mouse-size simulation, and it runs on a Blue Gene/L supercomputer with 8,192 processors, four terabytes of memory, and 1 Gbps of bandwidth running to and from each chip. "Even a mouse-scale cortical simulation places an extremely heavy load on a supercomputer," Modha explains. "We're leveraging IBM's technological resources to the limit."
Written with ordinary C code, this initial simulation is a remarkable proof of concept. As neuroscience and computing power continue to advance, Modha and his team are confident they can build cortical simulators of even greater complexity. And as they do, they hope to advance neuroscience even further, learning more and more about the inner workings of the brain and getting closer and closer to their ultimate goal.
Once they've simulated a mouse brain in real time, the team plans on tackling a rat cortex, which is about three and a half times larger. And then a cat brain, which is ten times larger than that. And so on, until they've built a cortical simulator on a human scale.
What's that good for? Anything and everything. "What we're seeking with cognitive computing is a universal cognitive mechanism, something that can give rise to the entire range of mental phenomena exhibited by humans," says Modha. "That is the ultimate goal."
"The scientists ran a "cortical simulator" that was as big and as complex as half of a mouse brain on the BlueGene L supercomputer. In other smaller simulations the researchers say they have seen characteristics of thought patterns observed in real mouse brains. Now the team is tuning the simulation to make it run faster and to make it more like a real mouse brain.
Life signs Brain tissue presents a huge problem for simulation because of its complexity and the sheer number of potential interactions between the elements involved. The three researchers, James Frye, Rajagopal Ananthanarayanan, and Dharmendra S Modha, laid out how they went about it in a very short research note entitled "Towards Real-Time, Mouse-Scale Cortical Simulations". Half a real mouse brain is thought to have about eight million neurons each one of which can have up to 8,000 synapses, or connections, with other nerve fibres. Modelling such a system, the trio wrote, puts "tremendous constraints on computation, communication and memory capacity of any computing platform". The team, from the IBM Almaden Research Lab and the University of Nevada, ran the simulation on a BlueGene L supercomputer that had 4,096 processors, each one of which used 256MB of memory. Using this machine the researchers created half a virtual mouse brain that had 8,000,000 neurons that had up to 6,300 synapses. The vast complexity of the simulation meant that it was only run for 10 seconds at a speed ten times slower than real life - the equivalent of one second in a real mouse brain.
On other smaller simulations the researchers said they had seen "biologically consistent dynamical properties" emerge as nerve impulses flowed through the virtual cortex. In these other tests the team saw the groups of neurons form spontaneously into groups. They also saw nerves in the simulated synapses firing in a ways similar to the staggered, co-ordinated patterns seen in nature. The researchers say that although the simulation shared some similarities with a mouse's mental make-up in terms of nerves and connections it lacked the structures seen in real mice brains.
Imposing such structures and getting the simulation to do useful work might be a much more difficult task than simply setting up the plumbing. For future tests the team aims to speed up the simulation, make it more neurobiologically faithful, add structures seen in real mouse brains and make the responses of neurons and synapses more detailed."
Dharmendra Modha, manager of cognitive computing at the IBM Almaden Research Center, believes that this transition is part of the next hundred years of technology leadership. The current data paradigm is structured data management, but in "real life" we deal with unstructured data (of which emotion is a single element). That is, we recognize a friend's face no matter how she's dressed or despite her mood; we detect patterns with a large amount of sensory data and we act appropriately. According to Modha, "We are at a crucial juncture in history where two trends are converging: the tremendous availability of computational power, and the amount of neuroscience knowledge that has exploded over the last few years."
"Cognitive computing is about engineering the mind by reverse engineering the brain," Modha explains. If the brain is the biological wetware, a collection of neurons and a set of interconnection between the neurons, then neuron by neuron and synapse by synapse, computer science is putting together the architecture of the mind. Thus, IBM Research is simulating collective dynamics; researchers are studying how a very large population of interconnected neurons evolve in order to characterize them mathematically and then to synthesize them for harnessing in synthetic computations. Says Modha, "Today, on a 4096 processor Blue Gene supercomputer with 256MB of memory per CPU, we are able to simulate 8 million neurons and 50 billion synapses, 10 times slower than real-time." That's just a start, but then the Cognitive Computing project is new. A mouse brain has 8 million neurons in one hemisphere and 64 billion synapses.
This research aims to assemble the knowledge to build novel perception machines, or novel sensory systems. Business issues, says Modha, will eventually involve visual recognition, pattern detection in the stock market, or inventory management in neurological devices—systems that underlie a wide variety of applications.
It's all very blue sky, of course, as research is supposed to be. But those science-fiction computers-get-emotion stories spoke of both opportunity and horror. What dystopia do such researchers worry that they may unleash? Says Modha, "At this stage in science and technology, the power of possibilities overwhelms me more than the fear of misuse."
"Though most of the truly futuristic projects are probably years from the commercial market, scientists say that after a lull, artificial intelligence has rapidly grown far more sophisticated. Today some scientists are beginning to use the term cognitive computing, to distinguish their research from an earlier generation of artificial intelligence work. What sets the new researchers apart is a wealth of new biological data on how the human brain functions." ...
"There is a new synthesis of four fields, including mathematics, neuroscience, computer science and psychology," said Dharmendra S. Modha, an I.B.M. computer scientist. "The implication of this is amazing. What you are seeing is that cognitive computing is at a cusp where it's knocking on the door of potentially mainstream applications."
"The conference, which will continue Thursday, was organized by Dharmendra S. Modha a computer scientist at IBM's Almaden Research Center. Modha said the gathering comes at a time when scientists are closing in on an understanding how the processes of the mind -- perception, intelligence, memory, learning and ultimately consciousness -- arise from the physical structures of the brain, and by extension how to model these functions in electronics. "Cognitive computing is about engineering the mind by reverse-engineering the brain," Modha said.
"New caching technology from IBM Research called Adaptive Replacement Cache (ARC) is designed to help clients achieve dramatically greater throughput and faster response times than previous IBM TotalStorage Enterprise Storage Server 800 systems. ARC incorporates autonomic, self-optimizing technology and a more efficient and effective method for the widely used process of replacing data pages in computer cache memories. The breakthrough technology, available in both the DS6000 and DS8000 series, dynamically optimizes the storage system's performance for both sequential and randomly accessed workloads."
"The DS8000 series products feature three IBM Research-developed software innovations in caching that are designed to work together to deliver dramatically greater throughput and faster response times for a wide range of real-life workloads. A new prefetching feature preloads and manages sequential data in the cache so it always contains the needed data. This prefetching feature also enhances the previously announced Adaptive Replacement Cache technology that integrates and balances both of the critical caching and prefetching functions. The third innovation is designed to eliminate undesirable interactions between the read- and write-cache management while still allowing both caches to beneficially share memory resources."
         |