================================================================

Global Technology Outlook (GTO) 

C. Mohan
IBM Fellow and IBM India Chief Scientist

Block C, 6th Floor, A Wing
Embassy Golf Links (EGL) Business Park
Off Indira Nagar - Koramangala Intermediate Ring Road
Bangalore 560 071
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

The Global Technology Outlook (GTO) is IBM Research’s vision of the future for information technology (IT) and its impact on industries that use IT. It highlights emerging software, hardware, and services technology trends that are expected to significantly impact the IT sector in the next 3-7 years. In particular, the GTO identifies technologies that may be disruptive to an existing business, have the potential to create new opportunity, and can provide new business value to our customers. A number of architectural changes are occurring – all of which are expected to evolve into a new enterprise environment with new ways to deploy information technology. The 2008 GTO focuses on five topics: Core Computer Architectures, Internet Scale Data Centers , Community Web Platforms, Real World Aware and Enterprise Mobile. In this talk, I will share the GTO 2008 findings with the audience. Following are some details on the five GTO 2008 topics.

Reinventing the way computer systems are built Core Computer Architectures examines the convergence of technology, hardware and software that will need to evolve to maintain peak computing performance and respond to the changing needs of today’s – and tomorrow’s – business environments. A significant evolution of systems and software across several market segments, cost and power optimized systems, high-end servers and specialized domains will have to occur to take full advantage of these new computer architectures.

Answering business needs with a “cloud” The engine rooms of our information – our data centers – are wildly distributed, siloed and sub-optimized around the world. New Internet Scale Data Centers are emerging to address this issue. These large data centers can expand and grow rapidly. They will be more efficient and more interconnected – inside and outside their companies – because they will have the ability to access applications from common infrastructures, often referred to as cloud computing. This will provide a tremendous increase in flexibility for large companies because they now will be able to quickly and easily take advantage of IT tools like web delivery, business analytics, and business process services to help grow their business and better serve their customers.  

Social – and data – networking for the enterprise Community Web Platforms have introduced new forms of content contribution, leading to more users – because it’s easier to share information through these new tools – and more data – because users are finding more value in the collaborative nature of these platforms. As these new business models evolve, additional new capabilities will emerge to help sustain and grow the features and functionality that companies will require to take advantage of these new technologies.  

Real time information processing and analysis Real World Aware is all about a new class of applications that will move business beyond traditional analytics to a place where all data – past and present, from inside and outside a company – can be streamed, processed and analyzed in real time. New systems are emerging to support this trend, and business applications will need to be extended so companies can use these new technologies.  

Doing business anywhere, anytime Business requirements and technology advances are driving tremendous change across the Enterprise Mobile space. In many regions, mobile devices are becoming an increasingly viable alternative to PCs. With the rapid rise of mobile business, companies will be able to do more than just give their employees the option to access email remotely. They will be able to give them access to critical data and applications – anywhere, anytime – because the infrastructure and security features will be there to support them.

================================================================

Global Technology Outlook (GTO) 

C. Mohan
IBM Fellow and IBM India Chief Scientist

Block C, 6th Floor, A Wing
Embassy Golf Links (EGL) Business Park
Off Indira Nagar - Koramangala Intermediate Ring Road
Bangalore 560 071
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

The Global Technology Outlook (GTO) is IBM Research’s vision of the future for information technology (IT) and its impact on industries that use IT. It highlights emerging software, hardware, and services technology trends that are expected to significantly impact the IT sector in the next 3-7 years. In particular, the GTO identifies technologies that may be disruptive to an existing business, have the potential to create new opportunity, and can provide new business value to our customers. The 2007 GTO focuses on four topics: A New Era in Systems Design, Nanotechnology Update, Digital Communities, Managing Business Integrity. In this talk, I will share the GTO 2007 findings with the audience.

Here are some details on the four GTO 2007 topics:

A New Era in Systems Design: In this topic, we are looking at the trends that are driving the evolution of microprocessor design today. Consistent with GTO topics of the past, frequency scaling for CPUs has slowed dramatically due to problems with power dissipation. Performance increases had been obtained in the past by increasing the CPU frequency. In this power-constrained environment, increased performance is obtained through the utilization of multi-core architectures. While traditional scale-up performance is still needed for some workloads (such as database), a growing number of new workloads can be addressed through the use of multiple small processors running in parallel. The engineering cost of designing leading edge CPUs has been increasing. It is not unusual for state-of-the-art CPUs to require 1000 to 2000 engineering years of design effort. As the underlying silicon technology continues to deliver transistor density, the cost of designing to use all of the available transistors will continue to increase, unless, the industry undergoes a fundamental change in the way CPUs are designed. This radical change is to introduce modular design techniques to greatly enable design IP reuse. Additionally, design techniques will have to include resiliency at the circuit and micro-architecture level, as well as fundamental understanding and control of variability at the device technology level. Finally, with the increase in parallel small processors, the levels of parallelism that were previously only seen within the high performance computing domain will now become mainstream. To utilize the available hardware resources, software will need to respond at all levels of the stack.

Nanotechnology Update : This year, the technology update is broken down into two sections.  The first section covers the topics traditionally covered in the technology update – the direction that the base silicon technology is headed. The second section is a broad survey of emerging nanotechnologies, looking at areas beyond our traditional IT focus. The core silicon technology upon which all systems are built will continue to evolve for at least another decade, taking us deep into the nano-scale regime. How will this be done? Just as we always have, we will continue to push the technology to its limits. Patterning will continue to be done with optical lithography (utilizing 193nm light, also known as ArF excimer for the light source). The “cost” of utilizing 193nm wavelength light to print features much smaller than that will be process complexity and ultimately cost. In the current power-constrained environment, innovation in the form of new materials and technology elements will be required to maintain performance improvements. Consistent with what we said last year, subsystem integration in the form of three dimensional structures will be a key component of performance improvements. This was covered in detail last year and we will not be going into this in any depth. Finally, nanotechnology is an exciting area.  As you will see, the era of nanotechnology within silicon technology is already upon us. Nanotechnology is and will be a part of the core silicon technology. In addition, nanotechnology holds many opportunities in areas far beyond information technology.

Digital Communities: Humankind has always benefited from geographically co-located communities as it provides social values by allowing members to feel a sense of belonging, to share knowledge and to collaborate and innovate. In recent years, we have seen an explosion in the number and the diversity of digital communities, e.g., MySpace, SecondLife, Massively Multiplayer Online Games (MMOG). Enabled by emerging technologies such as social software and 3D internet, these digital communities are providing new forms of interactions that are important to individuals. In this topic, we examine characteristics of digital communities and their potential in providing value to the globally integrated enterprise.  In particular, digital communities technologies such as tagging, blogs, wikis, reputation systems, social network analysis and virtual worlds can be leveraged in the enterprise to enable collaboration and learning, team building and interaction with customers and business partners. Finally, we witness the increased use of digital communities such as MySpace and SecondLife as venues for viral marketing.

Managing Business Integrity: The dynamics of disaggregated value nets, compounded by increasing regulations and data explosion, will require a new approach to manage business integrity. In particular, in addition to an imperative model of the enterprise based on the component business model, a declarative model of the enterprise is needed which focuses on consistent information about the core entities of the enterprise, such as customers, products, employees and services.  We believe that maintaining business integrity requires the management of policy integrity, process integrity and core entities information integrity. Policies consist of government regulations and rules, but also include business policies and ethical policies. Today, enterprises do not have a complete view of their policies, are not aware if processes are followed properly and have inconsistent information scattered throughout the enterprise. Emerging middleware solutions are addressing the integrity problem of policies, processes and core entities independently. What will manage and maintain business integrity will be an integrated platform which manages the integrity of policy, process and core entity and their interdependencies. This will be realized as an extension of the existing middleware platform. An integrated framework for business integrity can help deliver on the promise of SOA-enabled disaggregated IT by maintaining the integrity of SOA components and their interactions. Another opportunity is in the Governance, Risk and Compliance space, where business integrity solutions can help businesses satisfy regulatory compliance, while also delivering business clarity for performance gains and better strategic planning. Finally, the capture and discovery of information provenance will be important to business integrity and new provenance storage and management systems will emerge to address the near-line availability and long term preservation requirements.  

================================================================

Global Technology Outlook (GTO) 

C. Mohan

 IBM Fellow and IBM India Chief Scientist
Block C, 6th Floor, A Wing
Embassy Golf Links (EGL) Business Park
Off Indira Nagar - Koramangala Intermediate Ring Road
Bangalore 560 071
 mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

The Global Technology Outlook (GTO) is IBM Research’s vision of the future for information technology (IT) and its impact on industries that use IT. It highlights emerging software, hardware, and services technology trends that are expected to significantly impact the IT sector in the next 3-7 years. In particular, the GTO identifies technologies that may be disruptive to an existing business, have the potential to create new opportunity, and can provide new business value to our customers. The 2006 GTO focuses on five topics: Technology Update, The Event-Driven World, Application-Optimized Systems, The Accelerating Evolution of Software, Services 2.0. In this talk, I will share the GTO 2006 findings with the audience.

Here are some details on the five GTO 2006 topics:

Technology Update:  CMOS technology will continue to flourish for at least 10 years before radically new non-CMOS devices are called for. There are two main challenges for future CMOS devices: increasing leakage currents causing high power dissipation and increasing device variability. Near term new materials (high-K/metal gate, low-K dielectrics, and strained silicon), device structures and tailored layouts will increase performance. The future extension of CMOS will incorporate ultra-low voltage devices which will enable 3D silicon integration. Concurrently, random variability is becoming a dominant limiter for scaling and  advances in complex on-chip monitoring and  control will mitigate this. 

The Event-Driven World: The capability to handle events in software is an increasingly important business need in the areas of financial services, manufacturing, etc. Detecting and responding to events in a just-in-time basis is sometimes critical whether the events are business needs or public emergencies (e.g., credit card misuse, money laundering, real-time compliance, terrorism, internet services, critical asset tracking, detecting sand in oil intake pipeline, etc.) This topic addresses the increasingly sensor-driven world we live in and how events, rather than data, are beginning to be recognized as a unit of information. It addresses the requirement for systems to sense, analyze, act on, and deal with events in a deterministic, time-dependent fashion with due importance to throughput, latency, programmability, and scalability, by building time-dependent and throughput-sensitive middleware (application servers, databases, Java virtual machines, messaging, publish/subscribe entities), programming models, tools, and by extending standards to this evolving field.

Application-Optimized Systems: Modularity, scale-out, virtualization and the flattening of device performance growth are leading to an era of significant systems performance gains through optimization for classes of applications. New and evolving workloads from general purpose to ones optimized for specific workloads (e.g., anti-virus, event-driven applications, web services/XML, in-line analytics, etc.) will exploit these systems. New design approaches, architectural optimization, and power efficient processors are making it easier to design, develop, and deliver systems with an order of magnitude better performance than before. Virtualization will simplify the deployment and management of systems enabling ‘ready-to-go’ systems that are preconfigured for various needs.

The Accelerating Evolution of Software: Software development is going through a rapid evolution enabled by the ubiquity and ease-of-use of the web, simple to use software, tools, and techniques, dramatic rise in computer literacy, and the development of standards around Web Services. All these forces together are giving rise to a new paradigm for the collaboration, creation, manipulation of dynamic content with the web as the platform, a.k.a. Web 2.0. The building of situational applications – applications built with just enough function to satisfy a business need, usually by business users – by mixing and re-mixing existing components are becoming more and more common. These trends will force businesses to rethink how their applications and services are designed, developed, and managed. This in turn will put the onus on IT infrastructure companies to offer new tools for development, management and integration of situational applications and services. This will also act as a disruptive agent and will hasten the refactoring of monolithic applications into standardized and compartmentalized sub-components that can be mixed, matched, and replaced to deliver desired solutions.

Services 2.0: The decomposition of businesses continues apace providing business efficiencies and flexibility. Web 2.0 will impact business services by accelerating the move to services composition (“services mash-up”) particularly in small and medium businesses. Services mash-up is appearing as a classic disruptor to enterprise services business. New tools and methodologies for structured gathering of information about the extended value net, decomposing large complex business services, interfacing to legacy services, and managing relationship assets throughout the value-net will enable enterprises to also gain value from the composition of modular services. Overall these trends are growing the reach of business services further into IT and business services IT is responding by hiding more and more of the IT complexity.

 ================================================================

Global Technology Outlook (GTO)

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/ 

The Global Technology Outlook (GTO) is IBM’s projection of the future for information technology (IT). It highlights emerging software, hardware, and services technology trends that are expected to significantly impact the IT sector in the next 3-7 years. In particular, the GTO identifies technologies that may be disruptive to an existing business, have the potential to create new opportunity, and can provide new business value to our customers. The 2005 GTO focuses on six topics: Hardware Technology and Systems, A Revolution in Enterprise Software, Innovation in Services, Metadata, Speak to IT and Characteristics of On-Demand. In this talk, I will share the GTO 2005 findings with the audience. 

================================================================

Global Innovation Outlook (GIO)

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

The Global Innovation Outlook (GIO) is a global conversation organized by IBM to examine the changing nature of innovation and the areas in which it might generate the greatest benefits for business and society. GIO represents an outgrowth of IBM's Global Technology Outlook (GTO), an annual forecast on technology trends in the coming decade, and the industry Points of View produced by the company's Institute for Business Value. But rather than trying to replicate either process, the GIO blends them to reach a new level of insight on a time horizon of 5 to 10 years. In this talk, I will share the GIO findings with the audience. 

================================================================

DBCache: A Project on Database Caching Support for Web Applications

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In this talk, I will discuss IBM Almaden's DBCache project whose goal is to support database caching for use with web-based applications. I will introduce a new database object called Cache Table that enables persistent caching of the full or partial content of a remote database table. The content of a cache table is either defined declaratively and populated in advance at setup time, or determined dynamically and populated on demand at query execution time. Dynamic cache tables exploit the characteristics of typical transactional web applications with a high volume of short transactions, simple equality predicates, and 3-4 way joins. Based on federated query processing capabilities, we developed a set of new technologies for database caching: cache tables, "Janus" (two-headed) query execution plans, cache constraints, and asynchronous cache population methods. Our solution supports transparent caching both at the edge of content delivery networks and in the middle-tier of an enterprise application infrastructure, improving the response time, throughput and scalability of transactional web applications. This is joint work with M. Altinel, C. Bornhoevd, S. Krishnamurthy, H. Pirahesh and B. Reinwald. The paper on which this talk is based was published in the 29th International Conference on Very Large Databases (VLDB).

================================================================

Dynamic e-Business: Trends in Web Services

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In the last couple of years, the concept of a web service (WS) has emerged as an important paradigm for general application integration in the internet environment. More particularly, WS is viewed as an important vehicle for the creation of dynamic e-business applications and as a means for the J2EE and .NET worlds to come together. Several companies, including Microsoft, have been collaborating in proposing new WS standards. The World Wide Web Consortium has been the forum for many WS-related standardization activities. Many traditional concepts like business process management, security, directory services, routing and transactions are being extended for WS. This talk traces some of the trends in the WS arena. The paper on which this talk is based was published in the 3rd VLDB Workshop on Technologies for E-Services (TES'02).

================================================================

Tutorial: Application Servers and Associated Technologies

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

Application Servers (ASs), which have become very popular in the last few years, provide the platforms for the execution of transactional, server-side applications in the online world. While transaction processing monitors (TPMs) have been providing similar functionality for over 3 decades, ASs are their modern cousins. In this tutorial, I will provide an introduction to different ASs and their technologies. ASs play a central role in enabling electronic commerce in the web context. They are built on the basis of more standardized protocols and APIs than were the traditional TPMs. The emergence of Java, XML and OMG standards has played a significant role in this regard. Consequently, I will also briefly introduce the related XML, Java and OMG technologies like SOAP, Enterprise Java Beans and CORBA. One of the most important features of ASs is their ability to integrate the modern application environments with legacy data sources like IMS, CICS, VSAM, etc. They provide a number of connectors for this purpose, typically using asynchronous transactional messaging technologies like MQSeries and JMS. ASs integrate developments in a number of areas of computer science: software engineering, distributed computing, transaction processing, database management, workflow management, ... Of course, traditional TPM-style requirements for industrial strength features like scalability, availability, reliability and high performance are equally important for ASs also. Security and authentication issues are additional important requirements in the web context. ASs support DBMSs not only as storage engines for user data but also as repositories for tracking their own state. There are several on-going debates regarding ASs: Are they really needed when DBMS stored procedures, triggers and user-defined functions are available for implementing server-side logic? Even if they are needed, should they stay as independent middleware components or be integrated more tightly with DBMSs? I will address these questions and others in this tutorial. 

================================================================

Tutorial: Caching Technologies for Web Applications

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

The emergence of the Web has transformed the execution environment of transactional, server-side applications. 3 and 4-tier application environments involving browser-based clients and Web/application/database servers are the norm these days. The generation and distribution of dynamic web pages has also increased dramatically. Attaining good end to end performance under these circumstances requires exploitation of caching technologies. Caching is being deployed at different stages in the software and hardware hierarchies. Work is in progress to design caching standards. In this tutorial, I will provide an introduction to different caching technologies and their support by different products and specialized systems/vendors. I will also discuss the tradeoffs involved with different caching granularities and cache deployment points.

================================================================

Repeating History Beyond ARIES

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In this talk, I describe first the background behind the development of the original ARIES recovery method, and its significant impact on the commercial world and the research community. Next, I provide a brief introduction to the various concurrency control and recovery methods in the ARIES family of algorithms. Subsequently, I discuss some of the recent developments affecting the transaction management area and what these mean for the future. In ARIES, the concept of repeating history turned out to be an important paradigm. As I examine where transaction management is headed in the world of the internet, I observe history repeating itself in the sense of requirements that used to be considered significant in the mainframe world (e.g., performance, availability and reliability) now becoming important requirements of the broader information technology community as well.

This talk was originally prepared for delivery as an opening plenary talk at VLDB99 as a consequence of winning the 10 Year Best Paper Award for my work on the ARIES family of concurrency control and recovery algorithms. A paper which forms the basis for the talk and the talk's slides themselves are available on the web.

================================================================

DB2 UDB Family of Products and IBM Database Research:
Trends and Directions

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In this talk, I will introduce the various members of the DB2 UDB family of products. The broad themes that were the focus of the recent releases of the products and the specific features introduced in those releases will be reviewed. Then I will discuss the database work underway in IBM's research laboratories and thereby introduce the possible future directions of the IBM database products.

================================================================

Transaction Processing and Distributed Computing in the Internet Age

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In the last few years, there have been many developments in the transaction processing (TP) and distributed computing (DC) areas. This talk aims to introduce those developments and put them in perspective. Many areas of computing have influenced the trends in TP and DC: client-server computing, database management, object-oriented programming, groupware, internet and processor architectures, to name a few. While most computer professionals would be aware of these influencing factors in a general sense, mostly they tend to be unaware of their significance on transaction processing in general and electronic commerce in particular. The emergence of the web and Java has also had a dramatic influence on TP and DC. In addition to covering these topics, I will also address some of the debates in the TP and DC communities on the appropriate paradigms for program to program communications, and the role of TP monitors in the world of web servers and feature-rich RDBMSs. I will analyze the topics that the industrial and academic research communities have focused on, and the gaps between their work and the developments in the commercial world.

A longer version of this talk has been given as a tutorial many times.

================================================================

Tutorial: Transaction Processing and Distributed Computing in the Internet Age

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

DESCRIPTION

In the last few years, many paradigm shifts have occurred in the information technology (IT) arena. They relate to many areas of computing: client-server systems, database management, middleware, mobile computing, object-oriented programming and software development methodologies, groupware, internet and computer architectures, to name a few. These shifts impact transaction processing (TP) and distributed computing (DC) in significant and radical ways. The emergence of the worldwide web and Java has also had a dramatic influence on TP and DC. The Transaction Processing Council (TPC) has recently released a standard benchmark for the transactional web environment called TPC-W. This tutorial aims to introduce those developments and put them in perspective as we try to marry the legacy world with modern developments from a transactional and distributed systems viewpoint. I will discuss how TP systems are impacted by the need to support web access, electronic commerce and increased transaction rates/complexity, and how OO technology and Unix DBMSs need to be enhanced as they attempt to handle enterprise-class TP and DC applications. I will also address some of the debates in the TP and DC communities on the appropriate paradigms for program to program communications (transactional RPCs versus transactional messaging and queuing), and the role of TP monitors in the world of web servers and feature-rich RDBMSs.

OUTLINE


INTENDED AUDIENCE

Database and transaction systems' designers, implementers, users and administrators, students and researchers in industry and academia

================================================================

Lotus Domino/Notes: First
Semi-Structured DBMS of the World

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In this talk, I will describe the architecture and functionality of Lotus Notes/Domino which has traditionally been positioned as a groupware product. From its very beginning, this product has done its own persistent storage management by directly exploiting the underlying operating system's file system (i.e., without using any traditional DBMS as a repository). I will discuss why Domino/Notes can also be viewed as a semi-structured DBMS with its very general and flexible data model, powerful API and support for storage and manipulation of multimedia data. Even though Domino might not satisfy all the requirements that researchers have in mind for a semi-structured DBMS, this is an important analysis to do given the current popularity of research into semi-structured data management. I will trace the evolution of the product from its origins for use in a workgroup environment with very advanced functionality like replication and mobile computing to its current production deployment in enterprises as large as IBM (300,000 employees) for messaging and groupware applications in the intranet and internet environments. It has evolved from using only proprietary protocols in a number of areas to supporting the most relevant industry standards. With its support for multimedia data, authentication, encryption at a fine granularity and a flexible data model, it is being used effectively as a web server and a front-end to various data sources. Its scalability and reliability characteristics have been dramatically improved in the recent past. Domino is also being enhanced to provide support for the emerging area of knowledge management. I will discuss these recent developments also.

 

================================================================

Evolution of Groupware for Business Applications:
A Database Perspective on Lotus Domino/Notes

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

Lotus Notes was released in 1989 as a groupware product. Both the server (Domino) and client (Notes) versions of the product do their own persistent storage management by directly using the file system, without relying on a DBMS. While it was designed initially as a workgroup product for use by a small number of users, it has been enhanced extensively over the years, allowing it to be successfully deployed in many large enterprises with a year-end 1999 installed base of 50 million seats. Unlike in RDBMSs, support for semi-structured data management has been one of the unique features of Notes from the very beginning. From the first release, support for replication and disconnected operation has also been one of the most significant and innovative features. More recently, a major feature implemented in the latest release (R5) is a traditional log-based recovery scheme. This was done by extending the ARIES recovery method to take into account the unique characteristics of Notes's storage engine. Functionality and standards enhancements over the last few years have transformed Domino into a web application server. In this talk I will give a database perspective of Domino. Slides from an older presentation are available.

================================================================

ARIES: A Transaction Recovery Method Supporting
Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In this talk, I present a simple and efficient method, called ARIES (Algorithm for Recovery and Isolation Exploiting Semantics), which supports partial rollbacks of transactions, fine-granularity (e.g., record) locking and recovery using write-ahead logging (WAL). I introduce the paradigm of repeating history to redo all missing updates before performing the rollbacks of the loser transactions during restart after a system failure. ARIES uses a log sequence number in each page to correlate the state of a page with respect to logged updates of that page. All updates of a transaction are logged, including those performed during rollbacks. By appropriate chaining of the log records written during rollbacks to those written during forward progress, a bounded amount of logging is ensured during rollbacks even in the face of repeated failures during restart or of nested rollbacks. I deal with a variety of features that are very important in building and operating an "industrial-strength" transaction processing system. ARIES supports fuzzy checkpoints, selective and deferred restart, fuzzy backups, media recovery, and high concurrency lock modes (e.g., increment/decrement) which exploit the semantics of the operations and which require the ability to perform operation logging. ARIES is flexible with respect to the kinds of buffer management policies that can be implemented. It supports varying length objects efficiently. By permitting parallelism during restart, page-oriented redo and logical undo, it enhances concurrency and performance. ARIES is applicable not only to database management systems but also to persistent object-oriented languages, recoverable file systems and transaction-based operating systems. I show why some of the System R paradigms for logging and recovery, which were based on the shadow page technique, need to be changed in the context of WAL. I compare ARIES to the WAL-based recovery methods of DB2/MVS V1, IMS and Tandem systems. ARIES has been implemented, to varying degrees, in IBM's DB2 family of products, MQSeries, ADSM, Lotus Domino/Notes, Starburst and QuickSilver, in Transarc's Encina product suite, in many non-IBM products, and in the University of Wisconsin's Gamma, EXODUS, Shore and Paradise. Much more details on ARIES can be found in a paper in the March 1992 issue of the ACM Transactions on Database Systems and also in the IBM Research Report RJ6649. This is joint work with D. Haderle, B. Lindsay, H. Pirahesh and P. Schwarz.



================================================================

ARIES/IM: An Efficient and High Concurrency Index
Management Method Using Write-Ahead Logging

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

Even though concurrency in search structures (e.g., B+-tree indexes) was discussed frequently in the literature, the problem of providing recovery from transaction and system failures when transactions consist of multiple search structure operations received very little attention for a very long time. In this talk, I present a method called ARIES/IM (Algorithm for Recovery and Isolation Exploiting Semantics for Index Management) for controlling concurrency and logging changes to index data stored in B+-trees. ARIES/IM supports transaction semantics for locking (repeatable read or level 3 consistency) and uses write-ahead logging (WAL) for recovery. A transaction may consist of any number of index and nonindex operations. ARIES/IM supports very high concurrency by (1) not locking the index data per se (i.e., keys), (2) locking the underlying record data in data pages only (e.g., at the record level), (3) not acquiring commit duration locks on index pages even during index structure modification operations (SMOs) like page splits and page deletions, (4) allowing retrievals, inserts, and deletes to go on concurrently with even an SMO, and (5) optionally, supporting level 2 consistency of locking (cursor stability). During restart, any necessary redos of the index changes are always performed in a page-oriented fashion (i.e., without traversing the index tree) and, during normal processing and restart, undos are performed in a page-oriented fashion, whenever possible. The protocols used during normal processing are such that if a system failure were to occur any time and a logical undo were to be necessary during the subsequent restart, then, without resorting to any special processing, it is ensured that, by the time the logical undo is attempted, any incomplete SMO would have already been undone, thereby restoring the structural consistency of the tree. ARIES/IM, with variations, has been implemented in the DB2 family of products. Some of the ideas have also been incorporated in SQL/DS and the VM Shared File System, which were based on System R and which use the shadow-page technique for recovery. More details can be found in a paper in Proc. ACM SIGMOD International Conference on Management of Data, June 1992 and in the IBM Research Report RJ6846. This work was done jointly with F. Levine.

================================================================

Workflow Management in the Internet Age

C. Mohan

IBM Almaden Research Center
650 Harry Road, K01/B1
San Jose, CA 95120, USA
mohan@almaden.ibm.com, http://www.almaden.ibm.com/u/mohan/

In the last few years, workflow management has become a hot topic in the research community and in the commercial arena. Workflow systems hold the promise of facilitating the efficient everyday operation of many enterprises and work environments. Workflow management is multidisciplinary in nature encompassing many aspects of computing: database management, distributed systems, messaging, transaction management, mobile computing, collaboration, business process modeling, integration of legacy and new applications, document management, etc. Many academic and industrial research projects have been underway for a while. Numerous products are currently in use. The capabilities of these products are being enhanced in significant ways. Standardization efforts are in progress under the auspices of the Workflow Management Coalition and OMG. As has happened in the RDBMS area with respect to some topics, in the workflow area also, some of the important real-life problems faced by customers and product developers are not being tackled by researchers. Based on my experience founding and leading the Exotica workflow project at IBM Research, and my close collaboration with the IBM FlowMark (now called MQSeries Workflow) and Lotus Notes product groups, in this talk, I will discuss the issues relating to contemporary workflow management systems. I will also elaborate on various directions for research and potential future extensions to the design and modeling of workflow management systems.

A longer version of this talk has been given as a tutorial many times.

================================================================

Last updated on 19 March 2009. C. Mohan, mohan@almaden.ibm.com