IBM®
Skip to main content
    United States [change]    Terms of use
 
 
 
    Home    Products    Services & solutions    Support & downloads    My account    
IBM Research

Autonomic Databases

Computer Science


 Overview

With the cost of both hardware and software decreasing rapidly due to technological advancements and economies of scale, the total cost of ownership for database applications is increasingly dominated by the cost of people to manage them. Databases are growing rapidly in scale and complexity, while skilled database administrators are becoming rarer and more expensive.

The goal of Autonomic Databases is to make databases completely self-managing, i.e. self-configuring, self-healing, self-optimizing, and self-protecting. It is part of the IBM-wide effort, called Autonomic Computing, to make all IBM products self-managing. Almaden Research co-founded the DB2 Autonomic Computing project, originally called SMART (Self-Managing And Resource Tuning), a collaboration with IBM's Toronto and Silicon Valley Labs aimed at accelerating autonomic technologies in the DB2 family of data management products.

Most of our projects to date have dealt with self-optimizing databases, combining our expertise in query optimization and autonomic computing. These efforts seek to eliminate the need to "tune" SQL queries, by automating the physical database design and improving query optimization by actually learning from the database.

Much of this work has already been incorporated into the DB2 Universal Database product for Linux, Unix, and Windows, following the initial release of our Index Advisor in Version 6 of DB2 (1999). Version 8.2 of DB2 UDB, released in September 2004, contains two major autonomic features with significant contributions from Almaden:

  • Design Advisor, which automatically determines the best combination of indexes, materialized views, partitioning, and clustering to optimize a workload of SQL statements
  • Automated database statistics collection, which completely automates the scheduling, prioritization, throttling, and collection of statistics on the database (the RUNSTATS utility). This feature exploits LEO, the LEarning Optimizer, which automatically learns which statistics are needed based upon the accuracy of prior estimates by the Optimizer of the number of rows resulting from each step of a query.

Automated database statistics collection, which completely automates the scheduling, prioritization, throttling, and collection of statistics on the database (the RUNSTATS utility). This feature exploits LEO, the LEarning Optimizer, which automatically learns which statistics are needed based upon the accuracy of prior estimates by the Optimizer of the number of rows resulting from each step of a query.

Recently, we have extended our research to include self-healing systems, especially the challenges of problem determination, such as detecting, isolating, and determining the root cause of unknown problems, after first determining whether the problem is known or not by matching its symptoms to a database of known symptoms.

Additional information:

Bibliography:

General DB2 Autonomic Computing:

  • Usability and design considerations for an autonomic relational database management system, IBM Systems Journal 42, 4 (2003), pp. 568-581 (R. Telford, R.Horman, S. Lightstone, N. Markov, S. O’Connell, G. Lohman)
  • SMART: Making DB2 (More) Autonomic, VLDB 2002 (Guy M. Lohman, Sam S. Lightstone)
  • Toward Autonomic Computing with DB2 Universal Database, in ACM SIGMOD Record 31,3 (Sept. 2002) (Sam S. Lightstone, Guy M. Lohman, Danny Zilio)
  • A SMARTer DB2, in DB2 Magazine 7,4 (Q4, 2002) (Guy M. Lohman, Bryan Smith, Sam S. Lightstone, Randy Horman, James Teng)
  • More...

Learning Optimizer (LEO):

  • Automated Statistics Collection in DB2 Stinger, VLDB 2004 (Ashraf Aboulnaga, Peter J. Haas, Sam Lightstone, Guy M. Lohman, Volker Markl, Ivan Popivanov, Vijashankar Raman)
  • Progressive Optimization in Action (demo), VLDB 2004 (Vijayshankar Raman, Volker Markl, David E. Simmen, Guy M. Lohman, and Hamid Pirahesh)
  • CORDS: Automatic generation of correlation statistics in DB2 (demo). VLDB 2004 ( I. Ilyas, V. Markl, P. J. Haas, P. G. Brown, and A. Aboulnaga)
  • CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies. ACM SIGMOD 2004 (Ihab F. Ilyas, Volker Markl, Peter J. Haas, Paul Brown, Ashraf Aboulnaga)
  • Robust Query Processing through Progressive Optimization. ACM SIGMOD 2004 (Volker Markl, Vijayshankar Raman, David E. Simmen, Guy M. Lohman, Hamid Pirahesh)
  • Automatic relationship discovery in self-managing database systems (poster), IEEE Intl. Conf. Autonomic Computing (ICAC 2004) ( I. Ilyas, V. Markl, P. J. Haas, P. G. Brown, and A. Aboulnaga)
  • LEO: An Autonomic Query Optimizer for DB2. IBM Systems Journal Special Issue on Autonomic Computing, Jan. 2003 (Volker Markl, G. M. Lohman, and V. Raman)
  • Learning table access cardinalities with LEO (demo). ACM SIGMOD 2002 (Volker Markl, Guy M. Lohman)
  • LEO - DB2's Learning Optimizer. VLDB 2001 (Michael Stillger, Guy M. Lohman, Volker Markl, Mokhtar Kandil)

Design Advisor:

  • DB2 Design Advisor: Integrated Automatic Physical Database Design, VLDB 2004 ( Daniel Zilio, Jun Rao, Sam Lightstone, Guy Lohman, Adam Storm, Christian Garcia-Arellano, and Scott Fadden)
  • Recommending Materialized Views and Indexes with IBM’s DB2 Design Advisor, IEEE Intl. Conf. on Autonomic Computing (ICAC 2004) (Daniel C. Zilio, Calisto Zuzarte, Sam Lightstone, Wenbin Ma, Guy M. Lohman, Roberta Cochrane, Hamid Pirahesh, Latha S. Colby, Jarek Gryz, Eric Alton, Dongming Liang, and Gary Valentin)
  • Trends in Automating Database Physical Design, IEEE 2003 Workshop on Autonomic Computing Principles and Architectures, Banff, Alberta, August 2003 (Danny Zilio, Sam Lightstone, and Guy Lohman)
  • Automating Physical Database Design in a Parallel Database: the DB2 Partitioning Advisor, ACM SIGMOD 2002 (Jun Rao, Chun Zhang, Guy M. Lohman, and Nimrod Meggido)

Meta-Optimizer:

  • Estimating Compilation Time in a Query Optimizer, ACM SIGMOD 2003 (Ihab Ilyas, Jun Rao, Guy M. Lohman, Dengfeng Gao, and Eileen Lin)

    About IBMPrivacyContact