Cost-Based Optimization for Magic: Algebra and Implementation

Praveen Seshadri, Raghu Ramakrishnan, Joseph M. Hellerstein, Divesh Srivastava, Hamid Pirahesh, Peter J. Stuckey, T. Y.Cliff Leung, S. Sudarshan

Research output: Contribution to journalArticleResearchpeer-review

44 Citations (Scopus)


Magic sets rewriting is a well-known optimization heuristic for complex decision-support queries. There can be many variants of this rewriting even for a single query, which differ greatly in execution performance. We propose cost-based techniques for selecting an efficient variant from the many choices. Our first contribution is a practical scheme that models magic sets rewriting as a special join method that can be added to any cost-based query optimizer. We derive cost formulas that allow an optimizer to choose the best variant of the rewriting and to decide whether it is beneficial. The order of complexity of the optimization process is preserved by limiting the search space in a reasonable manner. We have implemented this technique in IBM's DB2 C/S V2 database system. Our performance measurements demonstrate that the cost-based magic optimization technique performs well, and that without it, several poor decisions could be made. Our second contribution is a formal algebraic model of magic sets rewriting, based on an extension of the multiset relational algebra, which cleanly defines the search space and can be used in a rule-based optimizer. We introduce the multiset θ-semijoin operator, and derive equivalence rules involving this operator. We demonstrate that magic sets rewriting for non-recursive SQL queries can be modeled as a sequential composition of these equivalence rules.

Original languageEnglish
Pages (from-to)435-446
Number of pages12
JournalSIGMOD Record (ACM Special Interest Group on Management of Data)
Issue number2
Publication statusPublished - 1 Jan 1996
Externally publishedYes

Cite this