SEADS: scalable and cost-effective dynamic dependence analysis of distributed systems via reinforcement learning

Xiaoqin Fu, Haipeng Cai, Wen Li, Li Li

Research output: Contribution to journalArticleResearchpeer-review

1 Citation (Scopus)

Abstract

Distributed software systems are increasingly developed and deployed today. Many of these systems are supposed to run continuously. Given their critical roles in our society and daily lives, assuring the quality of distributed systems is crucial. Analyzing runtime program dependencies has long been a fundamental technique underlying numerous tool support for software quality assurance. Yet conventional approaches to dynamic dependence analysis face severe scalability barriers when theyare applied to real-world distributed systems, due to the unbounded executions to be analyzed in addition to common efficiency challenges suffered by dynamic analysis in general. In this article, wepresent SEADS, a distributed, online, and cost-effective dynamic dependence analysis framework thataims at scaling the analysis to real-world distributed systems. The analysis itself is distributed to exploit the distributed computing resources (e.g., a cluster) of the system under analysis; it works online to overcome the problem with unbounded execution traces while running continuously withthe system being analyzed to provide timely querying of analysis results (i.e., runtime dependence set of any given query). Most importantly, given a user-specified time budget, the analysis automatically adjusts itself to better cost-effectiveness tradeoffs (than otherwise) while respecting the budget by changing various analysis parameters according to the time being spent by the dependence analysis. At the core of the automatic adjustment is our application of a reinforcement learning method for the decision making - deciding which configuration to adjust to according to the current configuration and its associated analysis cost with respect to the user budget. We have implemented SEADS for Java and applied it to eight real-world distributed systems with continuous executions. Our empirical results revealed the efficiency and scalability advantages of our framework over a conventional dynamic analysis, at least for dynamic dependence computation at method level. While we demonstrate it in the context of dynamic dependence analysis in this article, the methodology for achieving and maintaining scalability and greater cost-effectiveness against continuously running systems is more broadly applicable to other dynamic analyses.

Original languageEnglish
Article number10
Number of pages45
JournalACM Transactions on Software Engineering and Methodology
Volume30
Issue number1
DOIs
Publication statusPublished - Dec 2020

Keywords

  • cost-effectiveness
  • Distributed systems
  • dynamic analysis
  • reinforcement learning
  • scalability

Cite this