A quantitative analysis framework for recurrent neural network

Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, Jianjun Zhao

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

2 Citations (Scopus)


Recurrent neural network (RNN) has achieved great success in processing sequential inputs for applications such as automatic speech recognition, natural language processing and machine translation. However, quality and reliability issues of RNNs make them vulnerable to adversarial attacks and hinder their deployment in real-world applications. In this paper, we propose a quantitative analysis framework-DeepStellar-to pave the way for effective quality and security analysis of software systems powered by RNNs. DeepStellar is generic to handle various RNN architectures, including LSTM and GRU, scalable to work on industrial-grade RNN models, and extensible to develop customized analyzers and tools. We demonstrated that, with DeepStellar, users are able to design efficient test generation tools, and develop effective adversarial sample detectors. We tested the developed applications on three real RNN models, including speech recognition and image classification. DeepStellar outperforms existing approaches three hundred times in generating defect-triggering tests and achieves 97% accuracy in detecting adversarial attacks. A video demonstration which shows the main features of DeepStellar is available at: https://sites.google.com/view/deepstellar/tool-demo.

Original languageEnglish
Title of host publicationProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
EditorsJulia Lawall, Darko Marinov
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages4
ISBN (Electronic)9781728125084
ISBN (Print)9781728125091
Publication statusPublished - 2019
Externally publishedYes
EventAutomated Software Engineering Conference 2019 - San Diego, United States of America
Duration: 10 Nov 201915 Nov 2019
Conference number: 34th
https://2019.ase-conferences.org/ (Conference website)
https://dl.acm.org/doi/proceedings/10.5555/3382508 (Proceedings)

Publication series

NameProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
PublisherThe Institute of Electrical and Electronics Engineers, Inc.
ISSN (Print)1938-4300
ISSN (Electronic)2643-1572


ConferenceAutomated Software Engineering Conference 2019
Abbreviated titleASE 2019
CountryUnited States of America
CitySan Diego
Internet address


  • Coverage criteria
  • Model abstraction
  • Quantitative analysis
  • Recurrent neural netwrod
  • Similarity metrics

Cite this