Importance-Driven Deep Learning System Testing -ICSE 2020

About

This paper presents DeepImportance, a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion for DL systems. Applying IDC enables to establish a layer-wise functional understanding of the importance of DL system components and use this information to guide the generation of semantically-diverse test sets. Our empirical evalua- tion on several DL systems, across multiple DL datasets and with state-of-the-art adversarial generation techniques demonstrates the usefulness and effectiveness of DeepImportance and its ability to guide the engineering of more robust DL systems.

Repository

This repository includes details about the artifact corresponding to implementation of DeepImportance. Our implementation is publicly available in DeepImportance repository. This artifact allows reproducing the experimental results presented in the paper. Below we describe how to reproduce results. Before going further, first, check installation page (i.e. INSTALL.md).