Are Free Android App Security Analysis Tools Effective in Detecting Known Vulnerabilities?

Increasing interest to secure Android ecosystem has spawned numerous efforts to assist app developers in building secure apps. These efforts have developed tools and techniques capable of detecting vulnerabilities and malicious behaviors in apps. However, there has been no evaluation of the effective- ness of these tools and techniques to detect known vulnerabilities. Absence of such evaluations poses as a hurdle when app developers try to choose security analysis tools to secure their apps. In this regard, we evaluated the effectiveness of vulnerability (and malicious behavior) detection tools for Android apps. We considered 64 security analysis tools and empirically evaluated 19 of them -- 14 vulnerability detection tools and 5 malicious behavior detection tools -- against 42 known vulnerabilities captured by benchmarks in Ghera repository. Of the many observations from the evaluation, the key observation is existing security analysis tools for Android apps are very limited in their ability to detect known vulnerabilities: all of the evaluated vulnerability detection tools together could only detect 30 of the 42 known vulnerabilities. Clearly, serious effort is required if security analysis tools are to help developers build secure apps. We hope the observations from this evaluation will help app developers choose appropriate security analysis tools and persuade tool developers and researchers to identify and address limitations in their tools and techniques. We also hope this evaluation will catalyze or spark a conversation in the Android security analysis community to require more rigorous and explicit evaluation of security analysis tools and techniques.


Ghera: A Repository of Android App Vulnerability Benchmarks

Security of mobile apps affects the security of their users. This has fu...

SOURCERER: Developer-Driven Security Testing Framework for Android Apps

Frequently advised secure development recommendations often fall short i...

BenchPress: Analyzing Android App Vulnerability Benchmark Suites

In recent years, various efforts have designed and developed benchmark s...

CRYLOGGER: Detecting Crypto Misuses Dynamically

Cryptographic (crypto) algorithms are the essential ingredients of all s...

Mascara: A Novel Attack Leveraging Android Virtualization

Android virtualization enables an app to create a virtual environment, i...

Challenges in Net Neutrality Violation Detection: A Case Study of Wehe Tool

The debate on "Net-neutrality" and events pointing towards its possible ...

Discovering Flaws in Security-Focused Static Analysis Tools for Android using Systematic Mutation

Mobile application security has been one of the major areas of security ...

Please sign up or login with your details

Forgot password? Click here to reset