Search papers, labs, and topics across Lattice.
This paper introduces CAUSEC, a causal analysis framework designed to evaluate the validity of assumptions made by Static Application Security Testing (SAST) tools and their impact on performance. By systematically reviewing 57 assumptions related to crypto-API misuse, the authors demonstrate that many design choices in SAST tools compromise detection capabilities for performance gains, raising questions about the validity of these assumptions. The findings reveal critical insights into the causal relationships between assumptions and performance, providing actionable takeaways for improving future SAST methodologies.
Many SAST tools sacrifice detection capabilities for performance, but CAUSEC reveals that the underlying assumptions may not hold true, challenging the status quo in security analysis.
Static Application Security Testing (SAST) tools are widely used in both industry and academia. Such tools often make design choices that sacrifice detection to achieve higher performance, i.e., increased precision, decreased runtime, or increased scalability. These design choices rely on certain assumptions regarding the target code or the analysis technique itself. Hence, the assumptions directly impact the detection outcome through the design choices they influence. This motivates a key question: do the sacrifices in the detection capabilities actually help tools achieve the expected performance gains? That is, are the underlying assumptions valid? This paper seeks to address this question by relying on a key observation that the assumptions made by these tools are generally of a causal nature. We propose CAUSEC, a causal analysis framework that makes SAST assumptions testable and explains why the performance changes given certain assumptions, beyond simple correlations. CAUSEC formalizes the assumptions of the SAST tool into the abstraction of a security assumption and combines assumption-driven causal modeling with effect estimation and validation to test its validity and investigate the factors affecting it. To understand what security assumptions generally entail, we perform a systematic literature review of SASTs that detect crypto-API misuse, leading to the discovery and qualitative analysis of 57 assumptions. We then demonstrate the utility and robustness of CAUSEC by testing a popular assumption in four highly relevant tools, using a manually labeled ground truth dataset consisting of 57,038 alerts. Our analysis leads to several key findings that represent insights regarding assumptions and causal effects, which we distill into 3 takeaways for future work.
