SAMATE Logo NIST Logo The SAMATE Project Department of Homeland Security

Test Suites

Stand-alone Suites

Download Publication Date Title Version Description Contributor # of Cases
See Description Column May. 2015 IARPA STONESOUP Phase 3 - Virtual Machine 3.0 A collection of C and Java test cases based on 16 widely-used open-source software in which vulnerabilities have been seeded. It comes bundled in a virtual machine for ease of use.

This product contains or makes use of Intelligence Advanced Research Projects Activity (IARPA) data from the STONESOUP program. Any product, report, publication, presentation, or other document including or referencing the IARPA data herein should include this statement.

NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic.

Download the VM in 2GB slices below:
Part 00 Download testsuite Part 01 Download testsuite Part 02 Download testsuite Part 03 Download testsuite Part 04 Download testsuite Part 05 Download testsuite Part 06 Download testsuite Part 07 Download testsuite Part 08 Download testsuite Part 09 Download testsuite Part 10 Download testsuite Part 11 Download testsuite
IARPA C: 4582

Java: 3188
Download testsuite May. 2013 Juliet Test Suite for C/C++ 1.2 A collection of test cases in the C/C++ language. It contains examples for 118 different CWEs.

This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic.
NSA Center for Assured Software 61,387
Download testsuite May. 2013 Juliet Test Suite for Java 1.2 A collection of test cases in the Java language. It contains examples for 112 different CWEs.

This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic.
NSA Center for Assured Software 25,477

SARD Suites

Results: 34 Test Suites.


Test Suite ID View
Download
Manifest
Creation Date Title Description Contributor # of Cases
101View testsuite  Download testsuite  Download manifest2015-03-16C Test Suite for Source Code Analyzer v2 - SecureThis test suite replaces test suite 46 based on a collaboration with Alexander Hoole from University of Victoria, BC, Canada. The new test cases provided by these test suites contain the following improvements: removal of targeted weaknesses from 13 “GOOD” test cases in test suite 46, removal of extraneous weaknesses, replacement of test cases to align with the CWE’s specified in NIST SP 500-268 v1.1, creation of additional test cases to provide consistent BAD/GOOD pairings, application of minor improvements to code, renaming of files and the addition of FLAW/FIX comments to assist automation, and insertion of improved metadata to assist researchers using SARD. Please refer to the test case metadata fields to view additional information for each test case.Aurelien Delaitre96
100View testsuite  Download testsuite  Download manifest2015-03-16C Test Suite for Source Code Analyzer v2 - VulnerableThis test suite replaces test suite 45 based on a collaboration with Alexander Hoole from University of Victoria, BC, Canada. The new test cases provided by these test suites contain the following improvements: removal of extraneous weaknesses, replacement of test cases to align with the CWE’s specified in NIST SP 500-268 v1.1, creation of additional test cases to provide consistent BAD/GOOD pairings, application of minor improvements to code, renaming of files and the addition of FLAW/FIX comments to assist automation, and insertion of improved metadata to assist researchers using SARD. Please refer to the test case metadata fields to view additional information for each test case.Aurelien Delaitre96
99View testsuite  Download testsuite  Download manifest2014-08-01wordpress-2.0Content management system based on PHP and MySQL. Contains CVEs.SAMATE Team Staff17
98View testsuite  Download testsuite  Download manifest2014-08-01openfire-3.6.0Real time collaboration server that uses XMPP (Jabber). Contains CVEs.SAMATE Team Staff12
97View testsuite  Download testsuite  Download manifest2014-08-01jspwiki-2.5.124WikiWiki engine built around JEE components (Java, servlets, JSP). Contains CVEs.SAMATE Team Staff3
96View testsuite  Download testsuite  Download manifest2014-08-01jetty-6.1.16Web server and javax.servlet container with support for SPDY, WebSocket, OSGi, JMX, JNDI, JAAS, along with other integrations. Contains CVEs.SAMATE Team Staff6
95View testsuite  Download testsuite  Download manifest2014-08-01apache-tomcat-5.5.13Open source software implementation of the Java Servlet and JavaServer Pages technologies. Contains CVEs.SAMATE Team Staff37
94View testsuite  Download testsuite  Download manifest2014-08-01wireshark-1.8.0Network traffic analyzer containing CVEs.SAMATE Team Staff85
93View testsuite  Download testsuite  Download manifest2014-08-01wireshark-1.2.0Network traffic analyzer containing CVEs.SAMATE Team Staff44
92View testsuite  Download testsuite  Download manifest2014-08-01dovecot-1.2.0IMAP and POP3 email server for Linux/UNIX-like systems. Contains CVEs.SAMATE Team Staff9
91View testsuite  Download testsuite  Download manifest2014-08-01chrome-5.0.375.54Google web browser containing CVEs.SAMATE Team Staff10
90View testsuite  Download testsuite  Download manifest2014-08-01asterisk-10.2.0VoIP communication system with chat, conferencing, instant messaging, fax and other features. Contains CVEs.SAMATE Team Staff15
89View testsuite  Download testsuite  Download manifest2014-06-09A Taxonomy of Buffer OverflowsKendra Kratkiewicz developed a taxonomy of C buffer overflows and 291 test cases representing this taxonomy. Each test case has three flawed versions (with overflows just outside, moderately outside, and far outside the buffer) and a patched version (without buffer overflow). Examples of using these are in "A Taxonomy of Buffer Overflows for Evaluating Static and Dynamic Software Testing Tools" 2005. Eric Rosenberg1164
88View testsuite  Download testsuite  Download manifest2014-06-09Testing Exploitable Buffer Overflows From Open Source CodeZitser, Lippmann, and Leek extracted 14 model programs from internet applications (BIND, Sendmail, WU-FTP) with known buffer overflows. These models have the portion of code with the overflows. Patched versions are also included. Examples of using these are in "Using Exploitable Buffer Overflows From Open Source Code" 2004. Eric Rosenberg28
87View testsuite  Download testsuite  Download manifest2013-05-15Juliet Test Suite for Java (v1.2)This is a collection of test cases in the Java language. It contains examples for 112 different CWEs. NOTE: This package contains only individual test cases. We recommend to download the full test suite at the top of the "Test Suite" page.SAMATE Team Staff25477
86View testsuite  Download testsuite  Download manifest2013-05-15Juliet Test Suite for C/C++ (v1.2)This is a collection of test cases in the C/C++ language. It contains examples for 118 different CWEs. NOTE: This package contains only individual test cases. We recommend to download the full test suite at the top of the "Test Suite" page.SAMATE Team Staff61387
81View testsuite  Download testsuite  Download manifest2013-02-08Basic CWE Effectiveness, CWE-121: Stack-based Buffer Overflow, for C.These allow a prospective user to understand that a capability is effective in locating CWE-121: Stack-based Buffer Overflow in the most basic situations in C code.Michael Koo5
69View testsuite  Download testsuite  Download manifest2011-04-08Juliet Test Suite for Java (v1.0 - Deprecated)This is a collection of test cases in the Java language. It contains examples for 106 different CWEs. NOTE: This package contains only individual test cases. We recommend to download the full test suite at the top of the "Test Suite" page.SAMATE Team Staff14184
68View testsuite  Download testsuite  Download manifest2011-04-08Juliet Test Suite for C/C++ (v1.0 - Deprecated)This is a collection of test cases in the C/C++ language. It contains examples for 116 different CWEs. NOTE: This package contains only individual test cases. We recommend to download the full test suite at the top of the "Test Suite" page.SAMATE Team Staff45309
65View testsuite  Download testsuite  Download manifest2010-02-04Java Test Suite for Source Code Analyzer - weakness suppresionThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RO-2 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo10
64View testsuite  Download testsuite  Download manifest2010-02-04Java Test Suite for Source Code Analyzer - false positiveThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-6 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo27
63View testsuite  Download testsuite  Download manifest2010-02-04Java Test Suite for Source Code Analyzer - weaknessThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-1 through SCAN-RM-5 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo27
62View testsuite  Download testsuite  Download manifest2008-10-02Defence R&D Canada25 C++ test cases (plus a main including all of them) created in 2006 by Frederic Michaud and Frederic Painchaud, Defence Research & Development Canada, http://www.drdc-rddc.gc.ca/SAMATE Team Staff26
59View testsuite  Download testsuite  Download manifest2007-12-06C++ Test Suite for Source Code Analyzer - weakness suppresionThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RO-2 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo14
58View testsuite  Download testsuite  Download manifest2007-12-06C++ Test Suite for Source Code Analyzer - false positiveThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-6 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo39
57View testsuite  Download testsuite  Download manifest2007-12-06C++ Test Suite for Source Code Analyzer - weaknessThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-1 through SCAN-RM-5 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo41
47View testsuite  Download testsuite  Download manifest2007-02-05C Test Suite for Source Code Analyzer - weakness suppresionThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RO-2 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo21
46View testsuite  Download testsuite  Download manifest2007-02-05C Test Suite for Source Code Analyzer - false positiveThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-6 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo73
45View testsuite  Download testsuite  Download manifest2007-01-24C Test Suite for Source Code Analyzer - weaknessThis test suite tests against Source Code Security Analyzer based on functional requirements SCA-RM-1 through SCAN-RM-5 specified in "Source Code Security Analysis Tool Functional Specification"Michael Koo77
31View testsuite  Download testsuite  Download manifest2006-10-24Web Applications in PHPThe PHP Test casesRomain Gaucher15
27View testsuite  Download testsuite  Download manifest2006-10-18MSNoneEric D.25
17View testsuite  Download testsuite  Download manifest2006-08-09CANDIDATE Source Code Analysis Tool Functional Specification Test SuiteThis test suite contains all test cases that can be used to test a general purpose, production source code analysis tool implementation against the SAMATE Source Code Analysis Tool Functional Specification.SAMATE Team Staff34
9View testsuite  Download testsuite  Download manifest2006-07-11Test suite (2006/07/11 18:32:50)NoneRedge Bartholomew5
6View testsuite  Download testsuite  Download manifest2006-06-23ABM 1.0.1Fortify Software's Analyzer BenchMark v. 1.0.1Jeff Meister112


Archives

Download Publication Date Title Version Description Contributor # of Cases
Download testsuite Nov. 2012 IARPA STONESOUP Phase 1 - Memory Corruption for C 1.0 A collection of test cases in the C language. It contains examples of memory corruption issues, including input triggering the vulnerability. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. IARPA 212
Download testsuite Nov. 2012 IARPA STONESOUP Phase 1 - Null Pointer Dereference for C 1.0 A collection of test cases in the C language. It contains examples of null pointer mishandling, including input triggering the vulnerability. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. IARPA 115
Download testsuite Nov. 2012 IARPA STONESOUP Phase 1 - Injection for Java 1.0 A collection of test cases in the Java language. It contains examples of various injection issues, including input triggering the vulnerability. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. IARPA 36
Download testsuite Nov. 2012 IARPA STONESOUP Phase 1 - Numeric Handling for Java 1.0 A collection of test cases in the Java language. It contains examples of numeric mishandling, including input triggering the vulnerability. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. IARPA 59
Download testsuite Nov. 2012 IARPA STONESOUP Phase 1 - Tainted Data for Java 1.0 A collection of test cases in the Java language. It contains examples of tainted data mishandling, including input triggering the vulnerability. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. IARPA 35
Download testsuite Sep. 2012 Juliet Test Suite for Java 1.1.1 A collection of test cases in the Java language. It contains examples for 113 different CWEs. This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic.

v1.1.1 supersedes v1.1. It added methods needed for building test cases after adding/removing test cases. Does not affect using test cases as is.

NSA Center for Assured Software 23,957
Download testsuite Jul. 2012 Juliet Test Suite for C/C++ 1.1 A collection of test cases in the C/C++ language. It contains examples for 119 different CWEs. This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. NSA Center for Assured Software 57,099
Download testsuite Dec. 2010 Juliet Test Suite for C/C++ 1.0 A collection of test cases in the C/C++ language. It contains examples for 116 different CWEs. This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. NSA Center for Assured Software 45,324
Download testsuite Dec. 2010 Juliet Test Suite for Java 1.0 A collection of test cases in the Java language. It contains examples for 106 different CWEs. This software is not subject to copyright protection and is in the public domain. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guaranties, expressed or implied, about its quality, reliability, or any other characteristic. NSA Center for Assured Software 13,801