The SAMATE project is an inter-agency project between the U.S. Department of Homeland Security and NIST, and consists of two parts:
- Development of metrics for the effectiveness of software security assessment (SSA) tools.
- assess current SSA methods and tools in order to identify deficiencies which can lead to software product failures and vulnerabilities.
The first part classifies Software Security Assurance tools and develops metrics and tests for each tool class. Source/object code vulnerability scanners are an example of one possible class. A series of workshops is used to develop recommendations for:
- The order in which SSA tool classes requirements and tests are developed, and
- for each class of SSA tools in priority order, required and optional functionality.
Metrics and tests for these functionalities are developed. Classification and testing activities proceed simultaneously. As a result, a draft specification and test methodology for the highest priority tool class is developed. These activities develop the infrastructure needed for testing efforts in following years.
The SSA tool testing effort supports the second part of the project: identifying deficiencies in SSA methodologies and tools. Like the activities in the first part of the project, this part of the project develops an infrastructure for assessment and periodic reporting.