Why measurement is key to driving improvement in software security
By Paco Hope, member of (ISC)²'s Application Security Advisory Board, and Principal Consultant with Cigital
We recognise that insecure software is a major cause of security breaches; we as security professionals even know what to do to address the problem, but ironically, not many organisations are actually taking the necessary measures.
The Building Security in Maturity Model (BSIMM, http://bsimm.com), an observational model built from real-world software security initiatives, is evidence of this.
BSIMM-V (the fifth annual revision) is based on the software security initiatives of 67 well-known organisations (e.g., McAfee, HSBC, Fidelity, Nokia, etc.), so clearly there is significant knowledge on how security can be in-built into software. We just need more organisations to do the same. Embedding security in software is likely to be more robust than bolting it on at the end with features such as SSL or passwords.
This said, a lot more needs to be done. This study of 67 firms covers over 250,000 developers and shows that on average in the current data pool, firms have only 1.4 people dedicated to software security for every 100 people they have developing software. These stats speak for themselves, but it is a start.
Anecdotally, organisations corroborate that it is much easier to start software security initiatives and subsequently improve upon them if it is possible to measure the effectiveness of the programmes. This is another reason why security professionals and developers should consider BSIMM adoption—it is primarily a measurement tool. It allows a firm to benchmark their own programme against the average of others in their vertical, or generally across all verticals.
Companies learn best from comparisons with other real-world programmes rather by comparing themselves to abstract, theoretical models of software security. Firms succeed better by developing an individual, tailored approach to securing software based on an understanding of the problems that other companies face and how they tackle those challenges.
The BSIMM software security framework categorises security activities into four domains—governance, intelligence, SSDL touchpoints and deployment. Each domain has three practices. So for instance, the practices within the governance domain include strategy and metrics, compliance and policy and training; while SSDL touchpoints domain includes architecture analysis, code review and security testing. By comparing their own organisation’s initiative with a group of peers, businesses can see areas where they are expending similar effort to their peers and areas where more effort or more mature effort may be beneficial.
From there, they can devise a strategy and execute it. Over time, and over multiple measurements, such an approach provides strong intelligence and a clear trend of increasing maturity of security initiatives. In essence, organisations can measure their software security capability.
In all this, the proficiency of individuals involved greatly impacts on the success of such initiatives. Qualified security professionals are better placed to understand the developing nuances in the information security landscape and how best to deal with the challenges they throw up.
While BSIMM has a comprehensive view of software security initiative activities, there is no objective way to measure the impact of having employees involved in software security independently certified. The Certified Secure Software Lifecycle Professional (CSSLP) credential, though, can demonstrate that individuals have the knowledge required to understand and implement the activities in the BSIMM framework.
Today, BSIMM is the only measuring stick for software security initiatives based on science. It is a good representation of the software security issues that most organisations face—it describes the work of 975 software security professionals, working alongside 1,953 security champions and securing the software developed by 272,358 developers.
I’d encourage security professionals and software developers alike to use BSIMM to strategise and plan their software security initiatives, but also (and crucially) to measure the results. There are substantial incremental gains to be had, which cumulatively will contribute to the development of secure software. Keep in mind, also, that while BSIMM is primarily used to measure a centralised software security initiative, it also works very well for measuring individual business units.
These measures can show where software security uptake is high, where software security uptake is lagging, and where local ingenuity has created software security practices that others in the firm should know about.