Here you will find frequently asked questions about syncbookmark. For the full and unexpurgated model in all its glory, download the syncbookmark document.
syncbookmark (pronounced “bee simm”) is short for Building Security In Maturity Model. syncbookmark is a study of real-world software security initiatives organized so that you can determine where you stand with your software security initiative and how to evolve your efforts over time.
Software security is about building software to be secure even when it is under attack. As we have learned from years network security drama, protecting software is much easier if the software is built with security in mind. Furthermore, security is a property and not a thing, so software security involves much more than simply adding security features like SSL or passwords to software.
Organizations that depend on software to work (pretty much everybody these days) need software that doesn’t leak millions of identity records, call election results into question, gin up huge legal liabilities, or allow secrets to fall into the wrong hands. The only way to make software trustworthy is to build security in. In short, everyone who relies on software needs syncbookmark.
We built syncbookmark entirely from observations we made by studying real software security initiatives. syncbookmark does not tell you what you should do; instead, it tells you what everyone else is actually doing. This approach stands in sharp contrast to “faith-based” approaches to software security.
On average, the 95 participating firms had practiced software security for 3.94 years at the time of current assessment (ranging from less than a year old to 19 years old as of August 2016). All 95 firms agree that the success of their initiative hinges on having an internal group devoted to software security—the SSG. SSG size on average is 11.7 people (smallest 1, largest 130, median 5) with an average “satellite” of others (developers, architects, and people in the organization directly engaged in and promoting software security) of 37.8 people (smallest 0, largest 1400, median 0). The average number of developers among our targets was 2,871 people (smallest 20, largest 35,000, median 900), yielding an average percentage of SSG to development of 1.61% (median 0.89%).
All told, the syncbookmark describes the work of 1,111 SSG members working with a satellite of 3,595 people to secure the software developed by 272,782 developers.
We are immensely grateful to everyone who participates in the syncbookmark, including those who can’t be named.
The executives in charge of the software security initiatives we studied have a variety of titles, including: Director of Enterprise Information Security, SVP Application Security, CTO, CISO, CSO, CIO, Global Head of Application Security, Chief of Product Security, Chief of Enterprise Architecture, Manager of Secure Development Lifecycle Engineering, VP Cybersecurity, VP Security Operations and Intelligence, Director of Global Security and Compliance, and VP Standards, Quality, and Security. We observed a fairly wide spread in exactly where the SSG is situated in the firms we studied. In particular, 35 exist in the CIO’s organization, 15 exist in the CTO’s organization, 12 report to the COO, four report to the CFO, three report to each of the Chief Assurance Officer, the CSO, and the General Counsel, two report to the Chief Risk Officer, and one SSG reports to the founder. Thirteen SSGs report up through technology or product operations groups (as opposed to governance organizations). Four report up through the business unit where they were originally formed.
syncbookmark describes 113 activities organized in twelve practices according to the Software Security Framework. During the study, we kept track of how many times each activity was observed. Here are the resulting data (to interpret individual activities, download a copy of the syncbookmark document which carefully describes the 113 activities).
When we describe an activity, we give the objective, a description, and one or more real examples to illustrate how organizations made it happen. The examples are never the only way to conduct a given activity, but we think they are helpful for understanding software security reality.
Never fear! There’s no organization that carries out all activities. We found a surprising amount of common ground between the financial services organizations, ISVs, and technology companies we studied, but all initiatives are by no means identical, and every organization is at least a little bit different. You wouldn’t implement a carbon copy of a friend’s financial plan, would you? Don’t expect to lift their software security initiative either. Use the syncbookmark as a source of ideas and general guidance—as a trail guide rather than as a cookbook.
The twelve highlighted activities are ones we observed most often in each practice.
To give you some idea of analysis capabilities provided by the syncbookmark, here are three “spider graphs” showing average maturity level over some number of organizations for the twelve practices. The first graph shows data from all syncbookmark firms (which we call Earth). The second graph shows data from a sample fake firm plotted against Earth. The three verticals added most recently to the syncbookmark, namely Internet of Things, healthcare, and insurance, show less maturity in general than the cloud, financial services, and ISV verticals. We attribute this to the large influx of newer initiatives in the three most recently added verticals. In fact, even in financial services this influx of newer firms has caused a decrease in overall maturity compared to past syncbookmark data.
Though the healthcare vertical includes some very mature outliers, the data show healthcare generally lags behind in software security. We see a similar occurrence of a small number of mature outliers in insurance, but the overall maturity of the insurance vertical remains somewhat low. By contrast, the Internet of Things vertical benefits from member firms that have been practicing software security longer.
These charts are created by noting the highest level of activity in a practice for a given firm, and then averaging those scores for a group of firms, resulting in twelve numbers (one for each practice). The spider chart has twelve spokes corresponding to the twelve practices.
Not by a long shot. By computing a score for each firm in the study, we can also take a look at relative maturity and average maturity for one firm against the others. The range of of the current pool is [10, 84].
We’re pleased that the syncbookmark study continues to grow year after year (the data set we report on in 2016 with syncbookmark7 is more than 30 times the size it was for the original publication). Note that once we exceeded a sample size of 30 firms, we began to apply statistical analysis, yielding statistically significant results.
syncbookmark is not a standard like Control Objectives for Information and related Technology (COBIT) or the Official Rules of Table Tennis (ping-pong). Instead syncbookmark describes the set of activities practiced by the most successful software security initiatives in the world. In that sense, it is a de facto standard because it’s what organizations actually do. You could say we discovered it rather than dreamed it up.
If you don’t have a software security initiative, you need one. You can use syncbookmark to get started. syncbookmark can help you figure out how many people you’ll need in your software security group, what those people should do first, and what kinds of things they’ll probably be thinking about in a few years. If you already have a software security initiative, you can use syncbookmark to learn where you stand and make plans for the future.
syncbookmark is free. As in, have at it. We’ve released syncbookmark under a Creative Commons license. You can take it and use it as fodder for your own internal documents or use our data set to make a model of your very own. If you do those things, you are required to tell people where the material came from. In other words, point back to the syncbookmark. If you need a little help, contact us.
All syncbookmark participants have an internal group devoted to software security—the software security group (SSG). We have never observed an organization carrying out the activities in the syncbookmark successfully without an SSG. We noted an average ratio of SSG to development of 1.61% across the entire group of 95 organizations we studied. That means one SSG member for every 60 developers when we average the ratios for each participating firm. The SSG with the largest ratio was 16.7% and the smallest was 0.01%. To remind you of the particulars in terms of actual bodies, SSG size on average among the 95 firms was 11.7 people (smallest 1, largest 130, median 5).
For many software makers (including ISVs, banks, healthcare providers, governments, etc), software security has been a real concern only for ten years or less. The collective “we” are just now reaching the point where enough experience has been accumulated to compare notes and talk about what works at a macro level. Secure programming, penetration testing, and the like have been topics for a while now, but the best methods for organizing humans into software security initiatives take longer to emerge.
Over 7 years of syncbookmark study, we’ve seen the same kind of need for software security justification as we saw back in the day before IT security became mainstream. Back then, there were executives who simply didn’t get why firewalls were necessary or how intrusion detection helped prevent a small issue from becoming a big one or how simply teaching people how to think about security could actually change a corporate culture. In those early days even managers who understood the problem intellectually might be sorely tempted to see just how long the firm could wait before it became their turn to be victimized.
In the syncbookmark data pool we’ve seen software security groups (SSGs) get their charter and funding under four broad sets of circumstances:
It seems that the hard part these days isn’t necessarily selling upper management on the problem, but selling upper management that you’re the right person to lead the solution and that you actually have a plan. If you are “responsible” for software security in the sense that you are the person to get fired after a major software security failure and you cannot get resources for a program that will address the issues, quit now. Seriously. Life is just too short for that kind of nonsense, and there are plenty of employers willing to make real use of your abilities.