Creating actionable software standards for a portfolio of products (part 1)

Why is there a black diamond at 19.2 inches on my tape measure? And what does this have to do with software architecture and design? Well, the short answer is that 19.2 inches equally spaces studs of a normal thickness for an eight-foot section of wall framing, providing space for the starting and ending boards. The next question is really the focus of this article: the measuring tape is a way for the person designing a structure to communicate specific intent to someone else — or multiple other people — who may do the construction. In software we call this a standard, and it often helps communications between a designer, lead or architect role to a developer, who may never have seen one of these applications built before. One of the most important tools an architect has is the development of an organization-wide standard, and nowhere is this more useful than when the team is building and maintaining a portfolio of multiple products.

Just as if we were building multiple houses simultaneously — or maybe fixing the first one up which was built ten years ago and adding a new floor to a second home built only a couple years ago, and a third where we’re putting a new foundation on a dirt lot — the standard is a concise way to keep all three meeting the same customer expectations. To finish the analogy — and I’ll admit, we watch a lot of home improvement TV at my house — we have a number of craftspeople working around the job sites with varying degrees of skill and experiences. A software architect not only has to come up with the how we define a standard, but also has to ensure its adoption, through a socialization process to make a standard effective.

Developing the standards

“The good thing about standards is that there are so many to choose from.” — attributed to Andrew S. Tanenbaum

Let’s talk about the when, what, and why of standards first.

There are different times when we may want to develop a standard in an organization:

• We have a new team or a part of a team which is new to the technologies involved, the products or the domain[1]
• We have a project which has been in a different “mode” and is transitioning from maintenance to new customer demands for innovative feature sets (or conversely reducing technical debt)
• We are bringing product families together which have been developed in isolation from each other, and which may need to be part of a branded portfolio, especially when integrations are being considered[2]
• We need to adapt existing products to new markets, transforming assets in one domain to another and revalidating value propositions — the enterprise software expectations of our customers may be different over here than where we started, or
• We may have a new tool/process that adds value to our software supply chain which is not opinionated enough for our team to adopt without guidance[3]

Fundamentally, a standard will be designed around solving a particular problem, most likely a quality attribute of the products we want to improve which itself represents culture change in agile, sometimes called a “maturity step.” Common quality attributes we are concerned with, from the architecture side, are things like performance, security, interoperability, reliability, ease of use or deployment, test coverage and extensibility. These are the diamonds on our “ruler.” A software standard is just an agreement that we will build an application, component or service a specific way within a team.

Standards may contain a number of items,[4] so I’ll give you a quick checklist of what might be in one and follow with some principles I try to bear in mind while writing a standard. What goes into a standard?

• An identified scope for the standard, where it is applicable and where it is not
• Required aspects, a description of each and how it will be assessed or measured
• Optional aspects and guidance, and
• References

It’s that simple! Well, really, there’s a bit of art form in what comes next. Early on, I wrote a standard that looked like Figure 1 below.

Table of Contents of a sample standard

Making a great standard

My standard had some opinions in it, some consensus from the development team, some decisions (where I included the thought process in the standard itself), and no mention of how long this would be in force. There are a couple key learnings here: as we know from architecture decision records, all things have a finite shelf-life and should be revisited (refactored) from time to time, and the document is way too long for a developer — they really struggled with this, not from a technology perspective but in that they had to keep coming back asking for clarifications. I hadn’t made it easy to adopt. We’ll show version 2 — exactly how this evolved into something adoptable — in the next post, so don’t worry.

The first thing that we did right here is knowing where to start with a standard: this standard was based on the OpenAPI v3 specification. As an architect, we first look around for things that we can reuse — why create something new when it may already exist and it may have stood the test of time. We can look for starting seeds for our effort in places like:

1. Other organizations like ISO, NIST and IETF — for example, time and date ranges, REST HTTP codes

2. Framework, process and tool suppliers — for example, .NET Framework conventions, Docker file layouts, Angular learning guidance, ITIL processes which may be already known to our developers

3. “Well known” implementations — in the case of a language like Uber’s Go standards, or Netflix service descriptions

4. Legal and regulatory sources — for example, PCI DSS, SOX, SOC2 and HIPAA conventions and audit standards

5. Our own experience — previous projects or organizational (company) standards, and

6. De facto standards — where a tool may support one logical way, such as unit testing framework in our programming languages

The second thing we did right was to frame a first standard as one in a series of standards: identifying which we would follow-up with after the first achieved some adoption. This was important to show that systems on a roadmap converge over time, and that many aspects may be dealt with in the same development effort without having so many standards as to slow down development while we align. We limited our agile standards to:

• An Open-API standard since we have a number of products developed over the years to different integration methods and we want to standardize our approach for partners to extend our software
• A language standard for how we use Go (and soon Ruby) which includes starter applications for a couple common application types, since we are starting new development alongside old development and have learned some ways of doing things we would like to standardize — we also chose not to do standards for languages embedded in products we are only maintaining
• A container standard for how we envision the lifecycle of building and publishing Chef products and components; containers are a new frontier for us
• A source code management standard for how we do gitflow, so that developers can move between teams and keep their bearings, and
• A quality assurance standard for which functional, security and performance tools we’ll use as we ship various products — we leave test construction to developers or pair-programmers but customers, auditors and management need evidence of these efforts

Finally, let’s wrap up the best practice section with some guiding principles to the architect writing one of these up:

• Publish the standard in a process that encourages contributions from the developers[5]
• Make sure everyone understands the standard — this may require overcommunication, demos, socialization and/or training
• Put the measurability of required elements at the top
• Make the standard visible — don’t bury it in PowerPoint!
• Report on the alignment to the stakeholders — the developers and management need to see that someone cares and that there’s data to help make good decisions on
• Collect feedback on the standard after a few months to see if it needs to be updated

Conclusion

In this article, we covered some of the when, what and why we use standards in software architecture. In the next post, we’ll finish up with details of some examples and some of the interpersonal skills needed to get a standard adopted broadly by the team. Hope you’ve enjoyed this post and that it helps in your architecture journey!

[1] This could be a replacement team taking over from a previous team, or we may have hired less senior developers to augment a team, just that the culture or way of working is not shared between earlier folks on the project and the current development staff. At Chef, we have a globally distributed group of about 200 developers plus an external contributing community.

[2] This is the case with Chef.io, which started a decade ago and is now part of a bigger portfolio of infrastructure management products

[3] A good example here would be adding a strategic planning tool for SAFe implementation that requires a convention around definition of ready or definition of done

[4] Standards in software have a long history — check out the IETF ones at https://www.rfc-editor.org/search/rfc_search_detail.php?stream_name=IETF&page=All

[5] We use a request for comment (or RFC) process that allows for publishing, comment period, and disposition of comments before a draft is approved