GitArchitecture — a better way to capture Architectural decisions

By Kyle Brown

What is the problem?

There has been a conflict between software architects and developers almost since the discipline of Software Architecture was formed. There have been comparisons drawn between software design and architecture (of the bricks-and-mortar type) since at least the 1960’s (when the comparison appeared in the 1968 NATO conference on Software Engineering). Not surprisingly some of the negative views of bricks-and-mortar architecture (as in superstar architects designing things that could never be built) came along with the analogy. As Software Architecture became a fully formed discipline between the 1990’s to 2010’s, one of the most common complaints from developers became more prevalent — that the Architects were working in an “Ivory Tower” that was removed from the day-to-day work of the developer.

There have been other books and articles that have addressed many of the root causes of this disconnect (for instance, see Simon Brown’s “Software Architecture for Developers”) but there’s one specific problem in particular that I’d like to address in this article. That smaller problem lies is the fact that the artifacts produced by Architects are not often in a form that is easily consumable by developers and operations teams, and that the artifacts are removed from the tools suite that developers use. What I’m going to propose is a solution that we have been using inside the IBM Garage for the last several years (since at least 2016) that helps bridge that gap. But before I introduce the solution, I want to talk a little bit about the solutions that have been tried before and what the pros and cons of each have been.

What has been tried before?

There are as many different options that have been tried for addressing this issue as there are companies and teams that have faced the issue. However, in order to keep the discussion short, let’s look at three major cases of approaches that are among the most common.

General Purpose Drawing and Word Processing Tools

Probably the most commonly used tool for creating software architecture and design diagrams today is Microsoft Powerpoint, closely followed by Microsoft Visio. While there are plenty of free and open source drawing tools available (e.g. draw.io and others) and tools such as OmniGraffle have their adherents, the sheer ubiquity of Powerpoint and Visio make them heavily used in industry and academia.

However, setting aside the diagrammatic issues that you have when using a general-purpose drawing tool for drawing software diagrams, the first major issue is that you have to then introduce a separate tool for the inevitable text that has to accompany those diagrams. Thus, we often see software architecture documentation written in Microsoft Word, with Visio diagrams pasted in as needed.

The problem is that this technique results in a Microsoft Word or PDF file that is already separated from the normal workflow of the developer. They have to first find the document, read the document as you would read a book, and then figure out ways of annotating or reporting problems with the documentation if discoveries in the implementation of the concepts presented in the documentation crop up. That has led many over the years to try the second major approach, dedicated software architecture tools.

Dedicated Software Architecture Tools

Given the problem of using general-purpose tools for documenting software, it’s not surprising that there has been a robust industry in developing software engineering tools that are closer to the workflow of the developer. An example of this would be Rational Software Architect (RSA), an IBM product originally developed in the mid 2000’s that allowed architects to build documentation in the Unified Modeling Language, and then to check those diagrams into the same ClearCase or other code repository that the developers used for their code. What also became popular during this time was the idea of applying Model Driven Design (MDD) and Model Driven Architecture (MDA) techniques to generate initial code artifacts from the diagrams directly, linking the two parts of the workflow more closely together.

However, what often happened is that this type of approach usually devolved into a form of Big Design Up Front (BDUF) in which the architects would develop all of the diagrams for a system, and then use those to generate the code artifacts — which would then often become divorced from the realities of what the developers were learning in the implementation phase. While many tools (including RSA) included reverse-engineering features, these were often not used because of difficulties in reconciling the diagrams as originally intended with the actual implementation choices. What’s more, a division between the experiences of the architect and the developer with the tools often led the developers to disregard the artifacts produced by the architects. That led to the perception of architecture as an ivory-tower exercise we referred to earlier. This dissatisfaction is one of the factors that led many teams to try using Wikis to document their decisions, since that facilitated communication in a way that was thought to be more egalitarian and less formal.

Wikis

The original purpose of Ward Cunningham’s first Wiki Wiki Web was that it formed the basis of his Portland Patterns Repository. It was meant to provide an interactive tool to facilitate collaboration on capturing and documenting patterns of Software Development, Architecture and Design.

Wikis introduced the concept of working in a simple, intuitive markup language that allowed architects and designers to actively update documentation instead of allowing it to become static and out of date. What’s more, most Wikis have evolved to allow for both freeform text and graphics to be embedded in the pages. All in all, they were a huge leap forward in giving architects and developers to work together on interactive documentation that could change over time.

However, Wikis, while a major improvement over what came before, were not perfect. In particular, one of the issues with many Wikis pages is that they will often degrade into what people began to call “Thread Mode”.

In Thread mode, a contributor is unwilling to change the original content of a page (either for reasons of not being the original creator of a page, or for reasons of wanting to leave the original page contents in place for historic purposes). Instead, they add additional content at the bottom of the page, often under a separator. This begins to resemble a threaded conversation as in many email programs, chat rooms and platforms such as Reddit and Slashdot whose interaction features derived from BBS’s. The problem with a threaded conversation is that the most important information (such as a change or revision) is often well down the page instead of in the upper part of the page where it should reside. So while they are a definite improvement, the social mechanics of using Wikis often leaves them less useful than they could be.

However, the biggest problem is not in social mechanics, or in the features of a Wiki, but in the fact that the Wikis themselves are not part of the developer’s day-to-day workflow. So, while Wikis can always play a part in architectural decision making, especially as a format for structured discussions, they don’t serve the purpose of representing the primary documentation of architectural decisions, patterns and designs if that location should be one in which the developers themselves work.

Meeting Developers where they live

Instead, what we have concluded is that the best location for storing architectural decisions, architectural patterns and reference architectures, and design artifacts is in the very place that the source code (in all forms, including infrastructure as code and configuration as code) is stored for most development teams — in Git repositories.

Everyone should be familiar with the idea of Git as a code repository. You may also be familiar with the idea of GitOps (pioneered by Weaveworks) that leverages Git as a repository for code and configuration for Infrastructure as Code and Configuration as Code that uses the Git features and pipelines to drive infrastructure automation. What you may not be as familiar with is that Git has proven to be a remarkably flexible and powerful tool for developing design documentation and capturing design decisions. The primary benefits of using git that we have witnessed firsthand have been:

1. Markdown is a very simple language to learn and author content in. It is easy to write nicely formatted documentation just by using a markdown editor (either the one that is available on GitHub.com or a commercial tool like Xcode) without having to memorize complex markup commands.
2. Markdown makes it very easy to link documentation pages together through hypertext links. In fact, it can very easily be used as a simplified version of the wiki editing language used in wiki pages — while clicking on a blank local link does not create a new page, the visual differentiation between one with and without an active link (e.g. without a link in parenthesis in markdown) makes it easy to know which linked pages need to be filled in. That allows you to write linked wiki-style pages just as easily as you can in a wiki.
3. The built-in version control (it is, after all Git!) makes merging the work of multiple authors very simple. This is especially helpful in avoiding Thread Mode — you can always revert back or easily diff a markdown page to merge in multiple contributions. What’s more, once developers, operators and architects get used to the idea of treating markdown pages as just another type of code artifact, they more easily fall into the appropriate kinds of behaviors that they follow with code artifacts — cleanup becomes more second nature.
4. Graphics are handled just as easily as text in Github. You can create graphics using any number of online tools (such as draw.io) that check directly into Github, or you can create graphics locally on your laptop using whatever tool you like and check them into Git very easily.
5. The final advantage is of course, that it is simple to directly integrate your documentation and design decisions in the same repository, or a sub-repository as your source code. Keeping the two together makes it extremely unlikely that they will go out of sync — especially if you open GitHub issues against design documentation in the same way as you do against the code.

Lightweight architecture documentation rules

What these advantages bring is way to build very lightweight architecture documentation that avoids most of the issues we have seen with other tools. However, it’s important to keep some simple rules in mind in order to make the documentation effective:

• Your primary documentation should all be written in markdown stored in your git repo. You should use the same repo as for your code — it’s OK to create separate directories in the repo, but you want to keep the documentation in the same repository with the code in order to put it in front of the developers when they need it. The same technique applies to Infrastructure as code — keep your Kubernetes Operator code and Ansible scripts together in git with the design decisions that tell you what these infrastructure options should do. Another key thing to remember is that while formalisms are useful, simple prose narratives are often the most helpful. Tell the stories of why decisions were made along with a representation of the decision.
• Focus on a few key diagrams with a common notation. Nothing gets an architect more upset than to question his use of notation, and fewer topics are more fraught with disagreement. For the purposes of what we are referring to — almost any notation will work. The real key is to keep the number of diagrams very small, and to make them simple enough to be interpreted by those without special training. If your notation requires a complex key to decipher it, it’s probably too complicated. A favorite of ours is Simon Brown’s C4 notation, but a subset of UML can work just as well. We make our services notation available as template for several popular drawing programs.
• Don’t go overboard with templates and structures. Some structure is required — you can’t have every page of architectural documentation be completely freeform. But keep the structure simple. It’s often enough to just have a standard form for capturing an architectural decision, and another for capturing the elements of an example or reference architecture.

As an example, in the IBM Garage Architecture Center we’ve used just a couple of types of diagrams to represent our reference architectures. We use block diagrams for high-level concepts, and then a simple format for representing cloud services and concepts for the lower-level details. An example of one of the lower level diagrams can be seen in the figure below.

Example simple cloud services diagram (in Github)

Examples of both can be found in many of the architectures in the IBM Garage Architecture Center, like the one for Runtime Modernization of Java EE applications.

Standards for Architectural Documentation

Probably the most common need for a project team is in storing architectural decisions — this is just a record of a decision made at a point in time so that everyone can not only understand the decision, but so that it can be followed in the future.

Architectural Decisions are often about cross-cutting concerns — things like platform selection, language selection, framework selection, etc. Since they are cross-cutting, they often have impact on multiple squads. However, they still need to be easily accessible and consumable by developers. This means they need to be stored in a top-level git repository accessible by everyone in a tribe or guild.

One simple option is to create a markdown template that you “fill in” with details for a particular decision. For instance, we’ve used a template containing the following items in several client engagements (this roughly follows the scheme of an IBM internal work product sometimes called the “ARC100”). The sections are:

• Name
• Architectural Decision Summary Description
• Issue (or problem)
• Motivation
• Alternatives Examined
• Resolution (or ramifications)

As you can see, it closely resembles the structure for patterns documentation (see The Hillside Group for multiple examples). In fact, patterns are often a part of the architectural documentation we produce. For instance in our event driven reference architecture we have had to add definitions of common patterns such as CQRS that were used by the reference architecture. Another technique that we use in that architecture is to use GitPages for longer, tutorial-like content. If you’re telling an extended story, GitPages gives you better formatting and presentation options than markup in github alone.

Another option we have used at some clients is to use GitHub issues and take advantage of labels to represent the workflow of the decision. That is a bit more complex to set up, but helps in other ways like assigning clear ownership of the issue and decision.

Our Experiences with the technique

We have been using these approaches extensively since the creation of the first solutions in the IBM Cloud Architecture center in 2016. Roland Barcia was the person who pioneered the idea by leading a team to build our initial Microservices solution. We have since expanded out to more than 20 solutions built using these techniques, and have applied this approach at hundreds of clients through the IBM Cloud Garage and with our partners in the IBM Garage. Shahir Daya led in some of the earliest client engagements using the techniques above, including introducing the use of GitHub issues in addition to using GitHub to store architectural documentation pages. The architecture center pages have been viewed well over 600,000 times, and our solutions written using these techniques have proven to have lasting impact in communicating how to help customers adopt IBM technologies.

Bring your plan to the IBM Garage.
Are you ready to learn more about DevSecOps and delivering value with the IBM Garage? We’re here to help. Contact us today to schedule time to speak with a Garage expert about your next big idea. Learn about our IBM Garage Method, the design, development and startup communities we work in, and the deep expertise and capabilities we bring to the table.

Schedule a no-charge visit with the IBM Garage.