Debunking the Myths of Agile Documentation

By Mike Fernandez, Plaster Group Consultant

Introduction

Agile methodologies hold a lot of promise for increasing the business value of software development in a fast and nimble fashion. An important contributor to this promise, per the Agile Manifesto, is of value placed on working software over comprehensive documentation.  Rather than spending excessive time trying to create perfect and all too often encyclopedic requirements and specifications, the Agile team seeks to create the amount and level of documentation that truly provides value to their organization.  This approach keeps an Agile development team and the business closely aligned and communicating in a timely manner.

But what happens when the project ends, the project team disperses, and the application and user base is transitioned to Training, Support, Operations and Sustainment?  How do you ensure the information is captured for a successful knowledge transfer now or in the future?

Waterfall Project Documentation

In waterfall projects, a number of artifacts are created during the initial steps:  business requirements, functional specifications, technical designs, test plans and test cases.  These required written documents satisfy the stage-gate/handoff nature of the waterfall process, and are the primary method of communicating information between parties associated with a waterfall project. But the reality is that the business requirements were probably captured months, or even years ago, and have not been updated since. Technical specifications may not reflect the changes necessary to accommodate technical realities or in response to bugs found by the QA team and User Acceptance Testing.  While in theory there should be plenty of information, in reality it is often incomplete or out of date, or the members of the project team may no longer be available, and so additional effort is required to identify and capture accurate information about the system.

Agile Project Documentation

A smoothly running Agile project team will collaborate closely with the business, identifying needs and driving through iterations or sprints to develop and deploy user stories, adjusting their efforts and deliverables as the business needs and priorities change. The goal of these efforts is to provide value, so in documentation, as in all Agile activities, the team should always ask the question “where is the value in what I’m producing?”

One of the myths of Agile is that it requires no documentation, but this is patently not true.  The Agile paradigm shift from hand-offs to collaboration and face-to-face discussion naturally reduces the amount of documentation required and produced, but it does not eliminate it completely.  Agile stresses the importance of Just Barely Good Enough (JBDE) documentation to ensure delivery of quality software to the customer – the concept that the team only needs to know enough to deal with their current work.  In keeping with Agile principles, documentation is done at the lowest and most pertinent level. Some examples:

  • User Stories – commonly include acceptance criteria and some level of functional explanation
  • Code Comments – implementation details at a technical level, with some higher level explanation
  • QA Test Cases –should provide enough coverage and include enough detail and explanation to completely provide all the documentation needed on an Agile product

Agile Project Documentation for Post-Development Teams

A number of studies show that the greatest costs for an application accrue during the maintenance period.  The sources of maintenance costs are a separate and large discussion, but the accumulation of bugs, enhancements, and alterations to an under-documented codebase over the years will add up.  You can proactively minimize these costs by ensuring that the project team provides a clean, minimal information “pot of gold” for future teams.

While there may be projects in which a project team is actively engaged for the life of the application, there are many projects that will be passed from the project team to several potential teams that would need to understand the “what” and “why” of the functional and technical features of an application, including:

  • Training
  • Operations
  • Sustainment
  • Downstream systems
  • Future enhancement projects
  • Design rationale and intent

The types of information these groups would be interested in are varied, and too large to cover here.  Here are a few examples to start a discussion in your workspace:

  • Has the Product Owner included SMEs from the business during the project phase to ensure that they can maximize their internal expert knowledge and minimize formal training needs?
  • Do user stories include the information that lives behind some self-documenting application features like the use of good UI naming conventions, tooltips, built-in help files?
  • Have the “why’s” of decisions that would affect usage or future understanding been captured? Or only the “what” that was implemented?
  • Can the artifacts that the Agile project team created be easily found?
    • User Stories
    • Code (with commenting)
    • QA Test Cases and plan
  • Will project knowledge survive the dispersal of team members to other projects or roles or is embedded in their heads and/or email trails?
  • Can the system functions be easily located?
    • Source data
    • ETL – scheduled jobs, usernames and passwords
    • Destination databases
    • UI
    • User authentication

This is only the beginning of what might be needed to ensure that the application deploys, is used and maintained efficiently.  There may be additional company conventions or regulatory requirements that need to be considered as well.  The value of what an Agile team produces should extend beyond the deployed code – it should also allow current and future application consumers to easily leverage the current application for their future uses.

Depending on which of the above information is captured and what the organizational requirements are for documentation, there may be a need for additional work and/or reformatting of captured information.  Much like documentation created during project work, the documentation effort should wherever possible follow JBGE intentions.

Tools & ERPs

A smoothly running Agile project team will collaborate closely with the business, identifying needs and driving through iterations or sprints to develop and deploy user stories, adjusting their efforts and deliverables as the business needs and priorities change. The goal of these efforts is to provide value, so in documentation, as in all Agile activities, the team should always ask the question “where is the value in what I’m producing?”

One of the myths of Agile is that it requires no documentation, but this is patently not true.  The Agile paradigm shift from hand-offs to collaboration and face-to-face discussion naturally reduces the amount of documentation required and produced, but it does not eliminate it completely.  Agile stresses the importance of Just Barely Good Enough (JBDE) documentation to ensure delivery of quality software to the customer – the concept that the team only needs to know enough to deal with their current work.  In keeping with Agile principles, documentation is done at the lowest and most pertinent level. Some examples:

  • User Stories – commonly include acceptance criteria and some level of functional explanation
  • Code Comments – implementation details at a technical level, with some higher level explanation
  • QA Test Cases –should provide enough coverage and include enough detail and explanation to completely provide all the documentation needed on an Agile product

Depending on which of the above information is captured and what the organizational requirements are for documentation, there may be a need for additional work and/or reformatting of captured information.  Much like documentation created during project work, the documentation effort should wherever possible follow JBGE intentions.

Next Steps

Agile project teams have a JBGE approach to documentation that serves them well during the active project work, but which may fall short for subsequent teams in the future.  If your entire IT environment has embraced an Agile mindset, this may not be an issue.

If Agile is gaining (or looking for) a foothold in your organization, finding the right balance of documentation can be a challenge.  This can be a great opportunity to look at your current state, and evaluate if there may be better ways to accomplish your goals.  Even for areas or processes that are not currently Agile, use an Agile lens to view current process and documentation and ask, “does it provide value?”  Simply leveraging that viewpoint can help make your organization more effective.

Plaster Group’s Agile consultants have experience with evaluating and helping companies get started and improve their Agile practices – we’d love to help you out!

 

Maintaining System Metadata and Version History with SharePoint APIs


Utilizing SharePoint technology results in a familiar tool that meets our client’s precise needs at a greatly reduced cost

Challenge

In order to meet the needs of their team’s new goals and objectives, our client designed a new SharePoint site to store and manage their files. However, their legacy site contained all of their existing documents and files which needed to be retained and migrated to the team’s new site. Custom metadata fields, version data, and all default system information fields from the old site also needed to be migrated which is not possible using out-of-the-box SharePoint functionality.

The client purchased and implemented a third party product to do these things, but when deployed, it didn’t meet all of the requirements. Enhancing the performance of this third party product would require a lot of configuration, and, consequently, substantial time and additional funds. With a deadline approaching, our client needed a way to migrate the content with version data and metadata intact before their project timeline ran out.

Plaster Group’s SharePoint consultants were tasked with either fixing the performance issues of the third party product, or crafting a solution with other in-house technologies capable of migrating our client’s content while maintaining metadata and all version information.

Approach

Rather than attempt to use the complicated third party tool, Plaster Group suggested a custom solution using existing SharePoint APIs to provide thorough, cost-effective content migration. Because of the short timeline, Plaster Group consultants worked closely with management to identify and prioritize the functions that were most essential.

Solution

Using SharePoint and its available APIs, Plaster Group consultants developed a command line tool that allowed administrators to migrate content from the old site to the new site while maintaining existing system metadata and version information. The new tool is capable of:

  • Mapping document libraries in their entirety
  • Grabbing relevant document data, system metadata, and version data
  • Aggregating older data and metadata with new metadata automatically

Results

The new tool allowed for on-time delivery with a looming site migration deadline without the need for third party tools. Utilizing familiar SharePoint technology has resulted in a simple tool that meets our client’s precise needs at a greatly reduced cost. Our client has been able to use it to specify exactly which files needed to be migrated to the new site, with the added ability to:

  • Modify the metadata around the document if they need to change or specify new attributes for their content
  • Remap to a new metadata taxonomy
  • Move documents without losing historical version data, and maintain system information metadata
  • Reduce manual re-tagging

October 7 TDWI Event: Data Driven Decision Making with Indeed

Plaster Group is again looking forward to sponsoring our local Seattle chapter of the The Datawarehousing Institute this coming Tuesday, October 7th. This event features the topic “Data Driven Decision Making.” Douglas Gray (Sr. VP Engineering) and Chris Hyams (SVP Product and International Strategy) of Indeed, the #1 job site worldwide, will share how Indeed captures and integrates data into its business decisions effectively. Don’t miss out on this great opportunity to discuss and learn about topics in Business Intelligence while meeting other local professionals.

This event will be take place Tuesday, October 7th from 5:30 until 7:30 at Group Health Headquarters located at 320 Westlake Ave. N. For more information and to register for this event to attend, visit TDWI’s event page here!