Can formal requirement methods work for agile?
By Shobha Rangasamy Somasundaram & Amol Sharma
Formal methods adapted and applied to agile, provide clear and complete requirement, which is fundamental to the successful build of any product. The product might be developed by following Methodology-A or Methodology-B, which changes very few things as far as knowing what to build goes. So we could safely state that the development methodology used by the project team could be any, but good requirements are absolutely necessary. The manner in which we go about eliciting and gathering requirements would differ, and needless to say, this holds true for agile development too.
That said; let us take a look at what most inexperienced agile practitioners think about requirements.
It is a common belief that the requirements stage has become optional the day a project moves away from waterfall methods, and documenting requirements is not important anymore. While agile favors involvement of the entire team for creating the set of features, but in practice most development teams are involved only after first level requirement have been captured by the product owners and stakeholders. The team then creates a sprint backlog out of the feature list, based on business priority value and their velocity at the beginning of each iteration, and details out the features into user stories. Most requirement analysis at this level is done informally. Supporting this view are results of surveys done in the industry, with lack of documentation constituting 26% of the practitioners concerns (the only other bigger concern being that of lack of upfront planning). Most agile developers and managers feel there is no room for documentation whatsoever. This is only part of the story, and there is more to doing requirements in agile.
In traditional software development or waterfall process, the following methods are used during the requirements phase - brain storming, questionnaire, modeling, prototyping, observation, focus group, survey, reverse engineering, interview, document analysis, workshop for joint application development (JAD) - collaboration & domain model creation. In waterfall, requirements are sourced from the client, the BA and the product owner, wherein, they interact and prepare the final requirements document. Once the requirements are finalized, they are conveyed to the development team. There is little scope of negotiation within the team. It is majorly a standalone effort.
In agile, we do things differently from traditional or waterfall approaches. While gathering requirements in waterfall, the dependency or focus is on one person or selected persons. In agile, we would have an involvement of many members of the team. Put plainly, requirements in agile are no longer committed to the beginning of the project or limited to a few individuals, but are a perpetual driver for the entire software development lifecycle. Agile does not prescribe any one way to document the requirements, the focus is instead on "just enough" documentation. Details are discovered and unfold slowly only when they are required. The monolithic dedicated "requirements stage" is broken and dispersed, that it now becomes omnipresent, with the same traditional analysis methods, performed throughout the lifecycle.
Stages in agile requirements
Initial requirements elicitation is done at a very high level and usually comprise of the product vision, business need and the product roadmap. This involves creation of the theme of the product to be eventually developed. The next stage would lay down the set of features which would fulfill the product theme. UML modeling in the form of workflows and use case models are deemed useful in these two sub stages. Typically use case models would be used to establish the most useful features for the user and help in forming a priority value for the feature set in these stages. To re-iterate, use cases in agile are nothing different from the waterfall use cases, only that they are not created upfront, but in stages, as and when required.
Following the features, the team gets into the next level of details called the epics. An epic may be defined as a high level super feature, which might just be too big to be covered in a single sprint. It would be further broken down into user stories, a related bunch of which could form the major part of the next sprint. Stories are short scenarios, with acceptance criteria the user can understand. Each story would then be detailed into tasks, which are then taken up by individual team members to develop, test and deploy.
Each of these stages usually takes the form of a workshop, performed just before the specific detail is needed. For instance, the theme & feature workshops would be done in the pre-development, sprint-0, an epic workshop could be conducted before a release, a story workshop held before each sprint, along with an ongoing backlog grooming/ revisiting exercise which happens parallel to development work in a sprint. If we look closely at the stages we have mentioned above, most of the formal methods dealing with high level business requirements in waterfall, can be used (and they are) in agile's pre-development sprint. The pre-development sprint typically is undertaken to set the ground for future development, to thrash out the high level architecture etc.
Mapping of formal requirement methods to agile stages
Defining the product vision - Conduct market study/ survey/ brainstorming/ forming focus groups/ joint application design workshops/ models
To define the vision, a pre-study would be done for the product to be developed, by completing a market study, or a survey. A shared vision would be agreed upon by the team by brainstorming, by forming focus groups or by conducting JAD workshops. These sessions or workshops would enable the team to understand the overall scope of the project by drawing context diagrams, supported by entities & conceptual activities - which gives the scope and boundaries of the system. The high level process modeling which gives a picture of the entire application can be defined by using workflow diagrams at level-1, level-2 process, and level-3 process. On capturing these requirements value realization model can be utilized to find out the business value. The process can be re-used by referring to standardized framework such as APQC, e-Tom etc. The key here should be to define a wide view of requirements i.e. breadth first, depth later.
Feature workshop - use case diagrams/ context diagrams/ data modeling/ Logical diagram, domain analysis diagram/ functional diagram
Feature workshop - Release workshop should be conducted to understand the features for the release by keeping in mind the business needs of the client, the business value and architectural dependency of the features. The Feature workshop shall focus on high level backlog items which includes creation of use case diagrams, Workflow diagrams, personas and data modeling, quality attributes etc. The process models drawn can be associated with value realization model to calculate the business value. The prioritization of features can be done using value based requirements prioritization tool.
The Logical system architecture should be carved at this point which includes division of modules, layers and components within the boundaries of a group of features. The system components, their interrelationships, protection of data, should be clearly nailed down. This information will lay a strong foundation for architecture where the user will be able to visualize how the end product will look when deployed in a specific environment and specified technology.
This should be depicted using simple diagrams like Logical diagram, domain analysis diagram, functional diagram etc. The stories that are dependent on each other should be depicted using sequence diagram, so that the functionalities are listed appropriately. The Release runs for 3-4 months with 4-5 sprints which should be bounded with clarity in requirements and well defined architecture
Sprint planning & Backlog grooming - scenarios/ user stories/ work flow models/task flow models/ white-boarding/ flip-charts
Sprint workshop includes elaboration of requirements for each user story like User acceptance test, test cases, Business Logic, Wireframes, Constraints. Detailed Requirements workshop should be scheduled to detail out the requirements for the current sprint. Here the decisions about the specific design patterns are made and this is the place where the architecture becomes software. Based on the complexity of the user stories, use cases can be narrated which explains the main scenarios and also alternate scenarios for each story. Level 4 of the Process diagram and task flow diagrams should be modeled wherever applicable. The task flows drawn can be used further to generate test cases using Automated Test case Generator tool (GenTUF). Model checker shall be used to check the effectiveness and inconsistency of the models. The workshop will bring out all the minute detail of the stories which will be accompanied by flipcharts/ prototype, which gives the navigation flow so that the developer is aware of JUST in time requirements and is extremely clear what has to be implemented.
The formal methods of capturing requirements work for agile projects, helping to have clarity on project requirements. The diagrams and artifacts created provide references and complete details for all user stories, which could be traced. The successful agile teams usually team up for key decisions and make use of the requirements captured during agile iterations and workshops. This makes for an informed and proactive team, which is aware of feature road map for the product, and possesses an actionable release plan.