Controlled Agility

Quality in Requirements

For most of my time in software, I’ve been told that the requirements are the source of most defects (and misunderstandings). You can get this picture from many articles, magazines, books, etc. My experiences in this area tend to support this assertion – even when it’s me writing the requirements. Because of this, I’ve tried to improve my requirements gathering skills by applying practices from books such as this and this. I have participated in document reviews, mandated that people review my documents and lent heavily towards agile practices to eliminate this bug-ridden process from software development that I’m involved in.

What has surprised me most is that there has not been a good way of augmenting human abilities with Requirements Quality. What I’m really talking about is an automated assistant or consistent quality test when writing requirements. There are many options with unit, system and integration testing but none it seems for requirements. There are many requirements management tools, but they just seem to turn documents into databases. I heavily use word’s grammar checking capability. It’s helped me reduce passive sentences from my documents. I have also used word’s readability statistics to help me understand how complex my documents are. However, these practices wax and wane with my interest and commitment to other things, like deadlines.

This unease with requirements tooling is on it’s way to being resolved. This document from the SEI discusses a project that applies lexical and syntactic analysis to indicate the quality of a requirement.  A quick summary follows. There are three main requirement characteristics that this tool measures. Unambiguity, the ability of a sentence to have a single interpretation. Understandability, the ability of a requirement to be easily understood by any reader. Specification Completion, the ability of each requirement to uniquely identify its object or subject.

To break it further down, each characteristic group has the following characteristics.

Unambiguity
Vagueness – vague words
Subjectivity – opinions or feelings
Optionality - when the sentence contains a part that can or cannot be considered
Implicity - object of the sentence is generically expressed Weakness - sentence contains a weak verb

Completeness
Under-specification - when a sentence contains a class of object without specifying an instance of this class

Understandability
Multiplicity – when there is more than 1 subject or verb
Readability – statistical method to calculate readability.

The tool processes each sentence and analyses it for the above characteristics. Sentences that are possibly defective are highlighted and require the reviewer to make decisions. What this means is that reviewing or creating a document will be simplified, leaving human reviewers to concentrate on more high-level tasks.

There’s another potentially fantastic feature, that of views. Views are subsets of a document that deal with a particular area. For example, when viewing the “security” view of a document, it shows you all the statements that meet the criteria defining the security view. Obviously, the view is only as good as the criteria, but fine-tuning the criteria is a small problem to deal with once you have the view. Imagine centralizing all the requirements about scalability, or performance, or any of the other ‘iliities – brilliant.

Unfortunately, this tool is not available for download in any place I can see. Its home page can be found here. I would love to see a tool such as CaliberRM build this into their toolset, the quality of requirements would increase dramatically.

software requirements

what to count in EAI platforms

Danesh posts about estimation in EAI. A topic near and dear to me. In it he discusses the variability of artifacts in two different EAI products. Fundamentally both solutions offer the same kinds of constructs, but with some differences and ommissions. 

One way of estimating things, the most common way, is to decompose a toolset and count the number of each artifact, assign complexity then multiply it all out. The issue you have with this is that firstly, it requires some very detailed understanding of the design. At the project costing stage this type of information is difficult to come by. Secondly, it doesn't leave you with a customer (or value) focused estimate. It's technical hurdy-gurdy for the customer. Thirdly, it doesn't lend itself to easily to comparison across time or other toolsets. For example, new artifacts (eg BPEL composition) will require a new estimation metric making it difficult for comparisons with past data. Also as Danesh points out not all tools have the same artifacts.

One solution is to map Function Points onto EAI artifacts. It turns out that the core artifacts of FP map remarkably well to the things we manipulate in EAI. The issue is constructing some rules around what fits into which box and when.

So where does it leave you gentle reader? Possibly with yet another tried&true IBM technology invented for the mainframe, shoe-horned into everything else.

 

eai function points estimation

Function Point Inflation

The central problem I'm trying to solve is to apply a sizing metric to Integration style software projects. The kinds were we take one proprietary format and transform it to another format. For example, take an XML sales order request from a website, turn it into a SAP RFC ORDERS05 document and then the response back to the client from RFC to XML. My company does a lot of work like this.

After reading the David(s) book I was really left none the wiser about how to apply FP to this work. I accidentally joined the IFPUG thinking I needed to be a member to read the mail forums. Turns out the forums are public. In any case there are quite a few postings on "middleware" and even a few on "eai". I've read them all but still left with no clear feeling on this. The best posting I read was posted almost 3 years ago.

Additionally, the IFPUG has some guidance on counting middleware applications in a paper titled "Hints to Counting Middle-ware Applications" available to members. I found this whitepaper only mildly illuminating. The problem I had with a lot of these resources is the inclusive definition of "middleware". Basically, middleware is defined as anything that sits between an endUser application and the datastore. So a database driver is middleware, an OS is middleware, an EAI transformation and routing engine is also middleware. From a purely architectural standpoint this is true, but there is a vast difference between an eai transformation engine and an OS.

So, what I've done is composed a couple of common EAI scenarios and pointed out how I intend to count them. I've posted a message to the IFPUG mailing list to garner comments on this post. I hope the discussion continues either here or there.

The scenarios are designed to build upon each other in complexity and to exemplify the common things you might do with an eai engine. To give these scenarios "new-technology" polish [circa Oct '05] , you can visualise the scenarios as executing on an ESB to realise an SOA. Or you could just call it eai.

I hope you can understand my diagrams.

Scenario 1 - A simple XML input is posted to a legacy system

In this scenario an endUser application posts an XML document to the EAI server. From the EAI's point of view, this is an External Input. The inputXML is transformed to a canonical format which is considered an ILF. The ILF is never persisted but I count it so because the "hints to counting middleware" paper lets me. From the canonical the data is transformed into SAP RFC format and finally leaves the EAI system as an External Output.

The reason why I've broken it up into an EI and an EO will be explained in scenario 4.

Any comments on this?

Scenario 2 - an enrichment

Lets go back to scenario 1, and introduce data enrichment as part of the first transformation step. It's often a requirement that you need to lookup some value before passing it onto the backend. For example mapping zip to suburb or somesuch. This enrichment process I have chosen to model as an EIF. In most cases, this EIF will be a simple EIF because you often get only 1 or 2 fields.

Is this the correct way of thinking about this functionality? Or should I count this an EQ?

Scenario 3 - a simple xml request/response pair

Is the same as scenario 1 and we've lost the enrichment, but there is a response sent back to the endUser application. This essentially duplicates the FP count.

scenario 4 - addition of another endUser appplication

Scenario 4 illustrates why it's important to count the input and output as different constructs. Initially, I thought it would be adequate to count a single flow through the eai layer as an EO, but then it wouldn't allow me to count additional end user applications distinctly. Consider the fact that this addition comes at another time, so in some sense it's an enhancement. I want to count that fact that I only need to half of the work I previously did. In this scenario userAppB means there is an additional EI and EO that needs to be counted.

Is this the best way of going about this?

Conclusion
So, in conclusion, do experienced FP counters have any issues with this? My goal is not to compare across companies, but establish some kind of model within our delivery group.

the function point discussion can be found here

eai function points estimation

Why Function Points

In writing the next post, I realised that I needed to explain why it was I'm trying to understand and use Function Points.

I've spent a few months reading about and trying to get my head around various estimation techniques. I've read  Function Point Analysis  by the two Davids as well as joined the ifpug  (more about my experiences of this later on). I've also read various papers on other estimation techniques such as MarkII and other FP derived methods. Fundamentally, at this stage, it looks like FP is the most used and comprehensive method available. So I chose to take a deeper look at this.

In a nutshell, Function Points provide a common denominator across different technologies. The kinds of metrics I think are  useful to measure within a delivery organisation are things such as; 1) delivery-rate improvement over time in a technology, 2) size of assets under management, 3) costs of managing various technology stacks etc.

1. delivery-rate improvement over time in a technology
Consulting groups are the first to adopt new technologies. The first project will be a large learning effort for everyone involved. The following projects will each improve upon the one before - you hope, but have no way of knowing for sure. Knowing your delivery rate is a bit like understanding velocity.

2. size of assets under management.
We need to know how much functionality we actually manage, how much it grows. From an architecuture, or application portfolio perspective it's critical to know how big the thing is. When we need to replace it, what is it we are actually replacing? In business all assets are tagged, yet software has no inventory model. Could FP be this fit?

3. costs of managing various technology stacks
The company I work for completely supports some of the applications it builds. How do we determine how much effort is appropriate for a support team? How do we know if it's worth investing dollars to improve the quality of an application so that our costs of support are lower?

These are just some of the questions that I think FPs are well suited to answer.

The other aspect of function-points I particularly like is the end-user value perspective it takes. The Lines of Code, or size of database or whatever are of less concern than the value these things bring to the business. Function Points are more value based, it seems. I think other constructs such as FDDs Features or XP stories could play a similar role.

There is one huge caveat I wish to highlight. I'm not looking to compare productivity across organisations. I just want to be able to measure within our organisation