Controlled Agility

Total lockdown

I've been busy the last couple of weeks. Spent a week learning about IBM's Business Modeler tool. The last week I've joined an architecture group of a small bank to help them through their estimating backlog. My immediate thought was to David Anderson's drum-buffer-rope case-study but it doesn't apply to this world.

Anyway, as soon as I can figure out how to balance a totally locked down environment with my read/write habits.

Flat file parsing

Ok, I love flat files. I love them because I need to love them. Everywhere you go, you run into flat files. To be an XML purist is a painful plight. For such a dominant technology I am amazed that there isn't a java "best practices framework" for doing this. The number of times I've heard of people just writing "a little parser" for this simple flat file which turns into a monster java class that deals will all sorts of flat file vageries, is uncountable. In fact, there's a project right now at one of our client companies that is doing a full edi integration with java code and xslt.. the horror! the horror!

Anyhow, I've found a nice little engine that can. ServingXML seems to have created a generalised flat file processing engine. If you look at the example configurations you'll be amazed what it can do. Edifact no less!

ServingXML does a whole lot more, but the flat file bits are the bits that thrill me.

tags: eai

Event Driven Architectures in the wild

Just wanted to draw attention to two postings about this topic.

On darth.net, there is a comprehensive treatment of some challenges of this type of architecture.

On Panic From Fuzzy, there's an excellent series on lessons learned in implementing an EDA.  Lessons and confirmations for everyone. I look forward to the

tags:eda, tags:cep

you can lead a horse to water but you can't make them drink the Koolaid2.0

Denham Grey posts the new KM2.0 dream. A social software space for within the enterprise. His vision comes down to; "Imagine if every business built its own wikipedia!."

Nice dream.

I've been involved in an effort to do just such a thing. We used confluence which is a bliki(blog+wiki). It pretty much covers all the technical bits and pieces. The problems it leaves unsolved are unsuprisingly social and process based.

The organisation I worked for had approximately 50 people, working at software services, so let that be the context.

Skills2.0

People didn't use it(the bliki), and when they did, they didn't use it properly. Everyone is on a different point of the learning curve when using this stuff. Some people don't understand wiki markup, some people don't understand trackbacks. Some people don't understand the domain of the business. So don't feel confident to make pages, refactor mercilessly, etc. Some people aren't motivated to learn how all this works. In the self-selected 2.0 world, we all know how to do this stuff.. Some people just don't give a sh*t about it.

Content Issues

In a learning environment, what you get is that people write things into the wiki because they think it's important due to the fact that it's new to them. Once it's written, however unfactually, they don't revist it. Read the heading titled "organisational context" for comments on content refactoring.

Lack of contribution was another problem. We dealt with this by instituting(not mandating) a daily status report, where people would quickly sum up their day in their blog. Often this ended up being some kind of task list. We tried to encourage a contextalizing of the task list. So that people can add their own colour to the posting.. Some people are good at writing, some people suck. Some people understood that what we're really trying to do is add keyword clustering to work item history and helped, others couldn't give a rats.

Lack of Structure

Because it's so unstructured, it allows people to make things up as they go along. Lack of metadata within the page does not allow easy filtering of content. In some ways, this can be rectified by tags. you can read more about tag quality in a previous post. Tags aren't really metadata either, you can't say give me all the content where the person(s) involved are; x, y, z etc.. you rely on quality of search, quality of tagging, both of which are dubious.

Organisational context

Unlike wikipedia there is an implicit respect for someones work within an organisation. If I write something, people who consider themselves less experienced, or less skilled or whatever, are very hesitant to change anything. Wikipedia editing is merciless due lack of any personal context. In wikipediaLand it doesn't matter if your a CEO or a lineDroid; inside the firewall, it matters a lot! And, we were a flat organisation, with very little grey hair and still this deference to "respect".

Intranet/Internet divide is false(ish)?

Having a divide is hard. If you're a web2.0 person, with your delicious and bloglines and rss and ajax and blah blah blah, it's hard to contribute to two knowledge bases. I got to a point where wiki pages where subscribing to my various feeds.. but that felt a bit unnatural. It would be great if you could somehow plug your internet stuff into the intranet world, and then unplug it when you left.. I dunno.. it's hard.

Incentives

Why should some droid in Sector 7G bother to fix content he knows to be wrong? He's not there to fix that content, he's there to resolve some issue for a customer. His KPIs are around cycle time, not content quality. What's needed is some way of measuring contribution. A count of edits, creates, modifications, links, etc per person so that they're encouraged to contribute. All integrated into a 360degree review of the individuals interactions with the organisation..

So, even in social software, the problems are social. It's nice to have the dream, but it's really hard to change people. And they seem to have minds of their own too - often thinking contra to what they should be thinking. Contributions to the Enterprise Knowledge Base are easier in the web2.0 world, but the incentive problems are still the same. Wikipedia is it's own microcosm that works, the ecological profile inside an organisation is vastly different.

tags:km, web2.0, social+software

More on expertise locating

Nice article on Expertise Locating by some former IBM staff. Adds some nice context to my comments about how IBM are using new tagging techniques for the same thing.

tags:km

What is Quality XSD?

In my time in Integration, I’ve written a lot of XML schemas. But which ones have been "good quality"? Thanks to Priscilla Walmsley of "XML schema’s" fame I’ve written reasonably sophisticated xsds. But have my sophisticated schema’s actually been good? Or have they just been over-engineered?

The higher levels

At the highest level of schema design is the entity modelling. How do you model the business entities? Do you use the tried and true techniques of data modelling? Or do you use some fancy technique such as "Asset Oriented Modelling", which claim to be more specific to XML style work. I haven’t had the opportunity to (re)model an entire category of documents since I found out about this, but it looks interesting.

Another interesting method of XML design is put forward by the gentlemen who run the DocOrDie blog. Of course, they have a book too.

The lower levels

A couple of years ago, a colleague said my xsd style was the Venetian Blind style. What? xsd’s have styles? Exactly, what is Venetian Blind style, and should I be using some other style because it’s better?

Evolving a schema is hard work. Most web service implementations are brittle as buggery. When you generate wsdl2java what do you have other than highly coupled RPC mechanisms? I’ve talked about versioning schema’s before. This discussion goes further, into what is good schema?

In past projects I’ve tried to get some quality in my schemas. I’ve used IBM’s Schema Quality Checker as a first pass. This is a good tool, but doesn’t really help with higher level heuristics. I’ve also used MindReef’s SoapScope to make sure my wsdl (and by extension xsd’s) are WS-I compliant. Surely WS-I will help some way? In many ways, what I want at the lower levels is some kind of code inspection of my xml, much in the way intellij inspects your java code as you create it, the same thing for xml. At the higher levels, I want something that will speak to me about design trade-offs etc.

The first thing to deal with is this "Venetian Blind" thing. What is it, and is there anything better. I found an article on the web here and basically, no, the Venetian blind model is the best way to structure your xsd. It turns out that the Open Applications group use this style as well. And since they are the creators of the Universal Canonical Model of All things, who am I to argue?

In searching for quality heuristics, I find a gentleman by the name of Boonserm Kuyvatunya who’se done some deep thinking about this matter. The best of his articles is called XML Schema Design Quality Test Requirement which talks about how various groups, OAG, UBL, DoD, etc go about getting XML schema quality.  This is a good foundation document for your own schema quality initiatives.

On the matter of Schema Evolution, he doesn’t have much to say. On the same conference paper’s site where I found Boonserm, I also find Best Practices for XML Schema Evolution in Application Development. This is a paper, never submitted, by a BEA employee who works on XMLBeans. Looked really promising.

What it probably would not have mentioned, is how do you version schemas? Do you version the whole schema? Do you version the types individually? I still don’t know. I’m becoming close to the idea of versioning individual types and then composing them into a versioned schema – anyone else think like this, or think this is ridiculous?

In sum, the work of Boonserm allows the architecture group to automate some aspects of schema quality. In fact, Boonserm is developing a tool in support of these aims. 

XML Quality is a fertile area. I suspect that over time there will be much throwing away of XSDs as we muddle through to what is good in Schema quality.

tags:

schema+quality

WBS estimating - bad?

A random thinker draws some estimating archetypes. The archetypes are true, as you'll recognise. He has the obvious players, the optimist, the pessimist, the padder. Interestingly, he has the generalist being the one who can't be sure. In my experience, the generalist is often the person who'se been one of the other archetypes and realised the error. They've come to the conclusion that it's really hard to predict the future, especially in a field as soft as software.

His solution to the generalist's insight is to teach them the WBS process. Hmmm, can't say I'm a fan. I've never seen a WBS process work well. The unplanned tasks often swamp the time spent in planned tasks. I should get a PMI certification, that will make me a better planner. Here's a couple of other views on the scientific management of software processes, some of my thoughts and check out these two webcasts from someone much cleverer than me.

There some discussion of the psychological biases in estimating here, which bring to light some reasons for the failing of WBS approaches.

tags: estimation, project+management

SAP process models are broken

SAP process models are broken, if you consider the findings in this paper courtesy of BPTrends monthly email. I took a look at the tool that these people used to figure out that 5.6% of SAP models are broken but couldn't really figure out how it worked. The tool is designed so that you can convert any process model data and feed it in and it'll tell you if your process is broken.

It may well have some relationship to the common business event logging side of things too..

tags: bpm, logging

Advanced Team Foundation Webcasts

A couple of advanced agile(or lean) webcasts from David Anderson, author ofAgile Management for Software Engineering. These are logical continuations of the webcasts I mentioned before. As always, you can run these webcasts at double speed without losing understanding or accent.

Excellent discussion on how MS has combined the Agile and CMMI models. It’s interesting to see how much debt there is to people such as Deming and Crosby in modern day approaches to software development. Makes me wish I had aimed for more than a Conceded Pass in my Operations Management subjects at Uni.

David takes the audience through a nice high-level view of team foundation showing the various concepts that underpin the artefacts. Nice. Also shows how the CMMI-flavour is setup for a SCAMPI appraisal.

Also, on a related note, I’ve finally got my trial version of foundation server running on the virtual server system here. My advice on this is, RTFIM (read the f*g install manual), unless you know how all the MS portal/db/.net infrastructure all works. I’m mucking around with the method guidance, will try and build my own method flavour. Have pointed our method guy to the server, as his first intro to this idea of managing software properly, but the path to change is so hard, and long.

tags: agile, team+foundation+server, software+management

Emergent Navigation

The folks at Adaptive Path, make some interesting comments about emergent navigation based on tags. These comments add more to my post the other day about tagging and Enterprisy KM
.

Emergent navigation is definitely useful for the navigator of an internal knowledge base.