Problem Hunting with Identify Software's AppSight

I was recently lucky enough to be invited to a demonstration of Identify Software's AppSight J2EE Application Support System. I was first made aware of their product at JavaOne where I stumbled across their stand and was intrigued by what they had to offer.

The problem that AppSight addresses is how to track down, diagnose and resolve problems in production environments. In my experience when a customer reports a problem it can be very difficult to reproduce the problem in the development environment. Log files rarely contain the detail necessary to paint the picture, due to the runtime overhead that such detail entails. Even very basic diagnosis (is the problem user error? a configuration problem? a software defect?) can be difficult for anything but the most trivial system. Throw into the mix various in-licensed components provided by multiple vendors and the picture gets very murky indeed.

AppSight's solution is based on the software equivalent of an aircraft's black box recorder. This is a software module deployed on the application server that records the behaviour of the application at multiple levels. Using this information the actual steps performed by the user can be reproduced. Identify's own web site provides lots more information, which I don't plan to reproduce, but there are a couple of points which I think are of particular interest:

• I was immediately suspicious of the run-time cost of deploying such a module. However according to Identify themselves, there is a 2-4% overhead on the server and 1-2% overhead on the client.
• You can define alerts based on particular events (say database connection pool usage reaching 90%). That in itself isn't so interesting. But what I think is really neat is that you can specify that the level of information gathering be increased based on a particular alert. That is, if there is a problem, automatically start gathering more information, rather than wait for it to become a real issue.
  

There are a couple of things that occurred to me after the demo that are worth considering:

• Normally when I get a problem report, I like to use it as the basis for one or more test cases to add to my test suite. Is there any way this could be automated by AppSight?
• On a more abstract level, is there a risk of some kind of Schrodinger's cat phenomenon? That is, could the presence of the AppSight module affect the behaviour under observation?
  

After the demo I was left with the feeling that I had seen a product that I would like to use in anger. Just need to find the opportunity now...

[I should point out that I have no connection with Identify Software nor have I actually worked with AppSight in a production environment.]

Book Review: Beyond Software Architecture by Luke Hohmann

Beyond Software Architecture is a book about some of the business issues that need to be considered when developing a software system. It provides practical advice and insight into how software needs to be considered part of an overall business solution if it is to be successful. The emphasis of the book is on software products (mainly enterprise level) but there are many useful lessons to be learnt for project-based systems.

One of the key concepts introduced at an early stage in the book is the distinction between "marketecture" (marketing architecture) and "tarchitecture" (technical architecture). Under this categorization the traditional software architect (aka tarchitect) is responsible for tarchitecture and the business manager or program manager (aka marketect) is responsible for marketecture. This isn't a dry academic exercise in defining terminology: these two roles pervade the book. The underlying message is that the marketect and tarchitect need to be acutely aware of their own and each other's roles; the tarchitect needs to understand the business forces driving the product, and the marketect needs to appreciate the technical basis for the solution. Having the marketect and tarchitect roles defined is a necessary but not sufficient condition for creating winning solutions.

The book goes on to consider various issues which have implications for both marketecture and tarchitecture. For instance, which kind or product licensing model is suitable for the intended business model; what are the implications of using in-licensed technology in the solution; how should the product be branded; and so on. The point is that these are issues that the tarchitect needs to understand, be aware of and in some cases be involved in decision making for. However these aren't the kind of thing one normally reads about in books on software architecture.

For software architects working with software products, this book is essential reading; certainly in my own case, when I previously worked as architect on a product I would have found a lot of the information in the book directly relevant to my daily work. For those working on software projects, there are still some interesting practical issues such as installation, configuration and release management. However some of the other areas covered are purely related to products, so these can be skipped.

My only minor criticism of the book is that I would have enjoyed more real-world anecdotes; there are a number of sidebars in the book presenting the author's experience of a specific case supporting the more general message in the text. These were very interesting and I would have liked to see more of these.

Don't mix software and The Economist

I started reading The Economist recently and have found it to be an interesting weekly look at what is happening in the world, providing a slightly broader view than Time or Newsweek. As a rule it tends to stick to politics and economics, though occasionally subjects on the fringe slip through. For instance a few weeks back there was a special article about outsourcing that was very interesting.

When I got my issue this week I noticed there was a special article about the software development industry entitled "Managing Complexity". I went straight to it, intrigued to see what they had to say about my "back yard". Alas the normal erudition and clarity that I have come to expect from The Economist were notable by their absence in this article.

The article began by making the point that a large proportion of software projects go seriously wrong and according to a recent study 60% are considered failures by the organizations that initiated them. So far so good. In the light of the recent Child Support Agency fiasco it is also something that I have been thinking about.

Beyond this point things start to go wrong. According to the article the problem can be summed up as follows

"The Culprit: poor design"

That's it. The article proceeds to present a peculiarly US-centric survey of trends in software development such as open-source, agile development etc. The quotes provided are from US software tool vendors, so it is unsurprising that they are in agreement that the solution to the problem is to buy the newest tools they have to offer.

I could point out the inaccuracies and fallacies in the article in great detail (and I'm sure that this would provide some catharsis for me - if you hadn't already guessed. the article touched a nerve). However I think it would be more interesting to speculate on some of the issues that the article ought to have delved into.

If I had a magic answer I would of course have become rich from this knowledge; however I think there are some clear areas that cause problems:

• Poor management on the procurement side When an organization procures a software solution, there is some onus on them to know what they want. This may sound obvious, but I have worked on projects where a high-level management decision has been made that a group of users should use a new system that is to be developed, but when it comes down to it no one actually knows what the system is supposed to do. Large organizations often have a budget=power culture, so once the budget has been approved the procuring manager might not hang around very long, instead climbing further up the greasy pole.
• Poor management on the vendor side Software project management is an area strewn with lots of literature (good and bad). I don't intend to repeat any of that here. However one area often overlooked is the need to manage the expectations of the procuring organization. The vendor management needs to be intimately aware of the maturity level of the procuring organization and factor this into planning. Related to this, a less mature procurer will often come up with new requirements as their understanding of what technology can offer improves during the course of the project. Vendor management has the responsibility to explain to the procurer that expanding the scope of the project by introducing the new requirements has consequences for deadlines and budgets. Note also that increasing the budget to accommodate such an expansion will inevitably be reported by the press as a budget overrun.
• Overhyped Technology The one thing that unites otherwise warring software vendors is the claim that their new product will solve all of your development problems. This despite all previous experience to the contrary. Sometimes this overhype can get totally out of control which leads to inappropriate technology choices for a project. Witness the recent backlash against the use of EJBs.

These are the ones that occur to me straight off the top of my head. As I think of others I will add them to this list.