Problem Hunting with Identify Software's AppSight

I was recently lucky enough to be invited to a demonstration of Identify Software's AppSight J2EE Application Support System. I was first made aware of their product at JavaOne where I stumbled across their stand and was intrigued by what they had to offer.

The problem that AppSight addresses is how to track down, diagnose and resolve problems in production environments. In my experience when a customer reports a problem it can be very difficult to reproduce the problem in the development environment. Log files rarely contain the detail necessary to paint the picture, due to the runtime overhead that such detail entails. Even very basic diagnosis (is the problem user error? a configuration problem? a software defect?) can be difficult for anything but the most trivial system. Throw into the mix various in-licensed components provided by multiple vendors and the picture gets very murky indeed.

AppSight's solution is based on the software equivalent of an aircraft's black box recorder. This is a software module deployed on the application server that records the behaviour of the application at multiple levels. Using this information the actual steps performed by the user can be reproduced. Identify's own web site provides lots more information, which I don't plan to reproduce, but there are a couple of points which I think are of particular interest:

• I was immediately suspicious of the run-time cost of deploying such a module. However according to Identify themselves, there is a 2-4% overhead on the server and 1-2% overhead on the client.
• You can define alerts based on particular events (say database connection pool usage reaching 90%). That in itself isn't so interesting. But what I think is really neat is that you can specify that the level of information gathering be increased based on a particular alert. That is, if there is a problem, automatically start gathering more information, rather than wait for it to become a real issue.
  

There are a couple of things that occurred to me after the demo that are worth considering:

• Normally when I get a problem report, I like to use it as the basis for one or more test cases to add to my test suite. Is there any way this could be automated by AppSight?
• On a more abstract level, is there a risk of some kind of Schrodinger's cat phenomenon? That is, could the presence of the AppSight module affect the behaviour under observation?
  

After the demo I was left with the feeling that I had seen a product that I would like to use in anger. Just need to find the opportunity now...

[I should point out that I have no connection with Identify Software nor have I actually worked with AppSight in a production environment.]