7 years on

As I look back over the last seven years since I took a leap of faith and stepped away from the norm of full time employment to start my own company, CATCH. It's been interesting to see the change in the organisation from a start up with no set operational base to one that has premises and is selling globally. People have come and gone over the time period and what we stand for has been refined, redefined, and refined some more to the point we are now at. It also fascinates me when looking at my own personal thinking over this time and how it has changed. What I thought was the best way to do things I look at now and think was immature and needed refinement, what we delivered I still believe in so aren't unhappy with what we achieved but the biggest thing I look at is the strategic areas, especially the business model and how it has been refined over time.
 
To reminisce, we started supplying services to organisations and grew the business at a rapid rate through long term contracts and a bigger and bigger team, was it the business that I wanted, I wasn't so sure so three years ago we enhanced the strategy to include our own products. A bold move at the time to invest everything we made back into the development of a product. But I'd done my research and saw that IBM, Apple, Microsoft etc grew at greater rates than the normal market when they had majority ownership by an individual or family. The truth be told I was naive as to the journey I was starting on, I remember thinking lets build product and sell it to our customers, it will provide a revenue stream and enhance what we provide. Hmmm so what really happened. Well we started to develop our product 'TMS' (Test Management System) we allocated staff to the analysis, design, project management, development and testing of the tool and there was our first challenge we utilised existing staff - what do you mean this wasn't a good thing to do? - well yes it means that you can use your services staff that have downtime from client assignments on product development - a good thing, and then the big big big no no... these staff now have two masters 1 - clients that they are assigned to and 2 -the product project manager... one pays the organisation while the other costs the organisation - you won't need three guesses to know who wins when push comes to shove. We (finally) realised that only dedicated resources would produce a high quality product and allow us to realise the vision.

Lesson One: Use dedicated resources

So we renamed TMS to Enterprise Tester http://www.enterprisetester.com/ and released it to the market. Looking back it was too early, not ready for the market and way undercooked (I'm probably being very harsh but I'm allowed to be)  but you know what, it was the best decision we made. In releasing it to market we put ourselves out there smack bang in the middle of the market place for all to see.

Lesson Two: Get it out in the market as soon as you can so you can have more eyes on it and put the acid on the team.

We then sat back and waited for the sales to roll in... or not. What did happen was we received lots of people downloading and installing, then emailing us about enhancements, issues or feature requests they'd like to see.

Lesson three: Love the feedback

That was great, we had interest, what we needed now was to get sales to start the next part of the vision, but what we had to sort out first were the contact requests we had received, or did we? Well we decided to get further releases out to improve the software, and our thinking was the quicker the better so that in around four weeks we released a new version of the tool with new features, fixes and improvements. We received as many, if not more feedback from customers so the only thing to do was to continue implementing short releases, so we continued with around four weekly releases of the software. After several months we got into the groove and quick releases become a way of life, and not only for us but for our customers too. They looked forward to new releases, providing feedback and seeing the latest enhancements to the product on a significantly more frequent timeframe than anyone else in the Test Management space - pretty cool!

Lesson four: Release often and build community ownership

We continue to release new versions of Enterprise Tester in rapid fashion improving the feature set and bettering the way it works, making things easier and simpler for the users. The system is significantly bigger than when we started and the complexity has increased and I keep thinking there must be a point in time that it's finished... though I'm coming to the conclusion that it may never be finished, just continue to be improved and enhanced. We have a roadmap which sits at about 2 years and we (in my mind it's never fast enough) move through these planned releases on around a six to eight weekly timeframe now.

So after this time we have distilled things down to three principles we hold key:
1. Iterate quickly
2. Stay close to our customers
3. Platform play

We believe if we continue to be true to these principles then we'll continue to grow and develop a world leading test management tool that customers will greatly enjoy.

I personally hope you have enjoyed the journey so far, learnt a heap on the way and I look forward to sharing the next seven years, that I'm sure is going to be as much, if not more fun and contain many more growth opportunities.
Happy testing!

Incentives? Do they work?

Recently a number of areas of life have been discussing the application of incentives to drive performance.
This discussion continues to fascinate me and I personally swing from (what it feels like) one extreme to the other, maybe it's the blonde in me :)

Looking at North America based organisations they tend to offer options and share plans as a standard part of the package with staff picking up extremely attractive deals along the way, in fact I understand that some even make a career out of 'jumping' from one start up to the next with the hope of striking it big along the way. Almost like playing lotto in my mind, though with hopefully a much better shot at getting something out of it.
There is a book "The Great Game of Business" by Jack Stack which I picked up for the first time around 10 years ago and over the years keep going back too and re reading. Very briefly it's about open book management within a struggling machine shop that is at the point of closing down. Jacks approach was to give everyone a slice of the action, open the books, make them accountable for individual line items within the corporate accounts and meet every morning to go through the numbers. As you'd expected people started thinking bigger than just their individual roles and in fact roles become a label with people working and assisting in areas that had the most need as they all had a stake in the game. Great book, great story but does it really work in practice?

I have a lot of people who say it doesn't work at all and people will do as they please. Those that work hard have it in their DNA those that don't just don't and nothing will change it.
To jump a little, every Saturday I take my eldest son (5) to his rugby game, for you non fanatical New Zealander's think Gridiron with no pads or helmets, you only pass the ball backwards and a single team plays both offence and defence. Now I recently had my son run up to me and ask what I'll give him if he scores a try, my response was "nothing you do it because you enjoy playing the game". It was only after that did I find out that others in the team were getting ice creams, or toys if they scored try's or met certain try & tackle goals. Now has my blunt "you get nothing" response dented his enjoyment for the game? It doesn't appear to have, he's excited to go each Saturday getting up at 5:30am each morning and has scored at least one try a game so far... but could he have done better with an incentive scheme? or is the only thing that really matters if others get an incentive and you don't? Did business build a rod for its back by offering incentives so now everyone has to? It feels almost like that standard economics example they give you when you attend one of your first lectures: Two suspects get arrested by the police. The police split them up and question them. If both say nothing then they both get 6 months, if both tell on the other then both go to jail for five years and if only one talks that person goes free while the other gets ten years. Probability shows that they both talk getting the worst result for both more often than not. Now I'm not saying that offering incentives is bad, in fact it drives innovation around the world, look at the space challenges, professional sport, and businesses across the globe. I just wonder if it's the best way, or if there is another way to do this instead.

Really a moot point at five years old but still interesting to mull over.

The other extreme would I suppose be to offer a disincentive, though this doesn't sound great to me, you know something like "If you don't get the sale then you're fired" , or in my sons case "If you don't score a try you walk home..." yeah, I can't see this working and it's not part of my DNA but is there any other way?
Could it be an aligned vision as in Principle centered leadership by Stephen R. Covey so everyone strives for the same thing and the shear momentum excites them to the point that the experience and satisfaction of achieving the goal out ways all else?
I assume also that the person or people setting the incentive / disincentive scheme up need to be able to sleep at night which I'd guess would also have a bearing on its final makeup.

Not sure of the answer, but it's interesting to see different models and their creators at work around the place.
Let me know what side of the fence you sit.
Bryce

Test Management or Quality Management

Interestingly, I've been thinking about test management and whether this is the correct term to be used for Enterprise Tester, our test management platform. Test Management seems to me to be a limited term that is a subset of the true aim as to what we are looking to achieve. Sure adding tests, executing tests and reporting on tests are all part of the story. But what about intelligent information gathering around test cycles that have occurred (or sprints if you are using agile).
I like the term 'Quality Hub' since we take the view that test management is a part of the Quality initiative across an organisation, yes stopping bugs being released into the wild is one thing, but actually preventing bugs occurring through the life cycle of the product is another. The latter depends on systems, processes and people that are in concert with each other with an aim of quality output no matter what stage of the production process they are in. A sort of self healing way of working where adjustments are many and frequent as measurement data is feed back to each section or team. This also brings to mind an interesting question "What is quality?" Traditionally I would have said no (or many few) bugs found within the product would constitute a quality product. I'm wonder about this now days with the advent of agile practice, and while I'm not an avid agile fan I like some of the concepts though struggle with others so sit squarely in the middle borrowing techniques from one or the other in a more blended (and what I believe has greater benefits overall) approach.

But interestingly enough I've recently spent a good portion of time walking to and from a client site and having to cross roads at certain points. Now this one day I took the most direct route to the client site (not surprisingly) but interestingly another person (a stranger I didn't know) took a different route to mine but managed to get there first. Now it wasn't that they went the most direct way, nor were they moving faster instead they managed to get the luck of lights changing while I stood waiting to be able to cross the road. This while a simplistic example got me thinking... is it just better to keep moving toward your goal no matter whether it's the fastest or most direct way as long as you continue to move forward? Maybe this is the best thing to do.

This thinking also leads me in a roundabout way to have an answer to the question "What is quality?" Maybe quality has to be measured over a period of time? Maybe something that is perceived to be poor quality now could be high quality in the long term? or vice versa maybe something seen as high quality now could be low quality over the long term?
Take our approach to the product development of Enterprise Tester. We look to release a new version every 6 - 8 weeks and have been doing so for the last two years and have managed to release 18 versions over this time. Do we get it right every time? I'd like to think so, but if we don't what do we do? Our way around this was to have fast release cycles so in quick fashion we can make updates and get back to customers with resolutions or improvements. Call it an insurance policy if you will.
So how do other product manufactures get on with six or twelve month release cycles? To that matter, how do their customers deal with it? The customers business doesn't stop so do they just incur the additional cost through inefficiency? Surely it is madness to continue under this slow release model.

So how do customers look to ensure that they are buying into a 'quality' product? Do they have criteria that they look at to ensure the product they purchase has the commitment of the organisation to continue over the long term to develop, improve, support and stand by their customers ensuring that their customers business runs smoothly on the platform they provide?

Does this mean that existing products that have slow release cycles that are perceived to be of 'high quality' are now actually bad purchases? If they can't change and improve fast enough and leave their customers businesses with more cost then what is the justification? Could it be a perception of lower risk? It does seem that risk or perceived risk is top of mind with organisations, though in this case again maybe perceived low risk now could be an indicator of high long term risk.

Interested in your thoughts.

Test Management Tool - New Release of Enterprise Tester

Well it's been a busy couple of weeks for the team putting together the next release of Enterprise Tester.

Enterprise Tester (http://www.enterprisetester.com/) is a Test Management Tool that allows for traceablility between your UML analysis model through test cases and scripts to any issues in your issue management tool.

While other Test Management Tools attempt to do everything from requirements management to issue management we at Catch (http://www.catchlimited.com/) have taken a more focused approach believing that a Test Management Tool should be just that, not a defect tracker nor should it be a requirements analysis tool. Most if not all organisations have a defect tracking tool and / or an enterprise issue management tool so our view is why buy another and duplicate capability that you already have or if your organisation doesn't have one then why should your test management tool determine the issue management tool your organisation uses?

That's right you should be able to use the best of bread tool of your choice that fits with your enterprise's architecture and allows you to minimise your technology footprint not make it more complex.

The other thing we have tried to do is make the testers life easier, yes that's right, if you are buying a Test Management Tool then you should get some benefits from it besides that fact that you are transfering paper forms and information into an electronic format. We run a consulting company and so a big driver for us was to improve the speed and quality of the work we complete. From our internal studies we have found that Enterprise Tester will save you approximately 4% minimum of your total project cost. That's right run a 100K project and save 4k and at the same time pay for the Enterprise Tester license cost - simple :) .

Why is Enterprise Tester different to other Test Management Tools?

Because of our consulting background in the past we have ( far too many times) been given specifications to develop test cases and scripts from. This is the industry standard but we also find that our test analysts have to question, query and update these specifications and try to keep everything in sync with the analyst and developers. With our own team we use Sparx Systems, Enterprise Architect (http://www.sparxsystems.com/) and in fact we like it so much that we last year for fortunate enough to also be allowed to operate Sparx Systems New Zealand (http://www.sparxsystems.co.nz/).

Enterprise Architect is a UML modeling tool that allows an organsiations Enterprise Architecture to be modelled, including the ability to model projects and automatically generate specification documents. Enterprise Tester integrates with Enterprise Architect to automatically generate test scripts for you, no more duplicating the work of the analyst and no more having to keep things in sync, the tool does it for you.

In this release we improved the reporting capability so that you can associated a number of precanned reports to your projects and customise them by selecting the relevant criteria. From a base set of 5 report templates you can generate approx. 30 different customised reports. We have also allowed users to export any report into a number of different formats including CSV, RTF, PDF, XML, etc. so you can use them in your current document templates or save them in your project directory or document management system.

Along with the improvements in the reporting area we built on the tools reporting to allow users to reference the reports they have created on their dashboard. In this way each project dashboard and users personal dashboard can be customised by 'pointing' a report portlet to a separate report and displaying the associated chart giving the user real time reporting on the projects they are interested in and at a glance the status of the testing being undertaken.

Users can now see what happening with the testing side of a project in an instant. No more trawling through data reports to understand the current status.

Other features we have tried to improve is that we have given the user the choice as to executing a script in a step by step manner, may be for Use Acceptance Testing, or via a multi step grid that allows the test analyst to see all steps and update them quickly while being able to see any attachments and incidents associated with any of the steps.

In addition we attempt to make the test analysts life easier by automatically populating the project, script, step, expected & actual result and related information directly into the incident record to save time and effort in revording this information and speed up the testing process.

Enterprise Tester takes reuse into the testing arena by not only allowing automatic generation of test cases and scripts from your UML models but allowing you to reuse script templates and reports within and across projects.

We aim to add additional features on a monthly basis based on the requirements of our user community. Not only will a number of new interfaces to other popular incident management tools and automated tool be added but we have had requests to release a single user license. We will look to release this in the coming month.

To demo the tool go to http://www.enterprisetester.com/ to download a trial version or have a go on the online demo at http://demo.enterprisetester.com/enterprisetester
username: demo , Password: password

Let us know what you think
Happy testing
Bryce

UML - why wouldn't you?

UML has been around for a significant period of time. So why are so few organisations using it?

• We want to use a global standard for the work we do
• We want to use standard notations and be consistent in our deliverable
• We want the ability to use the same high standards anywhere in the world

As a consulting organisation with global aspirations there is no other choice, so why do organisations all around the world believe they can do it better by developing CASE tools and modeling tools that are not based on UML?

Additionally, don't organisations using non UML based tools realise that by doing so they limit themselves to continuing to use their customised standards limiting their ability to grow. Not only does the organisation need to develop custom training courses, frameworks and usage guidelines but from their employees point of view they are making them less marketable once they leave, while at the same time, ensuring any new employees must go through a period of up skill to have the ability to deliver outputs to the desired organisational standard. Cost of recruitment rises dramatically while still having little or no certainty as to whether the new staff member will be able to make the grade even after the period of training and up skill.

UML allows us to recruit in any market and know straight away the technical skill of the UML professional. You could argue that we don't even need to limit ourselves to English speaking countries since a picture tells and thousand words and a UML model allows us to do the same.

Model Driven Development can be achieved using UML Platform Independent Models (PIMs) speeding up the development cycle and allowing industry and situational patterns to be developed for reuse later on. There seems to be a large number of advantages of using UML while few or no disadvantages.

I'd be interested in your thoughts...

Bryce