The Eight Levels of Project Success

Different parties involved in projects define 'success' at different stages creating confusion as to what 'success' really is and causing real success to be missed

Just what is 'success' and how do you measure it?

We have a dichotomy between the reported success rates of projects and the perceived success rates by the project fraternity and business management.

The reported project success rates are poor. Forty-percent of projects leave their organizations worse off, 15% fail completely, over 60% are ‘challenged’ in one or more ways, and real success is achieved by only 5% of projects.

However, businesses don’t see this level of poor performance. True, projects are expected to be a hassle, and being ‘on time and on budget’ may be seen as a challenge, but overall businesses see their project results as “okay” and continue to invest.

I came across a similar dichotomy many years ago where the IT Department claimed they had never missed a delivery deadline while the business believed they had always been late. This dichotomy was resolved when I discovered that the IT Department measured when the project was installed and the business measured when the solution worked and could be used.

This illustrates the root problem here—that the business is using a different set of measures of success to those reporting project success rates.

When would you call ‘success’?

For any project there are eight points at which ‘success’ can potentially be claimed. The earlier in the cycle that you choose to claim success the more likely you are to achieve it—but the lower the value of this ‘success’ will be.

Let’s take a seemingly easy example—the installation of a major new production machine. Consider when you’d claim success for this project:

The machine is physically installed.
The machine is physically installed and working.
The machine is physically installed, working and the operators know how to operate it.

The equipment vendors will usually depart at this stage claiming ‘success’.
The machine is physically installed, working, the operators know how to operate it and the engineers know how to maintain and repair it. It is ‘in production’.

The project team is now wrapping up and preparing to go. The solution is in, up and running—that’s their job done. (NB In the case example above this was the IT Department’s measure of success.)
The machine is physically installed, working, the organization knows how to operate and maintain it and it is integrated into the end-to-end production process (ie it works in business-as-usual production).

This is usually the business manager’s measure of success—the solution is working and is now part of ‘how we do business’.
The machine is physically installed, working, the organization knows how to operate and maintain it, it is integrated into the end-to-end production process and the expected benefits (eg lower cost production or faster or greater throughput) are realized (ie it is operating to the expected performance specification).

This can often be senior management’s measure of success—they can see some (performance) benefits from the delivery of the project/machine.
The machine is physically installed, working, the organization knows how to operate and maintain it, it is integrated into the end-to-end production process, the expected performance benefits are realized and the resultant business outcomes (eg greater market share, increased margin) are achieved.

This is the real measure of success. Has the investment been translated into tangible improved business results? Did we achieve the business outcomes, benefits and value that we sought when we commissioned this project investment?

So you can see the different results that would genuinely be considered "success" by the different parties — and that no one is measuring results level 7 as success.

The project team is claiming result 4 as ‘success’ — the machine is in, up and working.

The immediate business managers are seeing result 5 as ‘success’ — as now the machine is part of the ongoing production process.

Senior management seeing result 6 as ‘success’ — it is working to the performance specification. But whether this is really ‘success’ or of value depends on whether this (internal) performance improvement translates into (external) market results.

Therefore, the only real measure of success is result 7.

When you consider the reason the organization commissioned this project (even if it never clearly articulated it) was to enable greater market share and increased margins, then this must be the true measure of success. In this example, generating lower cost products or increasing throughput capacity is of no value unless this translates into improved results in the market.

Indeed, if the business outcomes are not achieved the whole investment is a waste—you’ve just invested, say, millions, but not reaped the business rewards you wanted. That is failure regardless how well the project is managed and delivered.

Now the (fallacious) hope is that ‘build it (the project outputs) and they (the business outcomes, benefits and value) will come”. This is not true. You can deliver all of the ‘success’ measures from 1 to 6 but fail to achieve success level 7—which is the measure of success you set out to achieve.

For example, in the systems environment a major asset management system was developed and installed to meet a detailed functional specification. Each of the specified functions was ticked off as delivered. It was ‘successful’ at levels 1 to 5. However, the organization no longer worked. The end-to-end processes had been ignored and gaps existed in the functional specification so that, although the specified functions had ben fully delivered and (some of) the process performance improvements had been achieved, the business operations had been compromised and the business outcomes had not been delivered.

Unfortunately, this level of non-success is the norm because you are only ever likely to achieve the desired business outcomes that represent ‘true success’ when they have been clearly specified and targeted—otherwise you are relying on hope and (false) assumptions.

The repeatedly confirmed figure of around 40% of projects leave their organizations worse off than before the investment is where success measures 1 to 5 can be met but there is no performance improvement (6) or outcome achievement (7). This is what can happen when you choose to target and measure the wrong results.

Meanwhile, the full, clear, specific definition of measurable desired business outcomes (7) is still rare.

Why are the business outcomes so rarely identified?

The answer is quite simple—we’ve lost the plot.[1]

We lost the plot in the 1990s when the management of projects moved from being a business concern to being an expert specialization. This change led to the people responsible for delivering the project no longer living with the results (and often not even being employed by the organization). This in turn led to ‘projects’ taking on a life of their own outside and independent of the business instead of being tightly integrated and leveraging the business staff and their knowledge and knowhow.

For example, recently a major bank working on the complete overhaul of its banking systems moved the project team to a new building away from the rest of the bank “so it could focus on the systems”. This isolated the project team from everyday banking and bank people—a recipe for disaster.

On instructions from the ‘project experts’ the business has pulled back and left projects to the ‘experts’—the project managers, business analysts, technical architects, change managers, benefits delivery managers and so on. Add in the extensive use of contractors and consultants and you can see a whole range of activities going on outside of business-as-usual that line managers do not see, manage, control or even understand, but are then expected to welcome and adopt the outputs of and translate them into business improvements. This is a high-risk process that continues to fail on the true measure of success.

There is now a fundamental gap between the business and projects. Indeed, many project managers complain how difficult it is to get business staff assigned to projects. Meanwhile business managers speak in terms of projects as “alien constructs” rather than as the means by which they will improve the business and deliver their strategy.

It is time for the business to retake control of projects, to wrest back control from the ‘project experts’ and give it back to the ‘business experts’ so as to be more successful.

But the mindset, processes and skills required to achieve this are largely missing or have been usurped by the project fraternity (often in order to get the job done rather than as a power grab).

To date the project answer has been to tack-on a business aspect to the existing project processes (eg MSP[2]) to deal with the business issues. This is wholly the wrong approach as it misunderstands the role of projects.

Projects are business initiatives. They are how the business executes its strategy and improves the business. They are, therefore, a business concern and should be business led, directed and controlled. But how to do this?

How the business can take back control—three easy steps

Here is a simple, three-step process for businesses to regain control of their projects:

1 Insist on each project defining its desired business outcomes

Each project should start, not by defining the project, but by defining a set of clear, specific and measurable definitions of exactly what the business’ desired outcomes are. Desired outcomes define exactly where the organization wants to be after the end of the project when everything is working ‘just right’. Delivery of these business outcomes is the project investment’s primary measure of success, as achievement of these outcomes will then deliver the desired business benefits and value—a level 7 result.

Definition of an initial set of desired business outcomes for a project takes only a couple of hours. These initial draft outcome statements then need to be refined and validated which takes a while longer. But, basically, you can define a set of outcomes within a week.

2 Plan all required (change) activities from these desired outcomes

Once you have your list of desired business outcome statements you can identify all of the changes required to move from your organization’s current state to its newly defined future state. This sounds obvious but is rarely done as the business outcomes are not defined and the current state is too often ignored.

Instead everyone sets off to “do” without having identified all of the activities required up front. Then, as the project progresses, “new” activities are discovered that extend the time and increase the cost. This is totally unnecessary.

When you know your desired business outcomes you can identify your current state in detail (and what you don’t know about the current state) and then identify all of the activities required to move from your current to your future state. These activities will include, as relevant, systems activities, new structures, new job roles, changed processes, infrastructure changes and so on. Whatever is required to deliver your desired business outcomes. You can now deliver the level 7 result.

3 Measure real progress, step-by-step

Each change activity (identified in step 2) should be specified to deliver a measurable output—a plan, a training course, a new policy. Delivery of these outputs should be planned and scheduled so that you know each month exactly which measurable outputs should be completed and delivered that month.

Then you measure each output’s 100% completion to the quality required. If a scheduled-to-be-delivered output is not 100% complete AND to the quality required, then it has not been delivered and your project is running late.

This 100% complete measurement discipline is very simple and easy to administer, and accurately tracks real progress.

Now, as each of the change activities is linked to the business outcomes it will ultimately deliver, each completed output is a measurable step towards the delivery of your outcomes. This gives you a simple, straight-line progress tracking and measurement process to achieve result level 7.

Recap

Specify what you really want to achieve—your desired business outcomes
For each outcome identify ALL of the change (and project) activities required to deliver it.

(NB Only after both the outcomes and all of the change activities have been identified is the project defined with a subset of the outcomes to deliver and all relevant change activities allocated to deliver these ‘project outcomes’.)
Track the cumulative 100% delivery of each activity’s outputs until the business outcomes and their benefits are fully realized.

These three steps are simple but highly effective. They put the business back in control to achieve and desired level 7 measure success.

It really is that simple.

Two further scenarios - systems and organizational restructure - and the eight levels of success.

SYSTEM SCENARIO

When is a systems project successful?

The system is installed
The system is installed and working to its functional specification
The system is installed, working to its functional specification and the staff know how to use it
The system is installed, working to its functional specification, the staff know how to use it and the systems maintenance group knows how to maintain it
The system is installed, working to its functional specification, the staff know how to use it, the systems maintenance group knows how to maintain it and it is integrated into the end-to-end business process (ie it works in practice)
The system is installed, working to its functional specification, the staff know how to use it, the systems maintenance group knows how to maintain it, it is integrated into the end-to-end business process and the expected performance benefits (eg increased productivity, reduced costs) are realized
The system is installed, working to its functional specification, the staff know how to use it, the systems maintenance group knows how to maintain it, it is integrated into the end-to-end business process, the expected performance benefits are realized and the resultant business outcomes (eg new or enhanced capabilities or competitive advantage) are achieved

ORGANIZATIONAL RESTRUCTURE SCENARIO

WHen is an organizational restructure project successful?

The restructure is in place with the roles filled
The restructure is in place with the roles filled and people understand their accountabilities and performance measures
The restructure is in place with the roles filled, people understand their accountabilities and performance measures and have the skills to perform their roles
The restructure is in place with the roles filled, people understand their accountabilities and performance measures, have the skills to perform their roles and have the reporting processes to know what/how they are doing
The restructure is in place with the roles filled, people understand their accountabilities and performance measures, have the skills to perform their roles, have the reporting processes to know what/how they are doing and have the operating systems and processes to deliver on their roles and accountabilities
The restructure is in place with the roles filled, people understand their accountabilities and performance measures, have the skills to perform their roles, have the reporting processes to know what/how they are doing, have the operating systems and processes to deliver on their roles and accountabilities and the expected benefits (eg greater departmental performance and reduced staff turnover) are achieved
The restructure is in place with the roles filled, people understand their accountabilities and performance measures, have the skills to perform their roles, have the reporting processes to know what/how they are doing, have the operating systems and processes to deliver on their roles and accountabilities, the expected benefits (eg greater departmental performance and reduced staff turnover) are achieved and the resultant business outcomes (eg improved business performance, reduced customer turnover) are achieved

Importantly, you can deliver all of the ‘success’ measures from 1 to 6 but fail to achieve success level 7—which is the measure of success you set out to achieve. The main reason why this failure is not known or appreciated is that these business outcome measures are so rarely defined, targeted and then measured.

Now admittedly the earlier you call ‘success’ the more likely you are to achieve your measure of success (hence the claimed success of ‘agile’ techniques). However, the earlier you call success the less likely you are to focus on the real end business outcome measures of success (7) and therefore the less likely you are to achieve these real measures of success.