Monday, 27 May 2013

What Happened to the Fine Art of Business Analysis? - Revisited 2013

Back in 2008, I wrote a paper on Business Analysis. Recently, I've been revisiting this subject in my day-job and this made me realise how little things had changed and inspired me to write this post, which is, basically, the original article re-written, with additional thoughts (in italics).

The role of the Business Analyst has never been more important but needs refocus on Information Systems not the technical solution. Many of us can recall a time when a distinction was made between the hardware and software supporting the business and the information used by the business – there was a clear difference between IT, to describe the former, and IS to describe the latter. 

IS stood for Information Systems: 

IS: The landscape  of business  information used by people within an organisation 
and how they use information to deliver business outcomes. 

IT, in contrast, meant: 

IT:  The  hardware  and  software  technology  that  automates  or  otherwise  supports 
information processing. 

This distinction between these two concepts is all but lost, and the disciplines associated with Information Systems (such as Business Analysis and IS Architecture), are have become too obsessed with IT. 

Where did the Business Analysts go? 

The change of focus was probably, in part, caused by the rise in enterprise software packages and the associated promise that these out­-of­-the-­box solutions could work miracles – magically automating all kinds of business processes and spraying efficiency glitter over all parts of the organisation. Powerful promises! 

And to make them even more seductive, they contrasted sharply with often, expensive and unsatisfactory, custom­-build software projects. Small wonder then, the new-kids-on-the-block, ­ERP packages,­ were seen as being more business relevant: enforcing standardisation and promising to boost efficiency. Business Analysts started to abandon their 'real job' to become, in effect, package 'fit­-and­-configuration' specialists. Inevitably, their work began to focus on the features and functions of the software, rather than the realities of the business with its complexities: messy processes and human interactions. 

Since then, others with expertise in IS, have gravitated to the world of IT architecture. This discipline has proved increasingly important with the many large organisations who realise that packages – even so called ‘enterprise-­wide’ ones, need to coexist the rest of the IT estate. You can't buy architecture in-a-box!

At the same time, the revolution in Open Source and Web technologies meant that the focus of architecture began to move away from the organisation’s internal IS needs and towards the emerging global IT standards, tools and methods used by its customers, suppliers and partners. The sheer complexity of all aspects of ‘architecture’ meant that areas of technology­-focused specialism began to develop within the architecture community. 

This article suggests, it’s time to dust-off the concept of Information Systems, and return to  business analysis practice that focuses on:

Simplifying [as much as makes-sense], balancing  and consolidating requirements across all aspects of  the business, but with, the important business outcomes, front­-of­-mind. 

Identifying repeating patterns of business activity and seeking out opportunities for shared competencies and services. 

Understanding the behavioural aspects of the business that have been proven to be barriers to adoption of IT solutions. 

Studying the elementary aspects of the business’s Information System such as rules and constraints embedded in policies, the business­-meaningful, real­-world, events of interest and the information content being exchanged between individuals and groups. 

Communicating and agreeing upon the business ‘what’ before getting immersed in the technology ‘how’. 

What’s in a Word? The Problem with ‘Process’ 

There appears to be a problem with how we use the word ‘process’. To support business efficiency goals, many envisage the world as a set of neatly ordered, well planned, pre­determined and logically sequenced set of activities – just as Henry Ford did when he introduced the production line. This is hardly surprising as it is a tried and tested approach to manufacturing, and more generally reflects a simple but intelligent goal – to be more efficient at a task through understanding and optimising all the steps. However, optimisation and automation require all the steps and parties involved to be specifically defined, so this approach usually seeks to ‘decompose’ the task into detailed models of all the interacting parts within the end­-to-­end process. The process being defined is also typically mapped to an organisational framework and financial models which constrain it within central and departmental rules, polices and edicts, which are often inherited from corporate decisions unrelated to the process in-­hand. This is Six-Simga-Everything mentality! The most critical problem with this efficency-only focus, is in its  poor representation of the softer, more knotty problems associated with differing values, policies, and operational ‘realities’.

To add to this problem, the language of users differs significantly from that of the IT specialist. This difference in language is a reflection of apparently opposing values (value systems) in play. Fundamentally, the users want a system that works the way they do ­one which is often prone to change and which supports learning. The IT specialist seeks a high degree of functional certainty and actionable rules that can be configured or encoded. 

(see note about 'Agile' SDLC at little later).

To make matters worse, the gaps and inconsistencies in the expression of business requirements force a high degree of IT ‘engineering rigour’ that attempts to complete the picture. IT engineering relies upon techniques and tools that are designed to convert process and policy into precise rules and constraints. This can result in important behavioural dimensions ­such as the effect of variance in values and levels of trust – being left unspecified, and tension-points left unresolved. The dialogue between user and the IT specialist becomes focused on the detailed features and functions of the IT 'solution'. The business outcomes of the sponsoring executive become obscured by this drive towards detail and by the end-users’ individual, and collective, ideas on how the 'solution' will work. 

C-level 'Businesss Stakeholder Standardisation Edicts' or sent out in an attempt to 'Get Things Back on Track': ensure that operational efficiency goals are met and key business outcomes are delivered. This can result in an unwitting collusion between the C-level 'Business Stakeholder' and the IT specialist: both parties are motivated to build 'standardised' behaviour into the IT solution.

The top­-down edict approach to driving IT­-enabled business change projects suffers from three major problems: 

Adoption Barriers ­ 
The users are less inclined to adopt the IT solution, claiming that it doesn’t work for them – they continue to use and develop ‘shadow’ IT solutions 

Brittle Processes ­
The top­-down specification of process works against flexibility and becomes a barrier to future business change. 

Business/IT Misalignment ­
The organisational and management models of the business are not properly reflected in the IT solution. In, for example, a multinational business with diverse operations, it may be necessary and desirable to allow for local freedom­-to­-act, in say, Customer-facing processes. 

It seems we need a different approach to how we specify requirements for IT: an approach that balances the desired business outcomes of the stakeholder with the needs of the user, yet with the precision needed by the IT specialist. 

Significant improvements are being claimed by the 'Agile' development  community, however, the problem still exists when likes of SCRUM and XP don't apply, such as the deployment of a package solution or, increasingly, the purchase of SaaS. Not to mention, that  'Agile' approaches can be left wanting when in comes to cyber-security and data privacy.

It appears that the language-used, and a premature focus on IT, are at the root of this problem.  The language problem starts with how we express the concept of processes.  The word ‘process’ can have very different meanings and implications. These can range from the loose collection of unsystematic activities, that might support a contract negotiation, or an HR process, to the highly repetitious, and predetermined behaviour, of say, a production line. Maybe a common language for describing the range of business behaviour, across any type of process, would help?

A premature obsession with the details of the IT solution, seems to drive the analysis of the business problem towards IT engineering methods. Such methods often ignore or, at best, neglect softer, human, aspects. IT engineering methods and techniques are designed to drive completeness and precision for systems and, in doing so, introduce a language that is, by its nature, hard for the business to understand. This creates frustration on both sides of the Business/IT divide: neither party understanding the needs of the other.  The sheer volume of information produced during the engineering life­-cycle, means the overall business context gets lost in the production of  flow diagrams and technical plans. The complexity of the task requires a high degree of technical specialism. This can lead to a lack of focus on the nuances of real-world behaviour and human interaction ‘on­-the-ground’. Often the focus is slanted towards engineering ‘How’ rather than the business ‘What’: the business outcome. 

The reality is that many business scenarios are more organic, and unpredictable, than  pre-determinable, and mechanistic. In today’s connected world, many business scenarios encounter tensions ­ between corporate and local functions, between customer and supplier, and between partners. No one is in control of the end­ end processes; rather, a combination of interacting service agreements make up the overall value delivered to the customer. 

In contrast to usual business/IT dialogues, what’s needed is a natural-language that moves the focus away from any technologies and towards business behaviour – as expressed in values, policies, real­-world events and any type of information [Content] being exchanged.  

The original article bangs-on about VPEC-T and, our then, current, book 'Lost In Translation', which I'll skip now [sigh of relief for many!]. I would end the article differently now.  In hindsight, it missed-out some more fundamental (and frankly, more broadly applicable techniques) such as SWOT, Risk-Radar, Hypothesis-led and PESTLE analysis. Perhaps more importantly, I would emphasise the value of; 'abstraction' (conceptual and logical 'What' before 'How' thinking), an understanding of 'Complex Systems' - The Cynefin Model (Dave Snowden) would be a good starting point -  and  workshop techniques such as those described in 'Gamestorming' (Dave Gray, Sunni Brown and James Macanufo).

I recently came across the 'Brown Cow Model'  (Suzanne & James Robertson) when wrestling with a culture of 'How' before 'What' behaviour  - aka 'I have the solution, so what's the problem' ! I love the simplicity of 'Brown Cow':
I've found this simple model a great way to keep project teams focused on understanding the 'What' before the 'How' and the importance of knowing the AS-IS situation before leaping to the TO-BE. This, for example, has opened-up the requirements discussion to include the, often-overlooked, business requirements of the transition between 'Now' and 'Future' states. Maybe, this is more of a problem here in Asia, but given some experiences I had in the UK over the past five years, I suspect not!

 I about to run out of battery on my iPad now - so that's all for now. As always, I ask for your comments! Thx.


  1. Nigel,

    I am sorry your battery ran out when you were getting to (for me) is a very interesting part. If I may jump in, most of what you have been talking about in the article takes place "above the line" in the Brown Cow model. That is, the What of the work which is the essential view - what the work looks like if we leave out the technology used to implement it. The difference between What-Now and Future-What is important, as this is the benefit that the project brings. If there is no difference between Now and Future, then there can be no benefit.

    In addition, I urge my students and consulters, to spend a significant amount of time in the Future-What quadrant. This is where the business analysts work with the stakeholders to determine the desired future state of the business. Or in your terms, the future IS.

    It is possibly a little simplistic to say that IS exists above the line and IT below the line, but it is not a bad place to start.

    Best regards,

    James Robertson


    1. Hi James,

      Thanks for the comment. Perhaps a Part II will be worth exploring, if others show interest! Happy to share the toolkit we're developing (of which Brown Cow plays a part) if of interest. I agree, that ultimately, the simplistic divide between IS-What and IT-How is not adequate, but like you say, not a bad place to start! Ultimately, we need to work up to the Why layer that starts to question a businesses understanding of their strategy, which in my opinion, is often poorly described and lacks the key business axioms that provide the context for What. And, of course, when we examine the What of the Business Architecture, Hows are described that become the Whats of IS. Isn't the most important point here about the value of abstraction and how that helps us understand the 'requirements' at different levels, before we rush to a solution?


  2. Hi Nigel,

    The problem of solution coming before requirements is all too familiar, it is reassuring to hear others calling this out as a broader problem.

    I like the distinction you've drawn between IS and IT. I do wonder whether IT could benefit from removing the I (what's the difference these days between information technology and just 'technology' - how much of IT is really about the 'I' any more?)

    Your hypothesis on where BAs went (victims of ERP) is interesting. I am admittedly showing my age..or lack of it .. I perceive the 'golden years' of IS to be a time when technology was perhaps rather more mysterious, expensive and inaccessible and played a less integrated part of our lives. A lot has changed in the meantime, and most people are now comfortable with technology and are less worried about offering up their opinions on What Needs To Be Done - 'everyone's a CIO' because they all have an iPad, and why can't it be as simple as this?!

    I guess something else would be.. a lot of these systems are now so deeply ingrained into people's work lives, that the system *is* the work, and people naturally find it very difficult to separate the two.. it's only natural that they would have strong opinions about What Needs To Be Done... and hey, sometimes they might even have a great idea that I might never have considered..

    I would also suggest that the skill or habit of 'abstract thinking' is not something that comes naturally to everyone.. it must be learned and developed. As people who generally have a grounding/background with information systems our brains naturally form these habits, otherwise we would drown in the detail.

    While we might find this abstraction natural and comfortable, many others don't, to the point of actively fighting it ('But we know What Needs To Be Done - why are we wasting time?')

    I would see our role in all this is not necessary stopping people from thinking this way, but gently leading them up from the how to the what, then gently down again to the how (which quite possibly has changed materially on the journey).. any thinking tools and frameworks we can bring to bear on this the better (VPEC-T!). However I fear fighting the good fight in 'coercing' people to use a specific language, framework or way-of-thinking is doomed to frustruation.. and if I recall, LiT cautions against this too..

    Well! That was a ramble! Thanks for reading this far.


    1. Thanks Mike for you insightful comments. I agree with your points, particularly around how people see 'IT' today and the bit about 'abstraction' and leading gently towards it. In fact, that's exactly what we're doing in my IT shop today via lots of facilitated workshops with business colleagues. Your also correct that we IT'ers aren't the keepers of technology anymore and that much of the great innovation comes from others outside the IT department simply using the tech in ways we hadn't imagined. I do believe, however, we do need skilled 'Business Analyst' types who do understand why 'What' needs to be expressed before we get too buried in the 'How', and to be the facilitators of the discussion. Accepting that almost all conversations will start with a 'How' but need to be gently 'reverse engineered' to the 'What' and then back down to the 'modified How'. This doesn't have to be a lengthy exercise - in some cases just one 2 hour workshop. The important thing is to try to expand the thinking such that we get a full set of requirements that cover such things, as architectural requirements, non-functional, process impact, risk,etc.

  3. Thanks Nigel,

    I completely agree that the role/discipline of 'Business Analysis' is just as important as ever.

    One could argue its now more critical than ever before, in the days of being able to buy solutions 'under the radar' with your departmental credit card.. but only later realising you've got no way to get data in and out of this "cloud thingy" and you've now got 2 sets of addresses for the same customer contact in different systems with nobody really sure which one is the 'right' one.. and so on..

    I guess you could argue Architecture and BA are working to similar ends here.. in one way the BA is one of the 'front lines' of architectural thinking within projects and so on..

    Which makes me wonder - where does one draw the line between what's often referred to as 'Solution Architecture' vs 'Business Analysis' ?

    Is Business Analysis a role or a discipline that a range of roles need to 'do'?

  4. Mike,
    Interesting that you make the point about architecture vs BA roles.I see the two merging into an 'demand-side architecture' role that pulls together both 'feature/function' requirements and architectural requirements. I'm in the process of getting each of my domain architects to learn basic BA skills - starting with "what" before "how" and risk analysis etc. They are working more closely with 'BA-types' from the LoBs who we help gather the 'features/functions' and Business Process impact.

  5. Interesting! I'm arriving at the same destination from the other direction - Senior BAs coming across into my Architecture team!

    Requirements definition remains one of the most critical elements of any project, but seemingly one of the hardest to master. As per LiT's insight, it's the meeting of the 'analogue/fuzzy' world of business with the 'digital/crisp' world of technology..

    I do despair when business analysts are 'spawned' from knowledgeable SMEs in the business, but not given any grounding or support in the discipline of business analysis itself!

  6. Yes, I've often found SME's make terrible BAs - "Can't see the wood for the trees" syndrome can be hard to tackle. When developing Business Analysis competency, I tend to value foundational BA skills over domain knowledge (which can be obtained from SME’s in the business). One such skill is that the BA must be able to quickly understanding the language, pain-points and opportunities of the business domain in question.

  7. Very interesting, Nigel, and thank you for pointing me to this article.

    You and your respondents are obviously dealing with larger systems and issues than the ones that I generally come across, although I have my own experiences of the 'concrete blocks' of ERP systems landing on top of businesses, and squashing everything that lies beneath. Nonetheless, the principles you are outlining refer to organisations large and small.

    Because I have come to technology from the commercial and operational route, as opposed to a grounding in IT/ IS, I am encouraged by the fact that I recognise a lot of what is being said here.

    There is no doubt that technology is now so ingrained in our personal and business lives that there is really very little boundary between the two, especially for the increasing numbers of self-employed/ freelance/ contracting (call it what you will) providers.

    Also, we are past the tipping point where technology is an 'option' for businesses. Some years ago, it was often a cost argument - you could continue to interact with your customers or process your orders on bits of paper if you really wanted to, although it was often cheaper to employ technology to do some of that for you. Nowadays, it's not really an option - if you don't buy into the technology you are potentially limiting or damaging your business.

    It may be wishful thinking on my part, as it forms a significant chunk of my business model, but I am convinced that people like us who bridge the divide and understand both commerce and technology - the what and the how - and can support businesses that need that integrated approach.

    Interestingly, I find I do a fair amount of work for technology providers as well as end-users. A number of business (especially web providers) recognise that they lack 'business analysis' skills so work with people like me to make sure the client's expectations are met.

    I have seen my fair share of websites and systems that don't deliver, and interestingly it has got worse lately. I suspect (just my personal theory - I have no evidence for this) that there are a load of people suddenly on the job market in recent years who set up as web developers and the like, on the basis that they have some coding skills, but then rapidly get out of their depth.

    I am a fan, however, of Agile methodologies - although I use the term fairly loosely. I think it can allow for a more flexible approach to delivery, in collaboration with the client, and the main advantage from the developer's perspective is that it isn't easy to go too far 'off-piste', which can easily be the case with the traditional specification model.

  8. Thanks David, you theory about Web Developers does have a 'ring' about it! And I love your phrase "..experiences of the 'concrete blocks' of ERP systems landing on top of businesses, and squashing everything that lies beneath".

  9. Well this is exactly what thousands of people just like you are doing right now. They are called ‘Social Media Managers’ and they are getting paid great money to work online doing simple tasks on Twitter such as:

    Sending out ‘tweets’ for businesses
    Following customers and other related businesses
    Uploading photos and videos to Twitter
    Generating more followers
    Most of these jobs don’t require any prior experience and the only skills you need are a good knowledge of how to use Twitter. Click the link below to get started: