Distributed information systems

A week after posting on my UniPhi blog about capturing programme management data through the aggregation of distributed project data entered by project managers, an article about distributed energy and communication comes online at my preferred energy blog http://peakenergy.blogspot.com/2012/01/jeremy-rifkins-third-industrial.html. This guys positive views on the growth and potential of a third industrial revolution are very refreshing. There are several risks to this revolution coming to fruition. One of which is corporations lack of awareness of the distributed communication possible through web based platforms.

Organisations that realise the power of distributed data capture will definitely be able to invest in systems and processes that generate significant competitive advantage. The primary advantage is the ability to know when something goes wrong (or right) the moment it does so. Through the power of Twitter it was possible for any person with internet access anywhere in the world to know that the US were engaging with Osama bin Laden as it was happening. Although at first it was unknown who it was they were attacking, the fact that hundreds of millions of people could have known about a top secret mission as it was happening is an incredible example of distributed information systems. Suddenly, managing a multi-national business with 50,000 employees doesn't seem too difficult, and yet all the example companies I have worked with that fit the 50k plus definition would struggle to generate real time financial, customer and product related information. 

Most organisations still track their progress towards their vision through centralised systems. The main reason for this is the perceived need for security when all security is only as strong as the employee who wants to keep his or her password a secret (or even accidently release this information through the need of tracking the ever growing number of usernames and passwords). It seems that large IT business units pride themselves on restricting the implementation of a distributed data capture model. Actually, it seems large IT business units pride themselves on being as value destroying as possible rather than enabling the true power of software to transform the way businesses function.

These business units are supported by the product development of traditional enterprise software houses producing thick client software with complicated user interfaces that only the specialist business analyst, accountant or marketing guru can master.

Why is Oracle, SAP, Sun and others still insisting on developing centralised thick client financial systems? I know of one project manager that knows the true financial position of their project as at 31st of October 2011! He thinks they may have under invoiced $10m but isn't sure because he has to wait for the head office processes to catch up with where his multi billion dollar programme that crosses multiple countries and states is at. He is trying to catch up through the use of an excel shadow system. Good luck!

http://www.wired.com/wiredscience/2011/11/starling-flock/

I know this video has gone viral but I had to add it here using the weird science link for future reference. It's a great example of the complexity article http://mbhconsulting.blogspot.com/2006/09/project-management-and-complexity.htmlI referenced here http://www.red3d.com/cwr/boids/

project management and complexity theory

I am presenting on the topic of complex projects at the Australian CPA congress in Sydney on October 17 2006...actually, I'm double booked and will be roping in my colleague Sarah Quinton to do the presentation for me but I haven't told her that yet (it will be interesting to see if she reads these blogs and finds out through this communication medium or when I tell her on Friday this week). Anyway, one thing's for sure, I'll have to write the paper and do the presentation slides and as it was due a week ago, I better get cracking. The reason I have taken so long to get around to doing this task is that I'm not really sure what I want to say. Complexity theory and how it relates to management practices is something I have been reading about and musing over for 3 years. However, it is still a very disparate field of study and compiling coherent content is difficult. My gut feel says management complexity theory could solve a lot of issues I have encountered when managing projects.

Mixing complexity theory sciences with control based project management is akin to adding oxygen to magnesium. Adding a volatile substance to something that appears stable and watching it become extremely unstable. Project management appears stable and established in terms of practice, so why try to destabalise it? Because it doesn't work. Organisations that have mature project management practices and are at the forefront of project management still have projects that are a mess and perform poorly. In organisations that have less than best practice project management, project failure is more common and is blamed on the environment (we do it well but sponsors and managers don't understand it and hence it fails). And yet, surely, a project manager has to manage the environment they work in. This is the fundamental flaw in all things currently residing in that representation of how to project manage, the project management body of knowledge (PMBoK); It is built around inputs, processes and outputs not around outcomes. Outcomes are environmentally based, inputs, processes and outputs are control based. You can control the process to create the output based on the inputs, but, if the output is invalid, so too will be the outcome.

In terms of process, the PMBoK is excellent. There are tools and techniques to manage almost any process. In terms of outcomes, it is devoid of content. This has led to a culture in the project management community (represented in functions attended at the Australian Institute of Project Management and Project Management Institute) of people driven to deliver. Project managers will deliver at all costs. The concept of projects being cancelled not because they are performing poorly on a time, cost or quality basis but on an outcomes basis is an anathema to industry and will be seen as a blight against them.

A potential solution to the train wrecks that still occur in project work is applying complexity sciences to management principles. The area focused on with regards to complexity by us is the area of complex adaptive systems. The best example of this is in the method of creating automated characters in computer animation. This method was developed by Craig Reynolds in 1986. The basic premise was focused on how birds flock. To enable birds to flock in a computer animation, the focus of an individual bird is on the birds immediately around it. It ignores all other birds outside a specified region. This method of steering means that the birds are self organising and flocks of birds form spontaneously. This has been described as "the edge of chaos". How does this relate to project management? This has been the basis of my musing for the past three years. I have come up with three key points which when added to one point from the traditional project management body of knowledge and one point outside both complexity and PMBoK give you principles to apply to all complex projects.

Complexity principles
1) Each project is unique and hence cannot be managed in the same way. Blindly applying a project methodology to each project and following the body of knowledge input requirements, processes and outputs will not lead to a successful project every time. Complexity philosophies and principles must be overlayed on top of a project methodology.
2) The most effective way to manage a project is for the project team to be as self organising (on the edge of chaos) as their capability allows (again, will vary for each project as the allocated resources vary). High performing teams are those that need no or very little direction. Of course, if you are handed a team of novices, then it is going to take longer than if your team is experienced and more importantly, has experienced self organising projects before.
3) Communication is key in achieving the edge of chaos required to successfully manage complex projects. The medium, frequency and content of communications will change dramatically from project to project. There is some interesting research as to current forms, frequency and content properties used on complex projects (see Determinants for external communications of IT project managers by Ralf Muller) However, as this is current practice and we are proposing radically new methods be formed, one cannot rely on this as a basis for future practice. The best current research on complex communication processes within organisations is a book by Ralph D. Stacey: Complex Responsive Processes in Organizations. Learning and Knowledge Creation. We have included a link for a review of this book.. This area of complexity theory is where we will be turning our attention to over the coming months.

Most organisations structure themselves around themes or functions. In this way, information is communicated within certain known paradigm. Knowledge is not created, it is only transferred from one individual to another. The tacit knowledge of one person is shared with another. However, when information is given outside this theme or function to another theme or function, it is interpreted without the central paradigm or framework. In this way, its interpretation could lead to new knowledge being created as the receiver applies the information in a way outside the senders paradigm. Organisations that are silo based restrict the potential of new knowledge being created and when knowledge is transferred outside the silo, the chances of new knowledge being created is limited and the chances of misapplication of knowledge increased. Our number one proposition in this area is that cross functional teams are the means for knowledge creation. Informal communication from heterogeneous teams allows for the creation of new knowledge which could be crucial in achieving an outcome. Evidence of this would be projects delivering solutions that are significantly different from what was originally scoped but still delivering the benefits identified. Within these heterogeneous environments, acronyms would be of limited use as they would not be understood by all team members. New processes and methods will be formed as existing paradigms are challenged by those who don't have the cultural baggage of the current norm. All of this needs to be tested in the real world. The difficulty will be finding teams that are truly heterogeneous and have created that edge of chaos environment.

Project Management Body of Knowledge principles
1) Theory of Constraints is the number one method for estimating. Estimating is inherently poor on a lot of projects. Typically, project schedules are a pain to project teams, requiring them to do work in an order that is not valid (i.e. as soon as possible rather than when required) and hence the self organising principle is broken. Benefits, costs and time estimates are constantly changing. As we need to pre-plan things, we need to be able to manage this change and hence a project manager's job should involve the constant updating of the schedule, the benefits and the budget of a project and analysing its impact on the existence of a project. Typically, this review is done too infrequently to be effective. Alternatively, it is overdone to the point of micro managing.

Theory of Constraints scheduling is about managing the project to a project deadline not a task by task deadline. In Theory of Constraints, the project manager is efficient at altering the schedule and does so to determine the optimum path of work activities. This becomes a communication tool to the project team. It takes into account factors typical to teams that are not high performing such as multi-tasked resourcing and student syndrome and allows for the basic assumption that close enough is good enough. It also manages client or sponsor expectation through the project buffer. Theory of Constraints knows a schedule will always be wrong, hence doesn't go for perfection. Theory of Constraints is the most important control based method for project managers managing complexity.

Principle outside complexity and PMBoK
1) Only outcomes matter. Complexity and uncertainty should be managed through a regular re-evaluation as to the fundamental reason for the project's existence (i.e. its outcomes). The only reason the project exists is that its benefits outweighs its costs. This varies depending on the value proposition of the organisation and of the project itself (see Managing by Project article below). Re-evaluating timephased costs and benefits is fundamental to complexity management, it provides the means around which teams will re-organise themselves.