The shape of the future

August 2017

Businessman chairing meetingMark McKee, Director, Herongate Associates, asks if DevOps is the shape of IT for the future and, if so, do we dispense with more traditional project management?

If DevOps aims to create a culture where build, test and release can happen frequently and reliably, then most large IT organisations are not equipped for this kind of culture. Project management is also a topic that is ripe for major change due to the fact that IT is shifting towards products: after all, customers buy IT products and not projects.

In many large firms there are separately managed development, support and release teams with no incentives to work collaboratively or with a sense of shared values and outcomes. There can even be extreme cases where support/operational teams engage in a turf war with development teams to undertake minor enhancements and bug fixes and thus risk destabilising a codebase due to being outside of the wider knowledge-base where the code is built and maintained.

The same can apply to managing vendor systems too. I’ve also seen release teams that operate totally independently from development teams to manage a large suite of application releases with the upshot of not knowing enough about the systems to troubleshoot problematic releases or know which components to shut down and in what order as part of a release process. The problems created from these organisational silos mean relegating releases to weekends only so as to minimise weekday disruption.

When was the last time Google, Facebook or Amazon took their services down to perform an upgrade or release? These are the kinds of outcomes that DevOps aspires to. When the bar is raised to the level of end-users not knowing when a release is happening, except to see some new feature on a later login, then the rest of the software engineering community needs to sit up and listen. This also bears testament to the agile principle of self-organising teams who take complete ownership of delivering a feature from conception to release.

I would go as far as to suggest that the likes of release teams be abolished and that function sits within the development team. If they can’t be trusted to have access to production systems, or at least to have a level of access that enables them to start and stop process and execute release scripts, then why would you trust them to build the software in the first place?

Likewise, QA teams are going towards irrelevance when a self-organising team includes the unit and functional/behavioural tests needed to satisfy the functional and non-functional requirements. Graphical user interface (GUI) testing can also be fully automated. A traditional culture of developers handing over to QA testing creates intellectual laziness that results in sloppy work.

Well-crafted user stories with agreed acceptance criteria that the stakeholder has helped shape means that a self-organising team can get on with the job end-to-end. Operating in this manner means that at the end of a sprint a relatively small amount of change (but a change that is valuable) can be released. This greatly lowers risk. I know of teams who release quarterly or less. That is a huge risk to stability.

Even in the most regulated environment there needs to be a culture that facilitates frequent, incremental releases. It also enables firms to be competitive by getting to market fast and allowing stakeholders to shape the priorities. Naturally you need support teams to take phone calls, be at end-user desks where applicable and respond to alerts and triggers from monitoring tools. DevOps seeks to address the dichotomy caused by support teams being hived off into generalist teams that are separate from and almost ignorant of the product development teams. At the very least there needs to be rotations of developers working with the support teams and the support engineers working to populate the product backlog and help shape the sprint planning with product owners.

Project management is a topic that is under intense scrutiny as a discipline where methodology has become the tail that wags the dog. Too many times I have seen project management ‘professionals’ with very little context of the domain in which they operate. To them the underlying product could be anything, but they claim their process can fit any industry. This approach makes it hard to have a good dialogue with stakeholders when it comes to negotiating priorities, planning sprints and making sure that certain ground rules are adhered to in the development process.

If the project manager has no context of the user stories, their complexity or whether adequate acceptance criteria has been defined, and their role is merely tracking and reporting, then this is a low-value activity dressed up as something useful. A competent scrum master possesses domain expertise and the confidence to re-jig priorities in tandem with what the stakeholder envisages. Contrast that with a traditional project manager who wants to see everything locked down for many months ahead and can be thrown into a tailspin if features and priorities are adjusted after the ‘project’ has been planned.

We don’t work like this anymore in a dynamic business environment where everything from political, regulatory, economic and competitive events mean changes of strategy must be executed rapidly to ensure products hit the mark with customers.

The scrum master ensures the rules of agile product development are respected, e.g. not allowing an in-flight sprint to be adjusted. This is the point of sprints and planning for them. Upcoming sprints allow stakeholders to re-evaluate the list of user-stories and their content to his heart’s content, but once that sprint becomes the live one, it is going to be delivered and not tinkered with in-flight, otherwise effort is wasted. Many organisations have failed to grasp the significance of meddling with in-flight sprints in terms of lost productivity, damaged morale and financial repercussions to the budget.

The kernel of my argument is that if you have self-organising teams then you are able to do more with less people. Teams that operate in a truly agile manner don’t need a business analyst to hand them a requirements document or a QA team to point out their blunders. They are empowered to manage their destiny. This can include getting their own infrastructure (e.g. servers and databases with a cloud vendor on a pay-as-you-go model) that also reduces the need for those expensive in-house data centres and armies of infrastructure engineers.

My team won’t be coming to me and saying my feature is being held up by needing to wait three months for new hardware to be procured after they have filled in a 100-page capacity planning and architecture document for the infrastructure team.

Software engineering is about building great software products that fulfil end-user needs. Everything else (hardware, middleware, networks, databases) are a commodity and can be handed over to expert purveyors in all but the most specialised cases. Finally, self-organising teams, if empowered from top management, can be freed up to get on and deliver valuable software frequently, creating value, saving money, being highly motivated and helping their firm stay ahead of the curve.


Comments (7)

Leave Comment
  • 1
    Alec wrote on 9th Aug 2017

    While I agree with the sentiment here, I would like to highlight that software projects can often co-incide with other process or organisational transformation going on, where more traditional project management is being used, and there needs to be some kind of interface between the two.
    Support teams benefit from having knowledge, that many dev teams aren't interested in, preferring to follow the logic than a user story, that, without some knowledge of the 'ergonomic' use of the system is useless anyway.
    What you described is a great way for software, but it will be ideal when it can be used alongside transformational change methodologies without their being a lot of grief that the support and implementation teams are buffering.

    Report Comment

  • 2
    Steve Boronski wrote on 9th Aug 2017

    I'm biased I know but also have enough background to know that the argument is one sided and makes only negative assumptions about project managers and positive assumptions about scrum masters. Neither are necessarily correct but anyone could find examples to "prove" their own bias

    Report Comment

  • 3
    Craig rollason wrote on 10th Aug 2017

    Thanks for sharing this mark and good to see BCS publishing a disruptive article. I think we are seeing IT moving away from froma one size fits all model to meet new challenges businesses face. It's a multifaceted problem - I think the key is understanding the business environment then designing teams and roles, inc the role of PM in that.

    Report Comment

  • 4
    Edward Taaffe wrote on 10th Aug 2017

    20 years ago I was a developer at IBM ASC pioneering the agile approach. Since then I have delivered £40m business change programmes based around IT. There is no comparison at all between building an app for/with the guys in HR say and driving global transformation of process, culture and technology.
    It's a classic example of two people using the same words , and nodding in agreement while having no idea what the other understands by it. This DevOps agile is becoming a tool in the same turf war as mentioned.
    Separate point: I at least, am weary of waking in the morning to find an update has broken my system overnight and batch jobs have failed, broken software as a result of this shoddy approach is becoming to customers a way of life and there is nothing good about that.
    IT used to be an adviser that helped the customer understand the medicine before taking it. Becoming a hustler flogging them little bits of snake oil that will only lead to a spaghetti bowl is not professional, though understandable given the way we have treated by business stakeholders over the decades.

    Report Comment

  • 5
    David Kay wrote on 10th Aug 2017

    Whilst supporting (and have practised) developers being responsible for dealing directly with customers to own their mistakes, I'm not convinced that getting rid of release is the right thing in all environments.
    Part of the release process should be the independent testing responsibility. When leading development teams I always tested their stuff before moving to release and was glad I did. Mistakes can happen or get missed (recently there was a bug in Office365 which prevented files from being moved in one drive) and it is important to consider the impact of the risk.
    There is also a security issue and the prevention of access to live data for developers may be important in certain areas.
    I'm sure if I was responsible for the software on cashpoint machines, I'd be testing to the cows come home before releasing.

    Report Comment

  • 6
    Phil Orr wrote on 14th Aug 2017

    When I read articles about self-organising teams, doing more with less, continuous releases etc. I often find myself wondering if these are only ever suited to B2C products and that a significant proportion of IT professionals who work on B2B applications, with a usually less motivated user base, training overheads, complex workflows and transactions can really accept that things like QA and release management can be removed for the sake of speed and build efficiency.
    I am less a fan of project management than I am of smaller iterative releases, but as someone who has spent many an hour trying to unpick problems that have only been uncovered weeks after a release, I do think there is still value in wider product management governance beyond simply "the team".

    Report Comment

  • 7
    Tristan Iles wrote on 4th Sep 2017

    I echo the comment that it is so good to see BCS publishing a disruptive article. I don't necessarily agree with all the points (and avoid arguing over the internet like the plague!), but thank you for putting this together and putting it out there.

    Report Comment

Post a comment