Putting agility in your data strategy to power up business

Today a company must evolve its approach at high speed just to maintain its core operations to keep the plane in flight. This is principally because many new services, products and governance requirements are rapidly appearing due to changes in the market, thus making it a highly competitive, complex landscape for companies and experts. The fast evolution of technology is another key factor.

Agile principles have proved increasingly popular with software developers. There are many advantages of applying them to business and other activities in our day to day lives. Modern data platforms provide the opportunity to do this and can offer the transparency and up-to-date analytics to help move the agenda forward. Here, we will explain how agile ways of working can be beneficial to corporations and small businesses alike in terms of solving evolving data problems and navigating a changing organisational agenda using an incremental change strategy.

When it comes to the sphere of data we look closely at how agile working, using modern integrative data platforms and code bases, can help organisations overcome multiple market challenges more effectively — specifically, at the role of technology in shaping and powering dynamic agile platforms and business strategy in tandem.

We will explain what we ourselves have done in this area to innovate a flexible strategy to take us through the process in a practical, effective way.

What does agile working really have to do with it?

Let’s take a closer look. The last 20 years have seen the agile working grow from a fringe movement to mainstream thinking and a commonly accepted way of working smarter and faster.

At the outset, agile working had one principal goal: to allow developers to build better software faster. We believe this is a core part of business success today, and that the modern data platform itself is a key vehicle for driving agile working in different parts of the enterprise — and for the dynamic evolution of the very architectural designs and analytics that it uses.

Organisations have to consolidate their resources and operate from strong fundamentals. To do this in our firm, we looked at the performance strengths and specialisms of the best platforms we could find internally, and benchmarked them across an ideal ‘Super League’ model for a data platform, grading them on their different layers — from ingestion capability through to data warehousing, adaptive MI self-service reporting, AI capability and UX.

We applied agile thinking to our work, and then proposed a composable digital stack which we put in front of global teams so they could see and leverage the best quality code, design and platforms for their own customers. This acted as a launchpad, opening up access to our capability accelerators and provide everyone with a strong starting point. It also brought far flung teams together to share expertise, ideas and use cases to great mutual benefit, showing that applying agile principles to modern data platforms is highly beneficial.

The world of modern analytics

There are common agile principles running across the business analytics, software development and modern data platform domains.

A key aim of data analytics is to help people make decisions they trust, and the user journey can be just as important as the end results.

This means that when we apply agile principles to data analytics, we typically focus on three areas:

• Interactive decisions and customer experience: by focusing on what people want to do with data, we can progress their initial questions and formats to focus collaboratively on valuable iteration and follow-up questions, only moving on to predictive modelling once we have build trust and cultivated curiosity, meaning our responsive action offers a better outcome
• Rapid results and functional analysis over perfect outputs: to enable quick iterations, we need to spend less time crafting perfect outputs and focus on moving from one question to the next as quickly as possible, which flexible code stacks and agile platforms support
• Collaboration and sharing data: sharing responsibility for our data and data products with our business partners helps build trust and accountability for cultivating great data products and data-driven cultures, and agile modern data platforms foster collaborative teamwork

Agile methodologies place individuals and interactions over ‘processes and tools’. Agile platforms give us the means to adaptively reach out to new audiences, use new touchpoints to bring in new datasets, and create opportunities to evolve our approach at speed in order to transform our operations and thinking.

What else?

Advancing their product development activities is also integral to companies’ ability to deliver what the clients want, specific to the niche environment they operate in today. We looked at this in some detail too, finding main themes for product development and agile strategies that businesses are increasingly involved, including regular communication, an awareness of product evolution.

These are also at the heart of evolving modern data platforms, resulting in some key features:

• Real-time, based on a company-wide version of the truth
• Flexible - highly consumable in easy-to-read formats
• Well designed, on-demand and user-friendly customer journey
• Unified, solution fully integrative to company-wide systems
• Highly Adaptive to context and new scenarios
• Cloud-based for elastic storage space
• Highly accurate and reliable, with clearly defined, robust and transparent logic

For you

Be part of something bigger, join BCS, The Chartered Institute for IT.

Accurate data has long been a component of budgeting and planning. But today, strategic planning is not a periodic ritual: it requires accurate, up to the minute information. Hence, the underlying analytics must be seamless, digital and fast moving to keep up.

We looked at tactical ways to do this. As part of our global remit, we created a composable digital playbook of entire platforms, code and design frameworks and components. Our teams can now morph digital structures to meet specific business use cases by building custom platforms assembled from base frameworks taken from a common stack. In this way our teams can resolve many issues through essentially crafting custom platforms in a way that is similar to selecting shifting patterns in a kaleidoscope. This adaptability helps enterprises to remain competitive and allows them to share expertise across verticals internally.

Why you need modern data platforms: positioning your business

AI brings further possibilities for orchestrating insights and responses.

Today, modern data platforms offer the basis for purposefully bringing worlds closer through incorporating AI with other frameworks, opening up avenues for meaningful integrations between disparate data sets and federated ecosystems. Here, we can see how emerging technology is making it possible for multiple aggregations and intelligent comparisons to be done in real-time to accelerate our actions in a continuously fast-moving world landscape.

The composable digital stack we advocate, and are putting into place, will help address these challenges. At a conceptual level, it takes the form of a multi-layered cube with the ability to abstract and check code and digital structures at the different levels — including infrastructure, software code, applications, platforms, storage, analytics, design and AI, contributing to the dynamic design of a strategic thriving enterprise.

Leading players in tech, retail, media & comms, banking, healthcare & pharma, cyber-security and international organisations who can leverage this successfully will stand out from the crowd by operating from strong technology and business fundamentals. For example, the Ocean Cleanup project integrates data from multiple data sources collected world-wide from international and local bodies as well as members of the public and research institutions.

Legacy business technology challenges and how modern data platforms overcome them

Organisations that don't respond to changes with agility and do not change their technology strategy to use modern data platforms face three common challenges that will typically impede organisations with business models based on outdated data architectures:

1. Integrating multiple data sources and use cases: while new platforms that are involved in delivering analytics at scale are designed for this kind of work, older data platforms have evolved over time and are not prepared to manage more modern sources of information, such as machine to machine (M2M) data
2. Rapid scaling to handle large datasets: often in legacy setups, the enterprise pays over the odds for capacity, even if it’s unused. However, cutting back on capacity outright is high-risk and can leave the company unprepared for streamlining and rapidly scaling, as it needs time to upscale necessary hardware and software. By then, the opportunity may very likely be lost to a more agile competitor
3. Speed response times and eliminating reporting bottleneck: legacy reporting can lead to an inability to discover new insights rapidly when handling distributed and large datasets and ecosystems due to cumbersome workflows, and requires highly specialized IT resources. This can result in long lead times to obtain and ingest new sources of information. Without agile tools to create reports themselves, users can wait weeks in a queue for IT resources to become available. Bottlenecks in research can slow innovations and delay decisions for senior executives and front-line operatives in accessing high quality MI and their response to urgent and changing operational situations

We need modern data platforms using sophisticated data analytics to drive value from the company’s data holdings using a semantic layer with efficient governance. This will overcome complexity and latency challenges and provide critical decision-making capability to business users, instead of trying to operate a massive data blob that is both cumbersome and difficult to interrogate.

In the past, local teams or countries resorting to their own data tracking as they struggled to build new use cases from the ground up has led to information silos, protectionism and mistrust, creating inefficiencies and duplication of effort and resulting in a proliferation of sources/different versions of truth. The composable stack approach combats this by offering a real opportunity to leverage shared central and local intelligence to address new scenarios at pace.

The modern data platform

Enterprises now have more options than they did before. With modern data platforms, businesses can build flexible engines and systems which offer more agility than traditional solutions architectures, making data accessible to wider audiences. Here are some guidelines to help distinguish a modern solution from a more traditional hard-wired enterprise model:

• Results-driven: a modern data architecture is geared to be business centric, not IT focused. It isn’t about technology for the sake of technology or hard-wired structures, but about driving better business outcomes. It is focused on the needs of the business over the technology that enables it to power business success and adaptability
• Automated: next, a modern data architecture leverages automation. It seeks to augment and automate the most manual tasks to ensure we are not building brittle processes
• Flexible and elastic: a modern system should be flexible enough to address use cases which aren’t even envisioned yet. A modern data architecture is also elastic, leveraging the power of cloud computing to provide instant, on-demand scalability, ensuring that the right capacity is always available

Agile principles and engineering fundamentally place modern data platforms at the heart of business strategy. The businesses that do this are placed distinctly at an advantage in today’s market.