Enabling Digital Transformation
Continuous learning, feedback loops & chaos in containers.
The prevailing focus in Digital Transformation is the journey, customer centricity and outside-in being amongst the most popular buzzwords. The need for companies to have an effective and emotionally intelligent digital relationship with customers perceived paramount to their survival.
In order to develop that relationship companies need to listen and respond to their users. To do this quickly and reliably equal attention should be given to transforming the technical value-stream; the means by which those experience led transformational goals are achieved. Without the latter the former will be unable to break away from the mindset of playing to captive audiences with sequentially delivered products.
Broadly speaking the value-stream should be learning, outcome aligned, cloud based, highly automated, autonomous and transparent. Any boundaries facing out to 2-speed core systems should be intelligently stubbed, applications containerised, as much attention given to the quality of the pipeline and its telemetry as to what goes through it.
Significant transformation of value streams persist not exclusively but for the most part in the land of the behemoths. Those companies that have existed long enough to build up systems of IT that match organisations of teams, at extremes further fractured by activity and supplier e.g. historically Global Consultancy Alpha may own your end-to-end test function, while slightly cheaper Smaller Global System Integrator may provide your back end development function, with a bit more money and time spent on the front end agency. Quality becomes Conway’s law plus technical debt in proportion to the economics of investment across the stack.
There will be a mixture of architectures some well beyond saving, black-swans waiting for a 2am release. Others working towards an intended state that will never quite be met, change has been fought against instead of accepted as a constant, the current moment often neglected to thoughts of future needs. A team looking to undertake transformation in an environment like this will be faced with the sharp end of complexity theory, culture and technical archeology.
Given the evolved nature of the systems, automation and unit tests will be unlikely, (the unit-test being one of many victims of IT run as a cost centre) consequently changing any applications or services will be an act of faith, a London Taxi Drivers “Knowledge” a pre-requisite to finding the correct branch, method or property file, branches used as a means to satisfy scientific management.
Component based organisations often promote activity based practices, silo’d bunkers of anti-trust with long feedback loops as software trickles through on-prem test environments, it’s a lottery if the environments are up. Telemetry from test environments and production is patchy slowing down learning, tailing the logs can become a realtime sport, seconded only by defect ping-pong.
The resilience of these organisations relies on retained knowledge. The systems they suffer make them sacrifice a measure of fulfilment, quality. Technical debt compounds slowing the pace of change, the reaction is to scale, more teams compound the compounding. The system itself is wanting of being a learning value stream. If you are self-aware, happy to recognise your failings, open to learning and measuring daily the propensity to improve will be overwhelming.
For a new team to transform an environment like this their first problem will be a lack of knowledge, the sheer complexity of quickly discovering perhaps 10 years of IT systems. As knowledge is the obstacle learning must be the way. The current test systems would likely prohibit the quick feedback needed to support organisational changes towards cross-functionality. In order to be in a state to need transformation there would likely be an incumbent knowledge gap and some apathy towards continuous improvement.
A smaller single team morphing from a component organisation split across perhaps 4 teams can not survive without being able to inspect, adapt and automate. The work that has scaled out across many hands in the current organisation needs to be off-set with automation and a reduction in the cycle time of learning. Learning and quality need to become the defining ethics of the team. A level of autonomy and trust a necessity as is collaboration, paradoxical needs that can only be fulfilled through the bonds of familiarity and overtime friendship.
The cycle time of learning from change to test in a component organisation can be in the regions of days to weeks depending on the enterprise sauce of delivery. A team looking to drastically reduce that first needs to find means of bringing order to chaos, they need to remove the fog of war which covers tightly coupled enterprise architectures and infrastructures.
The chaos that this fog creates can be constrained through containerisation and whilst the problem is still complex, the complexity becomes framed. With each application contained we can automate tests and add telemetry to both the back and front-end providing the opportunity for continuous learning, continuous integration and culture permitting continuous deployment and delivery.
This practice assists discovery and learning of systems. The quick feedback when tests fail provide impactful lessons and code changes are fresh in the minds of fixing hands. Unintended consequences from tightly coupled services are understood at least as far as the container boundary, a stub of the core systems. Most importantly you have constantly working, releasable software. The Andon cord should pull itself.
Behemoth releases tend to be crescendo affairs. Quality is built up towards release go-live, the inefficiency of having to rebuild your house every time you decorate mostly rationalised away.
They are too big, too stressful, too brittle, too many defects. A feeling and recognition that some thing great has been achieved when the opportunity to see the product in action has only just begun and everyone is now exhausted not to mention the act itself may perpetuate bad practice, coercion to work weekends being the most toxic.
By having continuously working software you remove the hope associated with the climb towards a releasable state allowing focus to be spent on value creation. You shift a constraint holding back continuous delivery and take a step towards an immutable production, a place where releasing software can become mundane.
In the past the discussion on constraints has often centred on quality versus delivery as if a trade off was necessary to satisfy timeliness. I’m not sure if they were ever viable things to barter. I doubt it as the recurrent compounding of technical debt isn’t factored into the in-moment decision making.
We never think about how our decisions will come back and haunt us, making an equally difficult situation harder to resolve at some point in the future. We care about our future selves much less than our experiencing selves, yet our remembering selves spend most of the time worrying about our future in the present, when rationally speaking we should be enjoying the very moment we have prioritised.
Regardless the outlook for quality does seem to have changed and thanks to the maturity of Devops practice rather than being a constraint to delivery if left in the right combination of hands quality becomes the means to deliver.
There are lots of benefits to working in this way, work becomes pulled instead of pushed, capacity issues subside, you are able to deliver frequent higher quality change. Those departments that were once slow, reclusive and seen as overhead become sources of innovation and drivers of value.
Another significant side effect is its positive impact on culture, people are actively learning, seeing work getting done and knowing they are achieving things. Work doesn't get much better than being in a state of flow it completely changes the shape of conversations allowing focus to be kept on the frequent proven delivery of value and within that process becoming a self-aware, life long learning organisation.
