I’ve spent the last 3½ years at ThoughtWorks helping build infrastructure in platform teams and recently I’ve been reflecting on some of the common challenges that come up when building them.
Treating your platform as a product is a technique that’s been highlighted on the ThoughtWorks Tech Radar since late 2017 and is something that I’ve seen more and more of at clients over the past couple of years. I think it’s a really great approach and here I want to present the lessons that stand out to me when appying it and why I think they’re important.
But first, a few definitions. Platform is a bit of an overloaded term in the IT industry. In my opinion, if you’re in a team building infrastructure that someone else in your organisation is going to make use of in more-or-less a self-service manner, you’re a platform team. It may not be a PaaS like Kubernetes but it’s still providing the same function: a means to host or support an application.
Throughout this post I’ll be referring to customers. For a platform team, the customers are the people using the platform i.e. the development team(s). You may not treat them the same way you’d treat the customers of the product or service your organisation provides but there are definite similarities between them.
Make sure you know what your customers need
A couple of years ago I worked with a client on building out a big data platform. Having heard the ‘data lake’ buzzword, they wanted a place where they could consolidate all the data from the different parts of their business and have an easy way to query it.
The problem was, there was a lot of different datasets and many possible use-cases. Where should we prioritise our initial development efforts? We started by focusing on a single, well-defined use-case and built out the platform from there.
Without knowing what your customers are going to need, you may well end up building features no-one will use or that aren’t a priority. Having the customers feed in to your priorities allows you to answer questions like, ‘should we build an alerting system or a distributed tracing system first?’
The GDS PaaS team even perform user research on the platform they’re building to better inform where they spend their development efforts.
As the Tech Radar blip says, “Applying product management to internal platforms means establishing empathy with internal consumers (read: developers) and collaborating with them on the design […] avoid[ing] the pitfall of building yet another stagnant, underutilized service-oriented architecture.”
Defining boundaries is a really useful way of allowing an organisation to scale. By splitting up responsibilities between teams you allow them to operate independently from, and parallel to, each other. This is one of the touted benefits of adopting a microservices-based architecture.
For an infrastructure platform team, it is tempting to use containerisation as a boundary to separate responsibilities between the platform team and the customers. The platform team takes the responsibility of running the container and the supporting infrastructure, while the development team takes responsibility for what runs inside the container.
Regardless of the technology in use, it is imperative that both sides of the relationship understand the expectations of the other side. With microservices, a way to achieve this is with contract testing. The contract allows each side the means to communicate what they expect the boundary of the API to be.
With infrastructure, a similar contract is needed. What are the performance considerations of the platform? Who will be responsible for managing secrets? What data format will this API expose? How do I get logs into and out of the system? Where do I apply application configuration?
This won’t be as formal as the contract you might have between microservices, but can be communicated in various ways, such as service level objectives.
As the organisation grows, it may be necessary to spin up multiple platform teams. When this happens, it’s important that the responsibilities between the platform teams are well-defined. One trap it is easy to fall into is splitting the teams into ‘build’ and ‘support’, where one team is responsible for building out new functionality and the other is responsible for the day-to-day operational tasks.
The problem with this approach is that the support team doesn’t necessarily have the visibility and knowledge of the functionality the build team is producing, making supporting it a challenge.
Don’t build walls
However it is easy to allow walls to be built up along those boundaries. The DevOps movement has had a big emphasis on breaking down walls between operations and development teams and we don’t want to lose this in the world of platforms. To stretch the metaphor, whilst boundaries are useful, walls are opaque and don’t allow easy communication between the two sides. Make sure you build picket fences instead or a wall with plenty of open windows and doors in.
Too often I’ve seen platform teams slip into an us-and-them mindset where dysfunctional communication patterns with their customers and a misunderstanding of skills and intentions result in misplaced assumptions, tension and even hostilities. When everyone is working towards the same end-goal, this helps no-one.
Figure out communication routes with customers
To avoid the us-and-them mindset, communication with customers is key. It’s important to understand how they are using the platform.
Uncouple the platform from the customers
One of the main advantages of building out infrastructure in a platform team is to enable customers to solve problems for themselves.