Because of T-Mobile’s position in the cutthroat US mobile telephony marketplace, business agility is unquestionably strategic for the company, and it looks to its IT organization in large part to drive the agility they need to remain competitive. (
In fact, improving the culture of IT, and with it the velocity of their business, has led T-Mobile to expound upon their approach on the conference circuit, both at the
“We can’t beat
To that end, T-Mobile has undergone a transition to a popular new approach for building and running software: DevOps (see my recent article for a definition of DevOps). Right out of the gate, however, Krienke’s team had a problem.
“People had a hard time with the term ‘DevOps,’ because they thought it meant developers doing operations,” Krienke pointed out. “Instead, we focus our discussion on ‘continuous delivery’ and ‘Agile methodology.’”
Regardless of its name, T-Mobile’s emerging approach to building and delivering software rests on four pillars: standard, automated configuration of management processes, service virtualization, automated deployment environment management, and automated testing.
First Stop: Configuration Management
Cleaning up its configuration management was an important early step, as their build and deployment process varied from app to app. Meanwhile, “the need for more deployments and faster time-to-market for features keeps increasing,” according to Krienke. Moving to a centralized configuration management team “reduced resources by 40%” necessary for this part of the software effort.
Service virtualization was also an essential capability for resolving many of its software process issues, and led it to implement CA Service Virtualization from CA Technologies about two years ago. By simulating the behavior of parts of a complex application, developers can work in parallel, without having to worry about dependencies on production code or on the work of other teams.
With service virtualization, “we could get wins that benefited multiple groups,” Krienke says. “Let’s get environments up and let’s start testing sooner.” As a result, “development could start doing more parallel development.”
The business quickly saw value. “One of our key customers – retail sales – would benefit.” Service virtualization provided an important additional value beyond the acceleration of the software development process. “With service virtualization behind the user interface, we suddenly had a training environment for retail staff,” Krienke explains.
On to Deployment Automation
Automating software deployments is central to T-Mobile’s DevOps strategy. “The ability to utilize a modern workflow tool for deployment automation was key for us,” Krienke says. To that end, T-Mobile purchased CA Release Automation, which facilitates the building of core deployment patterns, making it easier for teams to see how the deployment flow worked.
The efficacy of any tool, however, depends upon how people use it. “The key to any process – including an automated one – is repeatability, which requires consistency.” T-Mobile increasingly focuses on consistent processes, tools, and practices, including the ability to repeat software builds quickly and reliably. “Deployment automation is now more consistent,” Krienke says.
Dealing with the Cultural and Organizational Change
The human side of T-Mobile’s DevOps transition was even more challenging than the technical aspects, as it is for most organizations. “Change is perceived as job insecurity,” explains Krienke – a common roadblock.
“If people see their role as producing a particular document or process, and if the discussion centers on the need for said item, that will often create ‘angst’,” Krienke explains. “People’s roles and identities often become intertwined with the processes they develop or support, and that can make change hard for team members.”
The software organization also had to rethink how they prevented recurring issues in the software. The existing approach was to conduct post-mortem meetings to discuss changes to procedures, with the goal of avoiding a recurrence of a particular problem. “This is what organizations have to watch out for and are things that we need to be cognizant of as we evaluate and strive to improve our processes,” Krienke says.
With the new approach, “we don’t layer on after-the-fact procedures,” Krienke says. “You need to look at the architecture.”
The team architected their core solution first, including application configuration, monitoring standards, and coding documentation. “Now the challenge is to get more teams to adopt this strategy,” Krienke adds.
In fact, software design and architecture underpin T-Mobile’s entire DevOps effort, as the figure illustrates, as are governance and process.
In fact, T-Mobile’s revamped governance approach promises to provide value beyond the development and deployment processes themselves, as everything the IT organization does must also be compliant with regulations like Sarbanes-Oxley (SOX).
For many IT shops, such regulatory compliance slows everything down. At T-Mobile, in contrast, “DevOps and SOX are not mutually exclusive,” Krienke says. “SOX expects processes and accountability, which you can still automate.”
Collaboration is Key
In the final analysis, the central element of T-Mobile’s DevOps efforts is collaboration. “People and process get very intentional,” Krienke points out. “Look at culture, which is where we are now.”
Many organizations fail on this important step, as successful collaboration isn’t automatic. “If they stuck us all in a big room like the junior high dance, with the boys over here and girls over there, people still wouldn’t talk to each other,” Krienke quips. “That’s the reason why DevOps often fails – you’ve created a ‘DevOps organization.’”
To avoid this “DevOps organization” trap, T-Mobile has laid out a core set of guiding principles: promoting innovation; ensuring mistakes become learning opportunities; being open to change and ideas; collaboration is critical, not just among development, test, and operations but also with the business; efficiency and effectiveness of the software; and leveraging process automation.
By following these basic principles, T-Mobile is well on its way to transitioning to an effective DevOps culture that promises greater velocity, quality, and business agility. However, the team’s work isn’t done. It’s important to “evangelize and champion the successes,” Krienke says. Also important: “We now as an organization have put Continuous Delivery and Agile methodology as stated goals,” Krienke adds.
As a result, they are actively looking for those transformational minded people who wish to participate. Krienke sums it up: “success is contagious – especially when it is being rewarded by management.”
Intellyx advises companies on their digital transformation initiatives and helps vendors communicate their agility stories. As of the time of writing, CA Technologies is an Intellyx customer, but had no input into this article. None of the other organizations mentioned in this article are Intellyx customers. Image credit: T-Mobile. T-Mobile and the magenta color are registered trademarks of Deutsche Telekom AG, © 2014 T-Mobile USA Inc.
