Software Development At SAP: DevOps And CI/CD

Software has a lifecycle: It starts with development, shipping, deployment, bug fixing, enhancements and ends with upgrading to the next release. For a software product to work flawlessly, it must consider all aspects. We, as the consumers of these products, face issues in this area every day. No useful error messages are provided, software is hard to install, or an upgrade to the next version produces errors – what might be an inconvenience for most people can be life-threatening for businesses.

Is DevOps the solution?

DevOps should solve that problem. At the core, the idea is to break up the silos between individual experts and different aspects of the software lifecycle. Traditionally, the development team builds the software, another team configures the installer, then there are testers, another group installs the software at the user company, and the support organization manages customer issues. In a DevOps philosophy, all these responsibilities lie with the development team, meaning all tasks are performed by the same people. Testers can fix the code themselves, or the developers can add automated tests.

This ensures software developers consider all aspects of the software lifecycle, and thus the quality of the product increases for us end users.

CI/CD pipeline

A natural consequence of breaking up these boundaries is automation. Whenever a code change works for the developer, the code is added to the main codeline, the software product is automatically complied, packaged, and deployed, and undergoes automated tests. This is called a Continuous Integration/Continuous Delivery pipeline and should be used for every kind of software development. From a large enterprise solution to a small open source project – even for in-house development, this approach pays off quickly.

Building such CI/CD pipelines has gotten easier in the past couple of months. Personally, I use GitHub actions for that. Whenever code is pushed into my GitHub repository, a set of repository specific actions are executed automatically. In one of my repositories, these actions are to download the required third-party libraries, compile the entire software, run predefined tests on the code, package it as a library file, release the library for others to download, build a docker image and upload that to docker hub.

Everybody does everything

For the other aspect of DevOps, where everybody is supposed to do everything, I have experienced mostly negative examples. Yes, it would be beneficial to have experienced developers on board who have seen it all, from UI development to backend, from coding to product management. However, this approach is a very expensive solution for this problem.

And it does not stop there, either. This approach would mean that the same software developers also have to use the program as consultants and work in customer support, answering trivial questions like: How do I change the color?

Is this cost efficient? Does it reflect reality? Not really. Even if initially all the developers get assigned equal parts of these tasks, over time, lead developers will have less time for it and junior developers will have to bear the brunt of the load, rendering the initial intention obsolete.

Root cause

From my experience, this part of DevOps is used to cover up a dysfunctional management aspect. The responsibility of managers – more precise: for every single person in a software development department – is to create a work environment for the developers(!) in which they are self-motivated, creative and love what they are doing. Period. Nothing else. It hurts me, as a software architect, to say, but I came to accept it as reality.

All roles beside the developers only act in a support function to make developers more productive. And yet, developers are often treated like juniors who should just write code. Obviously, their goal is to get out of this role as quickly as possible. This is not a sustaining setup.

Guiding developers and growing their experience gradually works better. When a developer starts, a simple, isolated, and non-critical feature is assigned to them. It is such a rewarding experience to see their own feature in the overall product later, and it makes them proud of their creation.

Next, a consultant is asked to join for a day, showing how they leveraged the product during a customer engagement to provide feedback. This again is a win-win situation. The consultant feels appreciated and the developers can see how the features are used. From now on, the developer will put themself into the shoes of the consultant when building the next feature.

The support staff can provide valuable input as well. How many tickets were created, which areas cause grief, what information is missing to speed up finding a successful customer resolution? Then the developers understand how expensive improper error messages and usability issues are. What also works out well is pointing developers to specific interesting questions in forums and letting them respond. Suddenly, they are visible, and customers appreciate the involvement of feature experts.

Slowly broadening the horizons of developers further, until they embrace architectural decisions and the strategic long-term goals of product management, is a much better approach than just saying “DevOps!” and calling it a day.

Difference to DevOps

Some people will love the challenges provided, others prefer a slower learning curve or have strengths in one area but not the other. We are all humans.

Another aspect is the amount of time it takes. To get insights into a role is a matter of one day. Working in a role on a rotational schedule requires a lot of time and creates, what macroeconomics calls, opportunity costs. Finally, not everybody has to have the complete picture. If some achieve the full end-to-end understanding over the years, that is enough. Those seniors will influence team standards and lead by example.

Granted, breaking up the silos this way requires an active management of the exchanges between the teams. Assigning tasks is simpler than creating a working environment where people trust each other and are motivated.