Good Practices to Build Security Automation for DevOps

The software application is now becoming more and more complex. The process of software development has been transformed itself that moving from the waterfall, RUP model (where the release cycle was measured in years, and the security process could take almost as long as the team wanted) to Agile and now is DevOps where things got very much speedier. The changes don’t only happen in formation but in work style & work mindset. Nowadays, we have heard a lot of DevOps terms that its goal is to give development teams more ownership in deploying and monitoring their applications. Automating how we provision servers and deploy our applications is at the heart of DevOps. Automation is a key factor to make success in DevOps manner. This automation is known as:

• Continuous Integration (CI)- is the principle that code changes are checked into the source code repository in small batches. With each check-in, the build system automatically checks out the latest version of code and goes through the build process. If the code that is checked in breaks the build, our changes get backed out and we get to figure out what caused the breakage.
• Continuous Development (CD) – is the principle & practices for how the results of testing are reviewed, and the system automatically makes the decision as to what to do with the build. We run a set of tests if the code passes, the build moves to a staging environment.

This accelerates a transformation in security testing. Security finds itself very much at the same crossroads. We cannot spend hourly on going through the process reconnaissance, scanning, enumerating and exploitation. Instead, security tests need to be automated to make effective software delivery pipeline.

There are many instances used as representatives for working between DevOps and Security. We can easily find a lot of names from Internet such as: DevSecOps, DevOpsSec, Rugged DevOps. Securing DevOps is all about bringing together the design, development, and operational aspects of security into one continuous discipline throughout the software product lifecycle. It’s not important how to call it. The key thing is to use it and gain the desired results. Following are tips to build our security automation effectively in DevOps manner.

1. To be a role of enabler rather than a blocker: If security testing plays its role as a blocker in the traditional environment where a fail identified to stop the progress of software delivery. In DevOps,  to be a success, we must be a supporter to help everyone go faster and assert the security quality at the same time. The core principle here is that if we are a blocker, we will be routed around. If we are enabler then we are adding value. A question for the security team is how to make the software development/ testing pipeline go faster and safer?
2. To Secure software Supply Chain: In this era when the rise of a lot of compliance framework, we add in a certain security control because compliance says we have to. However, we have to look into our software supply chain from start to finish and pinpoint areas where security can add value. A software supply chain security is viewed in different areas.
   • First, it usually starts off with software design. With this area, how to make our “sensitive” flows more secure. “Sensitive” flows can be identified by different criteria such as: what flows cause critical business down if a security flaw happens; what flows access to sensitive data that may cause data breaches; … 
   • Secondly, software dependencies – what library, external API our software using? What other layers our software stay on top? What other external systems that our software interacting with or exchange data together. And all these dependencies are meeting a good security level?
   • Thirdly, in the development stage of software engineering, are we writing security tests? Any practices like OWASP Top 10 we are using?
   • Next is build process, do we have security acceptance tests that they are covering a certain level of our application areas?
   • For the deployment stage, what happens to get software into production? Are we securing config and secrets for the application and infrastructure?
   • Finally, Operation stage, are we under active attack at this very moment? Do we even know what is getting attacked?
3. To create an effective feedback loop for security automation improvement. Feedback Loop with Using CI/CD with tools like Jenkins or Bamboo or SAS tools like Travis or CircleCI, adding security testing to CI finds flaws in development, and this is most often the best place to start with security automation. We will go into detail later on. Feedback Loop with using the deployment system. In general, we have a Staging Environment where changes get pushed before going to customers in a production capacity. The goal is to test against something that looks and acts exactly as production does. This is a good place to run security attack tooling and anything that attempts to exercise several attack vectors in conjunction. Finally, we need to monitor sensitive data flows against with common attack from production to answer are we under the attack? There are some tools can be used such as WAFTs or RASP for common web application security
4. Always keep the build process as fast as possible. In Agile and DevOps manner, the build is a skeleton of the whole process. If it gets down or breaks, the team cannot get work done or software released. Adding more security tests must not make the build slow down or taking too much time. Although it’s hard to quantitively identify how is too much, if our build takes more time than we make a cup of instant coffee, then we have a problem. A practice that is to keep its progress less than 5 minutes. If we find ourselves in a situation where security testing is taking too long, then we might consider splitting tests into fast tests and slow tests. The fast tests come into the main build job, while slow tests are executed in a parallel build. Most build system can do this, and we are good with using Jenkins to handle this situation at the company.
5. Don’t let our security tests preventing the release. We understand that security testing is to identify security vulnerability problems. So when saying “don’t let our security tests preventing the release” sounds like antithetical to our purpose, but security testing is tough to get right, we have to ensure what has been pointed out must be a real vulnerability, not a false positive. Therefore, we should be transparent to developers, communicate with them on the issues found but not breaking the build as a normal test failure would. However, in some cases, we cannot avoid blocking the build such as when sensitive data is exposing to unauthorized persons. We may build a team culture that we decide which tests are labeled as critical and most worth blocking for.
6. Build a ChatOps to convey the security info effectively. ChatOps is a new operational paradigm work that is happening in the background today is brought into a common chatroom. In a ChatOps environment, the chat client serves as the primary communication channel for ongoing work. Tools that developers and operations managers already use are integrated into a collaborative communication environment to improve ticket tracking and automated incident response. All the popular platforms have webhooks and integration that are easy for us to report with.

The two popular chat tools now are Slack and HipChat commonplace in organizations across every industry. Both tools have similar features, as well as an active community, and the marketplace of third-party services that we can easily integrate. Use ChatOps integrated with a lot of tools that we rely on such as GitHub, Salesforce, VictorOps,… is providing engineers with a deeper awareness of software and infrastructure. Support teams can use it to provide faster response times to customer requests. Security team can make informed about security flaws as fast as it is identified. Sales and marketing team can use it as a pulse on metrics and customer data like never before. And it’s all the result of shortening feedback loops

Securing DevOps is essential. However, there is no standard defined for it. Even having a chance of a standard developing are small because different organizations are doing things their way, they hardly to agree on common rules or standard. However, it is exactly the spirit of DevOps where is less prescriptive and no standard requirements.