Best Practices for Onboarding JFrog Xray

Note: A version of this blog post is also published on dev.to

Introducing, adding, or replacing a new SCA (Software Composition Analysis) tool such as JFrog Xray into your SDLC, if not handled correctly, can be very disruptive to the SDLC and organization.

This blog post provides recommended best practices for onboarding JFrog Xray; in order to reduce disruption, improve adoption, and shift left to create a DevSecOps behaviour.

First, let’s begin by describing a common scenario when onboarding security tools. What usually happens, especially when introducing a new tool, is that all screens go red and alerts appear from every direction. In such scenarios, it is a reasonable knee-jerk reaction to require a system lockdown; disqualify builds, reject dependencies, etc. However, such a reaction, even though it may seem valid, is counter productive. In theory, this behaviour will bring production to a halt, cause frustration and conflicts. Naturally, in real life, in most cases, no one will really stop the business. Rather, the organisation may develop Alert Fatigue and will ignore the tool, which is definitely not recommended.

5 Onboarding Recommendations

1. Involve R&D
   When you involve R&D in the process, you have a real chance at achieving a real DevSecOps process that is manageable and productive. There is approximately one security engineer per 100-200 developers. Reviewing and controlling all security issues by a single engineer is only possible when it is attached to development. Otherwise it will cause bottlenecks, delays, redundant work, and a lot of frustration.
2. Configure a Watch per application team and/or maturity stage
   A JFrog Xray watch groups together a set of resources (repositories, folders, builds, etc.) for the application of a Policy or Policies (which security and compliance governing rules).
   Creating a Watch per application team allows giving every team their own “world” and responsibility. In turn, this provides the ability to “shift left” the responsibility of security governance.
3. Integrate development tools
   Try to bring the information into the development tools (i.e. CI server, IDE), which will provide governance related information into the developmer’s environment. For Xray we have IDE integration plugins.
4. Start small and work in cycles
   • Choose one team to start the integration process with. Here are some considerations that you can use to choose the right team:
     • A vanguard team – a team which is open to changes and testing new ways to improve (especially if they care about security).
     • A new app team – if you have the option to, choose a team which is starting a new project/service (i.e. starting “greenfield”)
     • A team with less “integrations” or dependencies with other teams.
       Work with the selected team to establish and improve the process before expanding to other teams.
   • Start with the “Critical” issues. As mentioned above, implementing JFrog Xray will most likely generate tens or hundreds of alerts of all types and severities. Trying to handle everything at once might not be reasonable. If you see that even the number of critical issues is too high and your tool supports it, try to be even more granular than Low/Medium/High/Critical. Try to go by CVSS score; start with 9.9-10, then 9.8-10, 9.7-10, etc. The granularity of this filtering should be based on your needs.
   • Make a decision for each “Critical” issue. Decide whether the issue is needed  and can be fixed, or whether it should be ignored. You may decide to whitelist a group of issues temporarily, or define a deadline for fixing the issue.
   • Do not use brute force actions. JFrog Xray allows you to take actions with email notifications, webhooks, Slack messages, Jira issues. It also allows blocking download, failing builds, and even blocking the distribution of releases.
     Try to avoid any action that will disrupt the development lifecycle until you complete cleaning up manageable existing issues (i.e. issues that can be resolved or ignored). For example, actions such as blocking dependency downloads or failing a build based on the new tool’s scan results.
     • Only when the build passes without violations consider a hard action future.
5.  Add external notifications (such as Jira and Slack)
   • You probably want to add external notifications only for new issues, otherwise you may be risking “Alert Fatigue”.

After handling criticals repeat the same process for major issues. For minor issues, you should consider if you want to handle them at this point in time, or rather move to the next project/team and only then deal with them.

Hopefully the best practices in this blog post will help you remove any tension that you may have between Development and Security, as well as provide you with some DevSecOps foundation.