Policy Evaluation in Source Control Management

IQ Server | Reading time: 10 minutes

Is this article helpful?

Nexus IQ for Source Control Management (SCM) is a set of features that lets developers gain early insight into code changes by working in tandem with continuous integration to push policy information about an application’s components directly into their SCM.

Currently, Nexus IQ for SCM has the following features:

Features in IQ for SCM

  • Automated commit feedback: Nexus IQ for SCM puts the information needed to quickly remediate vulnerabilities in software solutions at the fingertips of developers by pushing policy evaluation information into SCM commits and pull requests (PRs), where developers work.
  • Automated pull requests: Nexus IQ for SCM will automatically create pull requests for policy violations on components that have an available version which remediates those violations.
  • Pull request commenting: Nexus IQ for SCM adds a comment to pull requests for repositories configured for source control when the PR introduces a new policy violation.

Early Feedback and Automation

In today’s software development lifecycle (SDLC), developers are asked to release complex software at an increasingly faster pace. Because of this, devs are relying more and more on OSS, utilizing openly available, popular components rather than writing unique code. Consequently, Sonatype’s State of the Software Supply Chain found that open-source software (OSS) components make up more than 80% of most modern applications. With this, the traditional capacity of a developer simply writing code has shifted to a holistic role that includes assembling and integrating code that supports both your software and business operations.

While assembling code, developers often use source control management systems, like GitHub, GitLab, and Bitbucket. SCMs are often the first place where a piece of code gets shared and reviewed by both humans and machines. Nexus IQ for SCM uses IQ Server to establish a relevant policy configuration. Scan evaluations in SCM are run against that policy configuration so that as developers build applications, they receive accurate and timely information to take action on reported violations.

The main value of SCM systems is their ease of collaboration. Developers are able to push quality control of their application development to a source control platform, where this collaboration exists in code reviews that contain both commits and pull requests.

Building on this, Nexus IQ for SCM provides suggested remediations for policy violations directly into the source control repository. We do this by automatically creating pull requests with the changes to an application’s component manifest. Using this type of early feedback and automation, we reduce rework and keep development teams focused on contributing business value rather than managing application component risk.

How IQ for SCM Works

To use Nexus IQ for SCM, you first need to configure the IQ Server to allow access to your company’s Source Control Management platform. For large organizations, we recommend enabling automatic source control which lets CI and CLI integrations configure application source control connections when running from a locally cloned repository (a common practice in CI systems).

Once configured, commits will immediately receive automated commit feedback. If enabled and appropriately configured, applications will also start seeing automated pull requests for any new policy violation with suggested remediation, and pull request comments that detail the summary of violations, affected components, and a description of the violations that were introduced in the PR.

A Continuous Integration system or build agent is required for these features to work. For example, if Jenkins is pulling and running the scan, the IQ Server can pick up the source control context and use that as a source. This eases control and gets policy information closer to developers.

We suggest using CI against all feature branches in your SCM, letting you see early feedback on the quality of your code as it’s written. For IQ Server to provide feedback to feature branches during development, CI must be configured to run a policy evaluation against these branches. We recommend that evaluations against the master branch use the ‘build’ stage in Nexus IQ, and that feature branch evaluations are performed against the ‘develop’ stage. Because feature branches are volatile and vary across the features developed, using the ‘develop’ stage helps reduce noise by not showing these evaluations on the dashboard or your reports. This is similar to our IDE integrations that use the ‘develop’ stage for the same purpose.

When deploying IQ for SCM, you should authorize IQ at the Root Organization level to communicate with your SCM tool (example: GitHub). Then, when you do any evaluation, we will look at the context in modern CI systems—meaning your Git repo gets cloned, then we run an evaluation in that directory, and we can get that information and see what was cloned. This lets us associate GitHub projects to applications in IQ Server.

Example Use Case: Automatic Pull Requests for npm

Using this example, you will be able to:

  • Regularly scan an npm project for violations of your organization’s stated policy for the Software Supply Chain.
  • Create GitHub pull requests and update project component dependencies to available versions that comply with organization policy.
  • Automatically verify and integrate Nexus IQ remediation suggestions in your Continuous Integration pipeline.

Prerequisites

  • The latest version of Nexus IQ Server.
  • A GitHub Repository hosting your project
    • This example uses an npm project. The configuration and workflow described here will also work for Maven projects.
  • A Continuous Integration (CI) system.
    • This example uses Jenkins with the default plugins installed. Details on the example pipeline are shown throughout the workflow.
    • We recommend that your feature branches use the IQ Server ‘develop’ stage and your master branch uses the ‘build’ stage. This setup reduces noise as evaluations made with the ‘develop’ stage aren’t displayed in the dashboard or reporting view.
  • Docker running on the host to be used by Jenkins - https://www.docker.com/get-started
  • Duplicate the sample repository at https://github.com/sonatype/nexus-iq-for-scm-demo/ using the GitHub ‘import’ feature. Nexus IQ for SCM operates on private repositories. The demo repository is public so that it can be duplicated to your account as a private repository. Note this process is different than a regular fork.

Note: Automated pull requests only work on private repositories. Currently, we support Auto PRs on GitHub, Bitbucket Server, and Bitbucket Cloud.

Configuration

  1. Configure your Nexus IQ Server:
  2. Configure a Jenkins Continuous Integration pipeline:
    • Install the latest version of the Nexus Platform Plugin in your Jenkins server.
    • Add credentials to authenticate with your Nexus IQ Server.
    • Add credentials to authenticate with GitHub.
    • Configure the Nexus Platform Plugin to communicate with your Nexus IQ Server.
    • Create a new multi-branch pipeline job for the demo GitHub repository created above. The sample pipeline will:
      a. Build and scan the repository on the master branch and create a pull request.
      b. Build the IQ for SCM created pull request, execute an npm install to automatically apply the change in the package.json to package-lock.json, then automatically merge changes to the main branch.
    • Note that the checkout to local branch option is used to match the structure of our sample Jenkinsfile.

Workflow

Once you’ve finished up configuration, it’s time to kick off a build of your npm application. When you do so, the following workflow takes place:

  1. The pipeline automatically runs on the master branch and the Nexus IQ server performs a policy evaluation.

    pipeline {
    agent {
    docker {
      // Docker image with node and git installed
      image 'tarampampam/node:13-alpine'
    }
    }
    environment {
      HOME = '.'
    }
    stages {
    stage('Build') {
      steps {
        sh 'npm install'
      }
    }
    stage('Test') {
      steps {
        echo "Run acceptance tests to ensure that IQ remediation changes function as expected in the application"
      }
    }
    stage('Policy Evaluation') {
      // Policy evaluation should only take place against the branch we intend to merge to
      when { branch 'master' }
      steps {
        sh 'npm run build'  // build script using webpack and the copy-modules-webpack-plugin for easy scanning
        nexusPolicyEvaluation iqStage: 'build', iqApplication: 'npm-example',
            iqScanPatterns: [[scanPattern: 'webpack-modules']],
            failBuildOnNetworkError: true
      }
    }   
    
  2. A policy violation is found for the react-dom 16.0.0 component. We also see that there is a newer version available we can replace it with.
    Vulnerable Component

  3. Nexus IQ automatically creates a pull request on GitHub, replacing the affected component with the newer, no-violation version. Details of the vulnerability and a link to the full Nexus IQ report are provided directly in the pull request description.
    Example Automated PR

  4. Nexus IQ suggests the smallest change possible to remediate the problem and preserves the formatting of your package.json file.
    Component bump to non-vulnerable version

  5. Our Jenkinsfile recognizes it as a Nexus IQ generated pull request based on the branch name. If the build successfully verifies the update, the package-lock.json file is updated and pushed back to GitHub. Note that this only occurs if a change was made to the package-lock.json.

    stage('Incorporate IQ remediation changes to package-lock.json') {
    when {
     // Only work with branches we recognize as created by Nexus IQ
     branch pattern: ".*-to-.*", comparator: "REGEXP"
    }
    steps {
     script {
       sh 'git status'
       // Look to see if package-lock.json file was updated as part of the build process
       def dirty = sh(script: 'git diff HEAD package-lock.json', returnStdout: true).trim().size() > 0
       if (dirty) {
         echo 'detected changes to package-lock.json, merging now'
         withCredentials([
             [$class          : 'UsernamePasswordMultiBinding',
              credentialsId   : 'github-user',
              usernameVariable: 'GITHUB_USERNAME', passwordVariable: 'GITHUB_PASSWORD'
             ]
         ]) {
           // Configure committer details 
           sh 'git config --global user.email sonatype-ci@sonatype.com'
           sh 'git config --global user.name Sonatype CI'
               
           // Only necessary for https urls, can use other mechanisms to authenticate with github as desired
           def updatedUrl = env.GIT_URL.replaceAll('github.com', "${GITHUB_USERNAME}:${GITHUB_PASSWORD}@github.com")
           sh "git config remote.origin.url ${updatedUrl}"
               
           // By default Jenkins will only checkout the specific branch so we configure to pull the branch we 
           // desire to merge to
           sh "git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'"
           sh 'git fetch origin master'
               
           // Commit changes to package-lock.json file, merge and push to the remote branch
           sh 'git commit -m "committing IQ remediation changes" package-lock.json'
           sh 'git checkout master'
           sh "git merge ${GIT_BRANCH}"
    
           // This should automatically close the open PR when pushed              
           sh 'git push origin master'
         }
       }
       else {
         echo 'no changes detected in package-lock.json, skipping merge'
       }
     }
    }
    }
    }
    }
    
  6. Now the pull request, which contains both the package.json update from IQ and the package-lock.json update from CI, can be merged.
    Merge automated PR

  7. Merging the changes into the master branch triggers another Jenkins CI build and an associated Nexus IQ Scan. The new scan shows the updated component version.
    Remediation of vulnerable component

Nexus IQ for SCM always bumps to a specific version, and pins it in your manifest. Only pinned versions give you control on remediation through security and quality policies. For example, version 1.3.7 might pass policy, but version 1.3.8 might not. A version range of ~1.3.0 does not allow this type of control—we don’t want to replace one security violation with another. For more details, see our IQ for SCM help docs.

The example workflow shown here used GitHub, Jenkins CI, and an npm project. We encourage you to utilize other systems you have in place to work well with our software.

Measuring Success

Success can be measured by a reduction in mean time to resolution (MTTR) which can be seen in our success metrics. IQ for SCM provides the ability to prevent new violations from entering your main branch by informing development teams of risk as soon as they introduce it. We use automation to fix any violations as they occur by providing pull requests with the version change, and both of these practices should reduce MTTR.

If you’re ready to start using IQ for SCM features, reach out to your customer success representative for guidance.

Resources

Need more help? We have you covered: