Vista Acquires a Majority Interest in Sonatype: A Great Day for our Customers, Partners and Community

Comprehensive Guide to Lifecycle Scanning

IQ Server | Reading time: 11 minutes

Is this article helpful?

NOTE: We update this guide as more languages are added to the Nexus IQ Server. Please check back often for updates.

In this guide:

Overview

This guide will help Sonatype customers evaluate their applications and deliver timely component intelligence to their developers throughout the software development lifecycle (SDLC). Part of this requires a high-level understanding of our integration points and our binary fingerprinting process so you can get the most value from scanning at the optimum point. We’ll take a comprehensive look at Sonatype’s supported languages to understand where we can scan an application during the SDLC, and how to deliver the most value to developers.

We also have a printable Quickstart Guide that you can use to get a brief overview of scanning Nexus Lifecycle supported languages.

Nexus Intelligence

The Nexus IQ Server uses data derived from our automated vulnerability detection system — basically a big funnel of sources (NVD, GitHub commits, Central Repository, Sonatype research, etc.) that is processed with automated techniques such as data filtering, aggregation, and machine learning algorithms. Next, Sonatype researchers triage the incoming data and determine if there is a vulnerability, creating a research ticket for further investigation when necessary. Tickets are prioritized and then entered into our human-curated research process. When research is complete, it goes into our data mart which feeds Sonatype’s Hosted Data Services (HDS). Data from the HDS is what you see in the IQ Server Application Composition report after an application scan.

Sonatype Data Flow

Figure 1: Sonatype Data Flow

Component Identification

In IQ Server, when an evaluation is performed, hashes of the components in your application are created. In simple terms, a hash is a mathematical way to make sure that data hasn’t changed. It is used to index a component’s original value and then used later each time the data associated with that value is retrieved. It works by passing data through a hashing algorithm which is a function that converts a data string into a numeric string output of fixed length. You can think of hashing algorithms as tools to amplify and detect changes in data.

This in many ways is like a fingerprint, which is unique to a component. That fingerprint (hash), is compared back to components known to the IQ Server, which will provide all the available component information. This includes: usage statistics, security vulnerability, and license information.

All of this information can be used as parameters in your policy, which translates to more understanding of the component usage in your organization. That data, however, can only be linked based on a matching of hashes, which can be exact or similar, and in some cases, unknown. For more information on matching, please see our Component Identification help topic.

Nexus Integrations

There are several methods for integrating scans: via the IQ command line interface (CLI), through continuous integration (CI) plugins, and integrated developer environment (IDE) plugins.

Command Line Interface

The most unobtrusive and non-tool specific approach is to use the command line version of the Application Evaluation Tool – commonly referred to as the Nexus IQ CLI. Any application can be evaluated against your policies simply by using the CLI. This includes adding a build step in your CI server and processing a simple shell script during the building of your application.

The IQ CLI lets you run in either normal (Lifecycle) mode or XC mode. Nexus Lifecycle currently has advanced component intelligence for Java, JavaScript, NuGet / .NET, and Python. The addition of Lifecycle XC brings basic-level intelligence to languages like Ruby, Swift, CocoaPods, and PHP. In XC mode, results come from unverified public sources, and do not include any Sonatype enriched information.

For more information on the CLI, please see our help topics on the Nexus IQ CLI.

Continuous Integration Plugins

Nexus IQ Server analyzes components used in your software development for security and license characteristics. When integrated with a continuous integration server, this becomes a dynamic analysis performed on a regular basis—occurring potentially with each build running on the server.

Maven

Our Maven plugin lets you evaluate any Maven-based software project. The plugin can run on a command line interface and can therefore be executed on any continuous integration server, as well as a number of popular IDEs.

For more information, see our IQ for Maven help topic.

Jenkins

The Nexus Platform Plugin for Jenkins integrates via Jenkins Pipeline or Project steps with Sonatype Nexus IQ Server. You can use the plugin to perform an IQ Server policy evaluation against files in a Jenkins workspace.

For more information, see our Jenkins Platform Plugin help topic.

Nexus Repository Manager

The integration between Nexus Repository Manager and IQ Server provides access to detailed Repository Results identifying components that represent potential risk to the components and applications your teams are developing it.

For more information, please see our IQ Server and Repository Management help topic.

Bamboo

The Bamboo plugin lets you evaluate applications with the IQ Server and provides a summary of the results on the Job Summary page. In addition to counts for each of these categories, a status for the success of the evaluation is provided, as well as a link to the summary report located on your IQ Server.

For more information, see our Nexus IQ for Bamboo help topic.

GitLab

The GitLab Nexus IQ docker image provides the ability to run Nexus policy evaluation against build artifacts in GitLab and produces a summary report with policy violation counts and a link to a detailed report on the IQ server.

For more information, see our Nexus IQ for GitLab CI help topic.

IDE/Dev Environments

Nexus IQ Server integrations for Integrated Development Environments (IDEs) provide development teams with direct access to Sonatype’s comprehensive component intelligence. Developers can quickly vet components used in an application against their organization’s open source policies, greatly reducing time wasted with complicated and exhaustive research.

Eclipse

The Sonatype Eclipse plugin lets you analyze components used by your software development project and take action to resolve any issues you discover. For more information, please see our Eclipse help documentation.

Visual Studio

IQ for Visual Studio provides component analysis for the Community, Professional and Enterprise versions of Visual Studio. The plugin can be installed from within Visual Studio using the Extensions manager or via the Microsoft Visual Studio Marketplace. For more information, see our Visual Studio help topic.

IDEA

IQ for IDEA provides component analysis for both the Community and Ultimate edition of IntelliJ IDEA. For more information, go to our IDEA help topic.

DepShield

Sonatype DepShield is a GitHub App used by developers to identify and remediate vulnerabilities in their open source dependencies. DepShield Continuously monitors projects and auto-creates issues for security vulnerabilities, and is currently available for Apache Maven and Node.js npm projects. For more information, please see DepShield help on GitHub.

Chrome Extension

Another option is the Chrome Extension for Sonatype Nexus IQ. This lets you inspect a package before you download it. The plugin requires a valid Sonatype Nexus Lifecycle instance. Once installed, it can scan packages from multiple repositories.

NOTE: The Chrome Extension is a Nexus Community plugin, and is not officially supported.

Running Scans

For each language / ecosystem, we’ll provide an explanation of where a scan should take place to provide you with the most value and an accurate bill of materials. This will help you analyze your applications as they’re being built, and deliver better value to developers.

Java (Apex, Scala, Kotlin, any JVM language)

Application types Where to scan Tips
jar
war
ear
tar.gz
zip
tgz
bz2
Maven plugin
CLI
CI plugins
Use the post-build artifact as the target (what you are planning to deploy). You can run the normal build and add a post build step before you deploy to the repository.

A scan will pick up all dependencies packaged into that file (unless it’s an uber jar).
Uber (shaded) jar Maven plugin
CLI
To scan your Uber/Shaded jars, the easiest integration is at build time.

If you are building with Maven, use the Maven plugin, which is downloaded when invoked. The index goal produced with the Maven plugin can generate a dependency index of direct and transitive dependencies referenced in the project POM. The generated module.xml file contains information that’s incorporated into the IQ Server evaluation. The final scan in the CI tools uses components in the .m2 cache for fingerprinting to enforce accuracy of what is ultimately in your project.

If you do not wish to integrate an evaluation at build time, the Maven copy dependencies command can be used: Mvn dependency:copy-dependencies. This will copy the dependencies of your project to a separate directory. This directory can then be scanned via the CLI.

NuGet (.net, C#)

Application types Where to scan Tips
nupkg
dll
zip
tar.gz
Visual Studio Plugin
CLI
CI plugins
Scanning .nupkg files provides the best results.

Scanning the deployment artifact (zip file usually) is useful at the release stage to catch anything not directly brought in through nuget.

JavaScript

Application types Where to scan Tips
js
zip
tar.gz
tgz
CLI
CI plugins
We are identifying components from the npm registry.

If using webpack to build, use copy-modules-webpack-plugin. This lets you isolate javascript components in the final application and scan them directly.

Python

Application types Where to scan Tips
requirements.txt CLI
CI plugins
Only requirements using the “==” operator and version without wildcards will be considered.One requirement can be matched to multiple python packages.

Use pip freeze to create the requirements file. Then, add optional environmental markers and run a scan.

For more information, please see our Python help documentation and checkout this video on scanning Python in the CLI:

 

Go

Application types Where to scan Tips
go.sum CLI
Jenkins
Bamboo
Maven
Go coordinate-based matching provides the ability to scan and evaluate Go module dependencies found in the go.sum file.

We recommended following the proper maintenance of the go.sum file. This ensures better matching results when scanning a go module project.

To run a scan in the CLI, scan the directory or subdirectories containing go.sum files.

Android

Application types Where to scan Tips
jar
war
ear
tar.gz
zip
tgz
bz2
Maven plugin
CLI
We do not support scanning a .apk directly due to the minification performed via the dalvik byte code process. For this reason, scanning prior to the assembling of the .apk is required.

Scanning APK directly is not supported: Compilers convert source code into DEX (Dalvik Executable) file.

Maven Build: Use the maven plugin.

Gradle Build: Maven copy dependencies, then scan folder using the CLI.

Docker (Kubernetes / Helm)

Application types Where to scan Tips
tar CLI
CI plugins
Nexus Lifecycle scans the application layer of your containers, and provides precise component intelligence for Java, JavaScript, Nuget, and Python.

To scan a Docker image, you need to first save it as a tar file, and then run a scan in the CLI or as a build step using a CI plugin.

Node / node.js

Application types Where to scan Tips
js
zip
tar.gz
tgz
CLI
CI plugins
Scan the node_modules folder generated from npm install.

Delete the node_modules and run npm install --production to exclude dev dependencies in the package.json.

NOTE: only a-name matching happens in this case. For best results, do an npm pack and scan the tarball to do package matching.

PHP

Application types Where to scan Tips
composer.lock XC scan in IQ CLI Use the -xc, --expanded-coverage parameter to run an XC scan.

NOTE: XC results come from unverified public sources, and do not include any Sonatype enriched information.

C / C++

Application types Where to scan Tips
CMake files XC scan in IQ CLI Use the -xc, --expanded-coverage parameter to run an XC scan.

NOTE: XC results come from unverified public sources, and do not include any Sonatype enriched information.

Swift / Objective C / iOS

Application types Where to scan Tips
Package.Swift (for Swift)
.podspec (for CocoaPods)
XC scan in IQ CLI Use the -xc, --expanded-coverage parameter to run an XC scan.

NOTE: XC results come from unverified public sources, and do not include any Sonatype enriched information.

Ruby

Application types Where to scan Tips
gemfile.lock XC scan in IQ CLI Firewall and XC only, not available in Lifecycle.

Use the -xc, --expanded-coverage parameter to run an XC scan.

NOTE: XC results come from unverified public sources, and do not include any Sonatype enriched information.

YUM

Application types Where to scan Tips
.rpm
tgz
XC scan in IQ CLI Firewall and XC only, not available in Lifecycle.

Use the -xc, --expanded-coverage parameter to run an XC scan.

NOTE: XC results come from unverified public sources, and do not include any Sonatype enriched information.

Talk to Us

Sign up to receive email alerts each time Sonatype’s Customer Education team publishes new content.

Have more questions or comments? Learn more at my.sonatype.com, and join us in the Sonatype Community.