The first phase of a Nexus Lifecycle deployment includes the discovery of total organizational risk by building an inventory of your open source components used in all your applications. This most often starts with scanning applications during an automated CI build but can be done manually by scanning applications through the UI or with the Command Line Scanner, CLI.
These applications will need to be added to the IQ Server in order to have the correct policy applied and enable the correct access controls to the final report. The strategy for which method of onboarding you choose is a factor of the overall number of applications to be onboarded along with any limitations to report access required by the organization. Here are some common examples.
The IQ server allows for applications to be created from the Organizations and Applications section of the UI. This is typically recommended when working with a limited (<50) number of applications. It is possible to do more if there is not a centralized application management system in place, however it is not an effective long term strategy if continued growth is expected. This is more common for legacy applications which are not regularly built.
- Take time deciding the organization structure in the policy configuration UI but keep it simple. Outside of managing access control and for reporting purposes, there is little reason to make the structure of organizations and applications very complex. The only additional consideration would be in creating policy against a specific organization. However, this is often better handled using Application Categories.
- Default on fewer organizational groups. Applications can always be moved later if needed.
- If there is not centralized application management, plan for Application IDs and application names to be meaningful and unique across the organization.
- If the complete list of applications is available via a spreadsheet/document, consider using an onboarding script to add them all at once.
- Misconfiguration of builds by re-using a duplicate Application ID is common. Reusing the same application ID will confuse success metrics and notifications as different code bases are scanning for the same application.
- Manually adding applications can be a slow process and will end up delaying onboarding. This may lead to errors when ids are not matching IQ verses Build jobs.
- Help - Application Management
Automatic Application Creation
The IQ server can be configured to allow applications to automatically be added when using a new Application ID during a scan. These applications are added to a configured default Organization with the Application name being set to the Application ID used. This method is recommended for any number of applications although anything over 500 applications will most likely need additional automation to assist with the configuration.
- Someone will need to be assigned to regularly review the applications that are onboarded to correct accidental scans and assign Application Categories where needed.
- Applications can be renamed to more meaningful names and moved to respective Organizations if more granular access control if required. This can also be automated with the API.
- If limiting access to Application reports is required, only the required persons should have View Access to reports from the default Organization. This may leave the end-users without access to their reports until the Application has been moved to an Organization with which they have access to.
- This method makes it very easy to create new applications unintentionally and as the source cannot be tracked back to the scanner it can be difficult to find where they are coming from.
- Applications creation can only be done in a single default organization. Applications will need to be moved to a different organization via the API or through the UI.
- The first scan of an application may not have the correct policy applied if policy should be different from the default organization. It is also true for license policies which are typically only appropriate for applications labeled as ‘Distributed’ via Application Categories.
- The application will have to be re-evalutated once moved to the correct organization and the correct Application Categories are added. This may give incorrect results later in Success Metrics.
- Unless a global common policy is used then Application Grandfathering is not as effective and may need to be manually reset after the correction.
- Notifications will be triggered for the first scan and then only for new violations afterwards. This may make automatic notifications difficult unless they are planned in advance.
Onboarding through Source Control Managers
Starting in IQ Server release 109, applications can be onboarded by directly connecting to them from your source control (SCM). Easy SCM Onboarding will allow you to select which applications to onboard through a point and click menu in the UI. After selected, IQ can automatically perform a direct manifest analysis on the applications without requiring a build to be run.
- When scanning applications in SCM, the scanner will often look for dependency lock files and as well as any binaries stored in the code (this is uncommon).
- While the scanner does not need to be told which languages to look for it will look for common patterns which are language specific. Review the analysis documentation for details on what the requirements are for manifest scanning in a given language.
- Currently only a few applications scan be selected to onboard and scan at a time.
- While very useful for early feedback, manifest scanning does not provide a complete risk analysis of your built application. A complete scan will still need to be added during the build process.
Onboarding Scripts (REST API)
The IQ server includes an extensive REST API for automating the tasks needed to onboard in the IQ server. This allows for direct integration into application management systems to onboard and configure applications either in real-time during a build or through a configuration page within those external systems. This method for onboarding will take development time to customize to any requirements as well as any external look-ups for elements such as Organizations, Application Categories, and possibly identity provider users and groups. This setup is recommended for self-service onboarding requirements for a large number of applications.
- The most successful implementations integrate to match internal systems/processes. Use internal ids, application names, and existing organizational groups where possible.
- Application Categories are best when they match internal metadata that is already in place. Do not get carried away adding everything as Application Categories are mostly used to differentiate policy through prioritization and reporting.
- A pilot project is recommended to refine the onboarding details before a large influx of applications are added.
- We recommend taking a backup of this IQ server before running complicated scripts that add a large number of applications, in case any goes wrong.
- By default, the IQ Server license will limit the number of applications that can be onboarded to 5000 applications. This is inplace to keep automatic scripts from overloading the system. Discuss your plans with the account team if you are planning to onboard more than 5K applications so that your license can be updated.
- Unless this is prioritized and properly scoped, onboarding this way could delay or derail early adoption. Starting with the other 2 methods may be desirable until a tested script can be fully deployed through the CI.
- Major deployments are very difficult to rollback or correct after the fact, especially if you do not have a recent backup. Ensure that any script is fully tested in a development environment before deploying to production. Even after deployment, monitor systems for any performance irregularities to ensure stability of production.
Config-as-Code is the DevOps principle of treating configuration resources like versioned artifacts, meaning it’s checked into source control, assigned version numbers, and associated with versioned builds.
The Config-as-Code tool, available in a Github repository, is a Python script that supports this principle. It can “scrape” a target IQ Server’s existing configuration and save it as a JSON. JSON files are human-readable, so stakeholders can review configuration details without needing access to a IQ Server that’s currently in-use. More importantly, the scraped JSON can be applied to other IQ Server instances, ensuring that new IQ Servers match previous versions exactly.
The tool also contains configuration templates, also in the form of .json files, that align to current Sonatype best practices. These templates, saved at iq-config-as-code/onboarding/templates, can help new users rapidly prepare their first IQ Server.
Please note that this tool is not officially supported by Sonatype.
- Be sure to version your scraped .json files appropriately.
- Avoid editing the template .json files unless you’re an advanced user with some familiarity editing and running Python scripts. Debugging is limited, so it will be easy to make mistakes.
- Before applying the templates or a scraped configuration to an active production environment, apply them to a test environment and run a test scan. Don’t move forward until you’re confident in the results.
applycommand completely overwrites your IQ Server’s configuration with the data in the target .json file. Be cautious when applying new configurations, and scrape your existing configuration as a backup.
- This tool makes it easy for misonconfigurations to propogate. Version your scraped configurations so you can track changes and recover from misconfigurations.
- Onboarding Organizations - python script
- Adding Applications - python script
- IQ Server configuration as code
Talk to Us
And visit my.sonatype.com for all things Sonatype.