circle-info
Release Announcement: NSO 6.6 is here!
Check out the release highlights to see what's newarrow-up-right
search
NSO DevCenterTry NSO

README v1.1.2 2025-12-17

hashtag
Table of contents

1. General
   1.1 Extract the NED package
   1.2 Install the NED package
       1.2.1 Local install
       1.2.2 System install
   1.3 Configure the NED in NSO
2. Optional debug and trace setup
3. Dependencies
4. Sample device configuration
5. Built in RPC actions
   5.1. rpc add-filter-path
   5.2. rpc clean-package
   5.3. rpc clear-cached-capabilities
   5.4. rpc clear-filter-paths
   5.5. rpc compare-config
   5.6. rpc compile-modules
   5.7. rpc export-package
   5.8. rpc get-modules
   5.9. rpc import-filter-paths
   5.10. rpc list-filter-paths
   5.11. rpc list-module-sets
   5.12. rpc list-modules
   5.13. rpc list-profiles
   5.14. rpc patch-modules
   5.15. rpc rebuild-package
   5.16. rpc remove-filter-path
   5.17. rpc show-default-local-dir
   5.18. rpc show-loaded-schema
   5.19. rpc verify-get-config
   5.20. rpc xpath-trace-analyzer
6. Built in live-status show
7. Limitations
8. How to report NED issues and feature requests
9. How to rebuild a NED
10. Configure the NED to use ssh multi factor authentication
11. Tips and Tricks for Resolving Common Issues with IOS XR Devices
    11.1 Interface Name Changes When a New Line Card Is Inserted
    11.2 Addressing Disappearing Interface Instances
12. Run arbitrary commands on device

hashtag
1. General

This document describes the cisco-iosxr_nc NED.

IMPORTANT: This NED is delivered without any of the device YANG models bundled to the NED package.

It is required to download the YANG files separately and rebuild the NED package before the NED is fully operational. See the README-rebuild.md for further information.

In summary, the below steps are needed to have a fully functioning NED:

Additional README files bundled with this NED package

Common NED Features

Verified target systems

Verified YANG model bundles

hashtag
1.1 Extract the NED package

It is assumed the NED package ncs-<NSO version>-cisco-iosxr_nc-<NED version>.signed.bin has already been downloaded from software.cisco.com.

In this instruction the following example settings will be used:

  • NSO version: 6.0

  • NED version: 1.0.1

  • NED package downloaded to: /tmp/ned-package-store

  1. Extract the NED package and verify its signature:

  2. In case the signature can not be verified (for instance if no internet connection), do as below instead:

  3. The result of the extraction shall be a tar.gz file with the same name as the .bin file:

hashtag
1.2 Install the NED package

There are two alternative ways to install this NED package. Which one to use depends on how NSO itself is setup.

In the instructions below the following example settings will be used:

  • NSO version: 6.0

  • NED version: 1.0.1

  • NED download directory: /tmp/ned-package-store

  • NSO run time directory: ~/nso-lab-rundir

A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory:

IMPORTANT:

This NED is delivered as an “empty” package, i.e without any device YANG models bundled. It must be rebuilt with the device YANG models to become operational.

The procedure to rebuild the empty NED (described in the README-rebuild.md) shall typically be done in a lab environment. For this step a “local install” of the NED shall be used. It is not suitable to use “system install” here since it is intended for production systems only.

Once this NED has been rebuilt with the device YANG and exported to one or many separate tar.gz customized NED packages, a “system installation” can be used on them.

hashtag
1.2.1 Local install

This section describes how to install a NED package on a locally installed NSO (see "NSO Local Install" in the NSO Installation guide).

It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.

  1. Untar the tar.gz file. This creates a new sub-directory named: cisco-iosxr_nc-<NED major digit>.<NED minor digit>:

  2. Install the NED into NSO, using the ncs-setup tool:

  3. Open a NSO CLI session and load the new NED package like below:

Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like below instead:

Set the environment variable NED_ROOT_DIR to point at the NSO NED package:

hashtag
1.2.2 System install

This section describes how to install a NED package on a system installed NSO (see "NSO System Install" in the NSO Installation Guide).

It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.

  1. Do a NSO backup before installing the new NED package:

  2. Start a NSO CLI session and fetch the NED package:

  3. Install the NED package (add the argument replace-existing if a previous version has been loaded):

  4. Load the NED package

hashtag
1.3 Configure the NED in NSO

This section describes the steps for configuring a device instance using the newly installed NED package.

  • Start a NSO CLI session:

  • Enter configuration mode:

  • Configure a new authentication group (my-group) to be used for this device:

  • Configure a new device instance (example: dev-1):

    IMPORTANT:

    The device-type shall always be set to generic when configuring a device instance to use a 3PY NED. A common mistake is configuring it as netconf, which will cause NSO to use its internal netconf client instead.

  • Finally commit the configuration

  • Verify configuration, using a sync-from.

If the sync-from was not successful, check the NED configuration again.

hashtag
2. Optional debug and trace setup

It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting)

This can be achieved by configuring NSO to generate a trace file for the NED. A trace file contains information about all interactions with the device. Messages sent and received as well as debug printouts, depending on the log level configured.

NSO creates one separate trace file for each device instance with tracing enabled. Stored in the following location:

$NSO_RUNDIR/logs/ned-cisco-iosxr_nc-gen-1.0-<device name>.trace

Do as follows to enable tracing in one specific device instance in NSO:

  1. Start a NSO CLI session:

  2. Enter configuration mode:

  3. Enable trace raw:

    Alternatively, tracing can be enabled globally affecting all configured device instances:

  4. Configure the log level for printouts to the trace file:

    Alternatively the log level can be set globally affecting all configured device instances using this NED package.

The log level 'info' is used by default and the 'debug' level is the most verbose.

IMPORTANT: Tracing shall be used with caution. This feature does increase the number of IPC messages sent between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should normally be disabled in production systems.

An alternative method for generating printouts from the NED is to enable the Java logging mechanism. This makes the NED print log messages to common NSO Java log file.

$NSO_RUNDIR/logs/ncs-java-vm.log

Do as follows to enable Java logging in the NED

  1. Start a NSO CLI session:

  2. Enter configuration mode:

  3. Enable Java logging with level all from the NED package:

  4. Configure the NED to log to the Java logger

    Alternatively Java logging can be enabled globally affecting all configured device instances using this NED package.

IMPORTANT: Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not affected. However, all log printouts from all log enabled devices are saved in one single file. This means that the usability is limited. Typically single device use cases etc.

SSHJ DEBUG LOGGING For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. This will make SSHJ print additional log entries in $NSO_RUNDIR/logs/ncs-java-vm.log:

hashtag
3. Dependencies

This NED has the following host environment dependencies:

  • Java 1.8 (NSO version < 6.2)

  • Java 17 (NSO version >= 6.2)

  • Gnu Sed

Dependencies for NED recompile:

  • Apache Ant

  • Bash

  • Gnu Sort

  • Gnu awk

  • Grep

  • Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob)

hashtag
4. Sample device configuration

NONE

hashtag
5. Built in RPC actions

hashtag
5.1. rpc add-filter-path

hashtag
5.2. rpc clean-package

hashtag
5.3. rpc clear-cached-capabilities

hashtag
5.4. rpc clear-filter-paths

hashtag
5.5. rpc compare-config

hashtag
5.6. rpc compile-modules

hashtag
5.7. rpc export-package

hashtag
5.8. rpc get-modules

hashtag
5.9. rpc import-filter-paths

hashtag
5.10. rpc list-filter-paths

hashtag
5.11. rpc list-module-sets

hashtag
5.12. rpc list-modules

hashtag
5.13. rpc list-profiles

hashtag
5.14. rpc patch-modules

hashtag
5.15. rpc rebuild-package

hashtag
5.16. rpc remove-filter-path

hashtag
5.17. rpc show-default-local-dir

hashtag
5.18. rpc show-loaded-schema

hashtag
5.19. rpc verify-get-config

hashtag
5.20. rpc xpath-trace-analyzer

hashtag
6. Built in live-status show

NONE

hashtag
7. Limitations

hashtag
8. How to report NED issues and feature requests

Issues like bugs and errors shall always be reported to the Cisco NSO NED team through the Cisco Support channel:

The following information is required for the Cisco NSO NED team to be able to investigate an issue:

Do as follows to gather the necessary information needed for your device, here named 'dev-1':

  1. Enable full debug logging in the NED

  2. Configure the NSO to generate a raw trace file from the NED

  3. If the NED already had trace enabled, clear it in order to submit only relevant information

    Do as follows for NSO 6.4 or newer:

    Do as follows for older NSO versions:

  4. Run a compare-config to populate the trace with initial device config

  5. Reproduce the found issue using ncs_cli or your NSO service. Write down each necessary step in a reproduction report.

  6. Gather the reproduction report and a copy of the raw trace file containing data recorded when the issue happened.

  7. Contact the Cisco support and request to open a case. Provide the gathered files together with access details for a device that can be used by the Cisco NSO NED when investigating the issue.

Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when applicable. Such requests shall also go through the Cisco support channel.

The following information is required for feature requests and extensions:

  1. A detailed use case description, with details like:

    • Data of interest

    • The kind of operations to be used on the data. Like: 'read', 'create', 'update', 'delete' and the order of the operation

    • Device APIs involved in the operations (For example: REST URLs and payloads)

    • Device documentation describing the operations involved

  2. Run sync-from # devices device dev-1 sync-from (if relevant)

  3. Attach the raw trace to the ticket (if relevant)

  4. Access to a device that can be used by the Cisco NSO NED team for testing and verification of the new feature. This usually means that both read and write permissions are required. Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access through VPNs, jump servers etc.

hashtag
9. How to rebuild a NED

Check the README-rebuild.md file, chapter 1.3, for more information.

hashtag
10. Configure the NED to use ssh multi factor authentication

This NED supports multi factor authentication (MFA) using the ssh authentication method 'keyboard-interactive'.

Some additional steps are required to enable the MFA support:

  1. Verify that your NSO version supports MFA. This is configurable as additional settings in the authentication group used by the device instance.

    Enter a NSO CLI and enter the following and do tab completion:

    If 'mfa' is displayed in the output like above, NSO has MFA support enabled. In case MFA is not supported it is necessary to upgrade NSO before proceeding.

  2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the ssh server and expects to get a proper response in return.

    The executable can be a simple shell script or a program implemented in any programming language.

    The required behaviour is like this:

    • read one line from stdin The line passed from the NED will be a semi colon separated string containing the following info:

      The elements for device name, user, password and opaque corresponds to what has been configured in NSO. The ssh server name, instruction and prompt are given by the ssh server during the authentication step.

      Each individual element in the semi colon separated list is Base64 encoded.

    • Extract the challenge based on the contents above.

    • Print a response matching the challenge to stdout and exit with code 0

    • In case a matching response can not be given do exit with code 2

    Below is a simple example of an MFA authenticator implemented in Python3:

  3. Configure the authentication group used by the device instance to enable MFA. There are two configurables available:

    • executable The path to the external multi factor authentication executable (mandatory).

    • opaque Opaque data that will passed as a cookie element to the executable (optional).

  4. Try connecting to the device.

hashtag
10.1 Trouble shooting

In case of connection problems the following steps can help for debugging:

Enable the NED trace in debug level:

Try connect again

Inspect the generated trace file.

Verify that the ssh client is using the external authenticator executable:

Verify that the executable is called with the challenges presented by the ssh server:

Check for any errors reported by the NED when calling the executable

hashtag
11. Tips and Tricks for Resolving Common Issues with IOS XR Devices

This chapter serves as a reference archive, providing suggestions for troubleshooting common issues that may arise when using IOS XR devices with NSO. New information will be added as features and workarounds are introduced to this NED.

hashtag
11.1 Interface Name Changes When a New Line Card Is Inserted

When interface hardware is changed on an XR device, the interface names typically change as well. For example, if SFPs are upgraded from gigabit-capable to hundred-gigabit-capable, the interface names in the XR configuration will change from: GigabitEthernet0/0/0/0 to HundredGigEthernet0/0/0/0

This is standard behavior in IOS XR, but it can cause significant issues when working with NSO.

NSO maintains its own CDB database with the running network configuration and is unaware of the interface name changes.

As a result, if a compare-config operation is performed in NSO after the SFPs have been changed, it will appear as though the old interfaces have disappeared and been replaced by a completely new set of interfaces. This often disrupts services running in NSO, as they may have dependencies on specific interface names.

This NED provides a workaround for this issue, but it should be noted that this is a best-effort approach -there is no foolproof solution.

The workaround involves using inbound and outbound replacer transforms based on regular expressions.

  • For inbound configuration, replacers are applied after the NED receives the configuration in XML format from the device, but before it is parsed and relayed to NSO. This affects NSO operations such as sync-from and compare-config.

  • For outbound configuration, replacers are applied as the last step before the XML configuration is sent to the device.

Example: Initially, interface names are prefixed with GigabitEthernet. After switching SFPs, the names change to HundredGigEthernet.

The interface list in the device's XML configuration initially appears as follows:

The approach is to have the NED transform interface names to a generic prefix, such as MyEthernet. To achieve this, the real interface name prefix is converted to My inbound, and for outbound operations, the prefix My is replaced with the original interface prefix.

This is configured through the NED settings:

  • Inbound transform: devices device dev-1 ned-settings cisco-iosxr_nc transaction config-replace-patterns inbound (<interface-name>)(Gigabit)([^<]+) replace $1My$3

  • Outbound transform: devices device dev-1 ned-settings cisco-iosxr_nc transaction config-replace-patterns outbound (<interface-name>)(My)([^<]+) replace $1Gigabit$3

To verify the inbound transform, perform a sync-from in NSO followed by show running-config. The interfaces should now be prefixed with My, as shown below:

To verify the outbound transform, edit an interface configuration and perform a commit dry-run outformat native. The output should now include the real interface prefixes:

Now, assume the SFPs are replaced on the XR device. The new interface list from the device will look like this:

At this point, you need to update the regular expressions in both the inbound and outbound transformers:

After making these changes, performing a compare-config should show no differences.

This approach allows you to keep the CDB database unchanged, ensuring that services continue to function without interruption.

hashtag
11.2 Addressing Disappearing Interface Instances

The IOS XR platform exhibits a unique behavior with interfaces that can cause synchronization issues when using NSO.

Specifically, if you configure an interface with the no shutdown command but do not add any additional sub-configuration, the interface may spontaneously disappear from the global device configuration. This behavior leads to immediate out-of-sync issues in NSO.

The following example demonstrates this issue:

To address this out-of-sync problem, the NED includes a built-in config caching feature. When an interface state is changed and new configuration is deployed (outbound), the NED caches the interface configuration. During subsequent read operations, the cached configuration is injected back, ensuring that NSO remains in sync-even if the device no longer displays the interface in its running configuration.

You can enable this feature using the transaction/enable-config-caching NED setting.

The config caching feature currently supports interface configurations accessed via the native classic, native UM, or openconfig YANG models.

hashtag
12. Run arbitrary commands on device

Some commands that are available to a user logged in to an interactive CLI session on the device might not be available through NETCONF. For situations like this the NED provides the feature to run arbitrary commands through an interactive SSH login to the device. This SSH session is handled internally in the NED and connected in NSO to a live-status action called 'exec any'.

There are some ned-settings to control the behaviour of this feature, see the section 'ned-settings cisco-iosxr_nc live-status cli' in README-ned-settings.md for details on this.

Specifically, to be able to handle the interactive session towards the device, the NED needs to know the format for the device prompt. It also assumes that pagination is turned off before reading output from command sent (i.e. that the device doesn't pause terminal output, waiting for interactive response). The ned-settings 'prompt-pattern' and 'no-pagniation-cmd' are used to control this. These might have proper default values, please check that this matches your device though before trying this feature, since if not configured correctly the NED will hang until timed out.

As an example, to run the command 'show running-config' on the device, and get the resulting output as a string from the ncs_cli, run the following:

Note that when using ncs_cli, the command-line given might need to be quoted if it contains characters that are interpreted by the ncs_cli itself.

PreviousREADME-rebuildNextcisco-iosxr_netconf

Last updated

Was this helpful?