This document describes the rad-vx NED.
This document describes the NED for Radware ETX-203 devices.
The NED connects to the device CLI using either SSH or Telnet. Configuration is done by sending native CLI commands to the device through the communication channel.
Additional README files bundled with this NED package
Common NED Features
Verified target systems
It is assumed the NED package ncs-<NSO version>-rad-vx-<NED version>.signed.bin has already
been downloaded from software.cisco.com.
In this instruction the following example settings will be used:
NSO version: 6.0
NED version: 1.0.1
NED package downloaded to: /tmp/ned-package-store
Extract the NED package and verify its signature:
In case the signature can not be verified (for instance if no internet connection), do as below instead:
The result of the extraction shall be a tar.gz file with the same name as the .bin file:
There are two alternative ways to install this NED package. Which one to use depends on how NSO itself is setup.
In the instructions below the following example settings will be used:
NSO version: 6.0
NED version: 1.0.1
NED download directory: /tmp/ned-package-store
NSO run time directory: ~/nso-lab-rundir
A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory:
This section describes how to install a NED package on a locally installed NSO (see "NSO Local Install" in the NSO Installation guide).
It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.
Untar the tar.gz file. This creates a new sub-directory named:rad-vx-<NED major digit>.<NED minor digit>:
Install the NED into NSO, using the ncs-setup tool:
Open a NSO CLI session and load the new NED package like below:
Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like below instead:
Set the environment variable NED_ROOT_DIR to point at the NSO NED package:
This section describes how to install a NED package on a system installed NSO (see "NSO System Install" in the NSO Installation Guide).
It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.
Do a NSO backup before installing the new NED package:
Start a NSO CLI session and fetch the NED package:
Install the NED package (add the argument replace-existing if a previous version has been loaded):
Load the NED package
This section describes the steps for configuring a device instance using the newly installed NED package.
Start a NSO CLI session:
Enter configuration mode:
Configure a new authentication group (my-group) to be used for this device:
Configure a new device instance (example: dev-1):
If the sync-from was not successful, check the NED configuration again.
It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting)
This can be achieved by configuring NSO to generate a trace file for the NED. A trace file contains information about all interactions with the device. Messages sent and received as well as debug printouts, depending on the log level configured.
NSO creates one separate trace file for each device instance with tracing enabled. Stored in the following location:
$NSO_RUNDIR/logs/ned-rad-vx-cli-1.0-<device name>.trace
Do as follows to enable tracing in one specific device instance in NSO:
Start a NSO CLI session:
Enter configuration mode:
Enable trace raw:
Alternatively, tracing can be enabled globally affecting all configured device instances:
Configure the log level for printouts to the trace file:
The log level 'info' is used by default and the 'debug' level is the most verbose.
IMPORTANT: Tracing shall be used with caution. This feature does increase the number of IPC messages sent between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should normally be disabled in production systems.
An alternative method for generating printouts from the NED is to enable the Java logging mechanism. This makes the NED print log messages to common NSO Java log file.
$NSO_RUNDIR/logs/ncs-java-vm.log
Do as follows to enable Java logging in the NED
Start a NSO CLI session:
Enter configuration mode:
Enable Java logging with level all from the NED package:
Configure the NED to log to the Java logger
Alternatively Java logging can be enabled globally affecting all configured device instances using this NED package.
IMPORTANT: Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not affected. However, all log printouts from all log enabled devices are saved in one single file. This means that the usability is limited. Typically single device use cases etc.
This NED has the following host environment dependencies:
Java 1.8 (NSO version < 6.2)
Java 17 (NSO version >= 6.2)
Gnu Sed
Dependencies for NED recompile:
Apache Ant
Bash
Gnu Sort
Gnu awk
NONE
summary-inventory:
Displays the output of show configure system summary-inventory
inventory:
This command has a mandatory parameter node.
Displays the output of show configure system inventory {node} status.
run: Displays the output of the command + parameters provided by the user. The arguments are forwarded as they are.
info:
Displays the output of info
Deactivating persistent data save to the device:
During development the NED user might want to disable persisten writes to the device. In this mode the NED will load push the configuration and the device will behave as expected for the current session. The original device configuration is restored by a reboot. This feature is disabled by default.
To enable persistent writes (default mode):devices device rad-vx ned-settings development_settings enable_device_save true
To disable persisten writes:devices device rad-vx ned-settings development_settings enable_device_save false
In certain circumstances (like ETX-205 device) some "port * ethernet * max-capability" instances have dual defaults:
For such a case the max-capability rj45 / sfp entries must be entered both when configuring from leaf style of config to list
Ex:
config on device :max-capability 100-full-duplex"
to modify, we must enter BOTH entries:max-capability 1000-full-duplex rj45 max-capability 1000-x-full-duplex sfp
If we would like to set both capabilities to same value, as to set the same capability value on both config parts, or if one
entry is set to a value and we want to set both capabilities to same value, we have to use the leaf/single capability format
Ex:
config on device:max-capability 1000-full-duplex rj45 max-capability 1000-x-full-duplex sfp to modify both entries to 1000-full-duplex, user muset enter:max-capability 1000-full-duplex (single format without sfp or rj45 teermination)
If the user wants to set both capabilities to same value, with regard that none of the entries has that value, user has to follow same scenario as above, entering the singleu format command line (max-capability 10-full-duplex or max-capability 100-full-duplex etc)
The "admin software install" live-status command device behaviour:
The following command can be used to trigger a device firwmare update:devices device rad-vx live-status exec admin_software_install sw-pack-4 no-restore-point
The arguments following "admin_software_install" are the same as the ones accepted by the device. Please note that on some devices a reboot will be performed before the expected command reply is send out. In that case the ned will time-out while waiting for the reply from the device, and you will see a message allong the lines of:
In that case the user should check that the firmware update was performed properly.
Note regarding 'configure qos queue-block-profile ' and 'configure qos queue-block-profile Scheduling':
The NED user must take care when he's controlling device degerated elements, and especially the 'queue-block-profile Scheduling*' list and it's containers. The device will remove by itself those element and, because NSO / the YANG model can't fully cover all cases, the system will get out-of-sync. For example, for the following configuration:
When the 'queue 0' element needs to be deleted via the device CLI command 'configure qos queue-block-profile Scheduling2 queue 0 scheduling wfq 100', the entire 'queue-block-profile Scheduling2' will be removed. The NED is able to follow this behavior up to a point:
At this point the proper delete sequence is generated, NSO is aware that 'queue 0' will be removed and the proper device CLI sequence is generated. There will be a compare diff after the commit because the removal of 'queue-block-profile Scheduling2' can't be modeled:
The service layer should handle these, or, if possible, completley ignore device auto-generated config.
Issues like bugs and errors shall always be reported to the Cisco NSO NED team through the Cisco Support channel:
The following information is required for the Cisco NSO NED team to be able to investigate an issue:
Do as follows to gather the necessary information needed for your device, here named 'dev-1':
Enable full debug logging in the NED
Configure the NSO to generate a raw trace file from the NED
If the NED already had trace enabled, clear it in order to submit only relevant information
Do as follows for NSO 6.4 or newer:
Do as follows for older NSO versions:
Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when applicable. Such requests shall also go through the Cisco support channel.
The following information is required for feature requests and extensions:
Set the config on the real device including all existing dependent config and run sync-from to show it in the trace.
Run sync-from # devices device dev-1 sync-from
Attach the raw trace to the ticket
List the config you want implemented in the same syntax as shown on the device
To rebuild the NED do as follows:
When the NED has been successfully rebuilt, it is necessary to reload the package into NSO.
This NED supports multi factor authentication (MFA) using the ssh authentication method 'keyboard-interactive'.
Some additional steps are required to enable the MFA support:
Verify that your NSO version supports MFA. This is configurable as additional settings in the authentication group used by the device instance.
Enter a NSO CLI and enter the following and do tab completion:
If 'mfa' is displayed in the output like above, NSO has MFA support enabled. In case MFA is not supported it is necessary to upgrade NSO before proceeding.
Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the ssh server and expects to get a proper response in return.
The executable can be a simple shell script or a program implemented in any programming language.
The required behaviour is like this:
In case of connection problems the following steps can help for debugging:
Enable the NED trace in debug level:
Try connect again
Inspect the generated trace file.
Verify that the ssh client is using the external authenticator executable:
Verify that the executable is called with the challenges presented by the ssh server:
Check for any errors reported by the NED when calling the executable
1. General
1.1 Extract the NED package
1.2 Install the NED package
1.2.1 Local install
1.2.2 System install
1.3 Configure the NED in NSO
2. Optional debug and trace setup
3. Dependencies
4. Sample device configuration
5. Built in live-status actions
6. Built in live-status show
7. Limitations
8. How to report NED issues and feature requests
9. How to rebuild a NED
10. Configure the NED to use ssh multi factor authenticationIf configured protocol is ssh, do fetch the host keys now:
Finally commit the configuration
Verify configuration, using a sync-from.
Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob)
copy:
The arguments are forwarded as they are.
Displays the output of copy {cmds}.
The command will automatically send yes if the devices asks for it.
admin_software_install:
The arguments are forwarded as they are.
Displays the output of admin software install {cmds}.
It will automatically send yes if the devices asks for it.
Reproduce the found issue using ncs_cli or your NSO service. Write down each necessary step in a reproduction report.
In addition to this, it helps if you can show how it should work by manually logging into the device using SSH/TELNET and type the relevant commands showing a successful operation.
Gather the reproduction report and a copy of the raw trace file containing data recorded when the issue happened.
Contact the Cisco support and request to open a case. Provide the gathered files together with access details for a device that can be used by the Cisco NSO NED when investigating the issue.
SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification of the new feature. This usually means that both read and write permissions are required. Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET.
read one line from stdin The line passed from the NED will be a semi colon separated string containing the following info:
The elements for device name, user, password and opaque corresponds to what has been configured in NSO. The ssh server name, instruction and prompt are given by the ssh server during the authentication step.
Each individual element in the semi colon separated list is Base64 encoded.
Extract the challenge based on the contents above.
Print a response matching the challenge to stdout and exit with code 0
In case a matching response can not be given do exit with code 2
Below is a simple example of an MFA authenticator implemented in Python3:
Configure the authentication group used by the device instance to enable MFA. There are two configurables available:
executable The path to the external multi factor authentication executable (mandatory).
opaque Opaque data that will passed as a cookie element to the executable (optional).
Try connecting to the device.
admin@ncs(config)# devices device dev-1 ssh fetch-host-keysadmin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 sync-from
result true[<device name>;<user>;<password>;<opaque>;<ssh server name>;<ssh server instruction>;<ssh server prompt>;]> ncs_cli -C -u admin
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# devices authgroups group <name> default-map mfa executable <path to the executable>
admin@ncs(config)# devices authgroups group <name> default-map mfa opaque <some opaque data>
admin@ncs(config)# commit+---------------------------+------------------------------------------------------------------------------+
| Name | Info |
+---------------------------+------------------------------------------------------------------------------+
| README-ned-settings.md | Information about all run time settings supported by this NED. |
+---------------------------+------------------------------------------------------------------------------++---------------------------+-----------+------------------------------------------------------------------+
| Feature | Supported | Info |
+---------------------------+-----------+------------------------------------------------------------------+
| netsim | yes | See README.md for further info on how to configure the NED to |
| | | use netsim as a RESTCONF target. |
| | | |
| check-sync | yes | Disabled by default. Can be enabled on devices that meet certain |
| | | requirements. See README-ned-settings.md for further info. |
| | | |
| partial-sync-from | no | - |
| | | |
| live-status actions | yes | Supported: summary-inventory, inventory, run, copy, |
| | | admin_software_install |
| | | |
| load-native-config | no | |
| | | |
| live-status show | yes | Supported: info |
+---------------------------+-----------+------------------------------------------------------------------++---------------------------+-----------------+--------+---------------------------------------------------+
| Model | Version | OS | Info |
+---------------------------+-----------------+--------+---------------------------------------------------+
| RAD-VX ETX205A-4E1/T1-SYE | Hw: 1.3/E, Sw: | VxWork | - |
| | 6.6.1(0.23) | s | |
| | | | |
| RAD-VX ETX-2I-10G-LC | Hw: 0.2/B, Sw: | VxWork | - |
| | 6.7.1(0.53) | s | |
+---------------------------+-----------------+--------+---------------------------------------------------+> cd /tmp/ned-package-store
> chmod u+x ncs-6.0-rad-vx-1.0.1.signed.bin
> ./ncs-6.0-rad-vx-1.0.1.signed.bin> ./ncs-6.0-rad-vx-1.0.1.signed.bin --skip-verification> ls *.tar.gz
ncs-6.0-rad-vx-1.0.1.tar.gz> export NSO_RUNDIR=~/nso-lab-rundir> tar xfz ncs-6.0-rad-vx-1.0.1.tar.gz
> ls -d */
rad-vx-cli-1.0> ncs-setup --package rad-vx-cli-1.0 --dest $NSO_RUNDIR> ncs_cli -C -u admin
admin@ncs# packages reload
reload-result {
package rad-vx-cli-1.0
result true
} > ncs-setup --package ncs-6.0-rad-vx-1.0.1.tar.gz --dest $NSO_RUNDIR
> ncs_cli -C -u admin
admin@ncs# packages reload
reload-result {
package rad-vx-cli-1.0
result true
}> export NED_ROOT_DIR=$NSO_RUNDIR/packages/rad-vx-cli-1.0> $NCS_DIR/bin/ncs-backup> ncs_cli -C -u admin
admin@ncs# software packages fetch package-from-file \
/tmp/ned-package-store/ncs-6.0-rad-vx-1.0.tar.gz
admin@ncs# software packages list
package {
name ncs-6.0-rad-vx-1.0.tar.gz
installable
}admin@ncs# software packages install rad-vx-1.0
admin@ncs# software packages list
package {
name ncs-6.0-rad-vx-1.0.tar.gz
installed
}admin@ncs# packages reload
admin@ncs# software packages list
package {
name ncs-6.0-rad-vx-cli-1.0
loaded
}> ncs_cli -C -u adminadmin@ncs# configure
Entering configuration mode terminal
admin@ncs(config)#admin@ncs(config)# devices authgroup group my-group default-map remote-name <user name on device> \
remote-password <password on device>admin@ncs(config)# devices device dev-1 address <ip address to device>
admin@ncs(config)# devices device dev-1 port <port on device>
admin@ncs(config)# devices device dev-1 device-type cli ned-id rad-vx-cli-1.0
admin@ncs(config)# devices device dev-1 state admin-state unlocked
admin@ncs(config)# devices device dev-1 authgroup my-groupadmin@ncs(config)# devices device dev-1 protocol <ssh or telnet>> ncs_cli -C -u adminadmin@ncs# configure
Entering configuration mode terminal
admin@ncs(config)#admin@ncs(config)# devices device dev-1 trace raw
admin@ncs(config)# commitadmin@ncs(config)# devices global-settings trace raw
admin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 ned-settings rad-vx logger \
level [debug | verbose | info | error]
admin@ncs(config)# commit> ncs_cli -C -u adminadmin@ncs# configure
Entering configuration mode terminal
admin@ncs(config)#admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.radvx \
level level-all
admin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 ned-settings rad-vx logger java true
admin@ncs(config)# commitadmin@ncs(config)# devices global-settings ned-settings rad-vx logger java true
admin@ncs(config)# commitadmin@ncs(config-device-rad-vx)# live-status exec admin_software_install sw-pack-4 no-restore-point
Error: External error in the NED implementation for device rad-vx: Sucess:
read timeout after 60 seconds, blocked on "\n\rETX-2I-10G# " when waiting for ".*Cannot.*install.*software.*identical.*software.*" | ".*Device.*reboot.*in.*" | ".*Incorrect.*file.*name.*" | ".*Source.*file.*labeled.*invalid.*" | ".*Could.*not.*copy.*"
admin@ncs(config-device-rad-vx)# configure
qos
queue-block-profile Scheduling2
queue 0
scheduling strict
exit
exit
exit
exit # devices device RADVX sync-from
result true
# top show full-configuration devices device RADVX config radvx:configure qos queue-block-profile Scheduling2
devices device RADVX
config
radvx:configure
qos
queue-block-profile Scheduling2
queue 0
scheduling strict
exit
exit
exit
exit
!
!
# no devices device RADVX config radvx:configure qos queue-block-profile Scheduling2 queue 0 scheduling
# commit dry-run outformat native
native {
device {
name radvx-78
data configure
qos
queue-block-profile Scheduling2
queue 0
scheduling wfq 100
exit
exit
exit
exit
}
} devices device RADVX compare-config outformat cli
diff
devices {
device RADVX {
config {
radvx:configure {
qos {
- queue-block-profile Scheduling2 {
- }
}
}
}
}
}- A detailed recipe with steps to reproduce the issue.
- A raw trace file generated when the issue is reproduced.
- SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team.
This typically means both read and write permissions are required.
Pseudo access via tools like Webex, Zoom etc is not acceptable.
However, it is ok with device access through VPNs, jump servers etc though.ncs_cli -C -u admin
admin@ncs# configure
admin@ncs(config)# devices device dev-1 ned-settings rad-vx logging level debug
admin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 trace raw
admin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 clear-traceadmin@ncs(config)# devices clear-trace> cd $NED_ROOT_DIR/src
> make clean alladmin@ncs# packages reload> ncs_cli -C -u admin
admin@ncs# show running-config devices authgroups group default default-map <tab>
Possible completions:
action-name The action to call when a notification is received.
callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance.
mfa Settings for handling multi-factor authentication towards the device
public-key Use public-key authentication
remote-name Specify device user name
remote-password Specify the remote password
remote-secondary-password Second password for configuration
same-pass Use the local NCS password as the remote password
same-secondary-password Use the local NCS password as the remote secondary password
same-user Use the local NCS user name as the remote user name> devices device dev-1 trace raw
> devices device dev-1 ned-settings rad-vx logger level debug
> commitusing ssh external mfa executable: <configured path to executable>calling external mfa executable with ssh server given name: '<name>', instruction: '<instruction>', prompt '<challenge>'ERROR: external mfa executable failed <....>admin@ncs(config)# devices device global-settings ned-settings rad-vx logger \
level [debug | verbose | info | error]
admin@ncs(config)# commitadmin@ncs(config)# devices device dev-1 compare-config#!/usr/bin/env python3
import sys
import base64
# This is an example on how to implement an external multi factor authentication handler
# that will be called by the NED upon a ssh 'keyboard-interactive' authentication
# The handler is reading a line from stdin with the following expected format:
# [<device name>;<user>;<password>;<opaque>;<ssh server name>;<ssh server instruction>;<ssh server prompt>;]
# All elements are base64 encoded.
def decode(arg):
return str(base64.b64decode(arg))[2:-1]
if __name__ == '__main__':
query_challenges = {
"admin@localhost's password: ":'admin',
'Enter SMS passcode:':'secretSMScode',
'Press secret key: ':'2'
}
# read line from stdin and trim brackets
line = sys.stdin.readline().strip()[1:-1]
args = line.split(';')
prompt = decode(args[6])
if prompt in query_challenges.keys():
print(query_challenges[prompt])
exit(0)
else:
exit(2)