vrnetlab - VR Network Lab

Run your favourite virtual routers in docker for convenient labbing, development and testing.

vrnetlab is being developed for the TeraStream project at Deutsche Telekom as part of an automated CI environment for testing our network provisioning system.

It supports:

Arista vEOS
Cisco CSR1000v
Cisco Nexus NX-OS (using Titanium emulator)
Cisco XRv
Cisco XRv 9000
Juniper vMX
Juniper vQFX
Nokia VSR

I talk a little about it during my presentation about TeraStream testing at the NetNod autumn meeting 2016 - https://youtu.be/R_vCdGkGeSk?t=9m25s

Brian Linkletter has written a good introduction too https://www.brianlinkletter.com/vrnetlab-emulate-networks-using-kvm-and-docker/

Usage

You have to build the virtual router docker images yourself since the license agreements of commercial virtual routers do not allow me to distribute the images. See the README files of the respective virtual router types for more details.

You need KVM enabled in your kernel for hardware assisted virtualization. While it may be possible to run without it, it has not been tested. Make sure you load the kvm kernel module: modprobe kvm.

Let's assume you've built the xrv router.

Start two virtual routers:

docker run -d --name vr1 --privileged vr-xrv:5.3.3.51U
docker run -d --name vr2 --privileged vr-xrv:5.3.3.51U

I'm calling them vr1 and vr2. Note that I'm using XRv 5.3.3.51U - you should fill in your XRv version in the image tag as the "latest" tag is not added to any images.

It takes a few minutes for XRv to start but once up you should be able to SSH into each virtual router. You can get the IP address using docker inspect:

root@host# docker inspect --format '{{.NetworkSettings.IPAddress}}' vr1
172.17.0.98

Now SSH to that address and login with the default credentials of vrnetlab/VR-netlab9:

root@host# ssh -l vrnetlab $(docker inspect --format '{{.NetworkSettings.IPAddress}}' vr1)
The authenticity of host '172.17.0.98 (172.17.0.98)' can't be established.
RSA key fingerprint is e0:61:28:ba:12:77:59:5e:96:cc:58:e2:36:55:00:fa.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.17.0.98' (RSA) to the list of known hosts.

IMPORTANT:  READ CAREFULLY
Welcome to the Demo Version of Cisco IOS XRv (the "Software").
The Software is subject to and governed by the terms and conditions
of the End User License Agreement and the Supplemental End User
License Agreement accompanying the product, made available at the
time of your order, or posted on the Cisco website at
www.cisco.com/go/terms (collectively, the "Agreement").
As set forth more fully in the Agreement, use of the Software is
strictly limited to internal use in a non-production environment
solely for demonstration and evaluation purposes.  Downloading,
installing, or using the Software constitutes acceptance of the
Agreement, and you are binding yourself and the business entity
that you represent to the Agreement.  If you do not agree to all
of the terms of the Agreement, then Cisco is unwilling to license
the Software to you and (a) you may not download, install or use the
Software, and (b) you may return the Software as more fully set forth
in the Agreement.


Please login with any configured user/password, or cisco/cisco


vrnetlab@172.17.0.98's password:


RP/0/0/CPU0:ios#show version
Mon Jul 18 09:04:45.261 UTC

Cisco IOS XR Software, Version 5.3.3.51U[Default]
...

You can also login via NETCONF:

root@host# ssh -l vrnetlab $(docker inspect --format '{{.NetworkSettings.IPAddress}}' vr1) -p 830 -s netconf
vrnetlab@172.17.0.98's password:
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <capabilities>
  <capability>urn:ietf:params:netconf:base:1.1</capability>
  <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</capability>
  <capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
  <capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
  <capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
  <capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-aaa-lib-cfg?module=Cisco-IOS-XR-aaa-lib-cfg&amp;revision=2015-08-27</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-aaa-locald-admin-cfg?module=Cisco-IOS-XR-aaa-locald-admin-cfg&amp;revision=2015-08-27</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-aaa-locald-cfg?module=Cisco-IOS-XR-aaa-locald-cfg&amp;revision=2015-08-27</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-aaa-locald-oper?module=Cisco-IOS-XR-aaa-locald-oper&amp;revision=2015-08-27</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-bundlemgr-cfg?module=Cisco-IOS-XR-bundlemgr-cfg&amp;revision=2015-08-27</capability>
...

The serial console of the devices are mapped to port 5000. Use telnet to connect:

root@host# telnet $(docker inspect --format '{{.NetworkSettings.IPAddress}}' vr1) 5000

Just like with any serial port, you can only have one connection at a time and while the router is booting the launch script will connect to the serial port to do the initialization of the router. As soon as it is done the port will be released and made available to the next connection.

To connect two virtual routers with each other we can use the vr-xcon container. Let's say we want to connect Gi0/0/0/0 of vr1 and vr2 with each other, we would do:

docker run -d --name vr-xcon --link vr1 --link vr2 vr-xcon --p2p vr1/1--vr2/1

Configure a link network on vr1 and vr2 and you should be able to ping!

P/0/0/CPU0:ios(config)#inte GigabitEthernet 0/0/0/0
RP/0/0/CPU0:ios(config-if)#no shutdown
RP/0/0/CPU0:ios(config-if)#ipv4 address 192.168.1.2/24
RP/0/0/CPU0:ios(config-if)#commit
Mon Jul 18 09:13:24.196 UTC
RP/0/0/CPU0:Jul 18 09:13:24.216 : ifmgr[227]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/0, changed state to Down
RP/0/0/CPU0:ios(config-if)#dRP/0/0/CPU0:Jul 18 09:13:24.256 : ifmgr[227]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/0, changed state to Up
o ping 192.168.1.1
Mon Jul 18 09:13:26.896 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

(obviously I configured the other end too!)

All of the NICs of the virtual routers are exposed via TCP ports by KVM. TCP port 10001 maps to the first NIC of the virtual router, which in the case of an XR router is GigabitEthernet 0/0/0/0. By simply connecting two of these TCP sockets together we can bridge the traffic between those two NICs and this is exactly what vr-xcon is for. Use the --p2p argument to specify the links. The format is X/Y--Z/N where X is the name of the first router and Y is the port on that router. Z is the second router and N is the port on the second router.

To set up more than one p2p link, simply add more mappings separated by space and don't forget to link the virtual routers:

docker run -d --name vr-xcon --link vr1 --link vr2 --link vr3 vr-xcon --p2p vr1/1--vr2/1 vr1/2--vr3/1

See topology-machine/README.md for details on topology machine which can help you with managing more complex topologies.

The containers expose port 22 for SSH, port 161 for SNMP, port 830 for NETCONF and port 5000 is mapped to the virtual serial device (use telnet). All the NICs of the virtual routers are exposed via TCP ports in the range 10001-10099.

Use docker rm -f vr1 to stop and remote a virtual router.

Handy shell functions

There are some handy shell functions in vrnetlab.sh that provides shorthands for connecting to ssh and console.

Load the functions into your shell

. vrnetlab.sh

Login via ssh to router vr1, you can optionally specify a username. If no username is provided, the default of vrnetlab will be used. If sshpass is installed, you will not be promted for password when you login with the default username.

vrssh vr1 myuser

Connect console to router vr1

vrcons vr1

Create a bridge between two router interfaces, the below command bridges interface 1 of router vr1 with interface 1 of router 2.

vrbridge vr1 1 vr2 1

To load these aliases on login, copy it to ~/.vrnetlab_bashrc and add the following to your .bashrc

test -f ~/.vrnetlab_bashrc && . ~/.vrnetlab_bashrc

Virtual routers

There are a number of virtual routers available on the market:

Cisco XRv
Juniper VRR
Juniper vMX
Nokia VSR

All of the above are released as a qcow2 or vmdk file (which can easily be converted into qcow2) making them easy to spin up on a Linux machine. Once spun up there are a few tasks one normally wants to perform:

set an IP address on a management interface
start SSH / NETCONF daemon (and generate crypto keys)
create initial user so we can login

There might be more things to the list but this is the bare minimum which makes the router remotely reachable and thus we can configure the rest from the normal provisioning system.

vrnetlab aims to make this process as simple and convenient as possible so that it may be used both by humans and automated systems to spin up virtual routers. In addition, there are scripts to help you generate topologies.

The virtual machines are packaged up in docker container. Since we need to start KVM the docker containers have to be run with --privileged which effectively defeats the security features of docker. Our use of docker is essentially reduced to being a packaging format but a rather good one at that. Also note that since we still rely on KVM the same amount of resources, if not sightly more, will be consumed by vrnetlab. A container is no thinner than a VM if the container contains a VM!

The assignment of a management IP address is ha

Vrnetlab

Install / Use

README

vrnetlab - VR Network Lab

Usage

Handy shell functions

Virtual routers