Lab: Taking the VPP Container Testbench for a Spin

Assuming the reader has already acquired the test bench build scripts, let's start with building it.

 1vagrant@ubuntu-focal$> make
 2# Client image.
 3DOCKER_BUILDKIT=1 docker build \
 4                --file Dockerfile.vpp_testbench \
 5                --build-arg HEALTHCHECK_PORT=8123 \
 6                --tag vpp-testbench-client:local \
 7                --target client_img \
 8                .
 9...
10...
11...
12DOCKER_BUILDKIT=1 docker build \
13                --file Dockerfile.vpp_testbench \
14                --build-arg HEALTHCHECK_PORT=8123 \
15                --tag vpp-testbench-server:local \
16                --target server_img \
17                .
18...
19...
20...
21 => exporting to image
22 => => exporting layers
23 => => writing image
24 => => naming to docker.io/library/vpp-testbench-server:local
250.0s
26Done.

Now, let's start up our newly built images as a pair of containers. The various hashes throughout this document will differ from those shown in your own console (perfectly fine). First, let's assume there are no running containers on your system. We'll verify via docker ps:

1vagrant@ubuntu-focal$> docker ps
2CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

OK: good initial conditions. Now, let's launch the containers:

 1vagrant@ubuntu-focal$> make start
 2# Create Docker bridge network.
 3bash -c " . vpp_testbench_helpers.sh; host_only_create_docker_networks; "
 46e071e533e239380b2fe92d6e0844c42736ec186226fbb20d89706f9a80f935f
 5# Launch the containers.
 6bash -c " . vpp_testbench_helpers.sh; host_only_run_testbench_client_container
 7vpp-testbench-client:local; host_only_run_testbench_server_container
 8vpp-testbench-server:local; "
 9720fed0a94fd715694d73a41317f05a3f36860a6d5ae54db2d7cb7f2dcaf7924
10ccf166993d09e399f7b10d372c47cc9e72ce6092ef70ea206c699263da844e1b
11# Entrypoint scripts will bring up the various links.
12# Use "docker ps" to check status of containers, see if their health
13# probes are working as expected (i.e. "health"), etc.

Now, we can use docker ps to verify if the containers are up and running:

1vagrant@ubuntu-focal$> docker ps
2CONTAINER ID   IMAGE                        COMMAND            CREATED STATUS                                  PORTS     NAMES
37d8e3ab35111   vpp-testbench-server:local   "/entrypoint.sh"   3 seconds ago Up 2 seconds (health: starting)             vpp-testbench-server
4cc01e64b12da   vpp-testbench-client:local   "/entrypoint.sh"   4 seconds ago Up 3 seconds (health: starting)             vpp-testbench-client

Looking good so far. However, note the "health" status of the containers. They're not yet ready. We can re-execute the docker ps command occasionally until the containers are ready:

1vagrant@ubuntu-focal$>  while true; do docker ps; sleep 1; done
2CONTAINER ID   IMAGE                        COMMAND            CREATED         STATUS                                     PORTS     NAMES
342e9bcea7c58   vpp-testbench-server:local   "/entrypoint.sh"   1 second ago    Up Less than a second (health: starting)             vpp-testbench-server
4710287b40bd3   vpp-testbench-client:local   "/entrypoint.sh"   2 seconds ago   Up Less than a second (health: starting)             vpp-testbench-client
542e9bcea7c58   vpp-testbench-server:local   "/entrypoint.sh"   30 seconds ago   Up 29 seconds (health: starting)             vpp-testbench-server
6710287b40bd3   vpp-testbench-client:local   "/entrypoint.sh"   31 seconds ago   Up 30 seconds (healthy)                      vpp-testbench-client
7CONTAINER ID   IMAGE                        COMMAND            CREATED          STATUS                    PORTS     NAMES
842e9bcea7c58   vpp-testbench-server:local   "/entrypoint.sh"   31 seconds ago   Up 30 seconds (healthy)             vpp-testbench-server
9710287b40bd3   vpp-testbench-client:local   "/entrypoint.sh"   32 seconds ago   Up 31 seconds (healthy)             vpp-testbench-client

Not the most elegant approach, but it works. Onward.

Note

How would one automate this step so that we're not having to manually watch the console until the containers are ready? What's something that we could put into a script our our Makefile to poll the containers until they're ready to use?

Spoiler
# "Direct" approach.
while true; do
    [ '"healthy"' = docker inspect --format "{{json .State.Health.Status }}" vpp-testbench-client] && break
done

# Could also use awk/grep/etc. against the output of "docker ps".

Now that our containers are up and running, let's drop a shell into the "client" container:

1vagrant@ubuntu-focal$> make shell_client

First, let's take a look at the default network configuration.

 1root@478ab126035e:/work# ip a
 21: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 3    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 4    inet 127.0.0.1/8 scope host lo
 5       valid_lft forever preferred_lft forever
 62: vxlan-vid-42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
 7    link/ether 3a:2c:19:cb:ca:35 brd ff:ff:ff:ff:ff:ff
 8    inet 169.254.10.1/24 scope global vxlan-vid-42
 9       valid_lft forever preferred_lft forever
103: vpp-tap-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN group default qlen 1000
11    link/ether 02:fe:c5:52:63:12 brd ff:ff:ff:ff:ff:ff
12    inet 169.254.12.1/24 scope global vpp-tap-0
13       valid_lft forever preferred_lft forever
14635: eth0@if636: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
15    link/ether 02:42:a9:fe:00:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0
16    inet 169.254.0.1/24 brd 169.254.0.255 scope global eth0
17       valid_lft forever preferred_lft forever

Let's also enumerate the interfaces managed by VPP. For the help of the reader, there is a shell function, vc, which just launches vppctl with some helpful default arguments.

 1root@478ab126035e:/work# type vc
 2vc is a function
 3vc ()
 4{
 5    vppctl -s "${VPP_SOCK}" "${@}"
 6}
 7root@478ab126035e:/work# vc
 8    _______    _        _   _____  ___
 9 __/ __/ _ \  (_)__    | | / / _ \/ _ \
10 _/ _// // / / / _ \   | |/ / ___/ ___/
11 /_/ /____(_)_/\___/   |___/_/  /_/
12
13vpp# show int
14              Name               Idx    State  MTU (L3/IP4/IP6/MPLS)     Counter          Count
15local0                            0     down          0/0/0/0
16memif0/0                          1      up          9000/0/0/0     rx packets                     7
17                                                                    rx bytes                     521
18                                                                    tx packets                     7
19                                                                    tx bytes                     503
20tap0                              2      up          9000/0/0/0     rx packets                     7
21                                                                    rx bytes                     503
22                                                                    tx packets                     7
23                                                                    tx bytes                     521
24vpp#

Note

One more exercise for the reader: 1. From the client container, how would you ping the server container on the Linux-managed VXLAN interface? 2. From the client container, how would you ping the server container on the VPP-managed TAP interface? 3. A couple trivial web servers (using netcat) are running on the server container. Besides looking at the Makefile recipes, how could one determine what ports and interfaces these servers are bound to, and how would one issue an HTTP GET query against them from the client container? (hint: you're allowed to log-in to the server container via make shell_server, and the netstat command may be of use).

Spoiler
1. ping 169.254.10.2
2. ping 169.254.12.2
3. make shell_server
     netstat -tulpn
       tcp        0      0 169.254.12.2:8000       0.0.0.0:*
       LISTEN      47/nc
       tcp        0      0 169.254.10.2:8000       0.0.0.0:*
       LISTEN      34/nc
     exit
   make shell_client
     root@478ab126035e:/work# curl 169.254.10.2:8000
       HOST:14f0df855445
       DATE:Fri Nov 19 16:36:57 UTC 2021
       Hello from the Linux interface.
     root@478ab126035e:/work# curl 169.254.12.2:8000
       HOST:14f0df855445
       DATE:Fri Nov 19 16:37:04 UTC 2021
       Hello from the VPP interface.
     exit

Now that we've done some quick exercises, let's clean-up the containers and their associated resources.

 1vagrant@ubuntu-focal$> make stop
 2# Terminate the containers.
 3bash -c " . vpp_testbench_helpers.sh; host_only_kill_testbench_client_container vpp-testbench-client:local; host_only_kill_testbench_server_container vpp-testbench-server:local; "
 4vpp-testbench-client
 5Error: No such container: vpp-testbench-client
 6vpp-testbench-server
 7Error: No such container: vpp-testbench-server
 8# Cleanup Docker bridge network.
 9bash -c " . vpp_testbench_helpers.sh; host_only_destroy_docker_networks; "
10vpp-testbench-net

That's it for this section.