FD.io VPP  v21.10.1-2-g0a485f517
Vector Packet Processing
Getting started

Concept (Connecting to VPP)

For detailed information on api calls and structures please refer to libmemif.h.

  1. Initialize memif

If event occurs on any file descriptor returned by this callback, call memif_control_fd_handler function. Since version 2.0, last two optional arguments are used to specify custom memory allocation.

memif_err = memif_control_fd_handler (evt.data.fd, events);

If callback function parameter for memif_init function is set to NULL, libmemif will handle file descriptor event polling.

Api call memif_poll_event will call epoll_pwait with user defined timeout to poll event on file descriptors opened by libmemif.

/* main loop */
while (1)
{
if (memif_poll_event (-1) < 0)
{
DBG ("poll_event error!");
}
}

Memif initialization function will initialize internal structures and create timer file descriptor, which will be used for sending periodic connection requests. Timer is disarmed if no memif interface is created.

  1. Creating interface

example app uses struct that contains connection handle, rx/tx buffers and other connection specific information.

If connection is in slave mode, arms timer file descriptor. If on interrupt callback is set to NULL, user will not be notified about interrupt. Use memif_get_queue_efd call to get interrupt file descriptor for specific queue.

int fd = -1;
err = memif_get_queue_efd (c->conn, data->qid, &fd);
  1. Connection establishment
    • User application will poll events on all file descriptors returned in memif_control_fd_update_t callback.
    • On event call memif_control_fd_handler.
    • Everything else regarding connection establishment will be done internally.
    • Once connection has been established, a callback will inform the user about connection status change.
  2. Interrupt packet receive
    • If event is polled on interrupt file descriptor, libmemif will call memif_interrupt_t callback specified for every connection instance.
      int
      on_interrupt (memif_conn_handle_t conn, void *private_ctx, uint16_t qid)
      {
      ...
      }
  3. Memif buffers
    • Packet data are stored in memif_buffer_t. Pointer data points to shared memory buffer, and unsigned integer *_len* contains buffer length.
    • flags: MEMIF_BUFFER_FLAG_NEXT states that the buffer is not large enough to contain whole packet, so next buffer contains the rest of the packet. (chained buffers)
      typedef struct
      {
      uint16_t desc_index;
      uint32_t len;
      uint8_t flags;
      void *data;
  1. Packet receive
    • Api call memif_rx_burst will set all required fields in memif buffers provided by user application, dequeue received buffers and consume interrupt event on receive queue. The event is not consumed, if memif_rx_burst fails.
      err = memif_rx_burst (c->conn, qid, c->bufs, MAX_MEMIF_BUFS, &rx);
      - User application can then process packets.
      - Api call memif\_refill\_queue will enqueue rx buffers.
      
      err = memif_refill_queue (c->conn, qid, rx);
  2. Packet transmit
    • Api call memif_buffer_alloc will find free tx buffers and set all required fields in memif buffers provided by user application.
      err = memif_buffer_alloc (c->conn, qid, c->tx_bufs, n, &r);
      - User application can populate shared memory buffers with packets.
      - Api call memif\_tx\_burst will enqueue tx buffers
      
      err = memif_tx_burst (c->conn, qid, c->tx_bufs, c->tx_buf_num, &r);
  3. Helper functions
    • Memif version
      uint16_t memif_ver = memif_get_version ();
      - Memif details
        - Api call memif\_get\_details will return details about connection.
      
      err = memif_get_details (c->conn, &md, buf, buflen);
      - Memif error messages
        - Every api call returns error code (integer value) mapped to error string.
        - Call memif\_strerror will return error message assigned to specific error code.
      
      if (err != MEMIF_ERR_SUCCESS)
      INFO ("memif_get_details: %s", memif_strerror (err));
          - Not all syscall errors are translated to memif error codes. If error code 1 (MEMIF\_ERR\_SYSCALL) is returned then libmemif needs to be compiled with -DMEMIF_DBG flag to print error message. Use _make -B_ to rebuild libmemif in debug mode.
      

Example app (libmemif fd event polling):

Optional argument: transmit queue id.

icmpr 1

Set transmit queue id to 1. Default is 0. Application will create memif interface in slave mode and try to connect to VPP. Exit using Ctrl+C. Application will handle SIGINT signal, free allocated memory and exit with EXIT_SUCCESS.

Example app:

ICMP Responder custom fd event polling.

Example app (multi-thread queue polling)

ICMP Responder multi-thread.

Simple example of libmemif multi-thread usage. Connection establishment is handled by main thread. There are two rx/tx queues in this example. One in polling mode and second in interrupt mode.

VPP config:

# create interface memif id 0 master
# set int state memif0 up
# set int ip address memif0 192.168.1.1/24
# ping 192.168.1.2

For multiple rings (queues) support run VPP with worker threads: example startup.conf:

unix {
interactive
nodaemon
full-coredump
}
cpu {
workers 2
}

VPP config:

# create interface memif id 0 master
# set int state memif0 up
# set int ip address memif0 192.168.1.1/24
# ping 192.168.1.2

Master mode queue number is limited by worker threads. Slave mode interface needs to specify number of queues.

# create memif id 0 slave rx-queues 2 tx-queues 2

Example applications use VPP default socket file for memif: /run/vpp/memif.sock For master mode, socket directory must exist prior to memif_create call.

Unit tests

Unit tests use Check framework. This framework must be installed in order to build unit_test binary. Ubuntu/Debian:

sudo apt-get install check

More platforms

on_connect
int on_connect(memif_conn_handle_t conn, void *private_ctx)
Definition: main.c:177
memif_conn_args_t::mode
memif_interface_mode_t mode
Definition: libmemif.h:308
DBG
#define DBG(...)
Definition: hash_lookup_private.h:30
control_fd_update
int control_fd_update(int fd, uint8_t events, void *ctx)
Definition: main.c:378
MEMIF_ERR_SUCCESS
@ MEMIF_ERR_SUCCESS
Definition: libmemif.h:38
INFO
#define INFO(...)
Definition: main.c:64
memif_conn_args_t::buffer_size
uint16_t buffer_size
Definition: libmemif.h:302
memif_create
int memif_create(memif_conn_handle_t *conn, memif_conn_args_t *args, memif_connection_update_t *on_connect, memif_connection_update_t *on_disconnect, memif_interrupt_t *on_interrupt, void *private_ctx)
Memory interface create function.
Definition: main.c:1065
memif_conn_args_t
Memif connection arguments.
Definition: libmemif.h:295
APP_NAME
#define APP_NAME
Definition: main.c:50
memif_get_details
int memif_get_details(memif_conn_handle_t conn, memif_details_t *md, char *buf, ssize_t buflen)
Memif get details.
Definition: main.c:2637
r
vnet_hw_if_output_node_runtime_t * r
Definition: interface_output.c:1089
memif_conn_args_t::interface_id
uint32_t interface_id
Definition: libmemif.h:306
on_disconnect
int on_disconnect(memif_conn_handle_t conn, void *private_ctx)
Definition: main.c:186
IF_NAME
#define IF_NAME
Definition: main.c:51
on_interrupt
int on_interrupt(memif_conn_handle_t conn, void *private_ctx, uint16_t qid)
Definition: main.c:287
len
u8 len
Definition: ip_types.api:103
memif_get_version
uint16_t memif_get_version()
Memif get version.
Definition: main.c:171
memif_conn_args_t::num_m2s_rings
uint8_t num_m2s_rings
Definition: libmemif.h:301
c
svmdb_client_t * c
Definition: vpp_get_metrics.c:48
memif_buffer_alloc
int memif_buffer_alloc(memif_conn_handle_t conn, uint16_t qid, memif_buffer_t *bufs, uint16_t count, uint16_t *count_out, uint16_t size)
Memif buffer alloc.
Definition: main.c:2265
events
static perfmon_event_t events[]
Definition: core.c:21
memif_conn_handle_t
void * memif_conn_handle_t
Memif connection handle pointer of type void, pointing to internal structure.
Definition: libmemif.h:110
data
u8 data[128]
Definition: ipsec_types.api:95
memif_get_queue_efd
int memif_get_queue_efd(memif_conn_handle_t conn, uint16_t qid, int *fd)
Memif get queue event file descriptor.
Definition: main.c:2798
index
u32 index
Definition: flow_types.api:221
memif_conn_args_t::num_s2m_rings
uint8_t num_s2m_rings
Definition: libmemif.h:300
memif_tx_burst
int memif_tx_burst(memif_conn_handle_t conn, uint16_t qid, memif_buffer_t *bufs, uint16_t count, uint16_t *tx)
Memif transmit buffer burst.
Definition: main.c:2444
memif_buffer_t
Memif buffer.
Definition: libmemif.h:325
memif_control_fd_handler
int memif_control_fd_handler(int fd, uint8_t events)
Memif control file descriptor handler.
Definition: main.c:1277
buf
u64 buf
Definition: application.c:493
memif_init
static clib_error_t * memif_init(vlib_main_t *vm)
Definition: memif.c:1094
ctx
long ctx[MAX_CONNS]
Definition: main.c:144
memif_conn_args_t::log2_ring_size
uint8_t log2_ring_size
Definition: libmemif.h:303
memif_conn_args_t::is_master
uint8_t is_master
Definition: libmemif.h:304
memif_conn_args_t::interface_name
uint8_t interface_name[32]
Definition: libmemif.h:307
memif_strerror
char * memif_strerror(int err_code)
Memif strerror.
Definition: main.c:157
memif_poll_event
int memif_poll_event(int timeout)
Memif poll event.
Definition: main.c:1503
memif_refill_queue
int memif_refill_queue(memif_conn_handle_t conn, uint16_t qid, uint16_t count, uint16_t headroom)
Memif refill queue.
Definition: main.c:2389
MAX_MEMIF_BUFS
#define MAX_MEMIF_BUFS
Definition: main.c:70
memif_rx_burst
int memif_rx_burst(memif_conn_handle_t conn, uint16_t qid, memif_buffer_t *bufs, uint16_t count, uint16_t *rx)
Memif receive buffer burst.
Definition: main.c:2545
flags
vl_api_wireguard_peer_flags_t flags
Definition: wireguard.api:105