HowTo Configure and Run a Ring Buffer Capture Session Using: "nstringbufcap"

From NST Wiki
Jump to: navigation, search

Overview

A new NST script: "nstringbufcap" has been developed with NST 24 for managing one or more network packet capture sessions that utilize a ring buffer storage mechanism. This capability allows one to capture network traffic pre and post a controlled event. Currently, an NST Network Interface Bandwidth Monitor 2 Threshold Pause State Notification Execs template: "/etc/nst/notifications/bwmon/tp_state_nstringbufcap_snapwuimerge.template" is provided for snapping a capture when a Pause event occurs and then the NST WUI Single-Tap Network Packet Capture page can be used for capture decode and analysis.

The nstringbufcap script has a specific mode used for installing and configuring a ring buffer capture session. Once install, a capture session can be started under the control of a systemd service. The life cycle of the capture session can then be controlled by different nstringbufcap modes that internally use systemd control commands (i.e., systemctl). At any point in time, a snapshot capture can be taking to preserve captured network traffic packets. nstringbufcap currently supports two (2) network capturing tools: dumpcap and netsniff-ng.

This page will describe the use of the nstringbufcap script by way of example use cases.

Mode: install

This mode will install a ring buffer capture session environment configuration. A valid and accessible ring buffer capture directory must be created prior to using this mode. This directory will be used to store the captured network traffic file components that make up the ring buffer.

Storage sizing of the ring buffer is very important because it establishes the rolling window in time (i.e., capture window) for the collection of captured traffic. The trick to sizing is based on the network traffic bandwidth usage. One needs to determine the size of the capture window so that network traffic of interest is captured and does not fall outside the boundary of the ring buffer. The size is calculated by multiplying the maximum number of capture file components times the allocated file size in Kilobytes (KB). When a capture file fills up to the allocated limit the file is then closed for writing and a new file is created for capture writing to continue. When the maximum file count is reached for the ring buffer, the oldest capture file will replaced by the new capture file to be written to. The dumpcap tool does this automatically but the netsniff-ng tool requires an additional systemd timer: "ringbuf-netsniff-ng-trim.timer" to perform this functionality that runs an instance of the systemd service: "ringbuf-netsniff-ng-trim.service" every 10 seconds. The entire sequence continues until the capture session is terminated.

A maximum file duration option: --max-file-dur INTEGER can also be used as an additional limiting ring buffer file size criterion. The capture file will switch to the next file after INTEGER seconds have elapsed, even if the current file has not reached its allocated size. This option may be useful to better quantify the ring buffer capture window especially during lower capture rates.

The example install mode shown below is used to create a configuration environment file (i.e., firewall located in directory: "/etc/nstringbufcap.d") for a ring buffer capture session that will capture packets on network interface: "fw0". The dumpcap capture tool will be used with the ring buffer directory set to "/opt/ringbufcap1". The ring buffer size consists of 20 capture files each with a maximum size of 2MB (i.e., A 40MB ring buffer sized rolling capture window.). The capture file prefix: "fw" was chosen for each file component name. The verbose parameter: "-v" is used to display the content of the created session configuration file.

[root@probe ~]# /usr/bin/nstringbufcap -m install -i fw0 -t dumpcap -d /opt/ringbufcap1 -s firewall -c fw --max-file-size 2000 --max-file-cnt 20 -v;

A new Ring Buffer dumpcap Capture session environment file was
created and installed: "/etc/nstringbufcap.d/firewall"
=========================================================================================

firewall (dumpcap): Ring Buffer Capture Configuration Directory: "/etc/nstringbufcap.d"
-----------------------------------------------------------------------------------------
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 479 Oct 19 17:03 firewall

Configuration: "firewall"
-----------------------------------------------------------------------------------------
/bin/cat "/etc/nstringbufcap.d/firewall";
#
# An nstringbufcap environment file (Installed on: Wed Oct 19 17:03:28 EDT 2016).
#
# Capture Tool: dumpcap
#
# Ring Buffer File Count: 20
#
# Required Syntax:
#   1) Make sure the entire OPTIONS variable value is enclosed
#      in double quotes (").
#
#   2) Make sure the Ring Buffer directory (-w) and/or the Capture Filter
#      expression (-f) parameters are enclosed in single quotes (').

OPTIONS="-q -i fw0 -b filesize:2000 -b files:20 -w '/opt/ringbufcap1/fw.pcap'"
=========================================================================================

[root@probe ~]#

 

Mode: install - With Capture Filter

This install mode includes a capture filer expression option. Filtering is commonly used to help reduce the size of the captured packet files, expand the time frame of the capture window and allows one to focus on traffic of interest.

The example shown below uses a capture file expression: 'ip and (host 24.97.151.197)' which states to only capture IPv4 packets from Host: 24.95.151.197.

[root@probe ~]# /usr/bin/nstringbufcap -m install -i fw0 -t dumpcap -d /opt/ringbufcap1 -s firewall -c fw --max-file-size 2000 --max-file-cnt 20 -f 'ip and (host 24.97.151.197)' -v;

A new Ring Buffer dumpcap Capture session environment file was
created and installed: "/etc/nstringbufcap.d/firewall"
=========================================================================================

firewall (dumpcap): Ring Buffer Capture Configuration Directory: "/etc/nstringbufcap.d"
-----------------------------------------------------------------------------------------
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 512 Oct 20 12:31 firewall

Configuration: "firewall"
-----------------------------------------------------------------------------------------
/bin/cat "/etc/nstringbufcap.d/firewall";
#
# An nstringbufcap environment file (Installed on: Thu Oct 20 12:31:38 EDT 2016).
#
# Capture Tool: dumpcap
#
# Ring Buffer File Count: 20
#
# Required Syntax:
#   1) Make sure the entire OPTIONS variable value is enclosed
#      in double quotes (").
#
#   2) Make sure the Ring Buffer directory (-w) and/or the Capture Filter
#      expression (-f) parameters are enclosed in single quotes (').

OPTIONS="-q -i fw0 -b filesize:2000 -b files:20 -f 'ip and (host 24.97.151.197)' -w '/opt/ringbufcap1/fw.pcap'"
=========================================================================================

[root@probe ~]#

 

Mode: install - With SnapLen

This install mode includes a Snap Length (SnapLen) option. Using a SnapLen will reduce the size of the captured packet files and expand the time frame of the capture window. The drawback to using a SnapLen is that packets that are longer than the SnapLen size will be truncated and not all the data from these packets will be saved in the capture file. Therefore, it might not be possible to decode upper-layer protocols or do reassembly of higher-level packets split across link-layer packets, such as IP_Reassembly.

A possible use case would be to discover the source and destination of network traffic at a particular time of day.

As an example, a SnapLen of "68" will only save header information for each IP packet captured and is shown below.

[root@probe ~]# /usr/bin/nstringbufcap -m install -i fw0 -t dumpcap -d /opt/ringbufcap1 -s firewall -c fw --max-file-size 2000 --max-file-cnt 20 -s 68 -v;

A new Ring Buffer dumpcap Capture session environment file was
created and installed: "/etc/nstringbufcap.d/firewall"
=========================================================================================

firewall (dumpcap): Ring Buffer Capture Configuration Directory: "/etc/nstringbufcap.d"
-----------------------------------------------------------------------------------------
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 512 Oct 21 08:22 firewall

Configuration: "firewall"
-----------------------------------------------------------------------------------------
/bin/cat "/etc/nstringbufcap.d/firewall";
#
# An nstringbufcap environment file (Installed on: Fri Oct 21 08:22:16 EDT 2016).
#
# Capture Tool: dumpcap
#
# Ring Buffer File Count: 20
#
# Required Syntax:
#   1) Make sure the entire OPTIONS variable value is enclosed
#      in double quotes (").
#
#   2) Make sure the Ring Buffer directory (-w) and/or the Capture Filter
#      expression (-f) parameters are enclosed in single quotes (').

OPTIONS="-q -i fw0 -s 68 -b filesize:2000 -b files:20 -w '/opt/ringbufcap1/fw.pcap'"
=========================================================================================

[root@probe ~]#

 

Mode: showenvs

The showenvs mode lists all configured ring buffer capture session environment files.

[root@probe ~]# /usr/bin/nstringbufcap -m showenvs -v;

Ring Buffer Capture Session Environment Files: "/etc/nstringbufcap.d"
=========================================================================================
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 479 Oct 21 14:32 firewall

[root@pktcap4 opt]# nstringbufcap -m showenvs -v

Ring Buffer Capture Session Environment Files: "/etc/nstringbufcap.d"
=========================================================================================
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 479 Oct 21 14:32 firewall

[root@probe ~]#

 

Mode: start

This mode starts one or more installed ring buffer capture sessions. The nstringbufcap script runs each capture session as a Systemd service. Since the session is under the management of Systemd, it can be controlled with the familiar Systemd utility: systemctl. In the example shown below the Systemd service for starting up capture session: firewall is "ringbuf-dumpcap@firewall.service". This service was derived by the Systemd service template unit file for the dumpcap capture tool: "/usr/lib/systemd/system/ringbuf-dumpcap@.service". A similar Systemd service template unit exists for the netsniff-ng capture tool: "/usr/lib/systemd/system/ringbuf-netsniff-ng@.service".

[root@probe ~]# /usr/bin/nstringbufcap -m start -s firewall -v;

A new dumpcap Ring Buffer Capture session: "/etc/nstringbufcap.d/firewall" running under
the control of Systemd Service: "ringbuf-dumpcap@firewall.service" will now be started.
=========================================================================================
/bin/systemctl --no-pager --full start "ringbuf-dumpcap@firewall.service";

/bin/systemctl --no-pager --full status "ringbuf-dumpcap@firewall.service";
● ringbuf-dumpcap@firewall.service - NST Ring Buffer dumpcap Service on firewall
   Loaded: loaded (/usr/lib/systemd/system/ringbuf-dumpcap@.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2016-10-21 15:07:04 EDT; 5ms ago
     Docs: man:dumpcap(1)
 Main PID: 26968 (dumpcap)
    Tasks: 1 (limit: 512)
   CGroup: /system.slice/system-ringbuf\x2ddumpcap.slice/ringbuf-dumpcap@firewall.service
           └─26968 /usr/sbin/dumpcap -q -i fw0 -b filesize:2000 -b files:20 -w /opt/ringbufcap1/fw.pcap

Oct 21 15:07:04 pktcap4 systemd[1]: Started NST Ring Buffer dumpcap Service on firewall.

A new dumpcap Ring Buffer Capture session: "/etc/nstringbufcap.d/firewall" running under
the control of Systemd Service: "ringbuf-dumpcap@firewall.service" has started normally.
=========================================================================================

[root@probe ~]#

 

Mode: status

This mode will show the Systemd service status for one or more installed ring buffer capture sessions. This capture session was running for more than 2 days. The dumpcap process ID is: "26968".

[root@probe ~]# /usr/bin/nstringbufcap -m status -s firewall -v;

Status for the dumpcap Ring Buffer Capture session: "/etc/nstringbufcap.d/firewall"
running as Systemd Service: "ringbuf-dumpcap@firewall.service".
=========================================================================================
/bin/systemctl --no-pager --full status "ringbuf-dumpcap@firewall.service";
● ringbuf-dumpcap@firewall.service - NST Ring Buffer dumpcap Service on firewall
   Loaded: loaded (/usr/lib/systemd/system/ringbuf-dumpcap@.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2016-10-21 15:07:04 EDT; 2 days ago
     Docs: man:dumpcap(1)
 Main PID: 26968 (dumpcap)
    Tasks: 1 (limit: 512)
   CGroup: /system.slice/system-ringbuf\x2ddumpcap.slice/ringbuf-dumpcap@firewall.service
           └─26968 /usr/sbin/dumpcap -q -i fw0 -b filesize:2000 -b files:20 -w /opt/ringbufcap1/fw.pcap

Oct 24 06:35:32 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04210_20161024063532.pcap
Oct 24 06:35:48 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04211_20161024063548.pcap
Oct 24 06:35:56 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04212_20161024063556.pcap
Oct 24 06:35:59 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04213_20161024063559.pcap
Oct 24 06:36:14 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04214_20161024063613.pcap
Oct 24 06:36:24 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04215_20161024063624.pcap
Oct 24 06:36:28 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04216_20161024063628.pcap
Oct 24 06:36:34 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04217_20161024063634.pcap
Oct 24 06:36:42 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04218_20161024063642.pcap
Oct 24 06:37:00 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04219_20161024063700.pcap
=========================================================================================

[root@probe ~]#

 

Mode: stop

This mode will terminate (stop) one or more installed ring buffer capture sessions. One can now perform a merge of the capture file components in the ring buffer directory: "/opt/ringbufcap1".

[root@probe ~]# /usr/bin/nstringbufcap -m stop -s firewall -v;

The dumpcap Ring Buffer Capture session: "/etc/nstringbufcap.d/firewall" running under the
the control of Systemd Service: "ringbuf-dumpcap@firewall.service" will now be terminated.
=========================================================================================
/bin/systemctl --no-pager --full stop "ringbuf-dumpcap@firewall.service";

/bin/systemctl --no-pager --full status "ringbuf-dumpcap@firewall.service";
● ringbuf-dumpcap@firewall.service - NST Ring Buffer dumpcap Service on firewall
   Loaded: loaded (/usr/lib/systemd/system/ringbuf-dumpcap@.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:dumpcap(1)

Oct 24 06:44:30 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04247_20161024064430.pcap
Oct 24 06:44:59 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04248_20161024064459.pcap
Oct 24 06:45:33 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04249_20161024064533.pcap
Oct 24 06:45:39 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04250_20161024064539.pcap
Oct 24 06:45:47 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04251_20161024064547.pcap
Oct 24 06:46:43 pktcap4 dumpcap[26968]: File: /opt/ringbufcap1/fw_04252_20161024064643.pcap
Oct 24 06:46:59 pktcap4 systemd[1]: Stopping NST Ring Buffer dumpcap Service on firewall...
Oct 24 06:46:59 pktcap4 dumpcap[26968]: [26B blob data]
Oct 24 06:46:59 pktcap4 dumpcap[26968]: Packets received/dropped on interface 'fw0': 9319391/2722 (pcap:2722/dumpcap:0/flushed:0/ps_ifdrop:0) (100.0%)
Oct 24 06:46:59 pktcap4 systemd[1]: Stopped NST Ring Buffer dumpcap Service on firewall.

The dumpcap Ring Buffer Capture session: "/etc/nstringbufcap.d/firewall" running under the
the control of Systemd Service: "ringbuf-dumpcap@firewall.service" has stopped normally.
=========================================================================================

[root@probe ~]#

 

Mode: list

This mode will list the content of each ring buffer directory for one or more installed capture sessions. A total of "20" capture component files are listed as part of the ring buffer that was installed above for session: "firewall". Note that there were 4,252 capture files created for this session and only the last "20" files of size "2MB" each were preserved. You can see the "40MB" size of the ring buffer capture window. The last file: "fw_04252_20161024064643.pcap" was short of the "2MB" size due to the early termination of the firewall capture session.

[root@probe ~]# /usr/bin/nstringbufcap -m list -s firewall -v;

firewall (dumpcap): Ring Buffer Capture Directory: "/opt/ringbufcap1"
=========================================================================================
/bin/ls -Al "/opt/ringbufcap1";
total 37392
-rw------- 1 root root 2000596 Oct 24 06:39 fw_04233_20161024063942.pcap
-rw------- 1 root root 2000840 Oct 24 06:39 fw_04234_20161024063952.pcap
-rw------- 1 root root 2001180 Oct 24 06:40 fw_04235_20161024063958.pcap
-rw------- 1 root root 2001540 Oct 24 06:40 fw_04236_20161024064005.pcap
-rw------- 1 root root 2000084 Oct 24 06:40 fw_04237_20161024064025.pcap
-rw------- 1 root root 2001172 Oct 24 06:41 fw_04238_20161024064054.pcap
-rw------- 1 root root 2000476 Oct 24 06:42 fw_04239_20161024064147.pcap
-rw------- 1 root root 2000340 Oct 24 06:42 fw_04240_20161024064206.pcap
-rw------- 1 root root 2000664 Oct 24 06:42 fw_04241_20161024064217.pcap
-rw------- 1 root root 2000812 Oct 24 06:42 fw_04242_20161024064227.pcap
-rw------- 1 root root 2000616 Oct 24 06:43 fw_04243_20161024064240.pcap
-rw------- 1 root root 2000028 Oct 24 06:43 fw_04244_20161024064310.pcap
-rw------- 1 root root 2000812 Oct 24 06:44 fw_04245_20161024064339.pcap
-rw------- 1 root root 2000468 Oct 24 06:44 fw_04246_20161024064401.pcap
-rw------- 1 root root 2001344 Oct 24 06:44 fw_04247_20161024064430.pcap
-rw------- 1 root root 2000460 Oct 24 06:45 fw_04248_20161024064459.pcap
-rw------- 1 root root 2000048 Oct 24 06:45 fw_04249_20161024064533.pcap
-rw------- 1 root root 2001184 Oct 24 06:45 fw_04250_20161024064539.pcap
-rw------- 1 root root 2001004 Oct 24 06:46 fw_04251_20161024064547.pcap
-rw------- 1 root root  230000 Oct 24 06:46 fw_04252_20161024064643.pcap
-----------------------------------------------------------------------------------------

Total *.pcap files: 20
=========================================================================================

[root@probe ~]#

 

Mode: show

This mode will show detailed information for one or more ring buffer installed capture sessions.

[root@probe ~]# /usr/bin/nstringbufcap -m show -s firewall -v;

Ring Buffer Capture Session Environment Files: "/etc/nstringbufcap.d"
=========================================================================================
/bin/ls -Al "/etc/nstringbufcap.d";
total 4
-rw-r--r-- 1 root root 479 Oct 21 14:32 firewall

firewall: Ring Buffer Capture Session Environment File: "/etc/nstringbufcap.d/firewall"
=========================================================================================

Configuration: "firewall"
-----------------------------------------------------------------------------------------
/bin/cat "/etc/nstringbufcap.d/firewall";
#
# An nstringbufcap environment file (Installed on: Fri Oct 21 14:32:14 EDT 2016).
#
# Capture Tool: dumpcap
#
# Ring Buffer File Count: 20
#
# Required Syntax:
#   1) Make sure the entire OPTIONS variable value is enclosed
#      in double quotes (").
#
#   2) Make sure the Ring Buffer directory (-w) and/or the Capture Filter
#      expression (-f) parameters are enclosed in single quotes (').

OPTIONS="-q -i fw0 -b filesize:2000 -b files:20 -w '/opt/ringbufcap1/fw.pcap'"
-----------------------------------------------------------------------------------------

firewall (dumpcap): Ring Buffer Capture Directory: "/opt/ringbufcap1"
-----------------------------------------------------------------------------------------
/bin/ls -Al "/opt/ringbufcap1";
total 37392
-rw------- 1 root root 2000596 Oct 24 06:39 fw_04233_20161024063942.pcap
-rw------- 1 root root 2000840 Oct 24 06:39 fw_04234_20161024063952.pcap
-rw------- 1 root root 2001180 Oct 24 06:40 fw_04235_20161024063958.pcap
-rw------- 1 root root 2001540 Oct 24 06:40 fw_04236_20161024064005.pcap
-rw------- 1 root root 2000084 Oct 24 06:40 fw_04237_20161024064025.pcap
-rw------- 1 root root 2001172 Oct 24 06:41 fw_04238_20161024064054.pcap
-rw------- 1 root root 2000476 Oct 24 06:42 fw_04239_20161024064147.pcap
-rw------- 1 root root 2000340 Oct 24 06:42 fw_04240_20161024064206.pcap
-rw------- 1 root root 2000664 Oct 24 06:42 fw_04241_20161024064217.pcap
-rw------- 1 root root 2000812 Oct 24 06:42 fw_04242_20161024064227.pcap
-rw------- 1 root root 2000616 Oct 24 06:43 fw_04243_20161024064240.pcap
-rw------- 1 root root 2000028 Oct 24 06:43 fw_04244_20161024064310.pcap
-rw------- 1 root root 2000812 Oct 24 06:44 fw_04245_20161024064339.pcap
-rw------- 1 root root 2000468 Oct 24 06:44 fw_04246_20161024064401.pcap
-rw------- 1 root root 2001344 Oct 24 06:44 fw_04247_20161024064430.pcap
-rw------- 1 root root 2000460 Oct 24 06:45 fw_04248_20161024064459.pcap
-rw------- 1 root root 2000048 Oct 24 06:45 fw_04249_20161024064533.pcap
-rw------- 1 root root 2001184 Oct 24 06:45 fw_04250_20161024064539.pcap
-rw------- 1 root root 2001004 Oct 24 06:46 fw_04251_20161024064547.pcap
-rw------- 1 root root  230000 Oct 24 06:46 fw_04252_20161024064643.pcap
-----------------------------------------------------------------------------------------

firewall: Systemd Service Capture Status: "ringbuf-dumpcap@firewall.service"
-----------------------------------------------------------------------------------------
/bin/systemctl --no-pager --full status "ringbuf-dumpcap@firewall.service";
● ringbuf-dumpcap@firewall.service - NST Ring Buffer dumpcap Service on firewall
   Loaded: loaded (/usr/lib/systemd/system/ringbuf-dumpcap@.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:dumpcap(1)
-----------------------------------------------------------------------------------------
=========================================================================================

[root@probe ~]#