SUMMARY

Note: This document is part of the libpcap Git and was derived from 'pcap.3' (circa Aug/07).

The ACN provides a customized/distributed version of this library that allows SMPs to interact with the various IOPs within the site providing a standard mechanism to capture LAN and WAN message traffic.

SMP	The Supervisory Management Processor where Wireshark (or equivalent) runs in conjunction with a libpcap front-end.
IOP	I/O Processors where the monitored ports exist in conjunction with a custom device driver/libpcap back-end.

Each IOP will be capable of supporting multiple connections from an SMP enabling monitoring of more than one interface at a time, each through its own separate connection. The IOP is responsible to ensure and report an error if any attempt is made to monitor the same interface more than once.

There are three applications that will be supported by the ACN version of libpcap. They each use a slightly different mode for looping/capturing and termination as summarized in the following table:

Application	Capture	Termination
wireshark	pcap_dispatch(all packets in one buffer of capture only)	pcap_breakloop()
tshark	pcap_dispatch(one buffer of capture only)	Since a CTRL-C was used to terminate the application, pcap_breakloop() is never called.
tcpdump	pcap_loop(all packets in the next buffer, and loop forever)	pcap_breakloop()

Note: In all cases, the termination of capturing is always (apparently) followed by pcap_close(). Pcap_breakloop() is only used to stop/suspend looping/processing, and upon close interpretation of the function definitions, it is possible to resume capturing following a pcap_breakloop() without any re-initialization.

ACN Limitations

1. Monitoring of backup IOPs is not currently supported.
2. Ethernet interfaces cannot be monitored in promiscuous mode.

ROUTINES

The following list of functions is the sub-set of Pcap functions that have been altered/enhanced to support the ACN remote monitoring facility. The remainder of the Pcap functions continue to perform their duties un-altered. Libpcap only supports this mode of operation if it has been configured/compiled for SITA/ACN support.
pcap_findalldevs
pcap_freealldevs
pcap_open_live
pcap_close
pcap_setfilter
pcap_dispatch
pcap_loop
pcap_next
pcap_next_ex
pcap_stats

These subroutines have been modified for the ACN specific distributed and remote monitoring ability perform the following basic functions. More detail is provided in the "SMP/IOP Inter-Process Communication Protocol" section.

pcap_open_live()	Used to obtain a packet capture descriptor to look at packets on the network.
	SMP -> IOP The SMP will open a connection to the selected IOP on its 'sniffer' port to ensure it is available. It sends a null terminated string identifying the interface to be monitored. IOP -> SMP After any required processing is complete, the IOP will return a null terminated string containing an error message if one occurred. If no error occurred, a empty string is still returned. Errors are: "Interface (xxx) does not exist." "Interface (xxx) not configured." "Interface (xxx) already being monitored."
pcap_findalldevs()	It constructs a list of network devices that can be opened with pcap_open_live().
	SMP It obtains a list of IOPs currently available (via /etc/hosts). SMP -> IOP The SMP will sequentially open a connection to each IOP on its 'sniffer' port to ensure the IOP is available. It sends a null terminated empty interface ID followed by the query request command. IOP -> SMP The IOP returns an error response and its list of devices. SMP -> IOP The SMP closes the TCP connection with each IOP. SMP The SMP adds the received information to its internal structure.
pcap_freealldevs()	Used to free a list allocated by pcap_findalldevs().
	SMP The SMP frees the structure it built as a result of the previous invocation of pcap_findalldevs().
pcap_dispatch()	Used to collect and process packets.
	SMP -> IOP On the first invocation of pcap_dispatch(), pcap_loop(), or pcap_next(), or pcap_next_ex() following a pcap_open_live(), the SMP will pass down the monitor start command and various parameters the IOP should use. IOP -> SMP The IOP now sends a stream of captured data. SMP The SMP will read the reverse channel of the connection between the SMP and the IOP that provides the captured data (via 'p->read_op' which is 'pcap_read_linux()' until the select() call returns a 'no more data' indication. It will the process (at most) the next 'cnt' packets and invoke the specified callback function for each packet processed. IOP The IOP continues to listen for additional commands as well as capturing and forwarding data to the SMP.
pcap_loop()	Is similar to pcap_dispatch() except it keeps reading packets until the requested number of packets are processed or an error occurs.
	SMP -> IOP On the first invocation of pcap_dispatch(), pcap_loop(), or pcap_next(), or pcap_next_ex() following a pcap_open_live(), the SMP will pass down the monitor start command and various parameters the IOP should use. IOP -> SMP The IOP now sends a stream of captured data. SMP The SMP continuously reads the next packet from the reverse channel of the connection between the SMP and the IOP that provides the captured data (via 'p->read_op' which is 'pcap_read_linux()' until 'cnt' packets have been received. The specified callback function will be invoked for each packet received. IOP The IOP continues to listen for additional commands as well as capturing and forwarding data to the SMP.
pcap_next()	It reads the next packet (by calling pcap_dispatch() with a count of 1) and returns a pointer to the data in that packet.
	SMP -> IOP On the first invocation of pcap_dispatch(), pcap_loop(), or pcap_next(), or pcap_next_ex() following a pcap_open_live(), the SMP will pass down the monitor start command and various parameters the IOP should use. IOP -> SMP The IOP now sends a stream of captured data. SMP The SMP reads only the next packet from the reverse channel of the connection between the SMP and the IOP that provides the captured data (via calling pcap_dispatch() with a count of 1) and returns a pointer to that data by invoking an internal callback. IOP The IOP continues to listen for additional commands as well as capturing and forwarding data to the SMP.
pcap_next_ex()	Reads the next packet and returns a success/failure indication.
	SMP -> IOP On the first invocation of pcap_dispatch(), pcap_loop(), or pcap_next(), or pcap_next_ex() following a pcap_open_live(), the SMP will pass down the monitor start command and various parameters the IOP should use. IOP -> SMP The IOP now sends a stream of captured data. SMP The SMP reads only the next packet from the reverse channel of the connection between the SMP and the IOP that provides the captured data (via calling pcap_dispatch() with a count of 1) and returns separate pointers to both the packet header and packet data by invoking an internal callback. IOP The IOP continues to listen for additional commands as well as capturing and forwarding data to the SMP.
pcap_setfilter()	Used to specify a filter program.
	SMP -> IOP The SMP sends a 'set filter' command followed by the BPF commands. IOP -> SMP The IOP returns a null terminated error string if it failed to accept the filter. If no error occurred, then a NULL terminated empty string is returned instead. Errors are: "Invalid BPF." "Insufficient resources for BPF."
pcap_stats()	Fills in a pcap_stat struct with packet statistics.
	SMP -> IOP The SMP sends a message to the IOP requesting its statistics. IOP -> SMP The IOP returns the statistics. SMP The SMP fills in the structure provided with the information retrieved from the IOP.
pcap_close()	Closes the file and deallocates resources.
	SMP -> IOP The SMP closes the file descriptor, and if the descriptor is that of the communication session with an IOP, it too is terminated. IOP If the IOP detects that its communication session with an SMP has closed, it will terminate any monitoring in progress, release any resources and close its end of the session. It will not maintain persistence of any information or prior mode of operation.

SMP/IOP Inter-Process Communication Protocol

• Communications between an SMP and an IOP consists of a TCP session between an ephemeral port on the SMP and the well known port of 49152 (which is the first available port in the 'dynamic and/or private port' range) on an IOP.

• Following a TCP open operation the IOP receives a null terminated 'interface ID' string to determine the type of operation that follows:

• Every command received by an IOP implies a 'stop trace/stop forwarding' operation must occur before executing the received command.

• A session is closed when the SMP closes the TCP session with the IOP. Obviously monitoring and forwarding is also stopped at that time. Note: All multi-octet entities are sent in network neutral order.

  pcap_findalldevs()	SMP -> IOP	Open socket (to each IOP), and sends: Name/ Purpose Size (in bytes) Description Interface ID 1 A NULL to indicate an empty 'interface ID'.
  	IOP -> SMP	Send its (possibly empty) NULL terminated error response string.
  	SMP -> IOP	Sends the 'interface query request': Name/ Purpose Size (in bytes) Description Interface ID 1 A 'Q' (indicating 'interface query request').
  	IOP -> SMP	The IOP returns a list of sequences of information as defined by the return parameter of this function call (as shown in the following table). Elements are specified by providing an unsigned byte preceding the actual data that contains length information. Notes: Name/ Purpose Size (in bytes) Description   length 1 The number of octets in the name field that follows. Name 1-255 The name of the interface. The format of the name is an alphabetic string (indicating the type of interface) followed by an optional numeric string (indicating the interface's sequence number). Sequence numbers (if needed) will begin at zero and progress monotonically upwards. (i.e. 'eth0', 'lo', 'wan0', etc.) For an IOP, the alphabetic string will be one of: 'eth', 'wan', and 'lo' for Ethernet, WAN ports and the IP loopback device respectively. An IOP currently supports: 'eth0', 'eth1', 'lo', 'wan0' ... 'wan7'. Note: IOPs and ACNs will not currently support the concept of 'any' interface. length 1 The number of octets in the interface description field that follows. Interface Description 0-255 A description of the interface or it may be an empty string. (i.e. 'ALC') Interface Type 4 The type of interface as defined in the description for pcap_datalink() (in network neutral order). Loopback Flag 1 1 = if the interface is a loopback interface, zero = otherwise. count 1 # of address entries that follow. Each entry is a series of bytes in network neutral order. See the parameter definition above for more details. Repeated 'count' number of times. length 1 The number of octets in the address field that follows. Address 1-255 The address of this interface (in network neutral order). length 1 The number of octets in the netmask field that follows. Network Mask 0-255 The network mask used on this interface (if applicable) (in network neutral order). length 1 The number of octets in the broadcast address field that follows. Broadcast Address 0-255 The broadcast address of this interface (if applicable) (in network neutral order). length 1 The number of octets in the destination address field that follows. Destination Address 0-255 The destination address of this interface (if applicable) (in network neutral order).
  	SMP -> IOP	Close the socket.
  	IOP -> SMP	Close the socket.
  pcap_open_live()	SMP -> IOP	Open socket, and sends: Name/ Purpose Size (in bytes) Description Interface ID 'n' 'n' octets containing a NULL terminated interface name string.
  	IOP -> SMP	Send its NULL terminated error response string.
  pcap_dispatch() pcap_loop() pcap_next() pcap_next_ex()	SMP -> IOP	On the first invocation following a pcap_open_live() or pcap_breakloop() additional information is sent: Name/ Purpose Size (in bytes) Description command 1 'M' (indicating 'monitor start') snaplen 4 snaplen timeout 1 timeout value (in milliseconds) promiscuous 1 A flag indicating that the interface being monitored show operate in promiscuous mode. [off(0) / on(NZ)] direction 1 A flag indicating the direction of traffic that should be captured [both(0) / in(1) / out(2)]
  	IOP -> SMP	Sends captured packets.
  pcap_setfilter()	SMP -> IOP	At any time, the SMP can issue a set filter command which contains an indicator, a count of the number of statements in the filter, followed by the sequence of filter commands represented as a sequence of C-style structures. Name/ Purpose Size (in bytes) Description command 1 'F' (indicating 'filter') count 4 The number of command in the Berkeley Packet Filter that follow. BPF program 'n' 8 bytes of each command (repeated 'n' times). Each command consists of that C-style structure which contains: Name/ Purpose Size (in bytes) Description opcode 2 The command's opcode. 'jt' 1 The 'jump if true' program counter offset. 'jf' 1 The 'jump if false' program counter offset. 'k' 4 The 'other' data field. Refer to the bpf(4) man page for more details.
  	IOP -> SMP	In return the IOP will send its (possibly empty) NULL terminated error response string.
  pcap_stats()	SMP -> IOP	At any time, the SMP can issue a 'retrieve statistics' command which contains: Name/ Purpose Size (in bytes) Description command 1 'S' (indicating 'request statistics')
  	IOP -> SMP	In return the IOP will send: Name/ Purpose Size (in bytes) Description ps_recv 4 The number of packets that passed the filter. ps_drop 4 The number of packets that were dropped because the input queue was full, regardless of whether they passed the filter. ps_ifdrop 4 The number of packets dropped by the network interface (regardless of whether they would have passed the input filter).
  pcap_close()	SMP -> IOP	At any time, the SMP can close the TCP session with the IOP.

Interface ID Naming Convention

Each interface within an IOP will be referred to uniquely. Since an currently contains 8 monitorable WAN ports and a monitorable Ethernet port, the naming convention is:

Interface #	Type	Name
1	WAN	wan0
2	WAN	wan1
3	WAN	wan2
4	WAN	wan3
5	WAN	wan4
6	WAN	wan5
7	WAN	wan6
8	WAN	wan7
9	Ethernet	eth0
10	Ethernet	eth1

Packet Trace Data Format

The format of the trace data that is sent to the SMP follows a portion of the libpcap file format and is summarized here. This format specifies the generic requirements needed to be able to decode packets, but does not cover ACN specifics such as custom MAC addressing and WAN protocol support.

Although a libpcap file begins with a global header followed by zero or more records for each captured packet, trace data sent to the SMP does NOT begin with a global header. A trace sequence looks like this:

[Packet Header]

[Packet Data]

[Packet Header]

[Packet Data]

[Packet Header]

[Packet Data]

...

Packet Header

Each captured packet starts with a header that contains the following values (in network neutral order):

 uint32 tv_sec;  /* timestamp seconds */
 uint32 tv_usec; /* timestamp microseconds */
 uint32 caplen;  /* number of octets in the following packet */
 uint32 len;     /* original length of packet on the wire */
		

tv_sec	The date and time when this packet was captured. This value is in seconds since January 1, 1970 00:00:00 GMT; this is also known as a UN*X time_t. You can use the ANSI C time() function from time.h to get this value, but you might use a more optimized way to get this timestamp value.
tv_usec	The microseconds when this packet was captured, as an offset to ts_sec. Beware: this value must never reach 1 second (1,000,000), in this case ts_sec must be increased instead!
caplen	The number of bytes actually provided in the capture record. This value should never become larger than len or the snaplen value specified during the capture.
len	The length of the packet "on the wire" when it was captured. If caplen and len differ, the actually saved packet size was limited by the value of snaplen specified during one of the capture directives such as pcap_dispatch().

Packet Data

The actual packet data will immediately follow the packet header as a sequence of caplen octets. Depending on the DLT encoding number assigned to the interface, the packet data will contain an additional custom header used to convey WAN port related information.

ACN Custom Packet Header

PCAP, Wireshark and Tcpdump enhancements have been added to the ACN to support monitoring of its ports, however each of these facilities were focused on capturing and displaying traffic from LAN interfaces. The SITA extensions to these facilities are used to also provide the ability to capture, filter, and display information from an ACN's WAN ports.

Although each packet follows the standard libpcap format, since there are two types of interfaces that can be monitored, the format of the data packet varies slightly.

• For Ethernet (like) devices, the packet format is unchanged from the standard Pcap format.
• For WAN devices, the packet contains a 5 byte header that precedes the actual captured data described by the following table:

Octet	Name	Mask/Value	Definition
0	Control / Status	xxxxxxx0	Transmitted by capture device	(see 'Errors' octets)
		xxxxxxx1	Received by capture device
		1xxxxxxx	No buffer was available during capture of previous packet.
1	Signals	xxxxxxx1	DSR asserted
		xxxxxx1x	DTR asserted
		xxxxx1xx	CTS asserted
		xxxx1xxx	RTS asserted
		xxx1xxxx	DCD asserted
		xx1xxxxx	Undefined
		x1xxxxxx	Undefined
		1xxxxxxx	Undefined
2	Errors (octet 1)		Tx	Rx
		xxxxxxx1	Underrun	Framing
		xxxxxx1x	CTS Lost	Parity
		xxxxx1xx	UART Error	Collision
		xxxx1xxx	Re-Tx Limit Reached	Long Frame
		xxx1xxxx	Undefined	Short Frame
		xx1xxxxx	Undefined	Undefined
		x1xxxxxx	Undefined	Undefined
		1xxxxxxx	Undefined	Undefined
3	Errors (octet 2)		Tx	Rx
		xxxxxxx1	Undefined	Non-Octet Aligned
		xxxxxx1x	Undefined	Abort Received
		xxxxx1xx	Undefined	CD Lost
		xxxx1xxx	Undefined	Digital PLL Error
		xxx1xxxx	Undefined	Overrun
		xx1xxxxx	Undefined	Frame Length Violation
		x1xxxxxx	Undefined	CRC Error
		1xxxxxxx	Undefined	Break Received
4	Protocol	0x01 - LAPB (BOP)   0x02 - Ethernet 1 0x03 - Async (Interrupt IO)   0x04 - Async (Block IO)   0x05 - IPARS   0x06 - UTS   0x07 - PPP (HDLC)   0x08 - SDLC   0x09 - Token Ring 1 0x10 - I2C   0x11 - DPM Link   0x12 - Frame Relay (BOP)   Note 1: Ethernet and Token Ring frames will never be sent as DLT_SITA (with the 5 octet header), but will be sent as their corresponding DLT types instead.