TCP Packet Processing

Let’s see how Snort++ processes a TCP Flow. There are some similarities with how Snort++ processes a UDP Flow but a few significant differences as well. Specifically, we’ll look at an HTTP connection
For this example, I show a GET request for a test page on my web site that contains some text and a GIF (a photo of me) .
The “S” flag indicates that the SYN flag is set. The source port is 58186 and the destination port is 80 (http). The initial SEQ numbers (isn) are 903853442 and 135109633.
The command executed to get the output below was:
andy@laptop:/var/www/html/images$ sudo tcpdump -n host 45.40.183.129 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 ...
:
name:

website:

comment:


SYN packet (no payload)

(SYN) 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 (SYN/ACK) 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 (ACK) 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 (CLIENT REQUEST) 11:43:09.568478 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 1:335, ack 1, win 29200, length 334: HTTP: GET /snort3/test.html HTTP/1.1 (ACK for request) 11:43:09.569005 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 335, win 65535, length 0 (SERVER PAYLOAD) 11:43:09.660309 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1:369, ack 335, win 65535, length 368: HTTP: HTTP/1.1 200 OK (ACK for payload) 11:43:09.660336 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 369, win 30016, length 0 (GIF REQUEST) 11:43:09.753042 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 335:625, ack 369, win 30016, length 290: HTTP: GET /snort3/test.gif HTTP/1.1 11:43:09.753586 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 625, win 65535, length 0 (GIF PAYLOAD) 11:43:09.843565 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 369:1749, ack 625, win 65535, length 1380: HTTP: HTTP/1.1 200 OK 11:43:09.843599 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1749, win 33120, length 0 11:43:09.843853 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1749:3129, ack 625, win 65535, length 1380: HTTP 11:43:09.843863 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 3129, win 35880, length 0 11:43:09.843890 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 3129:4509, ack 625, win 65535, length 1380: HTTP 11:43:09.843896 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 4509, win 38640, length 0 11:43:09.843986 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 4509:10029, ack 625, win 65535, length 5520: HTTP 11:43:09.844190 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 10029:11665, ack 625, win 65535, length 1636: HTTP 11:43:09.844199 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 11665, win 52540, length 0 (TCP TEARDOWN) 11:43:14.843079 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [F.], seq 11665, ack 625, win 65535, length 0 11:43:14.843494 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [F.], seq 625, ack 11666, win 52540, length 0 11:43:14.843885 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 626, win 65535, length 0
Here's the packet-processing pipe for the TCP SYN packet:
There are similarities with the initial UDP packet:
The SYN packet is looked at by the Codecs and then inspected by the packet Inspectors, stream Inspectors
StreamBase is the only stream Inspector whose eval() method does something. The other stream Inspectors (e.g., StreamIp) are used for configuration purposes but their eval() methods do nothing (e.g., StreamIp).
, the binder Inspector (which we’ll refer to as Binder from here on), and the probe Inspectors - just as was done for the first packet in a UDP Flow. The wizard Inspector (which we’ll refer to as the Wizard from here on), the service Inspector, detect (the Rule Detection Engine), and the second invocation of Binder, however, are all missing in the TCP SYN pipe. So why are they missing? On the other hand, the network Inspectors are new. What role do the network Inspectors play?
First let’s talk about what is missing and why it’s missing.
Binder and Wizard both attempt to determine the appropriate service Inspector for a Flow. Binder looks through its database for static entries and, if there are no matching entries, invokes Wizard to look for identifying tokens in a packet’s payload. Let’s look at the default bindings array again.
Since there are no static entries that match with the TCP ports (58186 or 80) of the packet, Binder cannot determine the appropriate service Inspector by itself. The Wizard must be invoked by Binder to look for identifying tokens in a Flow’s payload in an attempt to determine the appropriate service Inspector. Since a SYN packet typically
It is apparently possible for a SYN packet from MacOS to have payload. For our example, we assume that the SYN packet contains no payload.
contains no payload, the Wizard is not invoked
It’s a little more complicated than that. Even if there was payload in the packet, the Wizard still wouldn’t be invoked to determine the appropriate service Inspector for the Flow because Snort++ waits for the ACK packet that acknowledges the payload before it invokes the Wizard to look for tokens in the payload. We’ll discuss this in detail in a moment.
. Since neither Binder nor Wizard were able to determine the appropriate service Inspector for the Flow, no service Inspector is invoked.
The Rule Detection Engine (Detect) compares a packet’s payload with Snort++ rules' fast-pattern content (which are compiled into MPSEs). Since the SYN packet for our example has no payload, The Rule Detection Engine is not invoked.
The network Inspectors are invoked on any packet from a Flow for which the appropriate service Inspector has not been determined (i.e., the service of its Flow has not been set - we'll discuss this soon
The network Inspectors appear to be a miscellaneous collection of Inspectors. I suspect that a Inspector is typically categorized as a network Inspector because it doesn't fit elsewhere (for a number of reasons). For example, I’m guessing that the identification of a Flow as a BackOrifice Flow is too complex for the Wizard (which simply looks for strings in a packet that reveal a protocol – e.g. “GET” for the HTTP protocol). I’m also guessing that checking for ARP spoofing was regarded as too computationally expensive to put inside the ArpCodec Codec. Recall that Codecs are unconditionally invoked for the Codec’s appropriate packets (i.e., ArpCodec for ARP packets) whereas Inspectors must be enabled/configured in snort.lua or they will never be invoked. Since most Snort++ servers are placed between a firewall and router and not on a typical user LAN where most ARP spoofing occurs, it’s reasonable to expect that a Snort++ administrator can turn off the computationally expensive check for ARP spoofing.
.)
Appropriately enough, TcpSession handles a TCP Flow in the same fashion as UdpSession handled a UDP Flow. However, unlike UdpSession (which simply invokes the Wizard), TcpSession has a number of important duties. Since UDP is stateless, there’s not much for UdpSession to do other than to set some flags in the Flow. TcpSession, on the other hand, has a lot to do, getting help from the TcpTrackers and TcpReassemblers classes. TcpSession must verify that the TCP sequence numbers in the initial 3-way handshake make sense and also verify that all subsequent TCP packets in the Flow are reasonable (TcpTracker) and it must reassemble the packets from the Flows into pseudo packets (TcpReassembler with help from the service Inspector’s Splitter - e.g., HttpStreamSplitter - we’ll discuss this later).
Each TcpSession has two TcpTrackers, one for the client and one for the server. These Trackers keep track of their respective state in the TCP state machine
It’s important to understand that this diagram is actually two state diagrams (albeit interrelated), one for the client and one for the server. Next to each arrow are two strings separated by a forward slash. For example, the top-most blue arrow has “SYN/SYN+ACK”. This indicates that the server (because it’s a blue arrow - red arrows are for the client) moves from the “LISTEN” state to the “SYN RECEIVED” state if it receives a “SYN” and then sends out a “SYN+ACK”.
:
The client TcpTracker begins in state TCP_STATE_NONE (which corresponds with “CLOSED” in the diagram) and the server TcpTracker begins in state TCP_STATE_LISTEN. Each state has its own set of methods that handle different scenarios. For example, a SYN packet sent from a client whose Tracker is in the TCP_STATE_NONE state (the normal scenario) should be handled differently from a SYN packet sent from a client whose Tracker is in the TCP_STATE_ESTABLISHED state (an abnormal scenario). As TcpSession processes a packet, it uses the TcpStateMachine
to handle these differences. Each TcpStateHandler defines its overridden methods differently. For example, TcpStateListen defines syn_sent() differently than TcpStateEstablished, which result in the different treatment (i.e., handling) for the different scenarios described above (as it should of course be).
For the SYN packet, the TcpTrackers (client and server) just take note of the ISN (Initial Sequence Number) and advance to the appropriate state (TCP_SYN_SENT for the client and TCP_SYN_RECV for the server)
Snort++ makes some assumptions. Specifically, when Snort++ sees a SYN packet go to the server, it puts the server into the TCP_SYN_RECV state. If you look at the TCP state machine diagram above, you’ll see that for the server to move to the “SYN RECEIVED” state, it must receive a SYN and send out an SYN+ACK. Since Snort++ can’t know exactly what is happening with the server as it’s happening, it has to be either too early to move the server into the TCP_SYN_RECV (as is the case) or too late (which would be the case if Snort++ waited to see the SYN+ACK packet).
.
name:

website:

comment:


SYN/ACK packet (no payload)

(SYN) 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 (SYN/ACK) 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 (ACK) 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 (CLIENT REQUEST) 11:43:09.568478 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 1:335, ack 1, win 29200, length 334: HTTP: GET /snort3/test.html HTTP/1.1 (ACK for request) 11:43:09.569005 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 335, win 65535, length 0 (SERVER PAYLOAD) 11:43:09.660309 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1:369, ack 335, win 65535, length 368: HTTP: HTTP/1.1 200 OK (ACK for payload) 11:43:09.660336 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 369, win 30016, length 0 (GIF REQUEST) 11:43:09.753042 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 335:625, ack 369, win 30016, length 290: HTTP: GET /snort3/test.gif HTTP/1.1 11:43:09.753586 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 625, win 65535, length 0 (GIF PAYLOAD) 11:43:09.843565 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 369:1749, ack 625, win 65535, length 1380: HTTP: HTTP/1.1 200 OK 11:43:09.843599 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1749, win 33120, length 0 11:43:09.843853 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1749:3129, ack 625, win 65535, length 1380: HTTP 11:43:09.843863 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 3129, win 35880, length 0 11:43:09.843890 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 3129:4509, ack 625, win 65535, length 1380: HTTP 11:43:09.843896 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 4509, win 38640, length 0 11:43:09.843986 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 4509:10029, ack 625, win 65535, length 5520: HTTP 11:43:09.844190 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 10029:11665, ack 625, win 65535, length 1636: HTTP 11:43:09.844199 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 11665, win 52540, length 0 (TCP TEARDOWN) 11:43:14.843079 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [F.], seq 11665, ack 625, win 65535, length 0 11:43:14.843494 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [F.], seq 625, ack 11666, win 52540, length 0 11:43:14.843885 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 626, win 65535, length 0
The pipe for a SYN/ACK packet is very similar to the SYN packet. This time, however, Binder doesn’t look at the packet. Why not? For the SYN packet, the Binder looked in its bindings[] array for an entry that matches the source and/or destination IP addresses and/or TCP ports of the packet. It doesn’t need to do this a second time since all of the information was available for the SYN packet. Again, the Wizard isn’t invoked because there is no payload in the SYN/ACK packet for the Wizard to inspect.
The TcpSession again makes sure that the SEQ numbers of the TCP packet make sense. Note that there are two sequence numbers (one in each direction) that Snort++ must track. Furthermore, the SYN/ACK packet acknowledges (with the ACK) the SEQ number from the SYN packet (it actually adds one). The client and server TcpTrackers verify that these sequence numbers look good.
The SYN/ACK packet is also not run through the Rules Detection Engine (detect) since there is no payload.
The state of the client and server TcpTracker does not change.
name:

website:

comment:


ACK packet (no payload)

(SYN) 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 (SYN/ACK) 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 (ACK) 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 (CLIENT REQUEST) 11:43:09.568478 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 1:335, ack 1, win 29200, length 334: HTTP: GET /snort3/test.html HTTP/1.1 (ACK for request) 11:43:09.569005 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 335, win 65535, length 0 (SERVER PAYLOAD) 11:43:09.660309 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1:369, ack 335, win 65535, length 368: HTTP: HTTP/1.1 200 OK (ACK for payload) 11:43:09.660336 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 369, win 30016, length 0 (GIF REQUEST) 11:43:09.753042 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 335:625, ack 369, win 30016, length 290: HTTP: GET /snort3/test.gif HTTP/1.1 11:43:09.753586 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 625, win 65535, length 0 (GIF PAYLOAD) 11:43:09.843565 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 369:1749, ack 625, win 65535, length 1380: HTTP: HTTP/1.1 200 OK 11:43:09.843599 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1749, win 33120, length 0 11:43:09.843853 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1749:3129, ack 625, win 65535, length 1380: HTTP 11:43:09.843863 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 3129, win 35880, length 0 11:43:09.843890 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 3129:4509, ack 625, win 65535, length 1380: HTTP 11:43:09.843896 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 4509, win 38640, length 0 11:43:09.843986 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 4509:10029, ack 625, win 65535, length 5520: HTTP 11:43:09.844190 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 10029:11665, ack 625, win 65535, length 1636: HTTP 11:43:09.844199 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 11665, win 52540, length 0 (TCP TEARDOWN) 11:43:14.843079 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [F.], seq 11665, ack 625, win 65535, length 0 11:43:14.843494 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [F.], seq 625, ack 11666, win 52540, length 0 11:43:14.843885 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 626, win 65535, length 0
An ACK packet can have a payload but this isn’t a requirement. We look at the case where there is no payload since it’s easier to explain. The main thing that TcpTracker does is to verify that the sequence and acknowledgment numbers are valid
After the initial SYN packet was sent, Snort++ changed the state of the server to TCP_SYN_RECEIVED. This was a little presumptuous because Snort++ at this time cannot be sure that the server would actually receive the SYN packet. Here again, Snort++ is presumptuous in changing the server’s state because it can’t be certain that the server will receive the packet.
.
name:

website:

comment:


First packet with payload

(SYN) 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 (SYN/ACK) 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 (ACK) 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 (CLIENT REQUEST) 11:43:09.568478 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 1:335, ack 1, win 29200, length 334: HTTP: GET /snort3/test.html HTTP/1.1 (ACK for request) 11:43:09.569005 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 335, win 65535, length 0 (SERVER PAYLOAD) 11:43:09.660309 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1:369, ack 335, win 65535, length 368: HTTP: HTTP/1.1 200 OK (ACK for payload) 11:43:09.660336 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 369, win 30016, length 0 (GIF REQUEST) 11:43:09.753042 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 335:625, ack 369, win 30016, length 290: HTTP: GET /snort3/test.gif HTTP/1.1 11:43:09.753586 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 625, win 65535, length 0 (GIF PAYLOAD) 11:43:09.843565 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 369:1749, ack 625, win 65535, length 1380: HTTP: HTTP/1.1 200 OK 11:43:09.843599 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1749, win 33120, length 0 11:43:09.843853 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1749:3129, ack 625, win 65535, length 1380: HTTP 11:43:09.843863 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 3129, win 35880, length 0 11:43:09.843890 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 3129:4509, ack 625, win 65535, length 1380: HTTP 11:43:09.843896 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 4509, win 38640, length 0 11:43:09.843986 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 4509:10029, ack 625, win 65535, length 5520: HTTP 11:43:09.844190 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 10029:11665, ack 625, win 65535, length 1636: HTTP 11:43:09.844199 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 11665, win 52540, length 0 (TCP TEARDOWN) 11:43:14.843079 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [F.], seq 11665, ack 625, win 65535, length 0 11:43:14.843494 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [F.], seq 625, ack 11666, win 52540, length 0 11:43:14.843885 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 626, win 65535, length 0
This is where things begin to get interesting. This pipeline is the same as the ACK packet until TcpSession looks at the packet. There are 2 Reassemblers for each TcpSession, arranged in a somewhat convoluted way.
The payload from the packets going from the server to the client are placed in the Reassembler associated with the client TcpTracker and the payload from the packets going from the client to server are placed in the Reassembler associated with the server TcpTracker. Therefore, the client Reassembler reflects what the client receives and the server Reassembler reflects what the server receives.
The payloads are placed in seglist, which is a linked list of TcpSegmentNodes. Packet overlaps occur when successive TCP packets contain redundant data. For example, if TCP packets from the client to server have already been seen for sequence numbers 200-3199 and a packet with sequence numbers 700-1699 appears, Snort++ must make a decision (based on the receiving Operating System - as discussed below) as to what payload to keep and what payload to discard. Each Operating System (OS) has its own way of dealing with packet overlaps. The seglist linked list reflects how overlaps are handled by the receiving Operating System

It's a little messy:

. By default, Snort++ assumes that the Operating System is BSD. The methods of TcpReassemblerBSD (see diagram above) insert TcpSegmentNodes in a manner that is consistent with the BSD Operating System
Since many organizations use Windows, it's fortunate that the BSD and Windows Reassemblers handle overlaps the same. It nevertheless strikes me as odd that TcpReassemblerBSD is chosen as the default.
.
We'll discuss "pseudo packets" in a moment but for the time being, note that a pseudo packet is not generated yet
A pseudo packet is not generated for IDS (Intrusion Detection System) mode. A pseudo packet is, however, generated for IPS (Intrusion Protection System) mode. I do not discuss IPS mode on this website.
. The generation of pseudo packets is triggered by an acknowledgment of the payload (with ACK packets). The next packet (the “Second ACK packet” below) typically (but not always since the payload might not include a "logical break" - we'll discuss logical breaks in a moment) triggers the pseudo packet that contains the payload from the first data packet.
Note that the Wizard does not look at this packet since the Wizard only looks at acknowledged payload and, therefore, the service Inspector appropriate for this packet’s Flow is not determined at this time.
For the first time in the TCP connection (since there finally is payload), the packet is run through the Rule Detection Engine (detect). We talked a bit about the Rule Detection Engine in the discussion of a UDP Flow but some of the topics are better explained in our study of a TCP Flow. I will defer this discussion, however, until later so that we can see several different scenarios and what parts of the Rule Detection Engine are used. At this point, just understand that the packet will be run through the Rule Detection Engine.
TcpTracker again verifies that the sequence and acknowledgment numbers are valid.
name:

website:

comment:


Second ACK packet (with no data)

(SYN) 11:43:09.505138 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [S], seq 903853442, win 29200, options [mss 1460,sackOK,TS val 2396269750 ecr 0,nop,wscale 7], length 0 (SYN/ACK) 11:43:09.568062 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [S.], seq 135109633, ack 903853443, win 65535, options [mss 1460], length 0 (ACK) 11:43:09.568110 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1, win 29200, length 0 (CLIENT REQUEST) 11:43:09.568478 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 1:335, ack 1, win 29200, length 334: HTTP: GET /snort3/test.html HTTP/1.1 (ACK for request) 11:43:09.569005 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 335, win 65535, length 0 (SERVER PAYLOAD) 11:43:09.660309 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1:369, ack 335, win 65535, length 368: HTTP: HTTP/1.1 200 OK (ACK for payload) 11:43:09.660336 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 369, win 30016, length 0 (GIF REQUEST) 11:43:09.753042 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [P.], seq 335:625, ack 369, win 30016, length 290: HTTP: GET /snort3/test.gif HTTP/1.1 11:43:09.753586 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 625, win 65535, length 0 (GIF PAYLOAD) 11:43:09.843565 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 369:1749, ack 625, win 65535, length 1380: HTTP: HTTP/1.1 200 OK 11:43:09.843599 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 1749, win 33120, length 0 11:43:09.843853 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 1749:3129, ack 625, win 65535, length 1380: HTTP 11:43:09.843863 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 3129, win 35880, length 0 11:43:09.843890 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 3129:4509, ack 625, win 65535, length 1380: HTTP 11:43:09.843896 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 4509, win 38640, length 0 11:43:09.843986 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 4509:10029, ack 625, win 65535, length 5520: HTTP 11:43:09.844190 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [P.], seq 10029:11665, ack 625, win 65535, length 1636: HTTP 11:43:09.844199 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [.], ack 11665, win 52540, length 0 (TCP TEARDOWN) 11:43:14.843079 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [F.], seq 11665, ack 625, win 65535, length 0 11:43:14.843494 IP 10.0.2.15.58186 > 45.40.183.129.http: Flags [F.], seq 625, ack 11666, win 52540, length 0 11:43:14.843885 IP 45.40.183.129.http > 10.0.2.15.58186: Flags [.], ack 626, win 65535, length 0
A lot happens with the second ACK packet. As discussed above, ACK packets can have payload but, in order to keep things simple, we look at the case where there is no payload. This ACK packet acknowledges receipt of the payload from the first packet with payload. TcpSession is again busy. This time, there is no data that must be placed in a TcpReassembler queue. However, this second ACK packet (which is the first ACK packet to acknowledge payload) triggers the MagicSplitter (which is the Wizard’s Splitter) to look for identifying tokens in the payload.
For a TCP Flow, TcpSession::process() is called for each packet. As previously discussed, TcpSession gets help from its TcpTrackers and TcpReassemblers. When an ACK is seen for the payload of a Flow, the client or server TcpReassembler (indirectly) invokes a Splitter. Splitters typically break payload into logical components. For example, HttpStreamSplitter attempts to pick out the HTTP request, status, headers, and body from the payload of packets in an HTTP Flow so that HttpInspect, the service Inspector, can inspect a component (e.g., header) in its entirety
It’s important to understand that the payload inspected is not the payload of the current packet but the payload of packets that have traveled in the opposite direction as the ACK. For example, if the ACK is sent by the server to the client, the payload that a Splitter will process is the payload from packets sent by the client to the server.
.
In the case of the first ACK to acknowledge payload, the appropriate Inspector has not yet been determined. That’s the task of the Wizard and its associated Splitter, MagicSplitter. MagicSplitter simply looks for tokens (typically defined in snort_defaults.lua) in the payload that indicate a service
In this introduction, my preference is to give a high-level overview of the code rather than a detailed description of the code path. However, the invocation of MagicSplitter is fairly confusing and so I’m going to show the code path.
The server Reassembler “flushes” the ACK’ed payload to the Scanner by calling the Protocol Aware Flushing (PAF) functions (paf_check(), paf_eval(), and paf_callback()). Typically, these functions would send the ACK’ed payload to the scan() method of the appropriate service Inspector’s Scanner (e.g., HttpStreamSplitter::scan()). However, if the appropriate service Inspector has not yet been determined, MagicSplitter::scan() is called.
. If the Wizard finds a token indicating a service (e.g., “GET” for the HTTP protocol), the MagicSplitter simply sets the service field of the packet’s Flow to the appropriate service (e.g., “http”). Afterwards, the Binder is invoked a second time. If MagicSplitter found a token identifying a service and set the service field of the packet’s Flow accordingly, then Binder will look for an entry in its bindings array.
Let’s say the MagicSplitter finds “GET” in the Reassembler’s queue. The MagicSplitter will then set the service field to “http”.
For the default binder configuration, Binder will find an entry for the service of “http”. So Binder sets the gadget field of the Flow to a pointer to the HttpInspect object and clouseau to NULL (this indicates that the Wizard is done with its work and should not look at any subsequent packets in this Flow).
From this point on, any packet in this Flow will not be inspected by the Binder or the Wizard (or more precisely, the Wizard’s Splitter, MagicSplitter). Instead, HttpStreamSplitter::scan() splits the payload into components (e.g., request, header) and the HttpInspect Inspector inspects these components.
For example, out of the following packet
HttpStreamSplitter will find two logical chunks, creating pseudo packets out of each.
The Reassembler then generates a pseudo packet containing the request and sends it through the packet processing pipe. Then the Reassembler generates a pseudo packet containing the header that is sent through the pipe
These are recursive. In other words, the pseudo packets are sent through the pipe before the Second ACK packet reaches the end of its pipe.
.
name:

website:

comment:


First Pseudo packet

As mentioned above, the first pseudo packet contains the HTTP request.
Since the pseudo packet was generated by Snort++ itself, there isn’t any reason to send it through the Codecs and most of the Inspectors. Since Snort++ created the headers (Ethernet, IP, TCP) for the pseudo packet, it knows that the headers are not malicious. There is also no need to send the pseudo packet through the packet Inspectors . There is also no need to send the pseudo packet through the stream Inspectors because its TCP sequence numbers were generated by Snort++ and are therefore known to be acceptable.
So the only two Inspectors that actually inspect the pseudo packet are the appropriate service Inspector (in this case, HttpInspect) and probe Inspectors.
HttpInspect inspects the HTTP request, optionally sending information to AppId. (I cover HttpInspect and AppId elsewhere.)
name:

website:

comment:


Second Pseudo Packet

The second pseudo packet goes through the same packet-processing pipe as the first pseudo packet.
We discussed the Rule Detection Engine in the UDP example. Let’s go into more detail now.
name:

website:

comment:


Rule Detection Engine

The Snort++ Rule Detection Engine is typically made up of hundreds of Multi-Pattern Search Engines (“MPSEs”) as well as their associated detection_option_tree_nodes. The “fast-pattern”
Rules can contain more than one content. If a rule has more than one content, only the best (i.e., the “fast-pattern”) content is inserted into the MPSE. I discuss in a moment what constitutes “best”.
content for each rule is inserted into one or more MPSEs. A fast-pattern content within an MPSE points to its associated detection_option_tree_root, which contains the rest of the options for the rule. We discussed detection_option_tree_nodes in our UDP example and we'll talk about the MPSEs now.
MPSEs allow Snort++ to search for thousands of rule contents simultaneously. These contents are inserted into the MPSEs during Snort++’s initialization (i.e., before processing packets).
Let’s look at a rule:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS ( msg:"BLACKLIST User-Agent known malicious User-Agent ErrCode - W32/Fujacks.htm"; flow:established,to_server; http_header; content:"User-Agent|3A| ErrCode"; metadata:policy balanced-ips drop,policy security-ips drop,service http; reference:url, www.mcafee.com/threat-intelligence/malware/default.aspx?id=141161; reference:url,www.virustotal.com/latest-report.html?resource=f9dc0803ea4634256eae73b2db61a3c5; classtype:trojan-activity; sid:18247; rev:5; )
The fast-pattern content in this rule is "User-Agent|3A| ErrCode". Snort++ must find this content in a packet whose destination port is in the set of $HTTP_PORTS or in a pseudo packet that contains an HTTP header. Furthermore, all of the other options must match (e.g., the destination of the packet must be the server - flow:to_server, etc.) for this rule to match and for an alert to be sent to an analyst.
There are several ways to implement an MPSE. The default MPSE (and the only MPSE algorithm that I document) is the Aho-Corasick state machine. Aho-Corasick state machines are built during initialization from all of the rule contents and are used to simultaneously search for multiple strings in a packet
The importance of the Aho-Corasick state machines and other MPSE algorithms cannot be overstated. Without these algorithms, Snort++ as we know it would not exist.
. In Snort++, the MPSEs are built for specific source or destination TCP or UDP ports and are built for the specific components of specific services (e.g., HTTP headers). For example, the MPSE that is built for destination port 80
HTTP_PORTS, which is defined in snort_defaults.lua, includes port 80.
includes the content from the rule above. Furthermore, the MPSE for HTTP headers will also include this content since the rule contains the http_headers option. A packet whose destination port is port 80 will be compared against the destination port 80 MPSE. If this packet contains the HTTP header, a pseudo packet will be created from the HTTP header (by HttpInspect) when the DAQ-acquired packet (containing the HTTP header) is acknowledged with an ACK packet. This pseudo packet will then be compared against the HTTP header MPSE.
From here on, I will refer to MPSEs that are built for a specific port as “port-based” and I will refer to MPSEs that are built for a specific service as “service-based”.
Note that if the appropriate service Inspector is never determined (by Binder or Wizard) for a packet, the packet and subsequent packets in the packet's Flow will never be compared against a service-based MPSE
On a related note, Snort++ rules exist that do not have contents. For example, the following rule has no content:
alert ip any any -> any any (ip_proto:igmp;)
These rules are a tiny minority of all Snort++ rules. Obviously, since these rules have no contents, the rules can’t be inserted into an MPSE since MPSEs are made up of contents. Writing such rules is discouraged because rules without contents must be evaluated individually and therefore impact performance negatively.
. Both port-based and service-based MPSEs are associated with PortGroups, which we will discuss next.
name:

website:

comment:


PortGroups

Above is a PortGroup with 6 MPSEs. DAQ-acquired packets with a payload are checked against the PM_TYPE_PKT and PM_TYPE_FILE MPSE of a port-based PortGroup . Pseudo packets (which I discussed earlier) can be compared against any of these MPSEs. For example, a pseudo packet that corresponds to an HTTP header is compared against the PM_TYPE_HEADER MPSE.
As described above, one task that TCP service Inspectors (along with their associated Splitters) perform is to break a Flow into components. The TCP service Inspector appropriate for a Flow produces pseudo packets out of these components. And out of these components, the service Inspector selects what payload to send to what MPSEs. For example, HttpInspect (along with its HttpStreamSplitter) can break an HTTP Flow into an HTTP request, an HTTP status, an HTTP header, and an HTTP body and produce a pseudo packet out of each component. HttpInspect does not, however, send the entire HTTP request to an MPSE. Instead, HttpInspect picks the URI out of the HTTP request and sends it to the PM_TYPE_KEY MPSE. HttpInspect sends
I say that HttpInspect “sends” something to an MPSE. In more concrete terms, the Rule Detection Engine queries the service Inspector for the data of specific components. If there is indeed data, HttpInspect responds with the location of this data and the Rule Detection Engine will compare the data with the relevant (e.g., PM_TYPE_HEADER) MPSE.
the HTTP header and the HTTP body to the PM_TYPE_HEADER MPSE and PM_TYPE_BODY MPSE, respectively
HttpInspect chooses to wait to send the URI from the HTTP GET request to the PM_TYPE_KEY MPSE until after the HTTP request header pseudo packet has been assembled. Furthermore, HttpInspect also waits to send the HTTP response header to the PM_TYPE_HEADER MPSE until after the HTTP response body has been assembled. Why does Http do this? I don't know. The HTTP GET request and the HTTP request header are typically sent in the same packet and the HTTP response header and the HTTP body is typically sent in the same packet (when there's enough room in a packet) so there typically won't be any significant lag caused by the wait but I don't know why this is done.
.
name:

website:

comment:


Port-Based MPSEs

As mentioned above, PortGroups (and their corresponding MPSEs) are either port-based or service-based. The diagram below shows how a few of the port-based PortGroups are arranged.
Here we see the MPSE’s from several (port-based and service-based) PortGroup and that TCP source ports 80 and 81 share a PortGroup. Since their rule sets are so similar, TCP source ports 80 and 81 share PortGroups (and, obviously, the MPSE’s within the PortGroup). Using the same PortGroup for multiple ports is common in Snort++ and decreases memory consumption. The best example for this savings in memory is the MPSE associated with the HTTP ports. (TCP destination ports 80 and 81 are not the only HTTP-related ports that share the PortGroup.)
It’s important to understand that a packet matching with a content in an MPSE does not guarantee that the rule associated with the content matches. All of the options in a rule must match. As mentioned above, TCP destination ports 80 and 81 (as well as other HTTP ports) share an MPSE. If there is a content from a rule that is relevant to TCP destination port 80 that is not relevant to TCP destination port 81, it can still be inserted into the MPSE that is common to them both. In this case, if a packet with TCP destination port 81 matches with this content in the MPSE, the packet will not match with the rule since the packet has TCP destination port 81 and the rule requires TCP destination port 80. The rule’s ports will be checked after the MPSE finds the match and the port check will fail.
To summarize, a packet must not only match with a rule’s content in an MPSE but must also match with the rule’s other options. Remember – the content is only a single option in a rule.
name:

website:

comment:


Service-Based MPSEs

Once an appropriate service Inspector is determined (typically by the Wizard) for a TCP Flow, the service-based MPSEs come into play.
The rule option “service protocol-name” (e.g., “service http”) determines into which PortGroup a content is inserted. The content modifier (e.g., http_header) determines into which MPSE within the PortGroup the content is inserted. For example, the content for the following rule:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS ( msg:"BLACKLIST User-Agent known malicious User-Agent ErrCode - W32/Fujacks.htm"; flow:established,to_server; http_header; content:"User-Agent|3A| ErrCode"; metadata:policy balanced-ips drop,policy security-ips drop,service http; reference:url,www.mcafee.com/threat-intelligence/malware/default.aspx?id=141161; reference:url,www.virustotal.com/latest-report.html?resource=f9dc0803ea4634256eae73b2db61a3c5; classtype:trojan-activity; sid:18247; rev:5; )
will be inserted into the to_srv array because the rule has the to_server option and the http PortGroup because of the service http option and will be inserted into the PM_TYPE_HEADER MPSE of the http PortGroup because of the http_header content modifier.
name:

website:

comment:


content Insertion Into MPSEs

Let’s look at a few rules and see where they’re inserted.
Rule 1)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS ( msg:"BLACKLIST User-Agent known malicious user-agent string Opera/8.89 - P2P-Worm.Win32.Palevo.ddm"; flow:to_server,established; http_header; content:"User-Agent|3A 20|Opera|2F|8|2E|89"; metadata:policy balanced-ips drop,policy security-ips drop,service http; reference:url,www.virustotal.com/latest-report.html?resource=bc58e841f8a43072da7b3c7647828cb8; classtype:trojan-activity; sid:19756; rev:3; )
$HTTP_PORTS is defined in snort_defaults.lua and includes ports 80 and 81 (in addition to many other ports). The content from this rule will be inserted into the PM_TYPE_PKT MPSE of TCP destination ports 80 and 81
A rule’s IP address has no impact into which PortGroup/MPSE the rule is placed into. Only a rule’s source and destination port affect this.
.
This rule is also placed into the to_srv HTTP PortGroup’s PM_TYPE_HEADER MPSE because of the flow: to_server option and the service http option and the http_header content-modifier.
So if a TCP HTTP packet containing an HTTP header has a destination port of 80, then the DAQ-acquired packet will be compared against the TCP destination port 80 MPSE. When Snort++ sees an acknowledgment (an ACK) for this packet, a pseudo packet containing the HTTP header is created and that pseudo packet is compared against the to_srv HTTP PortGroup’s PM_TYPE_HEADER MPSE that contains this rule’s content
The pseudo packet is also compared against the to_srv HTTP PortGroup’s PM_TYPE_PKT MPSE. However, there are no HTTP rules in sample.rules that do not explicitly state where (e.g., the header) the content should be found so the HTTP PortGroup’s PM_TYPE_PKT MPSE is empty.
.
Rule 2)
alert tcp $HOME_NET any -> $EXTERNAL_NET any ( msg:"OS-MOBILE Android Androrat device information leakage"; flow:to_server, established; content:"sr|00 13|java.util.Hashtable"; content:"PhoneNumber"; content:"SimOperator"; content:"IMEI"; metadata:impact_flag red,policy balanced-ips drop,policy security-ips drop; reference:url,www.virustotal.com/en )
This is an example of a TCP “any-any” rule
Some rules have more than one content. Since only one content from a rule goes into an MPSE, a “fast-pattern” rule must be chosen. Typically, the longest content is chosen but there are several other factors that can be decisive in determining the fast-pattern content. In rule 2 above, “sr|00 13|java.util.Hashtable” will be chosen since it is the longest content.
. This rule’s fast-pattern content is inserted into the TCP any-any PM_TYPE_PKT MPSE. Every normal (i.e., non-pseudo) TCP packet is compared against the rules that are in the TCP any-any PM_TYPE_PKT MPSE
If the rule had the file_data option, it would be placed in the PM_TYPE_FILE MPSE.
.
For the sake of performance, the contents from the TCP any-any PM_TYPE_PKT and PM_TYPE_FILE MPSEs are by default also inserted into each and every TCP port-specific MPSE. Note that adding these contents to the port-specific MPSEs increases the amount of RAM used
Note that all port-specific MPSEs except for the PM_TYPE_PKT and PM_TYPE_FILE MPSEs are NULL. In other words, the port-specific PM_TYPE_ALT, PM_TYPE_KEY, PM_TYPE_HEADER, and PM_TYPE_BODY are NULL. This makes sense because only DAQ-acquired packets are compared against port-specific MPSEs and the payload of DAQ-acquired packets is determined later. (Recall that a service Inspector can only identify the specific parts of a packet after an ACK packet acknowledging the DAQ-acquired packet is received).
The PM_TYPE_FILE MPSE is an odd MPSE type that I discuss in a side note above.
.
If this is done, a TCP packet can be compared against the TCP port-specific MPSE (if it exists) without also comparing the packet to the TCP any-any MPSE. Comparing a packet against an MPSE with X+Y contents is faster than comparing a packet against an MPSE with X contents and then comparing the packet against an MPSE with Y contents.
Since there is no service option in the rule above, the content is not added to any service-based MPSE.
Note that not all TCP and UDP source and destination ports have associated port-specific MPSEs. In fact, only a small minority of source or destination ports have associated port-specific MPSEs. For example, sample.rules contains no rule with a TCP source port of 555 or a TCP destination port of 555. Therefore (provided that sample.rules is the only source of rules), there will be no port-specific MPSE for either TCP source port 555 or TCP destination port 555. So what happens if a packet’s source port and destination port do not have associated port-specific MPSEs? That packet is compared against only the TCP any-any MPSE.
So all DAQ-acquired TCP packets will be compared against this rule since this rule will be in every port-specific MPSE. If the source and destination TCP ports do not correspond with a non-NULL MPSE (all of which, by default, already contain all of the rules from the “any-any” MPSEs), then the packet will be compared against the TCP PM_TYPE_PKT and PM_TYPE_FILE "any-any" MPSEs.
Rule 3)
alert ip $EXTERNAL_NET any -> $HOME_NET any ( msg:"INDICATOR-SHELLCODE x86 fldz get eip shellcode"; content:"|D9 EE D9|t|24 F4|X"; metadata:policy balanced-ips drop,policy security-ips drop; classtype:shellcode-detect; sid:14986; rev:5; )
This is an example of an IP rule. Since there is no concept of ports in the IP protocol - as opposed to the TCP and UDP protocol - all IP rules are also IP any-any rules and so the content from this rule is inserted into the IP any-any PM_TYPE_PKT MPSE as well as the TCP, UDP, and ICMP any-any PM_TYPE_PKT MPSEs. After all, TCP, UDP, and ICMP packets are also IP packets. Since an IP rule will be checked against every normal (i.e., non-pseudo) packet, one should avoid writing IP rules if possible. As discussed above, these rules are inserted into the port-specific MPSEs
The layouts of the service-based MPSEs will probably change in the near future. Rules can begin with something other than alert tcp, alert udp, alert icmp, and alert ip. For example, alert file and alert http can be used. Since there are currently no rules in sample.rules that use these, I do not cover them in my documentation. I will, however, cover them when they do show up in sample.rules (or in a production rules file).
.
name:

website:

comment:


MPSE Selection

We’ve talked a little bit about which MPSEs a packet will be compared against. Let’s look at this in a little more detail. For our example, let’s look at an HTTP Flow using the typical service-side port of 80.
As can be seen by the diagram above, the packets involved in the three-way handshake (packets 1-3) are not compared against any MPSEs (i.e., “no detection”). (You can’t look for contents in a packet’s data/payload if there’s no data/payload.)
The first packet to be compared against an MPSE is the HTTP request (with its header) sent from the client to the server (packet 4). This packet is compared against the TCP destination port 80 PM_TYPE_PKT and PM_TYPE_FILE MPSEs.
The packet will not be compared against any service-based MPSE. That’s because the specific service Inspector is not known at this point. (This is reasonable - if you don’t know the appropriate service Inspector for a packet, you can’t know which service-based MPSEs to compare the packet against.) It’s important to understand that the Wizard is not invoked to try to determine a packet’s Flow’s appropriate service Inspector until Snort++ sees an ACK for the payload for this Flow
What about the TCP any-any MPSE? Shouldn’t every TCP normal (non-pseudo) packet be compared against the TCP any-any MPSE? In the default Snort++ configuration (as described above), the contents for all of the any-any rules for a given protocol are inserted into each port-specific MPSE of the protocol.
.
The next packet is the ACK for this packet (packet 5). This ACK prompts the Wizard to look at the acknowledged data (we’ll assume it finds the string “GET” in this packet and assigns HttpInspect as the service Inspector for this packet’s Flow). Now HttpInspect (actually, its Splitter, HttpStreamSplitter) can look for logical breaks in the ACK’ed packets of the Flow (in this case, it’s only a single DAQ-acquired packet) and start creating pseudo packets out of the data. The first pseudo packet (packet 6) contains the request from packet 4’s payload. However, HttpInspect waits to send the URI through the PM_TYPE_KEY MPSE until it also has the header (packet 6). After packet 7 has been created, HttpInspect picks out the URI from this pseudo packet and sends the URI through the PM_TYPE_KEY MPSE and the header through the PM_TYPE_HEADER MPSE of the http PortGroup.
The server then responds with the HTTP header and body (packet 8), which is compared against the TCP source port 80 PM_TYPE_PKT and PM_TYPE_FILE MPSEs
as well as the to_cli[] http PM_TYPE_PKT and PM_TYPE_FILE MPSEs
The first DAQ-acquired packet with payload was not compared against a service-based MPSE. Why is this DAQ-acquired packet compared against a service-based MPSE? Because the Wizard determined the appropriate service Inspector after seeing the ACK acknowledging the first DAQ-acquired packet. When it saw the first DAQ-acquired packet, the Wizard still hadn’t determined the appropriate service Inspector.
.
An ACK packet (packet 9) is sent from the client to the server acknowledging (ACK’ing) the DAQ-acquired packet (packet 8). This results in HttpInspect creating three pseudo packets: one pseudo packet for the status (packet 10), one pseudo packet for the header (packet 11), and one pseudo packet for the body (packet 12). HttpInspect sends the header through the PM_TYPE_HEADER MPSE and the body through the PM_TYPE_BODY MPSE
Since there are no rules in sample.rules with the http_body content modifier, this MPSE is empty.
and the pseudo packet corresponding to the body to the http PM_TYPE_PKT and PM_TYPE_FILE MPSEs of the http PortGroup.
HttpInspect does not send the status to an MPSE
It’s important to understand that HttpInspect inspects all of these pseudo packets itself before it sends some of the data (specifically, the URI from the request, the headers, and the body) to the MPSEs. For example, HttpInspect verifies that UTF7 (the use of which has been discouraged) is not being used in the header.
.
Here's an overview of what data is sent through which MPSEs.
name:

website:

comment:


Rule Option Evaluation

After the MPSEs find a match, all of the options must be evaluated to make sure that there really is a match between the packet and the rule. This was described in the UDP example.
name:

website:

comment:


Why This Arrangement Makes Sense

Snort’s rules are in transition, a transition that began before Snort++. Rules are now being written for specific service inspectors.
This is really nice because if the appropriate service inspector can be identified (typically by the Wizard) for a Flow, the source and destination ports become unimportant. For example, if the Wizard determines that the HttpInspect is the appropriate service inspector for a Flow, HttpInspect will inspect the different components (e.g., uri, header, body) of the Flow as well as run them through the corresponding MPSEs - regardless of the source or destination TCP ports of the Flow.
There’s also a potential performance advantage. For example, if a specific content indicates nefarious behavior only if it’s found in the HTTP header but is harmless if it’s found in the HTTP body, there’s no reason to insert the content in the PM_TYPE_FILE MPSE of the PortGroup. (The fewer contents in an MPSE, the faster the comparison between a packet and the MPSE is.)
Unfortunately, there are a lot of services for which service inspectors have not yet been written. And there are still plenty of rules that are not written for specific services. And there’s nefarious activity that doesn’t use any of the standard protocols. That’s why we still need the port-based MPSEs (or at a minimum, the any-any MPSEs).
name:

website:

comment: