Snort++ Thread Overview

Threads allow the multiple processors of a multiprocessor (or multicore) system to be leveraged. If a program does not take advantage of the thread capabilities offered by an OS, the corresponding process will only use a single processor, even if it's running on a multiprocessor system. Snort2 did not take advantage of multiprocessor systems but Snort++ does.
Snort++ currently creates threads in two scenarios: "packet processing" threads and "worker" threads. Packet processing threads are created for interfaces and PCAP files. For example, if Snort++ is listening on two different interfaces, two packet processing threads can be created (using the "snort.--max-packet-threads" setting in snort.lua). Worker threads are created for MPSE work. A configurable (detection.offload_threads) number of worker threads are created at Snort++ initialization and when a packet processing thread needs to do MPSE-related work, the packet processing thread itself can do the MPSE work or it can request that one of the worker threads does it.
Threads in general are discussed in this document. Packet processing threads and worker threads are discussed elsewhere.
name:

website:

comment:


Functors

There are several different ways that a new thread can be created
I prefer to talk about structs and classes and other data structures rather than functions. Unfortunately, when discussing threads, discussing functions (including system calls) is unavoidable.
. Using
"functors" is one way in which threads can be started.
So what's a functor? A functor is an object of a class that overloads the "()" operator. The analyzer object (of class Analzyer) is a functor (also called a “function object”) because it overloads the "()" operator. This means that one can call analyzer()
The code below (which is from the link to the left) is a little messy but the idea is that a new thread will be created and that thread will call a function at its creation.
athread = new std::thread(std::ref(*analyzer), ps, ++run_num);
In this case, that function will be analyzer() (the functor).
and its overloaded operator() method will be called.
If functors are used, there are two ways to do it. The first way is to pass a *COPY* of the entire functor object to the new thread. In this scenario, the thread executes the overloaded operator() method from a copy of the object and any changes only affect this new, copied object. The second way is to pass a *REFERENCE* to the functor object to the new thread by using std::ref() (Snort++ uses references - we'll discuss this in a moment). In this case, any changes are made to the object
It's actually a little more complicated than that. std::ref() wraps the pointer to the object (in this case, a functor) in a reference_wrapper object. Once the thread gets the reference_wrapper object, it can access the pointer using the object's access methods ((operator() and get()).
In the "Possible Implementation" in the link above describing reference_wrapper, look at the sole private field, which is a pointer to the object.
. In this scenario, no copy of the functor is made and the thread executes the overloaded operator() method of the functor object itself.
Let's look at an example using functors to start new threads.
andy@ubuntu-android:~/functors$ cat functor.cc #include <thread> #include <iostream> #include <functional> // for std::ref class FunctionObjectClass { public: FunctionObjectClass(void) { var1=0; } void get_var1(void) { std::cout<<"var1="<< var1 << " || " << this << std::endl << std::endl; } void operator()() { var1++; std::cout<<"var1="<< var1 << " || " << this << std::endl << std::endl; } private: int var1; }; int main() { FunctionObjectClass functionobject; std::cout<<"main thread"<<std::endl; functionobject.get_var1(); std::cout<<"thread t1"<<std::endl; std::thread t1(functionobject); t1.join(); std::cout<<"main thread"<<std::endl; functionobject.get_var1(); std::cout<<"thread t2"<<std::endl; std::thread t2(std::ref(functionobject)); t2.join(); std::cout<<"main thread"<<std::endl; functionobject.get_var1(); } andy@ubuntu-android:~/functors$ g++ -std=c++11 functor.cc -pthread -o functor andy@ubuntu-android:~/functors$ ./functor main thread var1=0 || 0x7fffead10b30 thread t1 var1=1 || 0x2016040 main thread var1=0 || 0x7fffead10b30 thread t2 var1=1 || 0x7fffead10b30 main thread var1=1 || 0x7fffead10b30
(Note that the join() method of a thread waits for the thread to finish terminating before returning.)
In the example above, two threads, t1 and t2, are created with std::thread()
std::thread() is a new function added to the C++ language in the C++11 version of the C++ standard. The --std=c++11 option is needed to compile code that uses this function. The -pthread option must also be given so that the program can be linked to the Posix thread (pthread) library.
. Thread t1 is created using a copy of the functor object whereas thread t2 is created using the functor object itself (i.e., a reference to the functor object is passed to the new thread). See how thread t1 has a different address than the main thread but that thread t2 has the same address? Furthermore, note how incrementing var1 in t1 has no effect on the main thread (i.e., var1 remains 0) whereas incrementing var1 in t2 affects the main thread (i.e., var1 is incremented to 1).
name:

website:

comment:


Snort++ Implementation

So how does Snort++ do it? The main thread and the packet threads must be able to communicate so Snort++ passes in a *REFERENCE* to an object of class Analyzer to the new thread. This communication is done by the main thread or the packet thread setting the members (in non-Object-Oriented terms, its "fields") of the Analyst object, analyst. For example, one important member in the Analyzer class to which both the main thread and the newly created packet threads need access is the state field. After DAQ has been initialized and the packet-processing thread can start processing packets, Snort++ sets state to STARTED. So just like thread t2 had access to the same data that the main thread had access to (specifically, var1), both the main thread and the packet processing threads in Snort++ have access to the state field
You have to be careful with passing in a reference. If you pass in a reference to the object rather than a copy of the object, you must make sure that the main thread doesn't delete the object while the other thread still needs it. In the code above, we're safe since we just wait for the thread to end in the main thread without doing anything else.
.
Using the process.threads option, a thread can be forced to run on a specific cpu. For example, the following configuration option:
process = { threads = { { cpu = 1, source = "eth0" } } }
moves the thread that processes packets on interface eth0 to cpu #1. As the Snort3 configuration files are being processed, the source_affinity map and the thread_affinity vector from the all-important snort_conf global configuration are built using the process.threads option. As the thread begins its execution in Analyzer::operator(), pin_thread_to_cpu() calls sched_setaffinity() to try to move the thread to the desired CPU
When studying the code, look out for global variables that are declared with the storage class specifier THREAD_LOCAL. For example, instance_id is declared as THREAD_LOCAL. This means that instance_id is specific for a given thread. Snort++'s first packet thread will have an instance_id of 1, the second packet thread will have an instance_id of 2, and so on. If the THREAD_LOCAL storage space specifier was not used, instance_id would be a single variable shared by all of the threads (which is obviously not what is desired).
.
name:

website:

comment: