Snort++ Modules

The Snort++ User Manual uses the word "module" somewhat loosely. For example, the "Inspector Modules" are described as "modules (that) perform a variety of functions, including analysis of protocols beyond basic decoding." Be aware that the Snort++ documentation is not always as picky in terms of terminology as the Snort++ code. In the code, the "Inspector Modules" would instead be referred to as "Inspector Plugins". (I’ll discuss "plugins" in a moment.)

In the Snort++ code, objects of Module-derived classes are used to process configuration settings in the Snort++ configuration files and rule option settings in the Snort++ rules files.
The topic of the Module class is not particularly sexy. That being said, a Snort++ administrator may find it occasionally useful to understand how configuration and rule option settings are processed in order to understand how a configuration or rule option setting impacts Snort++’s behavior. Furthermore, a programmer who wishes to write a Snort++ plugin (I’ll discuss plugins below) that is configurable (i.e., has configuration settings) must understand how Snort++ Modules work. But the most important reason to understand Modules is that the Modules are the definitive source for configuration and rule options
For example, if you want to know the valid options for search_engine, you would look at its SearchEngineModule (specifically, its search_engine_params[]).
.
There are a couple of different types of Module-derived classes that correspond to configuration settings - "basic" Modules and Modules that correspond with plugins (we’ll discuss rule option settings later).
name:

website:

comment:


Basic Modules

There are Module-derived classes that are so fundamental to Snort++ that the code that is relevant to the settings in the Module cannot be turned off. These Module-derived classes are found in src/main/modules.cc and are referred to as "basic" modules in the Snort++ documentation. These Modules simply process configuration settings.
Examples of "basic" Module-derived classes are SearchEngineModule (which corresponds to the search_engine configuration parameter) and AttributeTableModule (attribute_table). The code for both SearchEngineModule and AttributeTableModule is so fundamental to the operation of Snort++ that the functionality cannot be turned off (for example, you cannot disable the search engine).
name:

website:

comment:


Modules That Correspond With Plugins

In addition to the "basic" Module-derived classes are the Module-derived classes that correspond to "plugins". I said just a moment ago that objects of Module-derived classes simply process configuration settings. Plugins, on the other hand, "do something". Plugins can inspect packets (Codecs and Inspectors) and log events (Loggers) and take action (RejectAction and ReplaceAction). Also, unlike Modules, which all derive from the same class (Module), plugins derive from several classes: Codec, Inspector, Logger, and IpsAction.
Important examples of plugins are the TcpCodec Codec, the HttpInspect Inspector, and the FastLogger Logger.
name:

website:

comment:


Modules That Correspond With Rule Options

The last category of Module-derived classes are classes that process Snort++ rule file options. Examples are MsgModule and ContentModule. Objects of these classes are not used to parse a configuration setting within a Snort++ configuration file but rather a configuration option in the rules file (currently sample.rules for Snort++). For example, if the following rule is parsed:
alert tcp any any -> any 21 (msg:"FTP ROOT"; content:"USER root"; nocase;)
the MsgModule object parses msg:"FTP ROOT" and the ContentModule object parses content:"USER root" as well as nocase (nocase is a sub-setting for content).
name:

website:

comment:


How Modules Work

Module-derived classes override the set() method and (optionally) the begin() and end() methods from the Module class. A configuration setting typically corresponds to some data structure (variable, array, etc.) and set() sets the data structure to reflect the value from the configuration file. begin() is overridden if the data structures that set() affects must be allocated. end() is typically overridden to ensure that the configuration settings supplied by the Snort++ administrator make sense (i.e., an integrity check is done).
Let’s look at an example of a "basic" Module-derived class (as discussed above, a basic Module is a Module without a corresponding plugin). Like all basic Module-derived classes, SearchEngineModule is declared in src/main/modules.cc. Module-derived classes that correspond to configuration settings must declare these configuration settings. The settings for SearchEngineModule are declared in the search_engine_params[] array (note that these settings match up with the parameters given in the manual)
The search_engine_params[] array is passed into the Module-derived class’s instantiation method. This in turn invokes Module's instantiation method, which sets the params. I NEED TO FIGURE THIS OUT - THIS PROBABLY HAS TO DO WITH AN INTERACTION WITH LUA!!!! IT MAY BE THAT THIS IS EXPLAINED BELOW!!!! ).
.
As discussed above, all Module-derived classes (SearchEngineModule included) must override the set() method. The set() method is where the different parameters are set. For our SearchEngineModule example, if the following is specified in a configuration file:
search_engine.debug=true
then the Module-derived class that corresponds with the name "search_engine" will be located. (See figure above.) This class is SearchEngineModule and therefore SearchEngineModule’s set() method is called to process the debug parameter. Since the debug parameter is declared as follows within search_engine_params[]:
{ "debug", Parameter::PT_BOOL, nullptr, "false", "print verbose fast pattern info" },
a boolean value is expected so any values other than true or false will be rejected.
As I just mentioned, Module-derived class can also override the begin() and end() methods. As explained in the manual, begin() is used to allocate any required data structs and end() is used for integrity checks that are needed in addition to the basic parameter type that was described above. Since SearchEngineModule does not need any additional structs (all of its parameters are used to set values within the global configuration SnortConfig struct, which is allocated elsewhere), it does not need a begin() method. And because none of the values of the SearchEngineModule parameters depend on each other, an end() method that verifies that dependencies between different parameters are met is also unnecessary.
The same overridden Module methods that are used for processing configuration file settings - begin(), set(), and end() - are used to process rule file options.
name:

website:

comment:


Configuration Settings

Let’s look in detail at how the configuration settings are processed.
All plugins have an API. For example, the Ssh Inspector has the ssh_api API. Using the mod_ctor function in the API, objects of Module-derived classes are created (i.e., "instantiated") during initialization and, along with this API, are put in a linked list of ModHooks.
As Snort++ processes the Lua tables
An example of a Lua table in Snort3 is the default_wizard table. Lua "tables" are essentially associative arrays. In other words, the elements of the array can be indexed by strings - it is not a requirement that they are integers. In fact, most of the Lua tables in Snort3 are indexed by strings (e.g., default_wizard).
that correspond to the Module-derived classes,
Within the Snort++ configuration file(s) (typically snort.lua and snort_defaults.lua), the name of the Lua table (e.g., "ssh") is compared with the Module’s name field.
The name of the module is the first parameter passed into Module’s constructor (#LINKNEEDED# to Module’s constructor for Ssh). For plugins (oddly enough), the name of the Modules is not found in the Module’s API but instead is typically defined in a macro (#LINKNEEDED# to SSH_NAME in ssh.c) that is passed into Module’s constructor.
Later during Snort++’s initialization when Snort++’s configuration file(s) are being processed, if a ModHook is found whose Module matches up with the name of the Lua table, the plugin’s constructor (e.g., ssh_ctor) is looked up in the API and then ultimately called to create the plugin object.
The Codecs are unique in that all Codecs are created regardless whether they are found in the configuration files.
name:

website:

comment:


Lua Crash Course

Since few people have had exposure to Lua
I wasn't aware of Lua until I started studying the Snort++ source code.
, I’ve decided that a crash course in the Lua programming language and the Lua/C++ interface (i.e., how to call Lua functions from C++ and how to call C++ functions from Lua) would be useful. Lua has the advantage over other configuration file types (e.g., XML) in that Lua allows a Snort++ admin to dynamically determine configuration settings. For example, a Snort++ admin can
use arbitrary environment variables to configure settings.
I don’t really like to talk about functions (I prefer to discuss classes and structs). However, if we’re going to discuss the Lua programming language, we (obviously) need to discuss specific functions.
The processing of a Snort++ configuration file is kicked off by a call to parse_file(). This processing can be broken down into a 2 step process.
1) C++ code that invokes the Lua function snort_config().
2) Lua code that invokes the appropriate C++ methods in the objects of the Module-derived classes to process configuration settings as well as invoking C++ code that instantiates the Plugin object (e.g., an object of Ssh class).
Let's look again at the configuration of the Ssh plugin:
ssh = { max_encrypted_packets = 30, max_client_bytes = 18000, max_server_version_len = 90, }
In this case, Snort++ calls Lua code
This corresponds to (1) in the figure above.
(specifically the snort_config() function) which would, in turn, first find the appropriate Module (which is SshModule
(2) in the figure above.
) and then calls methods within the SshModule class to validate the settings
(3) in the figure above.
in the Lua table (which is shown above) and then call other functions to instantiate the appropriate plugin object
(4) in the figure above.
(in this case, Ssh), which would be used to inspect SSH sessions.
PART 1) C++ code that invokes the Lua function snort_config()
Let's look at the most important C++ functions in the code path
In the code below, note the use of the Shell class. Objects of the Shell class execute Lua code. An object of class Shell is created for the main Snort++ configuration file. (Recall that Snort++ configuration files follow Lua syntax and contain Lua tables as well as code. In other words, each Snort++ configuration file is a Lua script.) Also, for each configuration file specified by a Binder element, a Shell object is created that executes the Lua code in that configuration file.
that invokes the Lua snort_config() function:
parse_file() Shell::configure() config_lua(L, ...) load_config(L, ...) luaL_loadfile(L, file) lua_pcall(L, ...) run_config(L, ...) lua_getglobal(L, "snort_config") lua_pcall(L, ...)
The goal of this code is to execute the Lua function snort_config(), which was defined in file (previously set to include/snort/lua/snort_config.lua) and was loaded by luaL_loadfile().
Before we go further, it’s helpful to understand a little bit about the naming conventions of the Lua programming language and how (in general) its C/C++ interface works. The definition of any function that starts with "lua_" can be found in the Lua library (libluajit-5.1.a). Some of the functions in this library are C/C++ functions that are used to set up and call Lua code. The general mechanism is that Lua code is pushed onto a stack and then lua_pcall() executes a function found in this code on the stack. In the code above, luaL_loadfile() pushes a file (snort_config.lua) containing Lua functions and tables onto the stack. Next, the string "snort_config" (a Lua function) is pushed on the stack by lua_getglobal() and then lua_pcall() executes the function with this name snort_config().
In the code above, what is the L variable that is a parameter in many of the function calls? Starting with config_lua(), the L variable is always the first argument. L is a pointer to an object of class lua_State. A lua_State object contains the state of the Lua interpreter and allows code to be "reentrant." "Reentrant" means that C++ code can invoke the Lua interpreter with a call to a function in the Lua C/C++ interface library (and have the Lua code return) and then invoke the Lua interpreter again from the C++ code with a call to another function within this library and the second call will remember the results of the first call. In the code above, snort_config() is found by the Lua interpreter because the results of luaL_loadfile() (which loaded snort_config.lua) were remembered in L. Why is this necessary? Since the Lua interpreter is invoked with each C++ call to a lua_... function and then exits, the Lua interpreter must be reminded of what happened in its previous invocations.
Instead of just looking at the code path for run_config() as we did above, let's now look at the actual function.
static void run_config(lua_State* L, const char* t) { lua_getglobal(L, "snort_config"); lua_getglobal(L, t); if ( !lua_isfunction(L, -2) ) FatalError("%s\n", "snort_config is required"); else if ( lua_pcall(L, 1, 1, 0) ) { const char* err = lua_tostring(L, -1); FatalError("%s\n", err); } }
lua_getglobal() pushes values onto the stack. lua_getglobal pushes the string "snort_config" and then pushes the variable t, which will be the string "_G" (we will cover _G shortly). What’s the meaning of the "-2" in the call to the function lua_isfunction(L,-2)? "-1" is the last value pushed onto the stack and "-2" is the second to last value pushed onto the stack and so forth. So lua_isfunction(L, -2) is verifying that the second to last value pushed onto the stack (the string "snort_config") is indeed a function. And since the call to luaL_loadfile(L, file) previously loaded a file (typically etc/snort/snort.lua) which contains a function named snort_config() (since etc/snort/snort.lua includes include/snort/lua/snort_config.lua), lua_isfunction() will return without an error (a non-zero return value for lua_isfunction() indicates success).
I mentioned in the previous paragraph that the string "_G"
Note that identifiers (e.g., variable and table names) that begin with an underscore ("_") are names that are reserved for special uses in Lua and should be avoided by Lua programmers.
was pushed onto the stack - what is "_G"? The variable "_G" is a built-in table in Lua that contains all of the functions and global variables and tables previously defined
Although I could not find this in any documentation, "G" presumably stands for Global.
. So for the Lua file that was just loaded, all functions (including snort_config()) and global variables and tables can be found in _G. The call to luaL_loadfile(L, file) loaded not only all of the functions (e.g., snort_config()) found in snort.lua and its included files but also all of the variables and tables (e.g., HOME_NET, search_engine).
At this point, immediately before lua_pcall() is called by run_config(), the stack contains the string "snort_config" and _G. lua_pcall() then executes the first item on the stack (snort_config()) and uses in the second item on the stack (_G) as its only argument.
Let's look a little closer at lua_pcall(). lua_pcall() takes 4 arguments. The first argument is L, which was discussed previously. The second argument is the number of arguments passed to snort_config(), which is only "_G", so the second argument is equal to 1. The third argument is the number of results returned on the stack by snort_config
I believe that this value - 1 - is wrong because snort_config does not return any results. However, the Lua interpreter will push the value NULL on the stack and since the return value is never used, specifying 1 for the number of results does not cause any problems.
. The last argument to lua_pcall() is used to specify a different error handling routine. Since the value is 0, the default error handling is used (which is to print an error message). (The original values - the function name and arguments - on the stack will be removed from the stack after the function returns.) All of this was necessary in order to call the Lua function snort_config() with a single argument, _G.
It's important to understand that everything that was done up until this point was done so that the Lua function snort_config() could be called with _G as its only argument. This corresponds to step 1 of the 2 steps that are described above that are necessary to parse the Snort++ Lua configuration file.
PART 2) Lua code that invokes C++ methods in the objects of the Module-derived classes to process configuration settings as well as invoking C++ code that instantiates the plugin object.
The C++ code's sole task up to this point was to call the Lua function snort_config() with the Lua table _G (described above) as its only argument. snort_config() calls snort_traverse(), which (as its name implies) "traverses" a Lua file. The first time that snort_traverse() is called, this Lua table will be _G. In this context, traversing a table means that snort_traverse() calls snort_set() (often indirectly) for everything in _G, which includes all of the variables and tables in the Lua configuration files (typically snort.lua and snort_defaults.lua). The Lua variables (e.g., HTTP_PORTS) are typically used in rules and the Lua tables (e.g., ssh) are used to configure the objects of the Module-derived classes in the C++ code. (Recall that Module-derived classes - e.g., SshModule - are used for the configuration of basic modules and plugins.) snort_set() does this by calling set_bool() or set_number() or set_string() (whichever is appropriate) for each of these key/value pairs in _G and for each of the key/value pairs in the Lua tables within _G
In snort_set(), you'll find function calls like ffi.C.set_bool() and ffi.C.open_table(). (FFI stands for "Foreign Function Interface" - https://en.wikipedia.org/wiki/Foreign_function_interface.) At the beginning of snort_config.lua, you'll find the following code:
ffi = require("ffi") ffi.cdef[[ bool open_table(const char*, int); void close_table(const char*, int); bool set_bool(const char*, bool); bool set_number(const char*, double); bool set_string(const char*, const char*); bool set_alias(const char*, const char*); ]]
Furthermore, in the C/C++ managers/module_manager.cc file, you'll find the following code:
//------------------------------------------------------------------------- // ffi methods //------------------------------------------------------------------------- extern "C" { bool open_table(const char*, int); void close_table(const char*, int); bool set_bool(const char* fqn, bool val); bool set_number(const char* fqn, double val); bool set_string(const char* fqn, const char* val); bool set_alias(const char* from, const char* to); }
This is necessary for Lua to call C/C++ functions.
.
When Snort++ begins to process a Lua table (before Lua starts processing the key/value pairs with the set_...() functions), the C++ function open_table() is called. If the Lua table is not complex (i.e., there are no Lua tables within the Lua table), open_table() is straight-forward. open_table() simply verifies that there is an object of a Module-derived class appropriate for the Lua table being processed (e.g., SshModule for "ssh"). open_table also gets the Parameters for the Module-derived class. These Parameters define which the legal settings for a given Module.
An example can help explain Parameters. Here are the Parameter's for the SshModule class:
static const Parameter s_params[] = { { "max_encrypted_packets", Parameter::PT_INT, "0:65535", "25", "ignore session after this many encrypted packets" }, { "max_client_bytes", Parameter::PT_INT, "0:65535", "19600", "number of unanswered bytes before alerting on challenge-response overflow or CRC32" }, { "max_server_version_len", Parameter::PT_INT, "0:255", "80", "limit before alerting on secure CRT server version string overflow" }, { nullptr, Parameter::PT_MAX, nullptr, nullptr, nullptr } };
Here again is an "ssh" Lua table that one might find in snort.lua:
ssh = { max_encrypted_packets = 30, max_client_bytes = 18000, max_server_version_len = 90, }
Note how this "ssh" Lua table matches up with the Parameters above.
Let's look at one element within s_params:
{ "max_encrypted_packets", Parameter::PT_INT, "0:65535", "25", "ignore session after this many encrypted packets" },
This requires that the max_encrypted_packets field be an integer between 0 and 65535 and that its default value is 25. It also gives a summary of the field.
Parameters can be complex. For example, the parameters for the (very important) WizardModule class are complex, with Lua tables within Lua tables.
open_table() finishes by calling the begin() method for the appropriate object of a Module-derived class. In the case of SshModule, SshModule::begin() simply allocates the ssh plugin's configuration struct, SSH_PROTO_CONF:
struct SSH_PROTO_CONF { uint16_t MaxEncryptedPackets; uint16_t MaxClientBytes; uint16_t MaxServerVersionLen; };
After open_table() has been called, snort_traverse() (recursively) calls snort_traverse(), which in turn calls snort_set() to process the key/value pairs within the table. For each key/value pair, snort_set() calls one of the C++ functions set_bool(), set_number(), or set_string(), depending on the value's type (e.g., for the element "max_encrypted_packets = 30", set_number() will be called). Each of these functions then (indirectly) calls the set() method on the appropriate object of the Module-derived class.
Here is SshModule::set():
bool SshModule::set(const char*, Value& v, SnortConfig*) { if ( v.is("max_encrypted_packets") ) conf->MaxEncryptedPackets = v.get_long(); else if ( v.is("max_client_bytes") ) conf->MaxClientBytes = v.get_long(); else if ( v.is("max_server_version_len") ) conf->MaxServerVersionLen = v.get_long(); else return false; return true; }
In this manner, the configuration settings (stored in a SSH_PROTO_CONF struct) for the object of the Module-derived class (specifically SshModule) are set from the "ssh" Lua table in the Snort++ configuration file (typically snort.lua). Note that the actual plugin object has not yet been created (i.e., instantiated) yet. This comes a little later
When the Parameters for a Module-derived class are complex (i.e., tables within tables), the processing of the settings becomes very complex. snort_traverse() (and therefore also open_table() and close_table()) are called for each table (including sub-tables) encountered. Furthermore, the begin() method often becomes more complex since different types of structs must be allocated to store the different types of configuration settings.
.
After Snort++ is finished processing a Lua table (e.g., "ssh"), close_table() instantiates the plugin object with a call to PluginManager::instantiate(), passing in the object of the Module-derived class (e.g., SshModule). In this way, the newly created plugin object (e.g., Ssh object) has access to the configuration settings that the object of the Module-derived class just extracted. (In the case of the Ssh plugin, the plugin object simply re-uses the SSH_PROTO_CONF that was allocated by the object of the Module-derived class for its own configuration.) This object is one of the service Inspector objects that will inspect the DAQ-acquired packets and pseudo packets
Before close_table() instantiates the plugin object, it first calls the end() on the object of the Module-derived class. The SshModule class simply returns true (i.e., does nothing) whereas other classes (e.g., WizModule) are somewhat more complex. (In the case of WizModule, end() takes the configuration settings that it extracted and puts them into a container (specifically, a MagicBook) that allows for quicker lookups.
.
name:

website:

comment: