wiki:modularity

Modularity

We want to allow plugging hooks into various places to allow users to customise the behaviour. It should be possible to:

  • Add special action based on some condition (handle a different packet in a different way) without changing the behaviour for the rest.
  • Examine the data as they go by.
  • Modify the data before actual processing.
  • Filter data so it is or isn't processed.
  • Completely replace the default action.

It is desirable to have an ability to provide multiple instances of single hook (two different special actions, one filter before that all, etc). It is true that multiple hooks can be combined together manually, but it requires user to actually code, not download two different extensions and just plug them in.

It would be nice to be able to use different languages to write the hooks.

Ability to replace the hooks (add, remove, upgrade to newer version) is implied from the fact we want to have full runtime configurability.

Most code as an extension

If we had such system, we could write most of our code as extensions, even if it was core server functionality. The advantage would be that we could reload some of our code at runtime and would have the system more consistent.

It eases up the modularity design a bit.

Dividing into bits

When we are considering a functionality, the functionality can be divided into parts. For example, there is a (one possible, probably incomplete) division of authoritative server logic:

  • A part that listens for incoming connections and packets.
  • Something that parses a packet.
  • Some kind of ACL.
  • Actual provision of answer.
    • Something deciding what the user wants.
    • Some data storage to query for data (possibly called multiple times).
  • Some kind of post-processing (signing with TSIG would probably go here for example).
  • Rendering of the packet.
  • Sending the packet out.

If we wanted to plug something in (for example XFROUT, it is now handled by separate process, but something needs to decide where it goes, and, in theory, we could handle it completely inside one process. We could plug it into the „Actual provision of answer“.

Another example is how to build a recursive server. We would replace the „Actual provision of answer“ by a different logic, leaving the rest intact. In fact, if we wanted a hybrid (auth+recursive) server, we could combine these logics together somehow.

If we wanted to do some testing, we could replace the part that listens and the part that sends. Or, if we wanted to inject a packet, we could start the processing from the “Some kind of ACL“, mark the request and plugging something into „Rendering of th packet“ that would handle it back to the injecting tool if it was marked.

Action lists

This is inspired by the Apache web server.

Each of the places would be an action list. The list would hold series of functions (or functors, if needed, but functions seem better for performance). When the action list is executed, the functions are run one by one until the success of the list itself is decided.

Each function can return one of:

  • OK (and the actual answer). In that case this is the last executed function and the action list returns the answer to the caller (eg. terminates successfully).
  • Fail. No more functions are called and the action list fails.
  • Deny. It means this function denies to perform the action and another function in the list is called to try to handle it.
  • Reset. The action list is started from beginning (can be used for example when one function changes the input data and wants it to be processed again).

If the end of the list is reached without getting an answer, it is considered a failure.

In our example, the ACL list would contain a default successful action and anything else could either allow the packet directly (by returning OK), reject the packet (by Fail) or just let it be handled by other rules by Deny.

Suspension of a list

In some cases it is needed to suspend execution of the list for a while. One example of it would be the processing of recursive server. I have a slight idea how to do it, but the idea is not yet fully formed.

Extension-specific storage

We can pass a state object trough the processing of lists to handle notes one function from an extension can have for another function (in the example of packet injecting, it would be the mark). However, because we don't know what data will be needed by any future extensions, we need to do it differently.

Each extension will have the possibility to request a slot. It will receive a number. And the state object will contain array of pointers to abstract classes, each function can have a look into its object.

Actually, this could be handled in a way the extension functions don't have to care about it, by some template magic and inheritance.

Views

The views could be handled with some help of this extension mechanism. The state object will hold an id of view this request belongs to (it would be the default view at the beginning, some extension could move it to a different view by simply altering this id.

Then an extension could either handle views by itself somehow. Or, when an extension would have different configuration for a given view (or, there would be different set of extensions for a view), another instance of the extension would be created. The action list would have one default list and overwrite lists. If an overwrite list for current view is found when entering the list, the overwrite list would be used. Otherwise we would use just the default one.

Putting it together

It is possible to load and unload libraries fully dynamically (eg. controlled by the code of application). This could be used to load list of extensions provided from configuration.

Each extension would have an initialization and finalization function. It would link its functions into the lists, create IO actions in asio and so on. It would remove them in the finalization function.

So, at the lifetime of the process, it would:

  • Load the configuration.
  • Start loading extensions (one by one, sorted by dependencies).
    • Load the dynamic library.
    • Call its initialization function.
  • Run for some time.
  • Before shutting down: it would:
    • Signal the extensions it wants to shut down. As the extensions would signal back they are ready to (either right away or at some later time when it shuts down all threads, external processes, handles queries in progress).
    • Call finalization function of each extension.
    • Unload them from memory.

Actually, the unloading is not necessary at shutdown, but in case we needed to unload or reload an extension at runtime, it would be needed.

Manipulation with the lists

First problem is accessing the lists. The lists can be accessed directly if they are created by the same extension or extension we depend on (the auth server extension would depend on the extension providing listening, parsing of packets, etc). But to examine „foreign“ lists or access them from dynamic languages is harder. Therefore we will have a storage where the lists are registered under a name and can be found dynamically.

Second problem is modification of the lists at runtime. The list is not safe to be modified when something runs inside it. Therefore we will have a read-write lock. Most of the time it will be locked only for reading (eg. running the list) and it will not block. But when we want to modify the list, it must be locked for write. It will prevent anything to run at the time.

Last modified 7 years ago Last modified on Jan 29, 2011, 11:53:11 AM