[knot-dns-users] [Knot-Resolver] Async module unloading and cleanup event idea

Hi everyone, While working on a Knot-Resolver (kr) C module, I stumbled on a limitation of the current way layers are loaded and unloaded. My understanding is that, currently, C modules are loaded via a dlopen call, and then a callback "moduleName_init" symbol is searched and called. On unloading, a symbol "moduleName_deinit" is searched and called, and upon completion of the deinit function, a dlclose call is made to unload the module from memory. This occurs in lib/modules.c. As it happens, my C module provides callbacks (cb) for several layer hooks. Some of these cb perform potentially long-lived I/O operations. Since functions are not supposed to take a lot of CPU time, in order to maintain kr responsiveness and with respect to the event-oriented design, I chose to embrace libuv and use the uv_* functions to perform these I/O operations. This means I inserted in the uv_default_loop several cb for network or filesystem events. These cb need to be called for some operations to be successfully completed. Think about SQL commit statements and whatnot. Currently when a user wants to stop the daemon, the "graceful" procedure is to call quit() on the CLI. This procedure calls uv_stop on the uv_default_loop, thus stopping abruptly all registered cb from being ever called again. Then, modules are unloaded one by one. These unloading calls to moduleName_deinit can be blocking because no service requires responsiveness anymore. On the other hand, if a module is shutdown during normal operations, for instance, via a call to "modules.unload" from the CLI, the unloading procedure cannot perform blocking actions without generating a negative impact on the overall responsiveness. The problem is that it cannot perform non-blocking actions either because as soon as the deinit function returns, the code for the non-blocking actions "to wrap up" currently registered callbacks in the uv_default_loop is removed from memory. To solve this problem, I would suggest creating a cleanup event, sent to modules that should be unloaded. A module receiving a call to its handler for this event would be expected to wrap it up ASAP. It would, however not be expected to have completed everything at handler return time. The sender of the cleanup event could then register a uv_idle handler that would call the deinit function. The deinit function would then be in charge of ensuring the module no longer have any event handler registered in the uv loop. If no cb are present in the loop, it would return a status code allowing the caller to dlclose the module. With this approach, calling the "graceful" quit procedure would then send a cleanup event to all modules, including iterate, cache, etc. uv_stop would not need to be called anymore because the idle handler would be able to know that no handler are in the uv_loop except itself: it would then unregister itself and the uv_run call would terminate spontaneously. What is your take on this issue? Do you think the proposed solution would be acceptable? Would it work with all kind of modules (I'm thinking about the lua and go modules)? I, sadly, currently have no code to do this. I just thought it might be useful to document this on the mailing list and share ideas about it. Thank you. Cheers, Florian