Loadable kernel module with command line arguments (Linux kernel development – part 3)

  1. Preface
    Hi all,
    As I continue with my Linux kernel development and after I have talked very briefly about a minimal and simple example on how to implement a very basic kernel module in this previous post , in this post I will further elaborate about some “related” things worth noting when talking about kernel module development.
  2. Kernel symbol table
    The kernel holds a table of all public symbols (exposed by it or by other modules) – which are, for instance, functions and/or variables that some module wishes to “expose” (one can look at it as a public class member or function in OOP). The symbol table can be displayed using the command:

    cat /proc/kallsysms

    Note that when this command is invoked like so, all the addresses that will be displayed are zero’s –> this is essentially a security precaution of the kernel – cause this table contains important information about the system.
    In order to see the actual addresses of the entries in the symbol table, one should run the above command with sudo privileges.

  3. Exporting symbols from kernel module
    By default, all symbols (functions and variables) defined in a module (which means essentially in its source file(s)) are NOT exported (considered as “private”).If a module wishes to “expose” some function/variable for other module – it should use the following macro:

    const int exportedIntVarSymbol = 17;

    In this case, I have “exported” a simple const int global variable with the value of 17. Each other module that would like to “use” this variable will have to declare on this variable as extern. Below is the entire code of the module, with the C file named module1.c:

    #include <linux/init.h>
    #include <linux/module.h>
    MODULE_DESCRIPTION("A simple kernel module");
    MODULE_AUTHOR("Guy Avraham");
    const int exportedIntVarSymbol = 17;
    static int __init hello_init(void)
    	printk(KERN_ALERT "hello_init - module is loaded\n");
    	return 0;
    static void __exit hello_exit(void)
    	printk(KERN_ALERT "hello_exit - module is unloaded\n");

    Note that it is the exact same implementation as of the simple module form the previous post, only this time it also “export” the const int variable.

  4. Passing command line argument to kernel module
    It is possible to pass command line argument to a loadable kernel module when one load it (for instance with the insmod command). In the module below that is implemented in the source file named module2.c, the “global exported” variable from module1.c will be used (by declaring it as extern at the beginning) and it will receive a single command line argument of type int.
    The code for such a module is as follows:

    #include <linux/init.h>
    #include <linux/module.h>
    #include <linux/kernel.h>
    #include <linux/moduleparam.h>
    #define MAX_LEN 30
    MODULE_DESCRIPTION("A simple kernel module");
    MODULE_AUTHOR("Guy Avraham");
    const char* moduleName = "module2";
    //1)  this variable is defined in another module (in this case, in module1)
    extern const int exportedIntVarSymbol;
    // 2) this is a place holder with a default value of 9, for an int argument that will be
    //    passed via a command line argument once the module is loaded
    static int arg1 = 9;
    module_param(arg1, int, 0);
    static int __init hello_init(void)
    	printk(KERN_ALERT "hello_init - loading module:%s\n", moduleName);
    	printk(KERN_ALERT "hello_init - %s: the int got as a command line argument is:%d, and the exported variable from module1 is:%d\n", moduleName, arg1, exportedIntVarSymbol);
    	return 0;
    static void __exit hello_exit(void)
    	printk(KERN_ALERT "hello_exit - unloading module\n");

    4.1) Point 1 is where this module declares on the symbol that is defined in the other module (module1). Note that essentially in the “syntax” wise perspective, this is no different than two C source files sharing a “global” variable among them.
    4.2) Point 2 is where the place holder for the argument that will be received from the command line is declared.In this case I named it arg1.

  5. Source code folder structure
    In this example, I’m maintaining the source files and “build artifacts” in this structure:

    |- module1
    |---- module1.c
    |---- Makefile
    |- module2
    |---- module2.c
    |---- Makefile

    So when I’m building each one of the module, by running the command make in the module’s folder – for each one of them a different Module.symvers file is generated.

  6. Loading the modules (first attempt)
    6.1) Now after I have done writing the modules code it is time to load and “run them”. First thing is to load module1. This is done by the command:

    $ sudo insmod module1.ko

    Now module1 is loaded into the kernel.
    6.2) Now it is time to load module2, which can be done by the command:

    [gavraham@localhost build]$ sudo insmod module2.ko arg1=17
    insmod: ERROR: could not insert module module2.ko: Invalid parameters

    6.2.1) The name of the command line argument must be set exactly as it in the C file, followed by the equal sign (=) and then the value (no spaces between the name of the argument and the value of it – similarly as when defining a variable in a bash script).
    As can be seen the in the output – module2 was NOT loaded into the kernel.
    When inspecting the dmesg last “log prints” the following can be noticed:

    $ dmesg
    [23015.325098] hello_init - module is loaded and is exporting variable exportedIntVarSymbol with value:17
    [23170.356330] module2: no symbol version for exportedIntVarSymbol
    [23170.356333] module2: Unknown symbol exportedIntVarSymbol (err -22)

    It complains that it does NOT know the symbol of the exported variable.

  7. Enabling symbols “sharing” between kernel modules
    The reason that the error raised when I tried to load module2 is due to the fact that each one of the modules has its OWN Modules.symvers file cause it was built on “its own”, so they are “unaware” to “each other” symbols. There are three approaches to solve this situation that are described section 6. Module Versioning from the original kernel documentation.
  8. Solving the symbols issue
    In this case I will “tackle” the issue by “letting” module2 know about the symbols exported in module1 by adding to its Makefile the Modules.symvers file of module1.
    This can be done by adding the first line in the Makefile of module2 as follows:

    # Makefile for the simple kernel module that exports symbols from module1
    KBUILD_EXTRA_SYMBOLS := /home/module1/Module.symvers
    obj-m += module2.o 
    KDIR ?= /lib/modules/$(shell uname -r)/build
    PWD := $(shell pwd)
    	$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

    8.1) The first line in the above Makefile indicates to module2 about the existing and content of module1’s symbols.
    8.2) The Makefile for module1 is exactly the same expect that it does not need the first line and obj-m is set to : obj-m += module1.o

  9. Loading the module (second attempt)
    Now, I will do the following:
    9.1) Build module1 by running its Makefile (note that it needs to be built first cause module2 will use its Modules.symvers)
    9.2) Build module2 by running its Makefile.
    9.3) Loading module1: sudo insmod /home/module1/build/module1.ko
    9.4) Loading module2: sudo insmod /home/module2/build/module2.ko arg1=15
    After that sequence of actions, dmesg will display the following:

    $ dmesg
    [25167.282425] hello_init - module is loaded and is exporting variable exportedIntVarSymbol with value:17
    [25197.276125] hello_init - loading module:module2
    [25197.276128] hello_init - module2: the int got as a command line argument is:15, and the exported variable from module1 is:17


  10. Conclusions
    In this post the following were discussed:
    a) The notation of sharing symbols between external modules was introduced.
    b) What (in general) is the Modules.symvers  and how one should (can) use it when sharing symbols between several external modules.
    c) A simple Makefile to build a single source code module was introduced.
    d) Passing command line argument to a loadable module.

    a) Good Q&A on stackoverflow regarding how to compile two different kernel modules and be able to share (export) variable between them 
    b) Modules.symvers nice short explanation
    c) Another Q&A on Stackoverflow regarding sharing symbols between kernel modulesThe picture: Foz do Iguacu, state of Parana, Brazil.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s