-
Notifications
You must be signed in to change notification settings - Fork 21
LIMA Technical Documentation
Table of Contents generated with DocToc
LIMA configuration files are searched in the folder pointed to by the LIMA_CONF environment variable. If it is not defined, they are searched in the /usr/share/lima/conf folder (under GNU/Linux). This can be overridden on the command line using the --config-dir parameter.
The main configuration files are lima-common.xml and lima-analysis.xml. This can be overridden on the command line using the --common-config-file and the --lp-config-file parameters respectively. lima-common.xml defines some general information common to all languages and the names of the files defining language specific data (by default lima-common-<lang>.xml for the lang language). lima-analysis.xml defines the names of the files describing language specific pipelines and process units (by default lima-lp-<lang>.xml for the lang language). It also defines a mapping between global and language specific pipeline names.
The file that you will mainly have to look at to change the behavior of LIMA on a given <lang> language is lima-lp-<lang>.xml.
The last configuration file, that can be very helpful to help debugging or to understand the internals of LIMA is the log4cpp.properties file. It allows to activate several levels of debugging information.
All LIMA XML configuration files have the following structure:
<?xml version='1.0' encoding='UTF-8'?>
<modulesConfig>
<module name="moduleName">
<group name="groupName">
<param key="paramName" value="param value"/>
<list name="listName">
<item value="1st item value"/>
<item value="2nd item value"/>
<item value="..."/>
</list>
<map name="mapName">
<entry key="FirstKey" value="1st key value"/>
<entry key="SecondKey" value="2nd key value"/>
<entry key="..." value="..."/>
</map>
</group>
<group name="...">
...
</group>
</module>
<module name="...">
...
</module>
</modulesConfig>One can include configuration data from external files with the following syntax:
<group name="include">
<list name="includeList">
<item value="<filename to include>/<module name to include>"/>
</list>
</group>This must be placed as a child of a module tag. This will include the content of the target module in the target file into the current module where the include statement is.
This is the file defining all processing done during linguistic analysis and the resources they use. It contains two modules: Processors for pipelines and process units and Resources for the linguistic resources.
It contains several groups in four categories:
- Definition of pipelines
- Definition of process units
- Definition of loggers
- Definition of dumpers
In fact loggers and dumpers are kind of process units but with a special role, respectively to write log messages tracing the results of some process units and to write or print final results.
Each group has a name and a class, which is the identifier of the C++ class to instantiate using the dedicated factory.
The pipeline groups are all of the class ProcessUnitPipeline:
<group name="main" class="ProcessUnitPipeline" >They contain one list named processUnitSequence whose items are the names of process units, loggers and dumpers. When a pipeline is selected (main by default or the one selected with the --pipeline= or -poption), its elements are executed in sequence. There is no check of the possible dependencies between units. This is the role of the user to define coherent sequences. Please refer to process units reference documentation for details about the role, dependencies, and configuration of each process unit.
The beginning of the main pipeline at the time of this writing is:
<group name="main" class="ProcessUnitPipeline" >
<list name="processUnitSequence">
<!--item value="beginStatusLogger"/-->
<item value="flattokenizer"/>
<item value="regexmatcher"/>
<!--item value="fullTokenXmlLoggerTokenizer"/-->
<item value="simpleWord"/>
<item value="hyphenWordAlternatives"/>
<item value="idiomaticAlternatives"/>As you can see, several elements are commented out. Some of them are pipeline units that can be activated to do more things, like semantic analysis. The others are loggers that one can activate to see the results of previous modules and dumpers alternative to the default one, the conllDumper. Note that there can be several dumpers activated. Note also that some dumpers need to use a handler different than the default one. The correct handler is activated on the command line by using the --dumper= or -d parameter.
The other preexisting pipelines (but one can define others) are:
- limaserver: the pipeline for the LIMA HTTP/JSON server;
- easy: produce output for the Easy parsers evaluation campaign;
- none: an empty pipeline that does nothing, for tests.
All other Processors groups have parameters specific to each class. Some of the parameters are references to linguistic resources defined in the next module.
This module describes linguistic resources that are loaded at initialization time and that can be referenced from the process units using their name. Each one is described in a group that has a name and and a class which is the id of the C++ class used to instantiate it.
Resources are described on a dedicated page
This file is in the log4j format. It allows to setup several categories defined in the C++ code. Each debug message is emitted in one category and at one level. If the level set for this category in the log4cpp.properties file is lower or equal to this category, then the message is printed on the standard output.
The levels are:
NOTSET < TRACE < DEBUG < INFO < NOTICE < WARN < ERROR < CRIT < ALERT < FATAL = EMERG
Note that the destination of each category (file, standard output, system logs, etc.) should be configurable, but it is not the case currently.
In Lima, plugins are shared libraries that can be loaded and linked dynamically.
This task is handled by the class Lima::AmosePluginsManager. It allows to add new process units or resources without having to recompile all LIMA executables.
The class Lima::AmosePluginsManager is declared and defined in the following files:
lima_common/src/common/AbstractFactoryPattern/AmosePluginsManager.hlima_common/src/common/AbstractFactoryPattern/AmosePluginsManager.cpp
It is exported to the library lima-common-factory.
The class Lima::AmosePluginsManager publicly inherits from Singleton<AmosePluginsManager> allowing it to exists in the form of a single instance at runtime.
An instance of that class is responsible for the plugins loading.
This task is done by the call of its unique method loadPlugins() in its constructor
at instanciation time.
What loadPlugins method does is to look for a plugins directory under the $LIMA_CONF folder.
Then, for each file found under plugins directory, it reads plugin names and deduces
the shared libraries that need to be loaded.
Once a shared library is identified, Lima::AmosePluginsManager delegates the task to
DynamicLibrariesManager so that it can actually load the library.
Any Lima project (lima_common, lima_linguisticprocessings) can have a corresponding plugins file.
These files are automatically filled during project compilation step.
In order to declare a library as a LIMA plugin, the developer needs to use the LIMA cmake macro
DECLARE_LIMA_PLUGIN (defined in each project's root CMakeLists.txt). This macro appends
a line containing the library name in the project plugins file, and add a cmake rule to create
a shared library (using the add_library command).
The class Lima::DynamicLibrariesManager is defined in the following files:
lima_common/src/common/AbstractFactoryPattern/DynamicLibrariesManager.hlima_common/src/common/AbstractFactoryPattern/DynamicLibrariesManager.cpp
It is exported to the library lima-common-factory.
The class Lima::DynamicLibrariesManager publicly inherits from Singleton<DynamicLibrariesManager> allowing it to exist in the form of a single instance at runtime.
An instance of that class is responsible for the dynamic loading of shared libraries that are passed to it. Indeed, it provides a method loadLibrary(const std::string& libname) for that task.
This class use the Qt QLibrary class to dynamically load libraries in a portable way. In order to prevent loading a library multiple times, it holds a map, called m_handles, where the key is the library name and the value is the instance of QLibrary class.
Lima::AmosePluginsManager::single();Only one call is needed. As written above, the constructor of AmosePluginsManager will load the plugins found under $LIMA_CONF/plugins.
Table of Contents generated with DocToc