Technische Universität Wien
Orientation and Processing of Airborne Laser Scanning data
Department of Geodesy and Geoinformation - Research Groups Photogrammetry and Remote Sensing
This part describes the different categories and types of parameters supported by the OPALS framework, how they are specified via the 3 bindings and / or configuration files, and how to transfer parameter values of preceding module invocations to the current one using parameter files.
Each parameter is associated with a certain name, type, semantic, category, and optionality.
OPALS parameters belong to either of 3 categories:
Every parameter is associated with a certain data type. All parameter types exposed by OPALS modules are either built-in data types defined by the C++ standard (int, double, etc.), container types defined in the C++ Standard Template Library (opals::Vector, opals::List, etc.), or custom data types declared in OPALS header files (e.g. enumerators like opals::LogLevel::Type). Thus, when using the C++ - interface and dynamic linkage, values of these data types may be instantiated and passed directly. However, when using OPALS modules as executables or Python modules, values that are convertible to the respective internal type need to be passed, where the set of convertible types / values depends on both the data type and the binding in use: parameter values for
Parameter types can be classified into
Parameters that specify file paths are of type opals::Path. When encountering parameter values of type opals::Path that begin with the (case-sensitive) string $OPALS_ROOT, the OPALS framework internally replaces this sub-string with the installation's root directory. Thus, "$OPALS_ROOT/addons/pal/standardPalette.xml" is translated internally to the absolute path of the standard palette file.
A parameter's optionality defines if users must, may, or must not set its value:
The optionality of each parameter is given in the respective parameter description.
OPALS executables may be invoked from the operating system's shell / command prompt by calling the respective file path. The characters following this path are split into a list of character strings (tokens) according to the system's rules. Under Windows, blanks are interpreted as token separators. This separation may be circumvented by enclosure in double quotes (for further information, see the respective page in MSDN).
The list of tokens is then interpreted by the OPALS framework:
Values extracted from the passed tokens are converted and set in alphabetic order of parameter names, except for the following common parameters, which are processed first, and in the following order:
The following code submitted on a command-line on Windows calls Module Import, sets the minimum importance of log messages to be exported to file to opals::LogLevel::verbose, and imports 2 files:
In the Python shell, specific parameters are accessible as Python attributes of OPALS Python module instances. Of course, numeric types expect numeric Python types to be passed - the same holds for character strings. Other C++ types expect the following counterparts from the Python shell:
Unlike specific parameters, common and global parameters are not accessible as direct attributes of an OPALS Python module instance. Instead, every module instance has the attributes "commons" and "globals", which in turn have these parameters as attributes.
The following Python code snippet instantiates Module Import, sets the minimum importance of log messages to be exported to file to opals::LogLevel::verbose, and imports 2 files:
Note: by default, Python interprets backslashes contained in strings as escape characters. As shown above, this can be circumvented by prefixing the string definition with 'r'.
Parameter values may not only be specified on the command-line or via the Python / C++ interface, but all bindings of OPALS modules additionally parse configuration files at certain locations and set parameters from these, if present:
$OPALS_ROOT/cfg/opals.cfg$OPALS_ROOT denotes the root directory of the OPALS installationAs the same parameter may be present in more than 1 configuration file, a precedence has been defined: parameter values specified in configuration files mentioned later in the list above override those specified in files listed earlier. Kindly note that this also holds true for container types: values specified in different configuration files do not add (while multiple definitions in the same file do, see below). Furthermore, parameter values specified via the command-line / interface override any corresonding values found in (non-user-) configuration files.
Note that user-configuration-files are processed (and respective values are set) immediately. Thus, if a parameter value is specified both in a supplied user-configuration-file and via the interface, the order of respective member function calls is of importance.
Configuration files are INI-like ASCII text files that are interpreted line by line. Lines containing white-space only are ignored, as all characters including and following a hash mark ('#') are. Parameter values are generally specified using the syntax
i.e. the character string up to the first equals sign ('=') (excluding leading and trailing white-space) is interpreted as the name of the parameter that the value(s) represented by the character string following the equals sign shall be assigned to. The string representing the value(s) to be assigned is split into a list of tokens in the same way as is done on the command-line (see Specification of Parameters on the Command-Line). Multiple specifications of container type parameters in the same configuration file add up. Thus, several elements may be assigned to a container type parameter in either of two ways:
Using this syntax, the value of the parameter associated with the respective name will be set for any module that supports this parameter. However, setting the value can be restricted to a certain module only using the following sytax:
The same effect can be achieved by placement of the module name in squared brackets on a preceding line:
OPALS supports variable substitution within configuration files. It is possible to refer to other parameters in the value string as shown below (The concept is also known as (extended) interpolation in the configparser class in python). The feature can be applied e.g. within package script configuration files for defining parameter values only once or defining all adaptable parameters in a central section called [script.variables]. Parameters from the current section or the [script.variables] section can be referenced without section prefix. Reference from other sections require the full section prefix. E.g.
An example using the central variable section is in the following
For completeness it is mentioned that the substitution mechanism is working recursively up to a depth of 10. Hence, a variable that refers to an other variable is fully supported.
The substitute implementation searches for the ${...} pattern in all parameter values. Currently, there is no way of disabling the substitution mechanism. Since the distinctive pattern may not occur in OPALS filter syntax and is unlikely to be used within file names, this shortcoming should be of minor concern.
The invocation of each module's central function (run()) returns a list of all final module parameters, possibly including unchanged input-parameters, but more interestingly also (estimated) paths of exported files and computed results not stored to file. The list of final parameters may be
This list of final parameters may then be used to set the parameters of a later module call (probably of another module) by:
When passing a list of parameters as input to the current module, the common parameter 'paramMapping' comes into play, as it specifies which parameters in the list shall be assigned to which parameters of a module. The syntax used is somehow similar to the one used in configuration files.
Takes the last parameter with name 'ParamNameToUse' found in the (ordered) list of parameters, and assigns it to the parameter named 'ParamNameToAssign' of the current module. This mapping of output to input parameters may be restricted to output parameters from certain modules only:
Assigns the parameter named 'ParamNameToUse' of the last call of only the module named 'OutputModule' to the parameter named 'ParamNameToAssign' of the current module. Likewise, parameters may be mapped to parameters of certain modules only:
Assigns the parameter named 'ParamNameToUse' to the parameter named 'ParamNameToAssign' only, if the current module is named 'InputModule'. These two types of restrictions may be combined to result in a most restrictive mapping:
Assigns the parameter named 'ParamNameToUse' from the last call of 'OutputModule' to the parameter named 'ParamNameToAssign' of the current module, if it is named 'InputModule'.
Multiple mappings to the same parameter must be separated by commas:
Assigns the last ocurrence of any parameter either named 'ParamNameToUse' or 'AnotherParamNameToUse' to the current module's parameter named 'ParamNameToAssign'.
Mappings to different parameters must be separated by semicolons:
Concerning container types to be assigned, parameter mapping works in a greedy manner: while scalar parameters and the fixed-size container opals::Array take exactly as many values from the list as needed (1 or N, resp., beginning at the end of the list), all values found in the list that satisfy the mapping rules will be assigned to vector-type parameters.
As the OPALS framework does not check the types of parameters to be mapped beforehand, it is the user's responsibility to assure that values to be used are convertible to the types of parameters to be assigned.
Note: as the character string specifying the value of the common parameter 'paramMapping' is split into tokens like any other parameter value on the command-line or configuration file, it is necessary to enclose it in double quotes under Windows, if it contains white-space.
In addition to restrictions based on module and parameter names, the parameter values of the current module and the parameter in question may optionally be combined to a logical expression. If for a certain mapping, the restrictions on module and parameter names are fulfilled, then the value from list/file is only assigned to the corresponding value of the current module, if this expression evaluates to true. If given, expressions must follow the mapping, separated by white space, and be formulated in Python syntax. The current module's parameters are exposed through the variable named 'me', having attributes only for the currently set parameters. The parameter under question may be accessed via the variable 'you'. 'you' and the exposed attributes of 'me' are fully typed, why respective methods/member functions may be invoked. Depending on the complexity of the involved parameter types, elaborate expressions may be composed, eventually using functions built into Python. As the parameter mapping string itself must be double quoted, it is advisable to enclose string constants in single quotes.
In a configuration file, set the parameter mapping for all modules in a way that the final parameter 'outFile' of Module Import will be assigned to the parameter 'inFile' of Module Grid :
The same effect can be achieved by setting the parameter 'paramMapping' for Module Grid only:
The following code submitted to the command-line writes the final parameters of Module Import to a parameter file. The subsequent call to Module Grid uses this file in combination with the parameter mapping defined as above in a (default-) configuration file to set the ODM file path as input:
The code snippet below shows a parameter mapping being restricted by a data-dependent expression: a transformation of type opals::TrafPars3dAffine produced by opalsGeorefApprox shall be assigned to the parameter 'trafo' of opalsExport (having the same type). Separated by white space, the condition in Python syntax follows the actual mapping, restricting the set of candidate parameters to those whose member 'IdGridMov' is a prefix of the current module's inFile - strictly speaking, the respective file stems are compared:
The following Python script passes the list of final parameters of Module Import as input to Module Grid, using the variable 'results' for storage:
As in this simple example, only one scalar, numeric parameter is passed via the parameter file, of course the following approach is equally practical:
Possible values: none ...... Suppress all logging output. error ..... Some failure that cannot be handled. Program execution is aborted. warning ... Some weird program state which can still be handled (e.g. 'poor matrix condition', 'poor data distribution') info ...... Some progress that may be interesting in everyday-use verbose ... Something that may help in understanding normal program behaviour debug ..... Anything that may help 'debugging' by means of a release-buildThe minimum level of log message importance to print on the screen
Possible values: none ...... Suppress all logging output. error ..... Some failure that cannot be handled. Program execution is aborted. warning ... Some weird program state which can still be handled (e.g. 'poor matrix condition', 'poor data distribution') info ...... Some progress that may be interesting in everyday-use verbose ... Something that may help in understanding normal program behaviour debug ..... Anything that may help 'debugging' by means of a release-buildThe minimum level of log message importance to export to the xml-log
Possible values: uint8 ...... 8 bit unsigned integer uint16 ..... 16 bit unsigned integer int16 ...... 16 bit signed integer uint32 ..... 32 bit unsigned integer int32 ...... 32 bit signed integer float32 .... 32 bit floating point float64 .... 64 bit floating point cint16 ..... Complex 16 bit signed integer cint32 ..... Complex 32 bit signed integer cfloat32 ... Complex 32 bit floating point cfloat64 ... Complex 64 bit floating pointModules that create output grid/raster files may create these with this cell/pixel data type.
1.8.17