Module AddInfo
See also
opals::IAddInfo

Aim of module

Inserts or updates ODM attributes by combining existing attributes and/or grid models.

General description

AddInfo constitutes a very general and flexible tool for creating new or updating existing point attributes by combining point attributes and/or grid models (grid values interpolated at the point locations).

See also
Module AddInfo

The main feature of the OPALS Data Manager (ODM), besides the high performance spatial indexing, is the capability of storing arbitrary attributes (c.f. OPALS Software Concept). The attributes either stem from the analysis of the sensor data (e.g. amplitude, echo width, nr. of echos per laser pulse), but may also be a result of data processing. For the latter, the modules Module Normals and Module EchoRatio serve as examples. In contrast, the Module AddInfo provides a generic way of creating new or updating existing attributes by freely combining the attributes of a specific data point and its nearest neighbours with additional raster layers.

The additional attributes can be used in subsequent modules either for customized data selection via filters or even as basic information to control the model output (e.g. for data segmentation).

Among the potential applications of Module AddInfo are:

  • Normalized height (nH): relative height of a data point above a DTM grid, nH=z(pt)-z(DTM)
  • Object height (objH): height of DSM above DTM (does not require attributes of the data point itself, only its location in order to interpolate the DSM/DTM heights), objH=z(DSM)-z(DTM)
  • Relative height (relH): relative height of a data point w.r.t. DTM and DSM which can be used to quantify the relative location of a data point within a vegetation layer, relH=(z(pt)-z(DTM))/(z(DSM)-z(DTM))
  • Classification: simple data classification by evaluating existing point attributes. E.g. vegetation can be detected by thresholding the EchoRatio attribute
  • Statistical analysis of neighbouring points: e.g. mean instensity of all nearest neighbour points
  • Storing attributes from neighbouring points: e.g. generation of multi spectral lidar datasets by mapping the spectral information from different wave lengths stored in the same ODM.
  • ...


The main input for Module AddInfo are the ODM to operate on (inFile), the attribute assignment formula ( attribute), and a list of (optional) grid models involved in deriving the attribute values (gridModel). Additionally, Module Addinfo also provides access to the nearest neighbour (nn) points while iterating through the entire point cloud. The nn points are queried by specifying the number of neighbour points (neighbours) limited by a maximum search radius (searchRadius) and the general search and selection mode (searchMode, selMode). Please refer to Module Normals for a detailed description of the different search parameters.

The attribute names can be chosen from the list of predefined attributes with a fixed semantic and data type (c.f. ODM predefined attributes). User-defined attributes can be generated too, in which case (i) the name must be preceeded with an underscore and (ii) the data type must be provided in parentheses (e.g. attribute _myFloatAttrib(float)). A list of all supported attributes as well as more information concerning user-defined attributes can be found here. If no data type is assigned to a user-defined attribute, the type is estimated from the respective formula.

Additionally, one or two filter strings can be specified to select a subset of ODM data points (parameter filter). Please refer to Filters for a detailled description of available filters and their syntax). The first filter is interpreted as 'processing filter' and restricts the set of points for which additional attributes are to be calculated. The second filter serves as 'neighbourhood filter' and afftects the set of points returned in local neighbourhood queries. If only one (processing) filter is specified, the same filter is used as neighbourhood filter. If, however, the neighbourhood queries should consider all ODM points, the "Pass" filter or an empty filter (i.e. " " from within the commandline or "" in Python) must be used. Please note, that neighbourhood filters are only considered if spatial queries are activated (i.e. neighbours and/or searchRadius specified).

If grid models are involved in the formula, the respective grid values at the ODM data point locations are interpolated using either bilinear (default) or nearest neighbour resampling.

Attribute assignment formula syntax

For the assignment of additional ODM attributes the Generic filter syntax is used. Besides attribute based data selection, generic filters also allow assigning new attributes as the result of an arbitrary algebraic combination of other point attributes, the attributes of neighbouring points and/or raster layers. A simple example for assigning an estimate of the signal intensity as the product of the full waveform attriibutes Amplitude and EchoWidth is:

_Intensity(integer)=Amplitude*EchoWidth

Please note, that _Intensity is a user-defined attribute (indicated by the preceeding underscore) for which a corresponding type (i.e. 4-byte integer number) has to be specified in parentheses. If the type specification is omitted, the respective type is automatically estimated by the generic filter. Thus, we can also write:

_Intensity=Amplitude*EchoWidth

... in which case the corresponding type would be estimated as "float" as the both attributes on the right hand side (Amplitude, EchoWidth) are of float type.

A more complex example, assigning the LAS vegetation class identifier (medium vegetation=4) to all echoes with an echo ratio less than 75%, using the ternary conditional operator is:

Classification = EchoRatio < 75 ? 4 : Classification

This example illustrates conditional attribute assignments (if..then..else). The Classification attribute is set to 4 if EchoRatio < 75, else the attribute is unchanged (i.e. the old attribute value is re-assigned and in case the attribute does not yet exist, it is not inserted at all). This example also shows the operator precedence: First, the condition on the right hand side is evaluated (potentially including the original attribute value) and only then the new value is assigned. For a detailed description of the operator precedences within generic filters, please refer to here.

Generic filters also allow to query the existence" of a respective attribute. This is especially use full if the derivation of an attribute depends on the existence of certain other attributes. The follwoing example creates a boolean smoothness indicator depending on the existence of standard deviation of the normal vector estimation.

_smooth(boolean) = SigmaNormalFit ? SigmaNormalFit > 0.1 : false

The above example returns true (i.e. point is located in a smooth environment) only if the attribute SigmaNormalFit exists and is less than 10cm.

Generic filters allow for assignement of multiple attributes in a single (filter) string. Therefore, the individual attribute assignments have to be separated by a semicolon (;).

NormalizedZ = z - r[0]; _relativeHeight(float) = r[0]-r[1] ; _dist2nn(float) = dist2d(n[0],n[1])

In the above example:

  • the predefined attribute NormalizedZ and the user-defined attribute _relativeheight are added to the ODM
  • the point height (z) is for deriving a point attribute (the 3D point coordinates can be accessed as: x, y, z)
  • raster layers are used for the attribute derivation. r[0] refers to the first and r[1] to the second input grid/raster model (i.e.: r[0]=DTM, r[1]=DSM)
  • the distance from the dat point at hand to its nearest neighbour _dist2nn is calculated using a predefined generic filter function operating on two points of the nn point set.

The formulae can use all available information of the data point (coordinates and existing point attributes), its nearest neighbours as well as all values of the input grid models at the location of the data point. The following well-known variable names can be used in the formulae:

Variable name Description
x x-coordinate (Easting) of data point
y y-coordinate (Northing) of data point
z z-coordinate (Height) of data point
r[i] pixel value of i-th input raster/grid model (index starting with 0)
n[i] i-th nearest neighbour point (index 0 starting with 0, referring to the current data point itself
invalid general void indicator

Furthermore, all existing point attributes can be used by their respective names.

Parameter description

-inFileinput data manager file
Type: opals::Path
Remarks: mandatory
The path to the OPALS datamanager whose point attributes are to be extended.Read and Write access is required for the ODM file.
-attributeattribute assignment formula
Type: opals::String
Remarks: mandatory
The formula for assigning ODM point attributes is expected in 'Generic filter' syntax. Multiple attribute assignments need to be separated by semicolons. Both predefined and user-defined attributes are supported. For the latter, the type is either estimated automatically or can be specified in parenthesis (following the attribute name).
-gridFilegrid model file
Type: opals::Vector<opals::RasterFile>
Remarks: optional
A list of additional grid/raster files. The grid/raster layers can be used in the attribute assignment formula as: r[0], r[1]... according to their corresponding input order.
RasterFile: File path of a raster dataset.
See Supported grid/raster formats
-tilePointCountchunk size for data processing
Type: uint32
Remarks: default=5000
The parameter is used to restrict the concurrent number of ODM points for data processing. This is especially useful in case grid/raster layers are involved as the grid/raster values have to be interpolated for all ODM point locations and all layers. As this operation might become memory intensive, reducing the the number of concurrent points helps keeping the memory consumption moderate.
-neighboursnr of nearest neighbours to be considered
Type: int32
Remarks: optional
If specified, the nearest neighbour points (geometry and attributes) can be used for calculating new / updating existing attributes. C.f. documentation of generic filters for detailed information how to incorporate nn-points in the attribute formula.
-searchRadiusmaximum search radius for point selection
Type: double
Remarks: optional
Only points within the given search radius are considered: This parameter is only taken into account if -neighbours is specified as well.
Estimation rule: If not set, the search radius is estimated using the average point density and the number of neighbours
-selModedata selection mode
Type: opals::SelectionMode
Remarks: default=nearest
Possible values:  
  nearest .... pure nearest neighbour (nn) selection 
  quadrant ... quadrant-wise nn selection, ie. nn per quadrant, then 2nd nn per quadrant, ...
  octant ..... octant-wise nn selection, ie. nn per octant, then 2nd nn per octant, ...
Please note that quadrant wise data selection on the one hand reduces extrapolation effects, but on the other hand decreases the processing performance
-searchModedimension of nearest neighbor search
Type: opals::SearchMode
Remarks: default=d2
Possible values:  
  d2 ..... Search based on 2D coordinates (x and y) only 
  d3 ..... Search based on full 3D coordinates (x,y and z)
  d2_5 ... Search based on 3D coordinates but with different horizontal and vertical scales
Defines if 2D or 3D distances should be used for the nearest neighbor search
-filterfilter string
Type: opals::Vector<opals::String>
Remarks: optional
One or two filter strings in EBNF syntax (as described in section 'Filter syntax') can be passed to restrict the set of input points. The first filter specifies for which points an attribute should be calculated (if not specified, attribute calculation is carried out for all ODM points). The second filter defines which points are used for the spatial query and is only considered if at least one of parameters 'neighbours' or 'searchRadius' is set. If not specified, all ODM points are used for the spatial queries.
-resamplinggrid resampling method.
Type: opals::ResamplingMethod
Remarks: default=bilinear
Possible values:  
  nearestNeighbour ... nearest neighbour
  bilinear ........... bi-linear interpolation
Method to be used for interpolating grid values at the data point locations.

Examples

The examples presented in this section rely on the on the data sets strip21.laz and strip11.laz located in the directory $OPALS_ROOT/demo/ of the OPALS distribution. As a prerequisite, please import the LAS file into an ODM and derive a surface model using the following commands:

opalsImport -inFile strip21.laz
opalsImport -inFile strip21.laz strip11.laz -outFile strip2111.odm
opalsGrid -inFile strip21.odm -interpolation movingPlanes -filter Echo[last]

Example 1:

The first example shows how to derive the relative height of a data point with respect to a surface model.

opalsAddInfo -inFile strip21.odm -gridFile strip21_z.tif -attribute NormalizedZ=z-r[0] -nbThreads 1

For each point, the formula (attribute) calculates the difference between the point elevation (z) and the respective grid value of the specified surface (r[0]) at the planimetric position of the data point (x,y). The result is stored in the predefined attribute" NormalizedZ using the inherent data type float.

Please note, that the NormalizedZ attribute is only set, if a grid value exists at the x,y location of the data point, otherwise no such attribute is appended to the point.

After the (successful) execution of the above command, the new attribute is available for all subsequent operations based on the ODM. E.g. a histogram of all object heights can be derived by...

opalsHisto -inFile strip21.odm -attribute NormalizedZ

... resulting in the following output (excerpt).

CountData: 371682
CountUsed: 370607
Min: -78.066
Max: 18.413
Mean: 0.293
Median: 0.001
Std: 1.651
Rms: 1.677
SigmaMad: 0.057
Skewness: 0.598

Example 2:

This example shows how to combine/join information from distinct datasets stored in the same ODM. strip2111.odm contains laser echos of both strip21 and strip11. For each echo, the strip number is stored in the attribute PointSourcdId. This information is used in the subsequent command to process all points from strip21 (by specifying a first=processing filter) and calculate the elevation difference to the closest point of strip11 (by using a second=neighbourhood filter).

opalsAddInfo -inFile strip2111.odm -neigh 1 -searchRad 5 -attribute "_dz(float)=z-n[0].z" -filter "generic[PointSourceId==21]" "generic[PointSourceId==11]"
Author
gm
Date
23.06.2016
@ nbThreads
number of concurrent threads
@ NormalizedZ
point height above DTM
@ EchoWidth
Full width at half maximum [ns].
@ gridFile
grid model file
@ movingPlanes
moving (tilted) plane interpolation
opalsAddInfo is the executable file of Module AddInfo
Definition: ModuleExecutables.hpp:8
@ last
last stage to be processed (opalsStripAdjust)
opalsImport is the executable file of Module Import
Definition: ModuleExecutables.hpp:113
@ Amplitude
Linear scale value proportional to the receiving power.
opalsGrid is the executable file of Module Grid
Definition: ModuleExecutables.hpp:93
@ filter
string to be parsed in construction of DM::IFilter (various modules)
opalsHisto is the executable file of Module Histo
Definition: ModuleExecutables.hpp:103
@ Classification
See LAS spec.