This document describes the means provided by OPALS to filter vector data. In the context of OPALS, filtering denotes the generation of a new data set based on another data set using element-wise selection criterions and / or transformations.
The OPALS datamanager (ODM) is designed to store arbitrary attributes (see ODM predefined attributes) along with vector data. Hence, OPALS modules use the ODM as the central resource of vector data. Users may want to transform the data geometrically before import, or process only a subset of the data provided by an ODM, e.g. data within a certain region only. Likewise, only a subset of the processed data may need to be exported. This is where filters come into consideration.
For the purpose of filtering, OPALS provides a number of filter types that belong to one of two classes (except for Generic):
Most filters perform their operations based on proper parameters. The table below lists the implemented filter types together with their operations.
Name | Class | Inspects | Parameters | Operation |
Generic | selector, transformer | attributes, point coordinates | An expression consisting of constants, any of the supported attribute identifiers, and logical / comparison / arithmetic operators. | Whole geometry objects for whose attributes the expression evaluates to true pass through. |
Region | selector | geometry | A simple (non-self-intersecting) polygon. | Data (or datum parts) within the polygon pass through. |
Echo | selector | attributes | a set of acceptable echo descriptors | Points with acceptable echo attributes pass through. The given descriptors are transformed internally to valid combinations of the attributes EchoNumber and NrOfEchos. |
Class | selector | attributes | a set of acceptable classifications | Points with acceptable classifications (attribute Classification) pass through |
Synthetic | selector | attributes | – | Points having set the ‘synthetic’ bit (within the attribute ClassificationFlags) pass through |
KeyPoint | selector | attributes | – | Points having set the ‘keypoint’ bit (within the attribute ClassificationFlags) pass through |
Withheld | selector | attributes | – | Points having set the ‘withheld’ bit (within the attribute ClassificationFlags) pass through |
Semantic | selector | attributes | a set of acceptable semantic descriptors | Data with acceptable semantic descriptors (attribute ScopSemantic) pass through |
Affine | transformer | geometry | the 12 parameters of a 3d affine transformation given by a transformation matrix \( A = [a_{i,j}]_{3 \times 3}\) and a translation vector \( b = [b_{k}]_{3 \times 1}\) | The coordinates \(\{X,Y,Z\}\) of data vertices are transformed to \(\{X',Y',Z'\}\) by application of \[ \begin{pmatrix} X' \\ Y' \\ Z' \end{pmatrix} = \begin{pmatrix} a_{1,1} & a_{1,2} &a_{1,3} \\ a_{2,1} & a_{2,2} & a_{2,3} \\ a_{3,1} & a_{3,2} & a_{3,3} \end{pmatrix} \begin{pmatrix} X \\ Y \\ Z \end{pmatrix} + \begin{pmatrix} b_{1} \\ b_{2} \\ b_{3} \end{pmatrix} \] Attributes are passed unchanged. Inversion of this filter yields the application of the inverse transformation. |
EchoWidthDepOnAmp | selector | attributes | a continuous, piecewise linear function | The function defines the maximum echo width (attribute EchoWidth) for data to be acceptable, as a function of their amplitude (attribute Amplitude). It is thought to extend infinitely with constant value at its marginal nodes. Data featuring echo widths smaller or equal to this adaptive threshold pass through. |
PointSeq | selector | attributes | a set of acceptable indices | The order of appearance (e.g. as read from file) of data is inspected. Points that appear consecutively and feature the same GPS-time (attribute GPSTime) are regarded as a sequence. Data featuring acceptable indices i.e. positions within these sequences pass through. Thus, as an exception to the rule that filters apply element-wise criterions, PointSeq considers data appearing before or after. |
PointSeq2Echo | transformer | attributes | – | The order of appearance (e.g. as read from file) of data is inspected. Points that appear consecutively and feature the same GPS-time (attribute GPSTime) are regarded as a sequence. To each point in such a sequence, the attributes EchoNumber and NrOfEchos are assigned, based on the point's position within the sequence and the sequence length. |
Remove | transformer | attributes | none, one, or more attribute identifiers | If no attribute identifiers are given, then this filter removes all attributes from all data. This way, memory requirements may be lowered. If one or more attribute identifiers are specified, then only these attributes will be removed from all data. |
Primitive | selector | – | a set of acceptable primitive types | Data of the specified primitive types pass through. |
Pass | – | – | – | All data pass through unmodified. This filter cannot be combined with others, and its specification simply underlines that no filter is used. |
Filters are invertible and may be combined to composite filters i.e. arbitrarily large filter trees. OPALS filters and filter trees are created based on definition strings that adhere to a special syntax.
Module parameters containing the word 'filter' expect strings of this structure. Informal syntax definition with examples presents a rather colloquial definition of this syntax, together with exemplary filter definition strings. The syntax is defined formally in section Formal grammar
Filters are specified using their names. For filters depending on parameters, respective filter parameter definitions must be appended, enclosed in square brackets. These must adhere to filter-specific parameter syntaxes, which are given in the table below.
Filter name | Parameter syntax | Exemplary filter definition | |||||||||||||||||||||
Generic | see the separate page for Generic filter |
| |||||||||||||||||||||
Region | The query or analysis to be performed (optional), followed by
.
|
| |||||||||||||||||||||
Echo | One or more echo descriptors. Valid descriptors are:
Last : echo number == number of echoes Intermediate: NOT( First OR Last ) Single: First AND Last Multiple descriptors are combined with OR. |
| |||||||||||||||||||||
Class | One or more classification descriptors. Valid descriptors are:
| Class[Ground Water] terrain-data or data on water Class[64] user-defined class 64 | |||||||||||||||||||||
Synthetic | – | Synthetic Data whose synthetic-bit is set i.e. data that hold a ‘ClassBits’-attribute, and whose synthetic-bit within the class-bits is 1 | |||||||||||||||||||||
KeyPoint | – | KeyPoint Data whose keypoint-bit is set | |||||||||||||||||||||
Withheld | – | Not Withheld Data whose withheld-bit is not set i.e. data that either do not hold a ‘ClassBits’-attribute, or whose withheld-bit is 0 | |||||||||||||||||||||
Semantic | One or more semantics descriptors. Valid descriptors are:
|
| |||||||||||||||||||||
Affine | a sequence of 2, 3, 4, 6, 9, or 12 real numbers. If less than 12 values are supplied, then the other parameters are set to neutral values w.r.t. the transformation.
|
| |||||||||||||||||||||
EchoWidthDepOnAmp | a sequence of one or more 2-tuples of real numbers, separated by commas. In each tuple, the first element specifies the amplitude for which the second element specifies the maximum admissible echo width. | EchoWidthDepOnAmp[ 150 1.7, 100 1.75, 30 1.8 ] | |||||||||||||||||||||
PointSeq | One or more position descriptors. Valid descriptors are:
|
| |||||||||||||||||||||
PointSeq2Echo | – | PointSeq2Echo | |||||||||||||||||||||
Remove | none, one, or more predefined or custom attribute identifiers |
| |||||||||||||||||||||
Primitive | One or more primitive type descriptors. Valid descriptors are:
|
|
Filters may be inverted by prefixing the filter name with the unary operator NOT. Furthermore, two filters are combined logically by inserting either of the binary operators AND and OR between them. Finally, filters may be grouped by enclosing them in round brackets. Filters are generally evaluated from left to right. The operator precedence and hence the structure of filter trees will be familiar to users: NOT is evaluated first, then AND, finally OR.
For convenience, the three logical operators have aliases: ! for NOT, && for AND, and || for OR.
Following are two examples for composite filters. For the sake of better readability, linebreaks have been inserted.
Select last-echo-data that is either inside a window, or features some classification flag and either of two semantics:
Select data with acceptable echo width, translate them and finally remove any attributes:
For the graphical representation of formal grammars, we use railroad diagrams. These diagrams display
terminals (characters to be typed) as rounded rectangles filled with dark yellow, | |
non-terminals (syntax rules) as rectangles filled with medium yellow, and | |
regular expressions (rules that input text must stick to) as hexagons filled with light yellow. |
The generic filter syntax represented as railroad diagrams. The syntax for rules GenericFilterAttrib
and Attribute
can be found here: GenericFilter.
|
In computer science, the Extended Backus Naur Form (EBNF) is a metasyntax notation used to express context-free grammars: i.e., a formal way to describe computer programming languages and other formal languages. It is an extension of the basic Backus Naur Form (BNF) metasyntax notation.
EBNF was originally developed by Niklaus Wirth. However, many variants/dialects of EBNF are in use. The World Wide Web Consortium (W3C) has adopted an EBNF standard for the specification of XML, which is used to define the syntax of OPALS filter definition strings formally.
This W3C variant of EBNF uses the following conventions:
The W3C-EBNF conforming filter string syntax is defined as follows. The syntax for rules GenericFilterAttrib
and Attribute
can be found here: GenericFilter.