Generic filter is one of the Filters provided by OPALS to filter vector data. Among them, generic filter is a rather flexible one: it allows for evaluating arbitrarily complex expressions consisting of any of the supported attribute identifiers (see ODM predefined attributes), coordinate identifiers, raster identifiers, neighbor geometry identifiers, constants, scalarvalued unary and binary functions, and a bunch of arithmetic, relational, logical, conditional, and assignment operators.
For each object to filter, generic filter (generally) substitutes attribute identifiers contained in the expression for the object's corresponding attribute values, and coordinate identifiers for the object's coordinates. If the subsequent evaluation converts to true, then this object passes through the generic filter.
The expression syntax not only provides means to use attribute and coordinate values, but also to test the presence of attributes and the validity of coordinates.
Kindly note that filtering by coordinates is applicable only to point data i.e. any lines will not pass. Depending on the OPALS module, the set of admissible operators may be limited.
In the following, the expression syntax is discribed twice: once in a rather colloquial way (see Informal syntax definition with examples), including some exemplary expressions, once in a formal way (see Formal grammar ).
Generic filter expressions constitute rooted binary trees, where the leaves are either attribute identifiers, coordinate identifiers, or constants. If the root node is an attribute identifier, then the evaluation returns the presence of the attribute, not its value. Likewise, if the root node is a coordinate identifier, then the evaluation returns the validity of the coordinate (and not its value). Thus,
Generic[SigmaZ]
is a valid generic filter consisting of a leaf only: objects that feature the specified attribute pass through, regardless of the attribute's value.
Generic[Z]
is a valid generic filter, too. Objects featuring valid zcoordinates pass through.
Leaves may be combined to arithmetic expressions using resp. operators and functions (see the table below), regarding the usual operator precedence:
Generic[0.1 + 2*SigmaZ]
is a generic filter consisting of 3 leaves and 2 binary operators: as the attribute identifier is not the root node, the corresponding attribute value is evaluated. Objects that feature the specified attribute and for whose attribute value 0.1 + 2*SigmaZ
converts to true
(i.e. the result is different from zero) pass through. As one may expect, the attribute presence and value are evaluated first, then the multiplication is performed, followed by the addition, and finally, the resulting real number is converted to a boolean value.
Leaves and / or arithmetic expressions may further be combined to comparative expressions using resp. operators:
Generic[SigmaZ < 0.1]
Generic[2*SigmaX < SigmaY]
Generic[PointLabel == "Vienna"]
Generic[Z > 100]
Leaves, arithmetic expressions, and comparative expressions may be combined to logical expressions using the operators already mentioned in Filters :
Generic[SigmaZ < 0.1 AND PointLabel == "Vienna"]
lets data pass through that feature both the attributes SigmaZ
and PointLabel
, and for whom both comparisons evaluate to true.
Generic[SigmaZ < 0.1 OR PointLabel == "Vienna"]
lets data pass through that either feature the attribute SigmaZ
, and whose value is smaller than 0.1, or data that feature the attribute PointLabel, and whose value is Vienna
, or data that meet both conditions.
Generic[Z >= 0 AND Z <= 100]
lets (point) data pass through that feature valid zcoordinates within the (inclusive) interval [0, 100].
Like the root node, logical operators show a special behaviour concerning the evaluation of attributeleaves if applied directly as operands:
Generic[SigmaX AND SigmaY]
does not evaluate any attribute value, but combines the presence of the 2 attributes, i.e. data pass that provide both attributes, regardless of their values. In
Generic[SigmaX OR SigmaY]
objects that feature one or both of the specified attributes pass through, while in
Generic[SigmaX OR SigmaY < 0.1]
objects pass through that either provide the attribute corresponding to SigmaX
, or hold SigmaY
, with a value smaller than 0.1. Data that meet both conditions are also passed through.
Logical expressions may serve as a condition for the ternary conditional operator. This operator evaluates to its 2^{nd} argument if the condition (1^{st} argument) evaluates to true, and to its 3^{rd} argument otherwise:
Generic[x >= 0 ? y : z ]
lets data pass through whose xcoordinates are larger than or equal to zero and who have valid ycoordinates; otherwise, those who have valid zcoordinates.
Kindly note that currently, it is not possible to use parenthesis within the condition.
Finally, the assignment operator is supported, which may e.g. serve for setting attributes during import/export:
Generic[amplitude = 100*_myIntensity]
The assignment operator always assigns to its 1^{st} argument its 2^{nd} argument, regardless of the 2^{nd} argument's validity, and it lets all data pass through. Thus, in the above example, for objects that feature the userdefined attribute _myIntensity
, this userdefined attribute times 100 is assigned to the (predefined) attribute amplitude
. amplitude
results as invalid for objects that do not feature _myIntensity
, no matter if amplitude
was valid before or not. Regardless of the resulting validity of amplitude
, all objects pass through the filter.
Multiple assignments may be specified, separated by a semicolon.
For userdefined attributes, their data type may explicitly be given enclosed in parentheses, following the userdefined attribute identifier:
Generic[_myQuality(float) = pow( SigmaX*SigmaY*SigmaZ, 1/3 ) ]
For objects that feature the predefined attributes SigmaX
, SigmaY
, and SigmaZ
, this filter assigns to the userdefined attribute _myQuality
of data type float
a valid value. For all other objects, it renders the attribute _myQuality
as invalid.
Generic filters support a set of statistical operators via the general syntax:
operator(.)
where operator
is one of the identifiers that belong to rule Stat: min, max, mean, sum, stdDev, ... . With this syntax, statistical operators are applied to the valid items among the supplied ones. These items can either be the
r
, or then
. In this case, either a coordinate or an attribute identifier to be evaluated must follow, separated by a dot. The exception to this rule is count(n)
, which counts the number of supplied neighbor geometries, independent of their coordinates and attributes.As mentioned, only valid items among the supplied ones are considered in the statistical operations. Thus, e.g.
Generic[ _myStat(float) = max(r) ]
determines the maximum value of all valid rasters (i.e. not NODATA) and assigns that maximum to the userdefined attribute _myStat
of type float
, while
Generic[ _myStat(float) = max(n.z) ]
does the same for the valid zcoordinates (i.e. of finite value) of all neighbor points.
count(n)
considers all supplied neighbor geometries as valid, and hence, the following assigns the number of supplied geometries to the userdefined attribute _myStat
:
Generic[ _myStat(float) = count(n) ]
If only a subrange of the supplied rasters shall be evaluated, then use the following syntax, which follows the indexing syntax of Python sequence types:
operator(r[beg:end])
where beg
is the zerobased index of the first supplied raster to be evaluated, and end
is the index of one after the last supplied raster to be evaluated. Both beg
and end
are optional and default to 0 and the number of supplied rasters, respectively. Negative indices are added to the number of supplied rasters, and hence refer to the end of the array of supplied rasters.
Generic[ _myStat(float) = max(r[0:2]) ]
thus determines the maximum value of the valid rasters among the first 2 of all supplied rasters and assigns that maximum to the userdefined attribute _myStat
of type float
. The same indexing syntax can be used to select a subrange of neighbor geometries, e.g.:
Generic[ _myStat(float) = max(n[0:3].SigmaX) ]
assigns the maximum value of the valid attributes SigmaX
among the supplied neighbor geometries, excluding the last 3 neighbor geometries.
Random numbers may be generated using the general syntax:
random( distribution, arg1 [, arg2] )
where distribution
is one of the distribution names mentioned in column "Meaning" for rule random: uniform_int, normal, ... . The distribution name must be followed by the distribution parameters, given in column "Symbol(s)", separated by commas. Hence,
random( normal, 0, 1 )
will generate real numbers distributed according to the standard normal distribution (with mean 0.0 and standard deviation 1. ). Note that the combination of the uniform integer distribution and the modulo function provides a simple way of random subsampling. To e.g. select 1 out of 10 points, use:
random( uniform_int, 1, 10 ) == 1
For a different way to do subsampling, see Serial number generator .
The enumeration of nonnegative integer numbers in ascending order (0, 1, 2, ...) is gained with:
serial()
This offers a way to subsample data based on its order of evaluation. To e.g. import only every 4th geometry, one may call Module Import like this: opalsImport infile data.las filter "Generic[fmod(serial(), 4) == 0]"
For a different subsampling method, see Random number generators.
When using this nullary function in a more complex filter tree, mind that each serial number generator holds its own position in the sequence of nonnegative integers as state, and this position is advanced only when queried. This has implications when using
serial() + serial()
generates the sequence 0, 2, 4, ...
x >= 0 ? serial() : 1
returns 1 for points with negative xcoordinates. For other points, it enumerates the sequence 0, 1, 2, ..., irrespective of how many points with negative xcoordinates are evaluated in between them.
Generic filter divides the attribute data types supported by the ODM into 2 groups:
Internally, generic filter converts all numeric values to real numbers, and the full range of operators shown in the table below is supported. Stringtypeattributes support the addition and relational operators only. Identifiers of numeric and character string attributes must not be mixed within the same arithmetic or relational expressions, but they may be combined using logical operators. Hence,
Generic[SigmaX AND PointLabel]
is a valid expression, while
Generic[SigmaX + PointLabel]
is not.
The following table comprises all tokens understood by generic filters. The precedence defines the order of evaluation: the lower the precedence number, the sooner the evaluation. Operators and functions of the same precedence are evaluated from left to right. Kindly note that all trigonometric functions use radians as angular measure.
Group  Prec.  Rule  Symbol(s)  Meaning 

arithmetic  1  Attribute  <predefined attribute identifier> <userdefined attribute identifier>  depending on the parent node, evaluates to either the presence of the corresponding data attribute, or the attribute value. Userdefined attribute identifiers may be single or doublequoted. If quoted, they may contain white space. 
Coordinate  x, y, z  depending on the parent node, evaluates to either the validity or the value of the corresponding data coordinate. Returns invalid values for nonpoint data.  
Raster  r[i]  depending on the parent node, evaluates to either the validity or the value of the corresponding raster element. Indexing starts at 0.  
Neighbor  n[i].attribute  depending on the parent node, evaluates to either the validity or the value of the neighbor coordinate/attribute. Indexing starts at 0.  
Self  s  Used only as argument to rule NeighborBinary, to select the current geometry. This is the same as n[0] unless different processing and neighbor filters are used, as is possible with e.g. Module Normals.  
Real  <string literal convertible to a real number>  leaf that evaluates to the passed number  
String  <single or double quoted string literal>  leaf that evaluates to the passed string (unquoted)  
Factor    unary minus  
+  unary plus, evaluates to its argument  
(...)  grouping operator  
Constant  true  leaf that evaluates to the boolean value 'true'  
false  leaf that evaluates to the boolean value 'false'  
pi  leaf that evaluates to \(\pi\), the ratio of a circle's area to the square of its radius  
invalid  leaf that evaluates to an invalid value  
Nullary  serial()  evaluates to the enumeration of nonnegative integers in ascending order: 0, 1, 2, ... See Serial number generator  
Unary  abs(...)  unary function that evaluates to the absolute value of its argument  
acos(...)  unary function that evaluates to the arc cosine of its argument  
asin(...)  unary function that evaluates to the arc sine of its argument  
atan(...)  unary function that evaluates to the arc tangent of its argument, return values are in the range \([\pi/2,+\pi/2]\)  
ceil(...)  unary function that evaluates to the smallest integer not less than its argument  
cos(...)  unary function that evaluates to the cosine of its argument  
cosh(...)  unary function that evaluates to the hyperbolic cosine of its argument  
deg2rad(...)  unary function that converts its argument from degrees to radians i.e. it evaluates to \(\pi/180\) times its argument  
exp(...)  the exponential function \(e^x\), with argument as x, or Euler's number raised to the argumentth power, resp.  
floor(...)  unary function that evaluates to the largest integer not greater than its argument  
grad2rad(...)  unary function that converts its argument from gradians (gons) to radians i.e. it evaluates to \(\pi/200\) times its argument  
log(...)  unary function that evaluates to the natural (base \(e\)) logarithm of its argument  
log10(...)  unary function that evaluates to the base 10 logarithm of its argument  
rad2deg(...)  unary function that converts its argument from radians to degrees i.e. it evaluates to \(180/\pi\) times its argument  
rad2grad(...)  unary function that converts its argument from radians to gradians (gons) i.e. it evaluates to \(200/\pi\) times its argument  
round(...)  unary function that evaluates to the integer closest to its argument. round(0.5) == 1, round(0.5) == 0  
sin(...)  unary function that evaluates to the sine of its argument  
sinh(...)  unary function that evaluates to the hyperbolic sine of its argument  
sqrt(...)  unary function that evaluates to the square root of its argument  
tan(...)  unary function that evaluates to the tangent of its argument  
tanh(...)  unary function that evaluates to the hyperbolic tangent of its argument  
Binary  atan2(..., ...)  atan2(y,x) evaluates to the arc tangent of y/x, using the signs of the arguments to compute the quadrant of the return value that is in the range \([\pi,+\pi]\)  
fmod(..., ...)  fmod(x,y) evaluates to the remainder of x/y  
ldexp(..., ...)  ldexp(num,exp) evaluates to num * 2^{exp}  
pow(..., ...)  pow(base,exp) evaluates to base raised to the expth power  
NeighborBinary  SqrDist2D(n0, n1)  evaluates to the Euclidean distance squared, considering x and ycoordinates only  
SqrDist3D(n0, n1)  evaluates to the Euclidean distance squared, considering x, y and zcoordinates  
Dist2D(n0, n1)  evaluates to the Euclidean distance, considering x and ycoordinates only  
Dist3D(n0, n1)  evaluates to the Euclidean distance, considering x, y and zcoordinates  
Azimuth(n0, n1)  evaluates to the angle with the positive yaxis of the difference vector of 2 neighbors projected onto the x/y plane, counted clockwise i.e. \(atan2( n1.x  n0.x, n1.y  n0.y)\)  
ZenithDist(n0, n1)  evaluates to the angle with the positive zaxis of the difference vector of 2 neighbors i.e. \(atan2( sqrt( pow(n1.x  n0.x, 2) + pow(n1.y  n0.y, 2) ), n1.z  n0.z )\)  
Quadrant(n0, n1)  evaluates to the quadrant number (14) of n1 with n0 as coordinate system origin. The coordinate axes are included to the quadrant region in a counterclockwise manner. i.e. the positive xaxis is part of quadrant 1, the positive yaxis is part of quadrant 2, etc. Furthermore, the origin is included to quadrant 1.  
Octant(n0, n1)  evaluates to the octant number (18) of n1 with n0 as coordinate system origin. The octant definition is based on the quadrant definition above. if n1.z < n0.z than the quadrant value is increased by 4.  
Stat  count( . )  returns the number of valid items  
countUnique( . )  returns the number of distinct, valid items  
first( . )  returns the first valid item  
min( . )  returns the minimum of valid items  
max( . )  returns the maximum of valid items  
sum( . )  returns the sum of valid items  
mean( . )  returns the arithmetic mean of valid items  
median( . )  returns the median of valid items  
rms( . )  returns the root mean square of valid items as \(\sqrt{\sum_{i=1}^{N} r_i^2 / N}\) with \(N\) being the count of valid items  
stdDev( . )  returns the standard deviation of valid items as \(\sqrt{\frac{1}{N1} \sum_{i=1}^{N}(r_i  \bar{r})^2 }\) with \(\bar{r}=\sum_{i=1}^{N}r_i/N\) and \(N\) being the count of valid items  
stdDevMAD( . )  returns a robust estimation of the standard deviation of valid items: the median of the absolute values of the deviations from the sample median, scaled to make it a robust, consistent estimator of the standard deviation of normally distributed data (see here). It is computed as: \(1.4826 \cdot median_i(\leftr_i  median_j(r_j)\right)\) with \(i,j=1..N\) and \(N\) being the count of valid items  
minAbs( . )  returns the minimum absolute value of all valid items: minAbs(5,4,1)=1  
maxAbs( . )  returns the maximum absolute value of all valid items: maxAbs(5,4,1)=5  
meanAbs( . )  returns the arithmetic mean of the absolute values of all valid items  
minAbsSigned( . )  returns the signed minimum absolute value of all valid items: minAbsSigned(5,4,1)=1  
maxAbsSigned( . )  returns the signed maximum absolute value of all valid items: maxAbsSigned(5,4,1)=5  
Random  a, b  uniform_int  
a, b  uniform_real  
p  bernoulli  
t, p  binomial  
t, p  negative_binomial  
p  geometric  
\(\mu\)  poisson  
\(\lambda\)  exponential  
\(\alpha\), \(\beta\)  gamma  
a, b  weibull  
a, b  extreme_value  
\(\mu\), \(\sigma\)  normal  
m, s  lognormal  
n  chi_squared  
a, b  cauchy  
m, n  fisher_f  
n  student_t  
2  Term  *  binary operator that multiplies its operands  
/  binary operator that divides its left operand by its right operand  
3  Expression  +  binary operator that adds its operands  
  binary operator that subtracts its right operand from its left operand  
relational  4  Less  <=  binary operator that evaluates to true, iff its left operand is smaller than or equal to its right operand 
<  binary operator that evaluates to true, iff its left operand is smaller than its right operand  
>=  binary operator that evaluates to true, iff its left operand is greater than or equal to its right operand  
>  binary operator that evaluates to true, iff its left operand is greater than its right operand  
5  Equal  ==  binary operator that evaluates to true, iff its left operand is equal to its right operand  
!=  binary operator that evaluates to true, iff its left operand is not equal to its right operand  
logical  6  Inverted  ! not  unary operator that evaluates to true, iff its operand evaluates to false 
(...)  logical grouping operator  
7  And  && and  binary operator that evaluates to true, iff both of its operands evaluate to true  
8  Or   or  binary operator that evaluates to true, iff one or both of its operands evaluate to true  
conditional  9  TernaryConditional  ? :  a ? b : c evaluates to b if a evaluates to true, otherwise evaluates to c 
assignment  10  Assignment  =  assignment operator. Multiple assignments must be separated by ';'. To specify the data type of userdefined attributes to be assigned to, append the data type in parentheses. 
tbd
In the following, a formal definition of the generic filter syntax is given. For possible tokens denoting predefined attributes ( "PreDefinedAttribute" in rule Attribute), see ODM predefined attributes. For the production rule for real numbers ( "Real" in rule Constant), see Filter string syntax.
The generic filter syntax represented as railroad diagrams, as here: for an explanation of symbols, see Railroad diagrams.

For the formal definition of the generic filter syntax in EBNF, the same notation as in Filter string syntax is used.