Table of Contents
- See also
- python.workflows.preDataCheck
Aim of module
Performs a point cloud data check to secure validity and plausibility of input files.
General description
When processing large amounts of point cloud data it can be useful to assess the quality of the data beforehand. PreDataCheck aims to find possibly corrupt files using different approaches, making use of the overview-matrices (pcount and z), generated by opalsInfo.
An indicator for corrupt/faulty point cloud files is the amount of validpixels in the boundingbox of the overview-matrices. The default threshold of which is set to 5% but the user can control this value using the validpixels parameter. Some corrupt files show unrealistic z-values, if the user knows the approximate z-range of the area of interest, they can do so using the zlim parameter. By default it is set to the earthly possible z-limits. In the same way the user can specify an xylim bounding-box for the area in question to check if the points are all within the expected area.
The result of the script are tables, stored in the opalsLog, giving a statistic of the processed files and the result of each of the checks. Furthermore, two matrices of all the point cloud files are generated, for the pcount as well as for the z-values. This is stored in a QGis file together with bounding shape files of all the input files. An example of a QGis result, generated by the processing of two input files can be seen in Figure 1.
More details to come
- See also
- Script documentation
Parameter description
Remark : optional
Description: If the user knows the coordinates of the bounding box (x and y limits) roughly containing the area of interest (i.e. a country or a state), it can be provided in the format [ymin,xmin,ymax,xmax], where Y is the abscissa and X the ordinate.
| possible input | comment |
|---|---|
| s, a, v, e, s, , t, h, e, , a, r, e, a, , o, f, , t, h, e, , f, o, o, t, p, r, i, n, t, , a, s, , a, t, t, r, i, b, u, t, e, , [, i, n, , s, q, ., m, ., ] | area |
| s, a, v, e, s, , t, h, e, , s, e, m, i, m, a, j, o, r, , a, x, i, s, , [, i, n, , m, ] | semimajor |
| s, a, v, e, s, , t, h, e, , s, e, m, i, m, i, n, o, r, , a, x, i, s, , [, i, n, , m, ] | semiminor |
| s, a, v, e, s, , b, o, t, h, , a, x, e, s, , [, i, n, , m, ] | axes |
| s, a, v, e, s, , t, h, e, , a, n, g, l, e, , o, f, , i, n, c, i, d, e, n, c, e, , [, i, n, , r, a, d, ] | incidence |
| s, a, v, e, s, , t, h, e, , i, d, , o, f, , t, h, e, , o, r, i, g, i, n, a, t, i, n, g, , p, o, i, n, t | pointid |
| s, a, v, e, s, , t, h, e, , X, /, Y, /, Z, , c, o, m, p, o, n, e, n, t, s, , o, f, , t, h, e, , b, e, a, m, , v, e, c, t, o, r | beamvector |
| s, a, v, e, s, , t, h, e, , X, /, Y, /, Z, , c, o, m, p, o, n, e, n, t, s, , o, f, , t, h, e, , n, o, r, m, a, l, , v, e, c, t, o, r | normalvector |
Remark : optional, default: (-200, 9000)
Description: If the user knows the coordinates of the bounding box (x and y limits) roughly containing the area of interest (i.e. a country or a state), it can be provided in the format [ymin,xmin,ymax,xmax], where Y is the abscissa and X the ordinate.
Remark : optional, default: 0.05
Description: After a bounding box is created, the share of valid pixels are checked. If they are below the threshold set, the input file is rejected. Values should be between 0 and 1.
Remark : optional, default: True
Description: create mosaics
| possible input | evaluates to |
|---|---|
| 1, true, yes, Boolean(True), True | Boolean(True) |
| 0, false, no, Boolean(False), False | Boolean(False) |
Remark : optional
Remark : optional, default: False
Description: If True, script keeps running after encountering an error.
| possible input | evaluates to |
|---|---|
| 1, true, yes, Boolean(True), True | Boolean(True) |
| 0, false, no, Boolean(False), False | Boolean(False) |
Remark : mandatory
Description: As input either the name of an ASCII file (extension .txt) containing the names of the data files (one file per line) or an expression using wildcards to specify the respective data sets are accepted.
Remark : optional
Description: Specifies the directory where the final results are stored. If not specified, the results are created in the current working directory.
Remark : optional, default: current directory
Description: Specifies the directory where all intermediate results are stored. If not specified, a TEMP subdirectory is created in the current working directory. Additional subdirectories (one per involved module) are created.
Remark : optional, default: $OPALS_ROOT/cfg/preDataCheck.cfg
Description: The name of a configuration file containing all relevant calculation parameters is expected. If not specified, the default cfg files as provided by the OPALS distribution are used.
Remark : optional, default: current directory
Description: The default project directory is the current working directory, but can be easily changed by this parameter. All path parameters, if not specified as absolute paths, are interpreted relative to the project directory.
Remark : optional, default: True
Description: Skip processing if result already exists. In order to re-run current script it is useful to repeat the processing only if the respective output does not already exist. This allows for incremental processing of large projects.
| possible input | evaluates to |
|---|---|
| 1, true, yes, Boolean(True), True | Boolean(True) |
| 0, false, no, Boolean(False), False | Boolean(False) |
Remark : optional
Description: according to the number chosen as the value, independent processes are being started to parallelise (parts of) the program and therefore enhance the runtime.
Remark : optional, default: info
Remark : optional, default: info
Remark : optional
Description: Logger is usually provided by the opals framework.The user may provide their own logger object, but it has to function in the same way as the opals Logger.
Examples
Figure 2 shows an example of a result table of two non corrupt files. The inconsistent header error is raised by opalsInfo if the header information does not correspond with the bounding box that was extracted (also a sign of a corrupt file). The unspecific Error is raised if an error occured that is not covered by the other errors.
Example 1
In this first example we use a wildcard (*) to use strip11.laz and strip12.laz as input parameters. The z-range for these two point cloud files is known so we can specify it using the zlim parameter. The more diagonal a point cloud strip is, the less validpixels the overview-matrix has. We know that the two files in question are rectangular with the two sides being parallel to the coordinate axes, therefore we can set the validpixels parameter to a higher amount.
None of the errors were raised, the resulting table in the opalsLog can be seen in Figure 2.
Example 2
In Example 2 we want to purposefully trigger the zlim error, so we set the zlim to [300, 350] which does not cover the z-range of the point clouds.
Now that at least one error is raised, a second table is shown in the opalsLog, the Detailed Error Log, showing which files raised which specific errors.
Example 3
In this third example, we take the same input files, but use opalsTranslate before processing them to turn them into diagonal strips. The bounding shape files of those can be seen in Figure 1. The resulting pointcloud files are non corrupt files, yet due to the diagonal shape of the strips, the validpixel amount is reduced significantly.
As we can see in Figure 4, the threshold for the validpixels is now too high and the script raises an error. The validpixels parameter is to be set with caution.
1.8.17