Technische Universität Wien
Orientation and Processing of Airborne Laser Scanning data
Department of Geodesy and Geoinformation - Research Groups Photogrammetry and Remote Sensing
Performs format conversion for vector data files with additional read support of GDAL rasters.
Module Translate is a tool providing data format conversion functionality. Up to now, it supports the reading/writing of vector data formats (LAS, ESRI Shapefile, ODM, text /binary, etc.). In the longer perspective Module Translate will also support raster file conversion and coordinate system transformations.
The main parameters are a single or multiple input file names (inFile) and the output file name (outFile). The respective data format is automatically estimated based on the file contents (input) and/or extension (output) but can also be defined explicitly (iFormat, oFormat). In both cases, well defined standard formats as described in Supported data formats as well as arbitrary text or binary data formats based on OPALS Format Definition XML files are supported. In case that no output format is specified, the output file is created using the file format and structure of the input file. This is especially useful if only a subset of the data should be written to the output by specifying a filter. For details on filters and how they are defined please refer to the Filter manual. Please note that, in contrast to Module Import only a single input format is supported. Thus, if multiple input files are specified, all files must share the same data format.
An additional functionality of Module Translate is the application of a 3D affine transformation (trafo) in the course of file format conversion. The transformation parameters are typically computed by other OPALS modules like Module LSM, Module ICP or Module GeorefApprox.
Furthermore, Module Translate supports splitting of the input file contents into separate output files based on a specific attribute of the dataset splitbyAttribute. This feature is, e.g., useful to extract the original flight strips from tiled datasets via a strip identifier (e.g. PointSourceId, cf. Example 5: Extracting original flight strips from tiled data) or to divide a multi-return dataset into 1st, 2nd, ..., nth echo data files (EchoNumber, cf. Example 4: Splitting file contents by echo number). While the feature is supported for all numeric types, integer-valued attributes constitute the standard usage case. For each distinct attribute value, a separate file is created. Please note that real-valued attributes (float, double) are automatically casted to integer.
As a general rule within OPALS, data are expected in the same Coordinate Reference System (CRS) whenever multiple file inputs are supported. This applies to modules accepting file lists (e.g. inFile in Module Import , Module Algebra ...) and modules offering multiple (input) file parameters (e.g. inFile and trjFile in Module Import or inFile and gridFile in Module Snellius ...). Module Translate enables CRS transformations via the parameters inCRS, outCRS, and operation, available within the crsTrafo group. The implementation is based on the respective part of the GDAL/OGR library, which in turn internally uses the PROJ library.
In case the input datasets do not contain CRS information, the input reference system can either be provided via inCRS as a WKT string, a PROJ string, or an EPSG code. The same applies to the output reference system (outCRS), which is the only mandatory parameter for performing coordinate system transformations. Please refer to the CRS page for more information about reference system definition.
Please further note that the global parameter coord_ref_system is considered, if CRS information is neither stored on the data file nor explicitly set via inCRS. In case of multiple definitions, the following precedence is enforced: (i) inCRS, (ii) CRS information stored on the data file, (iii) coord_ref_system.
Starting from version 6.0, the PROJ library supports a flexible framework for constructing user-defined transformation pipelines. Built around this so-called "CoordinateOperation" feature, Module Translate offers a generic way for transforming data between different CRS including datum shift, change of height system, and gidi-shift based correction of inhomogeneous reference systems. The possible transformation paths are depicted in the following Figure 1.
In Figure 1, the dashed and solid boxes denote source and target reference systems including potential intermediate systems, the thick arrows indicate possible transformation routes, and the green dashed boxes finally represent the available atomic coordinate operations between two systems. Some of these operations require further parameters as detailed in the table below. Syntactically, a transformation pipeline (operation) is a list of strings, each containing the textual, function-style description of an individual operation. Each string consits of the name of the operation (i.e. labels of green boxes in Figure 1) followed by 0..N arguments in key=value enclosed in parentheses and multiple arguments are separated by commas.
The individual operations, their specific meaning and required parameters are listed in the table below:
| operation | arguments | dim. | description |
|---|---|---|---|
| projectedToGeodetic | arguments | 2D | Converts map projection coordinates (x/y or easting/northing, respectively) to geodetic coordinates (lon/lat). No arguments required (cf. above). |
| geodeticToProjected | arguments | 2D | Converts geodetic coordinates (lon/lat) to map projection coordinates (x/y or easting/northing, respectively). No arguments required as the necessary parameters are stored in the CRS definitions (WKT, proj, EPSG code). |
| customToOrthoH | hgtShift=<float>, hgtScale=<float>, hgtCorr | invHgtCorr=<path> | 1D | Converts custom heights h' (e.g. normal/tidal/etc. heights) to orthometric heights h. The transformation may include a a constant offset (hgtShift) and scale (hgtScale) as well as the application of a correction grids to account for inhomogenities of the source height system (hgtCorr = -invHgtCorr). Cf. Figure above for the transformation formulae. Georeferencing of the height correction grid file is expected in geodetic coordinates (lon/lat; unit: deg) of the source system. File format: GTX. |
| orthoToCustomH | hgtShift=<float>, hgtScale=<float>, hgtCorr | invHgtCorr=<path> | 1D | Converts orthometric heights h to custom heights h'. The same rules as above apply. Georeferening of the height correction file is expected in geodetic coordinates (lon/lat; unit: deg) of the target system. |
| ellipToOrthoH | undulation=<path> | 1D | Converts ellipsoidal heights H to orthometric heights h via application of a correction grid containing geoid undulations N (H=h+N). Georeferencing of the undulation grid file is expected in geodetic coordinates (lon/lat; unit: deg). File format: GTX. |
| orthoToEllipH | undulation=<path> | 1D | Converts orthometric heights h to ellipsoidal heights H. The same rules as above apply. |
| geodeticToGeocentric | arguments | 3D | Converts 2D geodetic coordinates + 1D-ellipsoidal heigths (lon/lat/H) to 3D Earth-Centered-Earth-Fixed (ECEF) cartesian coordinates (X/Y/Z). No arguments required (cf. above). |
| geocentricToGeodetic | arguments | 3D | Converts 3D ECEF cartesian coordinates (X/Y/Z) to 2D-geodetic coordinates + 1D-ellipsoidal heigths (lon/lat/H). No arguments required (cf. above). |
| paramDatTrans | float*0 | float*3 | float*7 | float*10, inverse = 0 | 1 | 3D | Converts 3D ECEF cartesian coordinates (X/Y/Z) from one geodetic datum to another via parametric transformation: 3P: shifts only, 7P: Spatial similarity transformation (Helmert: shifts[m], rotations[arc sec], scale[ppm]), 10P: Molodensky=Spatial similarity transformation with reduction point (shifts[m], rotations[arc sec], scale[ppm], reduction point shifts[m]). The optional inverse key can be used, if the transformation parameters are known in the opposite transformation direction. In this case, parameter inversion is performed internally. If no parameters are defined (i.e, float*0), the parameters are automatically extracted from the CRS definion strings (e.g.: WKT: TOWGS84). |
| gridShiftDatTrans | gridShift | invGridShift=<path> | 2D | Performs datum transformation of geodetic coordinates (lon/lat) potentially including homogenization due to irragularities of either CRS applying grid shifts (dLon, dLat) provided as an ntV2 correction grid (gridShift). Georeferencing of the grid shift file is expected in geodetic coordinates (lon/lat; unit: deg). Please use invGridShift, if the grid shift files denote the opposite transformation direction. |
| gridShiftHomogen | gridShift | invGridShift=<path> | 2D | Performs homogenization of geodetic coordinates (lon/lat) applying (ntV2) grid shifts. Please note that no datum transformation is performed in this step but the grid shift values (dLon/dLat) only correct potential inhomogenities of the source CRS (gridShift=-invGridShift). Georeferencing of the grid shift file is expected in geodetic coordinates (lon/lat; unit: deg). |
| gridShiftInvHomogen | gridShift | invGridShift=<path> | 2D | Performs inverse homogenization of geodetic coordinates (lon/lat) applying (ntV2) grid shifts. Please note that no datum transformation is performed in this step but the grid shift values (dLon/dLat) only restore potential inhomogenities of the target CRS (gridShift=-invGridShift). Georeferencing of the grid shift file is expected in geodetic coordinates (lon/lat; unit: deg). |
Please find specific examples here.
The data used in the following examples can be found in the $OPALS_ROOT/demo/ directory.
The above command converts the file strip31.laz from compressed LAS file format to the text file strip31.laz.convertedTo.fwf. The attributes Amplitude (i.e. LAS intensity), EchoNumber, and NrOfEchos are preserved as they are contained in the predefined FWF format. Please note that the name of the output file is automatically estimated based on the respective estimation rule.
Providing the proper input format definition XML file LAS_1.2._ew.xml which maps the LAS extry byte attribute PulseWidth to the predefined OPALS Datamanager attribute EchoWidth additionally preserves the echo width information in the text output file.
This command reads the LAS file, analyses the echo attributes via the specified filter, and writes all intermediate echo points to a binary XYZ file.
This example demonstrates the usage of splitByAttribute to extract the 1st, 2nd, ..., nth echo from multiple input files.
As a result, seven files named echo_1.las to echo_7.las are created corresponding to the unique echo numbers of the input datasets. Please note that it is not necessary to know the attribute range beforehand, as the uniqueness is checked for each point and new output files are automatically created if needed.
This example illustrates the usage of splitByAttribute to extract the flight strips from an entire flight block dataset based on the attribute PointSourceId.
Please note that in this example no output file name is specified. Module Translate automatically composes the output file names from the name of the input file (strips_), the attribute name (PointSourceId_) and the unique PointSourceId values (11, 21, 31). The following files are hence exported.
This example illustrates the usage of trafo to do 3D affine transformation on a raster dataset based on the result of alignment from Module LSM.
Module Translate automatically composes the output file name from the name of the input file.
This example shows how to perform a simple CRS transformation.
In this example, only the output CRS is set while the input CRS (ETRS89/UTM33) is automatically determined from the file header of the input dataset (strip11.laz). As no user-defined transformation pipeline is specified, the transformation path is automatically determined based on the given CRS definitions. In particular, the TOWGS84 specification of the output CRS (EPSG:31256, MGI, Gauss-Krueger East) is used for performing the datum transition from ETRS89->MGI via a 7P (spatial similarity transformation). Please notethat, as both system definitions are 2D (PROJCS), the transformations is only performed in 2D and consequently only the x,y, coordinates change but not the elevations.
In Module Translate , the parameters for the specification of coordinate reference systems are located within the option group crsTrafo. While it is not necessary to specify the option group on the command line, this is required in Python. The following code example illustrates the usage:
The script can be executed via:
This example shows how to use custom transformation pipelines (operation) to transform trans-national coordinates in UTM33/ETRS89 coordinate frame to the Austrian national system MGI in Gauss-Krueger projection.
Please note that no additional parameters are set in the individual transformation operations, especially not in the datum transformation step (paramDatTrans). Instead, the TOWGS84 parameters defined the WKT files (*.prj) are used. In the above example, the following line in mgi_gkm34.prj is considered:
These seven parameters of a spatial similarity transformation (3 shifts, 3 rotations, 1 scale) provide the datum transition from ETRS89 to MGI with an accuracy of around 1m in the resulting MGI coordinates for the entire area of Austria. Furthermore, as the input height system is ellipsoidal and no height transformations are provided, the resulting Z-component constitutes ellipsoidal heights above the underlying BESSEL ellipsoid.
The first three resulting MGI/Gauss-Krueger coordinates are:
For the same input as in the above example (ETRS89/UTM33) this example illustrates the full capabilities of the custom transformation pipeline concept including grid shift (ntV2) based datum transition and transformation of the height component from ellipsoidal heights above the GRS80 ellipsoid in the input to MGI (custom) heights..
The transformation pipeline starts with calculating the geodetic coordinates (lon/lat) from the projected UTM33 coordinates (x/y) followed by the height transformation from ellipsoidal to orthometric heights. At this stage, the geodetic datum transition from ETRS89 to MGI is performed via grid shift transformation, inversely applying the lon/lat corrections stored in the ntV2 grid shift file AT_GIS_GRID.gsb published by the Austrian national mapping agency BEV. Orthometric heights are invariant w.r.t. datum transitions, thus the remaining steps are to convert the orthometric heights to the custom MGI height system (i.e. small corrections stored in GV_Hoehengrid_V1.gtx, again published by BEV) and the final transformation of lon/lat to Gauss-Krueger system. The resulting planimetric coordinates (x,y) constitute a better approximation of the national Austrian MGI coordinate reference system (accuracy: better 15cm) compared to using the 7P-transformation demonstrated in the above example.
Currently only point datasets are supported, when running Module Translate with splitByAttribute and with CRS transformations.
1.8.17