Release notes
Technische Universität Wien (TU Wien)
Department of Geodesy and Geoinformation (GEO)
Research groups Photogrammetry and Remote Sensing
mail: Wiedner Hauptstr. 8-10, A-1040 Wien
phone: ++43-1-58801-12201
fax: ++43-1-58801-12299
email: opals@geo.tuwien.ac.at
OPALS
Orientation and Processing of Airborne Laser Scanning data
Release notes
Contents:

Version 2.6.0

Contents:

Preamble

Version 2.6.0 is the last release before version 3.0. It includes a large set of improvements, performance gains and corrections, featuring significant advances of the workflow package script, Module StripAdjust, Module Info and Module Bounds. Please see below for a detailed list of changes.

We plan to release version 3.0 by the end of 2024. This major release will bring important new features (e.g. generic spatial neighbour definitions, an advanced restructuring of option names, etc.) that will require adaptations of existing Python and batch scripts.

New features

Bug fixes

  • Script qltAccuracyPoint.py: deprecated ODM type "boolean" replaced by new type "bool"
  • Module ICP The optional transformation of the final point cloud files was not done correctly. The files were re-created without applying the computed transformation.
  • Rare endless loops in Module Histo corrected when analyzing attributes with very small ranges (max-min < 1e-13)
  • Module StripAdjust :
    • initialize the higher-order coefficients of trajectory correction polynomials to zero instead of replicating the zero-order one (offset).
    • Do not cut the trajectory to the time range of a strip and then report the first and last few points of that strip to lie outside of the trajectory's time range.
    • Correctly support control point cloud subsets.
  • SCOP DTM raster files were not created correctly
  • Module DTM didn't output structure lines of hybrid DTM with heights (only 2D)
  • Module Translate didn't consider full output path when splitByAttribute specified
  • Module PointStats failed to compute features (n)min, (n)max correctly, if attribute type was other than float
  • Rare crashes occurred while reading las/laz files containing GeoTIFF tags as CRS information. Affected mainly Module Import and Module Translate.
  • when writing LAS files with point data record format >= 6, correctly store classifications >= 32
  • In rare occasions the extension of ODM attributes didn't work correctly (Module PointStats and Module AddInfo). In those situations some attribute values, but mainly the statistic of added attribute(s) were incorrect (especially min/max values) and it could be corrected by re-running the same command on the same ODM file(s).
  • In Module Export a given transformation (parameter trafo) was not considered if no filter (parameter filter) was specified
  • When importing/reading multiple Riegl rdbx files, OPALS (e.g. Module StripAdjust) tended to crash irregularly.
  • Module FillGaps crashed at times if the raster row count was an integer multiple of the raster row block size.

Improvements

  • Significant performance improvement when running script preCutting.py on huge data sets
  • Module Histo improved precision management for attributes with (very) small ranges
  • Module StripAdjust :
    • Clutter removed from log output, and log enriched with additional information to allow for tracing the effects of various parameters.
    • Export .shp files as 3D (not 2D) points. Export .xyz-files with header.
    • Set coordinate reference system of exported strips.
    • If points actually lie outside of the trajectory's time range, then issue a warning.
    • To save RAM, load the control point clouds only if necessary. Hence, ignore workflow.controlPointClouds during import.
    • Got rid of distracting solver messages when using the spline correction model.
  • Setup and teardown of logging by many modules running in the same process accelerated.
  • Logging / Error Handling : for containers, log the actual elements instead of the all-element.
  • Log file : support concurrent logging into the same file from different processes possibly running on different machines.
  • Unpickling of modules accelerated.
  • Module Import : Improved handling of geokeys in LAS files as CRS information (Removes LASlib warnings that a geokey is not implemented)
  • Python interpreter that comes with OPALS updated to Python 3.8.10 (Support of python 3.2, 3.3 and 3.4 removed).
  • Riegl rdblib version increased to 2.4.1 (needed for reading and writing newer RDBX Riegl files)
  • Improved and homogenised raster file statistics and setting the band names for several modules: Module FillGaps, Module EdgeDetect, Module StatFilter, Module GridFeature.
  • Included R version increased to 4.3.0 (Windows version only)
  • Module Translate now checks and improves tile size of created ODM files if necessary
  • Updated LASlib to LAS_TOOLS_VERSION 110324
  • Updated PyCharm to version 2024.1
  • Included R version increased to 4.4.0 (Windows version only)

Version 2.5.0

Contents:

Preamble

Besides many small improvements and corrections, it features significant advances of Module StripAdjust, Module DTM and Module Histo. Please see below for a detailed list of changes.

We plan to release version 3.0 by the end of 2024. This major release will bring important new features (e.g. generic spatial neighbour definitions, an advanced restructuring of option names, etc.) that will require adaptations of existing Python and batch scripts.

New features

  • Support of Riegl RDB format added. Based on corresponding OFD files full attribute and coordinate system support for import and export is provided.
  • New Python script preDataCheck.py allows easy checking of correctness, completeness and validness of point cloud input files. This is especially useful before processing large projects with a huge number of input files.
  • An OPALS Format Definition can now be given not only as path of an XML file, but also inline as XML text. To allow for concise input, the XML schema of OPALS Format Definition has been changed: most XML elements with only a single attribute have become attributes of their parent element. Additionally, the XML element for Text Formats has been changed from <ascii> to <text>.
  • Support Python 3.10, see Configuring an external Python installation on Windows
  • Common parameter deleteEmptyOutFile renamed to deleteUselessOutFile. This reflects the new semantic which deletes newly created output files if they are empty or an exception during processing occurred.
  • Support of zstd compression within GDAL enabled
  • Module StripAdjust :
    • strips.strips.filter.iFilter now supports filters that change coordinates and / or attributes.
    • Temporary files:
    • Riegl RDB files can be directly used as sensor coordinate point cloud input
  • A major update of Module Histo that also effects the default behaviour:
    • sampleRange now supports relative value in percentage or quantile notation
    • Histograms are now limited to the 2% and 98% quantile per default (old behaviour was the full sample range)
    • Support of multi bands for raster files added
    • Support of non-continuos histograms added (relevant for category values as classification, etc.)
    • Change of output: histogram bins and all other statistic values are now presented in tables
  • Module DTM now supports a local and global TIN interpolation method
  • Parameter createAlpha added to Module Shade and Module ZColor which is useful to create a reliable nodata mask (using a alpha channnel) in case lossy compression is used in the output file. Breaking changes: Please note that Module Shade used a alpha channel already before. To have a uniform default behaviour in both modules, Module Shade is not creating a alpha channel per default any more. Set createAlpha to 1 to get the original behaviour.
  • New grid features (opals::GridFeature) slpPrec, exposRad, exposDeg introduced (relevant for Module DTM, Module Grid and Module GridFeature). Feature slpPrec replaces slope and feature exposRad replaces exposition. The old feature names are declared as deprecated but still usable.
  • Module Info can directly generate and output overview raster files from las/laz files (this was only possible from ODMs before)
  • Parameter nodata added to modules Module Algebra, Module Cell, Module DSM, Module DTM, Module Diff, Module Grid.

Bug fixes

Improvements

  • Included R version increased to 4.2.0 (Windows version only)
  • Python stub files (*.pyi) now define slots for all classes as an empty tuple. This tells static type checkers to flag the assignment of instance attributes. At run-time, this is prohibited, anyway.
  • Module StripAdjust : to limit RAM usage, do the geo-referencing of strips upon import in chunks.
  • Python script preCutting.py now supports individual filters for each input file and uses overview matrices to cut las/laz files more effectively.
  • Now also on Linux different python version are supported
  • Improved and homogenised raster file statistics and setting the band names for several modules: Module Algebra, Module Cell, Module DSM, Module DTM, Module Diff, Module Grid.
  • Improved thread management to reduce/eliminate "Unable to load new data tiles"-Exceptions when running Module AddInfo.

Version 2.4.0

Contents:

Preamble

Besides many small improvements and corrections, it features significant advances in multi-band raster support, and the default Python version has been raised to 3.6. Please see below for a detailed list of changes.

New features

Bug fixes

  • Module ZColor : bug with large palette (using more than 253 colors) fixed.
  • Module Simplify : The final compressed model did contain a diagonal shift if the raster input model had a grid size of lower than 1.
  • Module Rasterize : In rare cases the module crashed when rasterizing polygons.
  • Module Import :
    • Crash while importing las file containing empty WKT strings removed
    • Coordinate reference system got lost in case of re-tiling because of inappropriate tile size estimation
  • Module Translate : Improved syntax checking of the coordinate operation parser (in certain cases invalid operations were silently ignored)
  • Module FillGaps : In certain cases infinite values occurred in the resulting raster file when using the triangulation filling method
  • Module StripAdjust , Module ICP , Module View : Stop Matlab's Java VM from issuing errors upon process shut down by shutting down the VM before.
  • Module StripAdjust : keep Matlab going by explicitly re-starting its pool of parallel worker processes.
  • Module Info : Pixel count of non-square raster files was not correctly computed.
  • Module Cell : cellSize parameter changed to double to avoid out-of-round raster coordinates in case of binary out-of-round cell size values (e.g. 0.1)
  • Module DTM : When parameter neighbours wasn't specified, the plane interpolation was not performed if only 3 points were found.
  • Module Shade : horizontal black lines when pixelsize differs from gridsize of model fixed.

Improvements

  • Unified naming of basic data types: bool, uint8, int8, uint16, int16, uint32, int32, int64, float, double, string, cstr. This concerns the specification of attribute and output raster data types in DM::ICalculator, OPALS Format Definition, and the DM APIs (both C++ and Python).
  • Module StripAdjust :
    • Export of transformed strips up to 2 times faster.
    • Description of intermediate and result files and folders added to documentation.
    • screenLogLevel and fileLogLevel are now respected separately and can be changed for every stage independently.
    • When matching strips to control point clouds, normal vectors for strips are computed using the respective parameters for control point clouds (and not those for strips).
    • There is no maximum number of strips any more (was: less than 512).
  • Module Shade :
    • Calculate shadings for datasets with geographic reference system.
    • colormap added to output file.
    • enhanced performance (multithreading with up to 4 threads).
  • FAQ: Example command for coordinate transformation based on EPSG codes added.
  • Module Isolines : performance optimizations.
  • Module DBH : New parameter searchRadius introduced for allow controlling the data selection
  • Module Algebra : In case the formula is given in the Calculator syntax, the parameter skipVoidAreas is always activated if possible (overruling possible user input)
  • Module Translate :
  • Module Histo, Module ZColor and Module Isolines now use 0-based band indices for raster files (as in Module Algebra)
  • Included R version increased to 3.6.2
  • Python interpreter that comes with OPALS updated to Python 3.6.8.
  • Updated PyScripter to version 3.6.1
  • Updated PyCharm to version 2021.2.2
  • Updated LASlib to LAS_TOOLS_VERSION 190812. Unfortunately, this means that multi-element extra bytes cannot be exported any longer.
  • Updated GDAL/OGR library to version 3.1.2
    • SCOP driver improved

Version 2.3.2

Contents:

Preamble

In contrast to our original plan, version 2.3.2 constitutes another patch release introducing only minor breaking changes (cf. below). Version 2.3.2 introduces significant new features and improvements for the modules StripAdjust, Segmentation, Translate and RadioCal. Please see below for a detailed list of changes.

New features

  • Module StripAdjust :
    • Introduction of simultaneous orientation of images and laser scans (hybrid adjustment). The images may either by loosely coupled to laser strips, in which case the images do not have to be captured concurrently with the lidar, or tightly coupled to the strips in case of synchronous acquisition of scans and images from the same platform. Details and examples can be found here.
    • Support of stage-oriented, possibly distributed processing using new option group workflow
    • Support of the application of a previously computed calibration onto different (not thinned out) strip data.
    • Export of scan angles according to the LAS standard.
    • Additional export of the scanner system trajectory in addition to the INS system trajectory.
  • (Possibly breaking) changes within hybrid laser and image orientation of Module StripAdjust :
  • Module Segmentation :
    • Alpha shapes for segments can be derived
    • Module now provides programmatically access to segments incl. points after run (see example section for further details).
  • Major revision of Module RadioCal :
    • break of compatibility: parameter reflectivityFile removed
    • support of multiple calibration constant estimation within single run added (parameter splitByAttribute)
    • support of shape files added for defining calibration regions added
    • module can now process multiple input and calibration region files at once
    • filter wasn't applied in the actual calibration stage (originally it was only used in the calibration constant estimation stage)
    • possibility of accessing detailed statistics from the calibration constant estimation stage via Python or C++ (see opals::CalibrationStats)
    • simplified way of passing calibration constant(s) from estimation stage to calibration stage
    • handling of CRS information added as in line with other modules
  • Module Translate :
    • 3D affine transformation of raster files
    • Support of coordinate reference system (CRS) transformations via new parameter group crsTrafo (!!! currently in BETA state!!!)
    • CRS transformation bases on the PROJ library (version 6.1.0), which supports custom transformation chains. Please refer to the module documentation for details.
  • Module View :
    • new color palette "classpal_las" to colorize point clouds classified according to the LAS standard
    • automatic sub-menus for attributes, if the total number of attributes exceeds a user-defined size (setting of threshold in tools dialog window)
    • new dialog to select attributes displayed in the "data cursor" list (DataCursor->Select attributes)
  • Generic filter :
  • Explicitly clear/empty options of type opals::Vector on the command-line by giving its name without any values.
  • Support Python 3.7, see Configuring an external Python installation on Windows
  • New raster format supported for read and write access: JPEG 2000.
  • Power Line Extraction from ALS Data is added under OPALS use cases.

Bug fixes

  • Statistical operations :
    • "minAbs and maxAbs" did not result in absolute values.
    • correct handling of raster NODATA that is Not-a-Number
  • Module AddInfo : interpolated grid values were incorrectly assigned, if a process filter was used
  • Rename mpir.dll to mpir_geo.dll and mpfr.dll to mprf_geo.dll, to avoid conflicts with Matlab Compiler Runtime R2017b
  • correct conversion of enumerators from Python to C++ when running an external Python 3 installation
  • Module LineModeler : crash at the end of the run if no outFile was specified
  • Module StripAdjust : consider iFormat when reading original trajectories for final export of adjusted trajectories.

Improvements

Version 2.3.1

Contents:

Preamble

Version 2.3.1 brings a series of improvements and corrections compared to version 2.3.0. Please see below for a full list of changes.

OPALS 2.3.1 will be the last 2.x version. We plan to release version 3.0 by the end of 2018. This major release will bring important new features (e.g. generic spatial neighbour definitions, multi raster band support, switch to Python 3, etc.) that will require adaptations of existing python and batch scripts.

New features

  • A full featured setup program for Windows is provided
  • Displaying XML log file contents: Filtering of individual modules implemented
  • Script opalsQuality : new parameter -gridMask enabling additional user-defined exclusion areas (e.g. water bodies) for the analysis of masked strip differences
  • Module Translate now allows writing the data of one or more input files to separate output files based on specific attributer via splitByAttribute (use cases: extracting flight strips from tiled data, etc.).
  • Module Diff : new parameter -noData allows defining the no data value for the output gird (as in /ref ModuleAlgebra).

Bug fixes

  • Module Algebra : Structural identity and proper alignment of multiple input grids were not reliably detected.
  • Module Cell : Support feature sigmaMAD.
  • Module Export :
    • String attributes where not written correctly to Shape DBF files (requires an OFD file).
    • Module crashed when exporting polygons in conjunction with region filters.
    • Specifying the same file for input and output is not allowed. However, this was not checked by the module and a misleading error message occurred. Additionally, the input ODM got cleared so that all geometries objects where removed.
  • -helpHtml, -helpChm: support installation directories with blanks
  • Module Cell : feature pcount in conjunction with odm output caused the module to hang or crash
  • Module Info :
    • overview matrix output for ODMs where shifted by 1/2 pixel (mainly used in qpals).
    • the y pixel size of rasters were shown with a negative value (absolute value was corrected)
  • Module StripAdjust : correct handling of parameter -oFormat if an OPALS Format Definition is given.
  • export to LAS format: bring ScanAngle into the interval \([-\pi;\pi]\) by addition/subtraction of \(2n\pi\)

Improvements

  • Large OPALS (xml) log files are now automatically saved/renamed to opalsLog.YYYMMTT-HHMMSS.xml once a certain file size is reached. The critical file size can be defined by the global parameter max_log_file_mb.
  • Module Section : Negative overlap do now allow to specify section distances larger than the length/depth of the section.
  • Module Import : The ODM tile size was estimated incorrectly while importing GDAL raster files, causing an automated re-importing run (performance issue)
  • Module StripAdjust :
    • for the import of strip, trajectory and reference cloud data, skip the conversion to .bin files, but import directly to .mat files.
    • transforms strips in chunks, to lower memory consumption.
    • changed defaults and allowed values:
      • control.subsetRadius is unset by default, meaning that all control points shall be used.
      • all members of sessions.all.scanner are set constant by default, except range.scale.
      • correspondences.control2strip.selection.samplingDist may be 0.
  • Module Segmentation : Comparison of the normal vector of the segment plane and the normal vector of a candidate point is added to the plane extraction method to improve the results.
  • Classification script clfTreeModelApply now supports storing of probability values for each class.
  • Coordinate Reference Systems:
    • new global parameter –force_coord_ref_sys to explicitly override the use of dataset CRS information.
    • Implementation of automatic CRS validation and soft fixing strategies (e.g. correction of parameter ordering).
    • New script _validateCRS.py to validate and, as the case may be, fixing up CRS definitions.
  • Write opalsErrors.log only if an error has actually occurred.
  • When the file size of opalsLog.xml exceeds the maximum size given by the new global parameter max_log_file_mb, rename it, integrating the current time into the new file name, and start a new one.
  • Updated PyScripter to version 3.3.2
  • PyPI now provides all needed builds of NumPy, SciPy, and matplotlib. Hence, updates are fetched automatically, and these will not be logged here any more.

Version 2.3.0

Contents:

Preamble

Version 2.3.0 is a major release featuring the following release highlights:

New modules (now available in official release)

  • Module StripAdjust :
    • Improves the geo-referencing of ALS data and aerial images in a rigorous way combining strip adjustment and aerial triangulation.
  • Module View :
    • Provides an interactive 3d viewer for points (including arbitrary attributes) and lines.
  • Module TerrainMask :
    • Identifies open terrain parts of a DSM input raster using the volume based approach by Piltz et al. (2016).
  • Module Segmentation :
    • Provides methods for point cloud segmentation based on local homogeneity using either a generic region growing approach or an adapted region growing for plane extraction.
  • Module DTM :
    • Calculates a high quality Digital Terrain Model in hybrid grid structure (i.e. regular grid + structure lines).
  • Module Isolines :
    • Derives contour lines (isolines) from grid models and stores them as a vector dataset.
  • Module FillGaps :
    • Detects gaps (i.e. void pixels) in raster models and provides several interpolation methods to close them.
  • Module LineTopology :
    • Provides different line merging and topology correction tools for cleaning line networks derived via automatic edge detection.
  • Module LineModeler :
    • Models 3D structure lines based on a point cloud (ODM) and 2D line approximations.

New scripts

New features

Bug fixes

  • Module LineTopology : weight factors for angles (wf.angle) were erroneously set from the general maxAngleDev parameter
  • Module DSM : default file names for (intermediate) min and max model were interchanged.
  • Module StripAdjust : resolved problem with log file and custom outDirectory
  • Module View : avoid illegal struct element names resulting from attribute names read from LAS/LAZ and ODM files by replacing or removing illegal characters.
  • Command line parser :
    • Parsing of container types: support values interspersed with names and/or values belonging to other options, e.g. -infile a.txt -infile b.txt
    • Support parsing of values for a child of the default-element, and then parsing values for the same child of an actual element.
    • User limit definitions (parameter types opals::GridLimit and opals::MultiGridLimit, as e.g. used in Module Algebra) where not processed correctly if negative coordinates were specified.

Improvements

  • Re-organization of main doc page:
    • Reference documentation moved to a separate page. However, links to frequently used pages (e.g. filters) are still available on the main page.
    • New top level menu structure featuring links to the list of module including short descriptions and the reference documentation
    • Documentation of Licensing policy added
    • Documentation of packages extended
  • Documentation:
    • Open XML log files with exclusive write, but shared read access.
    • Use a monospace font for HTML display of log messages emitted in Matlab.
    • Print 0.1 as "0.1" instead of as "0.10000000000000001", etc.
  • Logging to screen :
    • Format tables with left-aligned first columns, and right-aligned subsequent columns, with constant width of each column.
    • Since by far most log records are of importance opals::LogLevel::info, do not print "info" after the time stamp for them, in order to save space.
  • Command line parser :
    • Print help, even if only -usageLevelMax has been specified.
    • Specific help: support non-absolute, possibly abbreviated option paths.
    • For top-level element of specific help:
      • Disregard maxUsageLevel.
      • Print the path, not only the name.
  • The OPALS license file opals.key can be alternatively placed in the directory $OPALS_ROOT/licence (beneficial when using OPALS within a Docker container).
  • Updated GDAL/OGR library to version 2.1.3, proj.4 to version 4.9.3, and GEOS to version 3.6.1.
  • Updated LASlib to LAS_TOOLS_VERSION 170419.
  • Python interpreter that comes with OPALS updated to Python 2.7.14.
  • Updated OPALS Python AddOns to NumPy 1.13.3, SciPy 1.0.0 and matplotlib 2.0.2.
  • Updated PyScripter to version 3.0.2
  • Updated PyCharm to version 2017.2.4
  • Improved the performance of gap detection in Module FillGaps with a new data structure for the handling of gaps.
  • Improved error messages in conjunction with ODMs
  • Python script qltStripDiff : Summary table of pairwise strip differences now includes sigma_MAD (median of absolute differences). sigmaMAD is a robust estimator of the (height deviation) dispersion.
  • Module RadioCal : The availability of surface normal vector information is now checked before running the module
  • Module ICP :
    • check if 100 is a whole multiple of voxelSize
    • Matlab implementation updated, with speedups.
  • Module StripAdjust :
    • Check if 100 is a whole multiple of voxelSize
    • Matlab implementation updated, with speedups.
    • For LS strips, hand only coordinates and time stamps over to Matlab, to save disk space. Afterwards, apply the transformation to the strips separately, preserving all attributes.
    • Estimators removed for parameters with internal defaults
    • strips.session: start indexing at zero, to be consistent within index into sessions
    • DM license no more needed for import of ODM files.
  • Module DirectGeoref : Rigorous handling of coordinate reference system information:
    • Read possible CRS information from trajectory files
    • Store CRS information in output file

Version 2.2.0

Contents:

Preamble

Version 2.2.0 is a major release introducing a series of new modules and a major change of the software framework which now supports parameter grouping.

New modules

  • Module DSM :
    • Calculates a land cover dependent DSM raster model based on the approach of Hollaus et. al. (2010).
  • Module EdgeDetect :
    • Provides different raster based edge detection algorithms (e.g. Canny).

New scripts

New features

Bug fixes

  • Module Cell didn't not consider the default filename postfix for the features variance, minority and majority
  • Module Openness : limits of output raster were not set correctly if the raster origin coordinates were not multipliers of the cell size.
  • Module DirectGeoref crashed if the input file was an ODM.
  • Module Info showed an incorrect estimation of the point density. Re-import of the ODM or processing the ODM with an OPALS module that opens the ODM in write mode (opalsNormals, opalsAddInfo, opalsPointStats, etc.) is required for obtaining corrected point density estimates.
  • Module GridFeature : Parameter limit is now fully functional.

Improvements

  • Switched to Matlab 2016b Compiler. Hence, Matlab dependent modules (opalsICP, opalsView, etc. ) require the 2016b Matlab Runtime
  • Python interpreter that comes with OPALS updated to Python 2.7.14.
  • Updated OPALS Python AddOns to NumPy 1.13.2, SciPy 0.19.1 and matplotlib 2.0.2.
  • Python-binding, module parameters: get_*, set_*, and isSet_* methods removed. Parameters remain accessible as properties only.
  • DM processor improved: Code basis for generic spatial search added
  • To reduce processing stops caused by 'unable to load new data tiles' exeptions, the DM processor now temporarily suspends processing threads (if possible) to avoid the abortion of the module. If threads have been suspended, a warning will be output after processing, since a higher 'points_in_memory' value could lead to a better processing. Modules that benefit from the new functionality are Module AddInfo, Module Normals and Module PointStats
  • opalsDirectGeoref now stops with an exception if GPSTime information is missing in the input file
  • support non-alphanumeric characters in string attribute values specified in Generic filter strings.
  • Module Grid :
    • Multi-thread support for the Delaunay triangulation interpolation added
    • Weight function IDW1/IDW2/IDM:n: Implementation changed to: 1/(1+(ad)^n), with a=shape factor, d=2D distance, n...power (IDW1: n=1, IDW2: n=2...). For more information please refer to the parameter description.

Version 2.1.6

Contents:

Preamble

The OPALS team is happy to announce the first Linux version of OPALS. From the current version 2.1.6 on, binary distributions for both Linux and MS Windows can be downloaded from the OPALS website. Please note that some limitations for the different plattforms exist. The Linux distribution was tested on several platforms (e.g., Ubuntu 14.04, Ubuntu 16.04). For more information about installing and running OPALS under Linux can be found here. Your feedback is highly appreciated.

New modules

  • Module Snellius :
    • Performs range and refraction corrections of raw laser bathymetry measurements according to Snell's law.
  • Module Vectorize :
    • Performs raster-to-vector conversion and exports polyline geometries.
  • Module Translate :
    • Performs format conversion for vector data files with additional read support of GDAL rasters.
  • Module Section :
    • Extracts sections of user defined width/length along a specified reference axis from a point cloud (ODM).

New scripts

  • forTopHeight.py script : Script for calculating the top height calculated from the 100 highest trees in a (1 ha) area
    • Calculates statistical measures (mean, std.dev., ...) for the 100 highest points within specific AOIs.
  • Python script forDSM :
    • Calculates a DSM for forested areas using the landcover dependent approach of Hollaus et al, 2010.
  • Python script forDelineation :
    • Derives forest boundaries from topographic models (DTM, DSM, ...) based on a specific forest definition.
  • Python script forBiomass :
    • forBiomass calculates and validates parameters that can be used for estimating forest biomass.
  • Python script preCalcFootprint :
    • Calculates the axes of the laser footprint on the target using a trigonometric approach.

New features

Bug fixes

  • Module Import : In case of multiple input files and a single trajectory file the trajectory information was not attached
  • remove control characters from log messages before storing them in the XML file.
  • OPALS Generic Format : consider to-internal conversion also for binary files.
  • Assignments in Generic Filters now make sure that if assignments fail, geometries including their attributes and attribute statistics remain unchanged.

Improvements

  • Updated OPALS Python AddOns to NumPy 1.11.1, SciPy 0.17.1, and matplotlib 1.5.2.
  • global configuration (opals.cfg):
    • Decimal precision for ESRI ASCII grid output increased (now using 8 digits)
    • The output transformation parameters of Module ICP are now saved automatically to an xml parameter file by setting the 'outParamFile" option
  • Generic filter leaves, operators, and functions : syntax rule NeighborBinary (functions Dist2D, etc.): provide proper error message upon parsing failure.
  • Module Histo : Multi attribute support added. Hence, histogram and statistics for multiple attributes can be derived within one run.
  • LAS Extra Bytes: support reading and writing documented extra bytes for all LAS versions.
  • Module AddInfo logs the first few exceptions that its calculator has thrown.
  • Module ICP
    • now accepts the point cloud data in arbitrary data formats (las, (binary) xyz, shp, ...)
    • now uses the new ModuleTranslate for pre-ICP preparation and post-ICP transformation of the point clouds
  • Module StatFilter now supports:
  • Module EchoRatio and ModuleAddInfo now support two filters: (1) processing filter, (2) neighbourhood filter.
  • Integration of PyCharm as alternative high-end Python IDE available for Windows and Linux (details can be found here)

Version 2.1.5

Contents:

New modules

  • Module Rasterize :
    • Performs vector-to-raster conversion for all types of simple feature geometries (points, lines, polygons).

New features

Bug fixes

  • Module Histo : In case of multiple input grids an exception was thrown.
  • Module Cell : Features nmin and nmax were not calculated correctly, but min and max was erroneously returned instead.
  • Installation Guide : Example code for checking the integrity of the OPALS installation corrected
  • Comparative Expressions of Generic filter : before comparing attributes of type float with real number constants, constants are down-casted to float. This makes such comparisons behave as expected, even if precision is lost during the conversion from double to float - which is the case for e.g. 0.1.

Improvements

Version 2.1.4

Contents:

New modules

  • Module GridFeature :
    • Derives feature models (slope, curvature, roughness, etc.) from grid models based on the local grid neighbourhood defined by a kernel.

New features

  • A full featured C++-API and python binding of the Datamanager added (see The C++ OPALS Datamanager Library and the examples section)
  • Variable support for configuration files introduced (see Variable Support in Configuration Files)
  • Option type opals::MultiGridLimit: Rounding of limit coordinates is now supported not only for explicitly defined limits but also when specified via: union|intersect|dataset:i
  • Flexibility of the trajectory import increased (relevant for Module Import and Module DirectGeoref)
  • Module Import : So far the data import with trajectory add the beam vector (in the world coordinate system) as attribute to each point. The new parameter storeBeamInfo adds the possibility to append the range, the scan angle and the beam vector in the sensor coordinate system to the data.
  • The OPALS format definition support transformation of values during import and export (e.g. for changing of units)
  • Module Grid :
    • Parameter weightFunc added allowing the specification of individual point weights for grid interpolation.
    • Documentation updated
  • GDAL/OGR library updated to version 1.10.1. Support of the geometry library GEOS (v 3.4.2) was added allowing to manipulate simple feature geometries via the python bindings of GDAL/OGR (eg. Buffer, Intersect, ....).
  • Region Filter supports reading polygons directly from file
  • Module Histo : User defined probabilities for quantile calculation can now be specified via probabilities as a vector of double values (0. <= p <= 1.). Please note probabilites are no longer expressed as percentiles (integer from 0-100%) but as floating point values between 0 and 1.
  • Module PointStats : Features minority and majority have been added. An additional reference model zeroPlane introduced
  • Module StatFilter : It is now possible to specify a bin width for the features minority and majority (So far the bin width was fixed to 1)

Bug fixes

Improvements

Version 2.1.3

Contents:

New features

  • Module AddInfo : Nearest neighbour points can now be considered for attribute calculation. By analogy to raster layers, the nearest neighbour points are accessed via n[i] in the attribute formula. For more details please refer to the documentation.
  • Module PointStats : The module now supports 2D (infinite cylinder), 2.5D (finite cylinder) and 3D (sphere) point selection queries.
  • Module Grid :
    • New Delaunay Triangulation interpolator added
    • New curvature grid features (kmin, kmax, kmean, kgauss) added (Moving Paraboloid only)
  • new statistical features (opals::StatFeature): "sum"
  • Generic filters support neighbour geometries. All statistical operators hitherto available for rasters can now also be applied to neighbour coordinates/attributes. Moreover, neighbour coordinates/attributes can be queried separately, and neighbours can be used in binary neighbour functions.
  • Module Overlap : Shape file output of the area covered by all strips (-borderFile) and the strip overlap boundaries (-overlapFile) added.
  • Base: default overall vector format (oformat_vector) changed to ESRI shape (.shp)
  • opalsAlgebra : Major performance improvement especially when using opalsAlgebra for mosaicing purposes. In case, that input rasters are not referenced directly via their respective indices (e.g. r[10]) only those rasters are considered which effectively contain data at a certain position. This way not the entire stack of input rasters ones but only the relevant rasters is processed. The decision to omit void parts of input rasters is made automatically based on the formula string but can explicitly be controlled via the parameter skipVoidAreas.

Bug fixes

  • Module Contouring : Reported number of polygons corrected
  • Generic filters: Mean(r) and MeanAbs(r) did not calculate the correct values but returned 0.0 in any case instead.
  • Module Grid : Moving planes interpolator: Grid points were accepted as valid for a minimum number of 3 control points (i.e., well defined tilted plane system). This resulted in potentially infinite sigmaZ and sigma0 values causing problems in follow up applications (e.g. opalsHisto). The original behaviour requiring at least 4 control points for a valid movingPlanes grid points was re-established.

Improvements

  • Module Bounds : memory consumption has been considerably reduced
  • Module Import : robustness increased to avoid crashes in case of totally wrong 2d coordinates

Version 2.1.2.SP1

Contents:

Bug fixes

  • If OPALS modules that change or add ODM attributes (opalsNormals, opalsAddInfo, etc.) were used in conjunction with filters, crashes often occurred or at least arbitrary attribute information was destroyed (without filters the modules are working fine). Hence, it is highly recommended to update from version 2.1.2 to 2.1.2.SP1
  • In certain cases modules failed to run with an 'Access denied' error message, in case upper level directories of the input data did not have read access to the current user (the error occurred in a path conversion function resolving relative paths).

Version 2.1.2

Contents:

New modules

  • Module PointStats :
    • Derives and stores statistical parameters describing the local distribution of a point attribute for each point of an ODM.

New features

  • Using options -iFormat and -oFormat, paths to OPALS Format Definition files can be specified. If these paths consist of file names only, then in addition to the current working directory, they are now also searched in $OPALS_ROOT/addons/formatdef
  • The same accounts for Module ZColor, option -palFile, and $OPALS_ROOT/addons/pal, resp.
  • Attributes can now be specified case-insensitive in any situations within OPALS (the exception of case-sensitivity for user-defined attribute in generic filters was removed)
  • relevant for Python-/DLL-bindings:
    • ODM files that have been opened with write access cannot be opened by other module instances at the same time.
    • opals._Base.run() (Python) or opals::IModuleBase::run() (C++), resp., has been augmented with the optional argument reset, which defaults to false. If true, then that module instance calls opals::IModuleBase::reset() on its own right after the module run, resulting in all options being reset, and any open (file-) handles being closed.
  • new Filter PointSeq2Echo that assigns the attributes EchoNumber and NrOfEchos to points based on their time stamp.
  • Filter Remove now optionally allows for removing specific attributes instead of all attributes.
  • OPALS DataManager (ODM) file format was adapted to detect corrupt ODM files. The changes are upwards compatible, which means that old ODMs are implicitly converted to the new format if opened with write access. Old OPALS versions (2.1.1 or older), however, can not access ODM files in the new format.
  • Module Grid and Module Cell : Please note, that the estimation rule for output file names has been changed slightly. Now, the specified attribute, or the default "z" attribute, are consequently appended to the base file name (derived from the input file name). The new output file name construction rule for both modules is: Base + "_" + Attribute + "_" + Feature + Extension
  • Scientific thesis license: The terms and conditions for obtaining a free OPALS license for the duration of a Bachelor / Master / PhD thesis have been defined. For further information see: Frequently asked questions

Bug fixes

  • Module Convolution : extrapolation of pixel values at the border of the raster dataset (i.e. -edgeHandling trend ) corrected
  • Module Morph : circular kernels were erroneously considered as diamonds
  • Module AddInfo :
    • raster datasets were not closed correctly, resulting in a Python script crash when using a single AddInfo module instance multiple times.
    • using multiple raster sources in a single AddInfo run yielded an error (exception occurred: Raster band(s) not properly initialized..)
  • Module Import : in case of import with trajectory, points with gpstime equal the lowest trajectory entry were considered to be outside the trajectory time range.
  • Module Algebra : with line oriented input formats the internal cache size for raster values was set too small resulting in bad performance.
  • Module Shade :
    • using the nodata value for heights internally also for slopes erroneously led to nodata (background) pixel values within shadings (typically only with inconvenient nodata values such as 0) in rare cases.
    • output format was always GTiff regardless of the specified format.
  • Module ZColor : output format was always GTiff regardless of the specified format.
  • Module Histo : raster datasets: specification of a certain raster band (number) via the -attribute parameter enabled (already stated in the help text of earlier versions but not yet functional)
  • All modules writing grids/rasters in SCOP data format:
    • All nodata values were written to the output file resulting in unnecessary big files.
    • Determination of GeoTransform has been changed from dynamic extents to static extents.
  • Module Export : in case of many input files memory problems arose.

Version 2.1.1

Contents:

New features

Bug fixes

  • Module Import :
    • The import of ODM files failed when importing with trajectory information
    • half-pixel displacement of GDAL rasters corrected
  • Module Normals :
    • Storage order of eigenvalues were handled differently depending on the estimation method (now always descending: lambda 1 > lambda 2 > lambda 3)
  • Module AddInfo , Module Algebra ... (Nearest neighbour resampling)
    • determination of nearest neighbour grid point index corrected
    • behaviour at pixel borders adapted to correspond to Module Cell (i.e., left and upper cell border included, right/lower border excluded)
  • Documentation: broken links corrected
  • Generic filters: Attribute "Reflectance" was not evaluated correctly

Version 2.1.0

Contents:

New modules

  • Module Info :
    • Extracts and reports information of an ODM, vector or raster file.

New features

  • OPALS shell:
    • OPALS now provides a command shell customized for immediate use of OPALS after unzipping the distribution archive (no additional environment settings necessary). Furthermore, the OPALS shell is set up for direct use with the built-in portable Python (2.7) version and provides additional command aliases (import -> opalsImport, grid -> opalsGrid...). Please refer to the Installation Guide for further details.
  • LAS IO :
    • user definable coordinate resolution via OPALS Format Definition (OFD) XML File
    • extra byte attributes: offset and scaling supported for better compression (via OFD Files)
  • OPALS packages
    • Package scripts have become fully parameterized: instead of values hard-coded in the script code, the scripts now use additional command-line options:
      • -s,–subScripts
      • -x,–skipIfExists
      • -m,–mosaic
      • -k,–mask
    • Package script configuration files: each script by default now uses only 1 configuration file where the parameters of all subscripts are specified.
  • data type opals::CellFeature renamed to opals::StatFeature
  • new statistical features (opals::StatFeature): "entropy", "minority", "majority", "quantile" and "range":
    • Entropy is calculated based on a histogram of the data samples as : \(E=-sum_{i=1}^n p_i * ln(p_i)\)
    • Range means the difference between maximum and minimum
    • Available in Module Module StatFilter and opalsCell .
  • Module Import :
    • reading raster data (in GDAL readable format) into the ODM
  • Module Export :
    • export of multiple ODM files into a single vector data file
  • Module AddInfo :
    • implementation of full functionality
    • new parameter "attribute" for arbitrary attribute assignments (generic filter syntax)
    • new parameter "resampling" [nearestNeighbour|bilinear]
  • Module Histo :
    • The general filter syntax is now also supported for grid/raster datasets. Therefore, grid masks are no longer supported.
  • Module Cell :
    • support for opals::StatFeature "entropy", "minority", "majority", "quantile" and "range" added
  • Module StatFilter :
  • Module Algebra :
    • formulas can now additionally be specified in generic filter syntax (better performance).
    • major performance improvement by directly reading input grid pixels if they are aligned to the output grid (instead of re-sampling the input grids)
  • Module GeorefApprox :
    • new parameter "convThreshold" to specify the convergence threshold for the strip adjustment
    • parameter "sigmaShift" now allow negative values indicating artificial Zero-observations of the shifts of all strips
  • Generic filter :
    • support statistical functions (e.g. min, max, sum, mean, median, ..) for raster datasets. More info.
    • provide a generic way of setting attributes as invalid by the respective "invalid" keyword (e.g. _myIntensity=invalid).
  • Module Convolution : now returns a nodata pixel if all kernel pixels are invalid (i.e. no data)

Bug fixes

  • General: ODM attributes with predefined semantics starting with "X","Y" or "Z" have not been identified correctly (eg. "YawAngle".
  • Module StatFilter : "minority", "majority" and "median" calculation corrected
  • Module Export : progress counter in conjunction with export filters corrected
  • read/read-write access validation of ODM files corrected.

Version 2.0.0

Contents:

New OPALS Data Manager (ODM)

With OPALS 2.0.0 a new OPALS Data Manager (ODM) structure is introduced. The main features are:

  • I/O Performance improvement (2-3x faster I/O)
  • Joining the previously two ODM files (.dat/.idx) into a single file (.odm)
  • Attribute statistics (list of all attributes with additional summary for each attribute: min, max, mean, std.dev.)

Please note, that the new OPALS Data Manager files are incompatible to prior versions! To convert data (coordinates and attributes) please use Module Export v. 1.1.0 (arbitrary binary) to export and Module Import 2.0.0 to re-import the data in the new ODM structure.

LAS 1.4 support

OPALS 2.0.0 supports the latest LAS version 1.4. With LAS 1.4 a standardized way of describing arbitrary point attributes was introduced via the "Extra Bytes" Variable Length Record (VLR). That way, it is now possible to export ODM contents (points and attributes) to LAS as well as to import the entire contents of LAS 1.4 files (potentially containing Extra Bytes) into an ODM. Furthermore, the LAZ file format (losslessly compressed twin of LAS) is supported. Please refer to the LAS Format Definition documentation for further details.

Embedded Python installation

Starting from 2.0.0 OPALS distributions contain a portable version of Python 2.7. This has become necessary as some OPALS components (framework, Module Algebra) internally use the Python Interpreter. Up to now, it was possible to run OPALS stand alone (i.e without Python installation using the Python shared library delivered with OPALS) with only a few limitations. However, to exploit the full functionality of OPALS a full Python installation is required.

The portable version of Python 2.7 is part of the OPALS distribution (self extracting exe) and is located in the directory $OPALS_ROOT/opals. Beyond the basic Python installation, additional Python packages (numpy, scipy, GDAL/OGR, ...) can be downloaded from the OPALS web site. Further more, to ease the development of OPALS Python scripts, the OPALS distribution also contains the open source Python development environment PyScripter. For more details, please refer to the installation manual.

Extended filter features

The filter concept for selecting and manipulating input data was substantially extended and supports assignment of attribute values as well as the definition of new user defined attributes. More details can be found here.

OPALS documentation with search capabilities

The OPALS documentation is now available in HTML and Windows Help File (.chm) format. The latter provides the contents in a tree structure, features an automatic index and enables full text search within the documentation pages. To run the Windows Help version of the OPALS documentation simply double-click on opals.chm (within the documentation folder).

New modules

  • Module TIN :
    • Creates a TIN surface from ODM vector data based on a 2D constrained Delaunay triangulation.

Bug fixes

  • Module Histo :
    • min/max values in case of a user defined sampleRange corrected.
  • Module Import :
    • Corrupt ODM files when importing data with user defined filters corrected
  • Module ZColor :
    • avoid crash in case of scaleByRange min==max

New features

  • Logging of ODM statistics (coordinates and attributes) was unified
  • LAS IO :
    • OPALS Format Definitions (OFD) have been extended to allow import/export of LAS 1.4 files with user defined attributes stored in LAS Extra Byte Records. The LAS OFD (xml file) controls which attributes to import/export from/to LAS.
  • Module Import :
    • New parameter -tileSize allowing the specification of a user defined ODM tile size
  • Module Export :
    • New parameter -limits allowing to export data within a user defined window.
  • Module Grid :
    • New parameter -tileSize (user defined computing unit size).
  • Module Normals :
    • New parameter selMode, controls the nearest neighbour point selection (nearest/quadrant/octant)
  • Module Shade :
    • Type of parameter inFile changed to vector<opals::Path> and
    • New parameter feature (data type: vector<opals::GridFeature>.
    • General remark: opalsShade now accepts either one height grid or two slope grids (normalx/normaly or slope/exposition) as input. Calculating hill shadings from slope data sources potentially reduces necessary on-the-fly re-sampling operations resulting sharper images. More details.
  • Module Overlap :
    • Check for doubly specified file names added
  • update to GDAL 1.8.1:
    • supporting XYZ ASCII export of grid/raster data.

Version 1.1.0

Contents:

New installation structure

The file structure of the OPALS installation was significantly changed. All binary files (*.exe, *.pyd, *.dll) are now located in a single directory ($OPALS_ROOT/opals/) in order to ensure that the correct dlls are loaded. The main code of an OPALS module is now contained in the respective DLL only. The command line executables and the python modules internally use the respective DLL, thus, the size of both, *.exe and *.pyd was decreased significantly.

Changes in Python bindings

The OPALS Python bindings have been revised fundamentally. A global namespace "opals" was introduced. As a consequence:

  • the statement: "import opals" is necessary to access the OPALS modules
  • the common "opals" prefix was removed (old: opalsImport --> new opals.Import). All OPALS specific data types including all common types (e.g. LogLevel) are now contained in the global opals scope and no longer at module level. Please refer to the documentation for further details.

64-bit version

From version 1.1.0 on, OPALS is available as a 64-bit application. For the time being, both, 32- and 64bit versions are provided. However, the 32 bit versions will only be maintained for a limited period of time (approx. end 2012). Please keep in mind, that when using the Python bindings of OPALS mixing of the bit platforms is not allowed. In other words, a 64-bit OPALS version strictly requires a 64-bit Python version.

New modules

  • Module AddInfo : The aim of Module AddInfo is to derive additional ODM attributes by algebraic combination of existing attributes. Optionally, grid models can be incorporated enabling to define complex formulae. !!! Caution !!! Please note that, at the moment, the module only supports the calculation of normalized heights (Input: ALS points (-inFile) and DTM model (-gridFile)). Since this operation is of crucial importance for a series of applications, Module AddInfo was included in the release although the module has not yet reached its final state.
  • Module Fullwave : The aim of the module is the decomposition of the full waveform signal. At the moment, Module Fullwave only supports Riegl SDF files from LMS-Q560 scanners as input. The output (x,y,z: in scanner own coordinate system + full waveform attributes: amplitude, echo width, echo number, nr. of echoes) is written either in the internal MB/SOCS or in Riegl SDC format, which can be further processed with Module DirectGeoref to obtain a georeferenced point cloud with additional attributes in the project coordinate system.
  • Module Openness : The aim of Module Openness is to provide a raster map of local view sheds (i.e. openness) based on a grid DTM.
  • Module Simplify : The aim of Module Simplify is to professionally thin out regular or hybrid grid models based on a user defined maximum height tolerance and maximum point distance resulting in an approximating TIN surface.

Bug fixes

New features

  • Module Algebra :
    • processing of very large grids enabled
  • Module Cell :
    • x and y coordinates can be used as attribute
  • Module Import / Module Export :
    • now supports arbitrary binary format (c.f. OPALS Generic Format documentation for further details)
  • Module Export :
    • support of a specific 'invalid-value-marker' added ('min', 'max', 'NaN', 'Inf')
  • Module Grid :
    • new interpolation method: movingParaboloid (2nd order polynomial)
    • experimental movingRobustPlane interpolation removed (will come in opalsRobustFilter)
    • movingPlanes interpolation: check for extrapolation (in vertical direction) added
    • additional measures for empty grids (grid file trunk deleted, appropriate error message)
  • Module Histo :
    • new parameter -limit (pre-selection of data)
    • computation of additional (robust) parameter: sigma_mad (median of absolute differences)
    • overflow/underflow bins are now drawn in red
    • creation of a scalable vector graphics plot file can now be suppressed with: -plotfile off
    • multiple inputs (grids/ODM files) enabled
  • Module Import :
    • extended logging information (eg. point index statistics)
    • single input format for multiple input files enabled
  • Module LSM :
    • new parameter: -robFactor (user defined factor for robust parameter estimation)
  • Module Normals :
    • new parameter: -direction (allows for specifying in which direction the normals shall point)
    • new interpolation method: Fast Minimum Covariance Determination (FMCD, very robust estimator)
  • Module Overlap :
    • new parameter: -minArea (allows for relative and absolute overlap specification)
  • Module GeorefApprox :
    • now provides a -maxIter parameter allowing control over the number of iterations
  • Module Shade :
    • estimation of local slope improved yielding more detailed (less smoothed) hill shadings.
    • parameter -resampling removed
  • Module Simplify :
    • Generic Panorama TIN Format added
  • Module ZColor :
    • parameter '-nClasses 0' allows using a palette with its natural number of entries

Remarks

Please note that ODM files produced with version 1.0.2 are not compatible with version 1.1.0. To maintain the processed data (especially the point attributes), export the ODM with version 1.0.2 using a user defined xml format definition (c.f. OPALS Generic Format for details) and re-import the resulting file with version 1.1.0. We apologize for the inconvenience.

Orientation
orientation enum
Definition: ColumnTypes.hpp:148
Contains the public interface of OPALS.
Definition: AbsValueOrQuantile.hpp:8