Shape Format Definition

OPALS supports reading and writing of files in the ESRI Shape format (see also here) with full attribute support using OPALS Format Definition (OFD) files. The Shape format supports different geometry types and is capable of storing attributes (for each geometry entry) in a separate dbf (dBASE) file. Whereas the OPALS Datamanager can manage geometries of different type (point, polylines and polygons) within one file, this is not possible for Shape files. OPALS will throw an exception if objects of an unmatching type are exported. This behavior can be disabled, by enabling the 'skipUmatchingTypes' flag (see below).

Generally speaking, Shape files are often used as data exchange format for polyline or polygon objects.

For completeness it should be mentioned that shape files can be imported and exported even without OFD files based on a default strategy (import: no attributes are read. export: since the shape standard demands for dbf file, a dbf file with one column containing an ascending id is written)

Shape geometry types

As mentoined above, a shape can only contain geometries of one and the same type, whereas four different geometry types are supported:

  • point
  • polyline
  • polygon
  • multipoint

Those four geometry object can be stored in three different variations

  • 2d
  • 3d + measure value
  • 2d + measure value

Currently, OPALS does not support measure values, limiting import and export functionality to 2d and 3d geometries (with attributes for each object).

The (export) shape OFDs may specify the shape geometry type or it is automatically determined based on the first exported geometry object. Using the correspoding xml element <geometryType> the following type values can be set

Shape OFD values for geometryType
valuedimensioncomment
auto 3d automatic geometry type selection based on first object
point 3d each point is written as separate object (attributes for each point can be written to dbf file)
pointSet 3d points are written in chunks to the shape file (attributes only for each chunk can be stored)
polyline 3d
polygon 3d
point2d 2d
pointSet2d 2d
polyline2d 2d
polygon2d 2d

The <geometryType> element may also contain the <skipUmatchingTypes> boolean attribute. Setting this flag to "true" prevents OPALS from throwing an expcetion, if the ODM contains geometry objects that cannot be exported to the shape file (unmatching geometry type). In this case such objects are simply ignored by the export.

Attributs within shape files

Similare to the ODM, Shape files can store arbitrary attribute for each geometry entry. As mentoined above, the attributes are stored in a separate dbf file. The dBASE file format supports null values and different data types. However, one should know that numbers are always stored as text, which is why it is important to specify an appropriate witdth (=string length of the dbf field) for any dbf field. Floating point numbers require an additional decimals field. If an attribute exceeds the reserved field witdth, the export stops and an exception is thrown.

Althouth it is not recommend to export large point sets with their attributes to shape files (use efficient las/laz files instead), it
may be valuable as data exchange format for GIS and CAD software packages.

Since the Shape standard demands for a dbf file for Shape files, the default Shape export (=no OFD XML-file is specified) creates a dbf file containing one integer column (named IdCounter) containing an ascending id value for each exported entry (This predefined column can also be specified in the an OFD XML-file using $IdCounter for xml attribute internalName).

For each dbf column an <entry> tag is required. The name of the dbf column is defined by dbfName or defaulted from internalName which relates to an ODM attribute or to an internal id counter (set internalName to $IdCount - its value is incremented for each exported object starting from zero). In general, only the XML-attribute internalName must always be specified, while further XML-attributes may be relevant or even mandatory - depending on the usage of predefined or user-defined ODM attributes (see here), and whether the Shape OFD is only used for import (I), or also for export (E):

Shape OFD XML-attributes for Extra Bytes
attributerelevancedescriptiondefault
dbfName I/E the name of the dbf column internalName
dbfType E data type in the dbf column derived from internalName's data type
dbfWidth E width of dbf column estimated from internalName's data type
dbfDecimals E use decimals for dbf column (revelant floating point types only) estimated from internalName's data type
internalName I/E the OPALS attribute's name or $IdCounter

As described above, appropriate width values for DBF columns are essential for the export to work. OPALS provides internal defaults for width and decimals based on the attribute type, which should be appropriate in most cases. However, it can be desirable to manually set \ dbfWidth and dbfDecimals, e.g. to generate more compact DBF files.

Default DBF Field Settings
ODM type Memory Size DBF type width decimals
int64 8 bytes integer 10
int32, uint32 4 bytes integer 10
int16, uint16 2 bytes integer 5
uint8 1 byte integer 3
int8 1 byte integer 4
bool 1 bit boolean 1
float 4 bytes double 9 5
double 8 bytes double 9 5
cstr(n) n bytes string n
string variable string 32
$IdCount integer 6

In the following examples for different cases are shown

Examples

Example OFD file 'fwfShape.xml'

This OFD file can be used to export a point cloud with fullwave attributes for each point. It can be found in the $OPALS_ROOT/addons/formatdef/ directory. Here its application is shown.

<?xml version="1.0" encoding="ISO-8859-1" ?>
<!-- Exemplary shape format definition that exports point coordinates with fullwave attributes -->
<opalsFormatDefinition>
<description>Shape export of single point with fullwave attributes</description>
<shape>
<geometryType val="point" skipUmatchingTypes="true" />
<attributes>
<!-- entry internalName="$IdCounter" dbfWidth="6" / --> <!-- If uncomment, this entry generates an 'IdCounter' column with ascending id values -->
<entry internalName="GPSTime" dbfType="double" dbfWidth="15" dbfDecimals="6" />
<entry internalName="Amplitude" dbfType="integer" dbfWidth="5" />
<entry internalName="_PulseWidth" dbfType="double" dbfWidth="4" dbfDecimals="1" dbfName="PulseWidth" />
<entry internalName="EchoNumber" dbfType="integer" dbfWidth="1" />
<entry internalName="NrOfEchos" dbfType="integer" dbfWidth="1" />
</attributes>
</shape>
</opalsFormatDefinition>

TO COME: Polyline export based on LineModeler example

Author
jo
Date
3.11.2017