Pre-Processing Data using Construo
WARNING These pages are out of date. The most recent documentation is on the KeckCAVES website.
Construo, the Crusta data pre-processor, processes DEMs (topographic data), imagery (Satellite, Aerial, etc - basically anything with three 8bit bands) and scalar layers (difference maps, slope maps, etc - basically anything with a single scalar band). The processor accept numerous types of data (TIFF, ArcGrid, etc. anything that is supported by the GDAL library).
Before getting into the details on how Construo is used it is important to understand why it is even needed. The high-quality data visualization provided by Crusta in real-time is possible due to optimization in how the data is organized. Crusta assumes that a globe is partitioned into 30 patches which are in turn refined into a multi-scale hierarchy as needed by the spatial resolution of the visualized data. The set of these 30 hierarchies, combined with a meta-data descriptor, form a crusta globefile. Construo is the tool used to build globefiles from traditional digital terrain data.
Construo runs on the command line of a Terminal. For a printout of how construo works, you can simply type:
construo -dem | -color | -layerf <globefile name>.globeFile
[-offset <scalar> | -noOffset] [-scale <scalar> | -noScale] [-nodata <value> | -defaultNodata]
[-settings <settings file>]
-dem | -color | -layerf — Choose only one depending on if the data you are processing is a digital elevation map, RGB image or single band scalar image respectively
<globefile name> — path to the desired output globefile. New input images can be added to existing, compatible globefiles; or if the globe file doesn't exist a new one will be created. You must specify the path to a globefile for Construo to work. Please follow the convention of explicitly ending your globefile names with the extension .globeFile.
-offset <scalar> — can be used to add the <scalar> value to the values that are read from all the <input files> following the specification of -offset. Use the -noOffset flag to disable offsetting for all following <input files>
-scale <scalar> — can be used to scale the values that are read from all the following <input files> by <scalar> (e.g. convert a DEM file in feet to meters with -scale 0.3048). Use the -noScale flag to disable scaling for all the following <input files>
-nodata <value> — overrides the nodata loaded by GDAL and assign <value> as the nodata for all the following <input files> (only use this if GDAL does not read in an appropriate NoDATA value). E.g. use -nodata “0 0 0” if you want to remove all black pixels (the quotes ARE important!!). Use the -defaulNodata flag to use the internal nodata value for all following <input files>
For now ignore -pointsampling and -areasampling
-settings <settings file> — is used to specify custom planetary settings (e.g. using Mars instead of the default Earth).
<input files> — paths to all the source image files to add to the specified globe file.
IMPORTANT - Creating a Crusta globefile is like painting. At first you need to paint a base layer and then layer other paint on top that improves the painting. Thus, when processing a Crusta globefile with different resolution data - the input file order is important. First specify the lowest resolution data and then the medium then the highest resolution (e.g. Blue Marble –> Aster –> LiDAR).
For the examples we assume that ouput globefiles should be placed in the /Data/Processed folder and all the input files can be found in /Data/Raw
construo -dem /Data/Processed/topo.globeFile /Data/Raw/dem0.tif /Data/Raw/dem1.tif
construo -dem /Data/Processed/topo.globeFile /Data/Raw/dem2ArcGrid
If you have a globefile that you would like to add additional data to, you can specify that globefile in <globefile name>. This will then write all new data on top of the existing one. The process is irreversible, therefore backing up the existing globefile is recommended before adding new data to it.
For Imagery Processing you can ignore most of the commands (-offset, -scale, -nodata) - e.g. construo -color <globefile name> <input files>
Construo can load in NoDATA values from ESRI GRID files automatically. If you are loading topographic (DEM) data this can be done by specifying the directory that contains numerous files. Construo will find the GRID file and load that (e.g. /gis-data/GIS/dems/Damage/sm_alirt/)
It seems that GDAL does not like RGB (imagery) GRID format. Apparently, this format export 4 folders - The 3 color bands that contain the w*.adf files that crusta loads and a fourth folder that ArcGIS recognizes as the combination of all those files. Unfortunately loading in each individual folder will overwrite the previous, so Do NOT use ESRI GRID format for COLOR IMAGERY.
It seems that GDAL has an issue with reading the JPEG2000 image formats. Affects some of the HiRISE imagery. Please convert the JPEG2000 to another supported image format, e.g. GeoTIFF. Help with converting Mars data can be found here: Data For CrustaMars
GDAL makes a distinction between the latitude of origin and the latitude of true scale. These were not used correctly in some of the HiRISE dataset that we used. Fortunately Construo outputs a .proj file that contains the projection it uses for processing the data when it first reads the input (it does not overwrite the file if it already exists). This way projections can be manually set (if there is no georeferencing) or tweaked (if there is). Thus, a first (cancelled) attempt to build the HiRISE data provides the .proj files where the value stored for latitude_of_origin can be set to a new standard_parallel_1 parameter to produced the correct projection