Operation

This document describes how to operate the Transform Tool. The following topics can be found in this document:

Note: The command-line examples in this section have been broken into multiple lines for readability. The commands should be reassembled into a single line prior to execution.

Example Products

The Transform Tool package contains an examples/ folder which contains example PDS3 and PDS4 data products. The intent is to provide example products to demonstrate the different transformations that can be performed by the tool. The table below lists the target inputs that can be passed into the tool and the format types that are valid for each target input listed:

Target InputDescriptionValid Format Types
20140804T205944Z_MAP_bias_V001.xmlPDS4 data product label of a FITS image.jpg, gif, jp2, tif, png, pnm, pvl, html, html-structure-only
BA03S183.IMGPDS3 attached label of a 16-bit image.jpg, gif, jp2, tif, png, pnm
C000M5232T493378259EDR_F0000_0134M1.xmlPDS4 data product label of an Array 3D Image.jpg, gif, jp2, tif, png, pnm, pvl, html, html-structure-only
ELE_MOM.LBLPDS3 data product label.pds4-label
FF01.LBLPDS3 data product label of an 8-bit image.jpg, gif, jp2, tif, png, pnm
FHA01118.LBLPDS3 attached label of an 8-bit image.jpg, gif, jp2, tif, png, pnm
N1727539187_1.LBLPDS3 data product label of a 16-bit image.pds
Product_Table_Binary.xmlPDS4 data product label of a binary table.csv, html, html-structure-only
Product_Table_Character.xmlPDS4 data product label of a character table.csv, html, html-structure-only
Product_Table_Delimited.xmlPDS4 data product label of a delimited table.csv, html, html-structure-only
Product_Table_Multiple_Datafiles.xmlPDS4 data product label that references multiple table data files.csv, html, html-structure-only
Product_Table_Multiple_Tables.xmlPDS4 data product label containing multiple table objects.csv, html, html-structure-only
b0090_p243401_01_01v02.xmlPDS4 data product label containing an Array 3D Specturm object followed by multiple Array 2D Image objects.jpg, gif, jp2, tif, png, pnm, pvl, html, html-structure-only
i943630r.xmlPDS4 data product label of an Array 2D Image.jpg, gif, jp2, tif, png, pnm, pvl, html, html-structure-only

Quick Start

This section is intended to give a quick and easy way to run the Transform Tool. For a more detailed explanation on other ways to run the tool, go to the Advanced Usage section. In general, to run the tool, pass in a PDS3 or PDS4 label along with one of the format types described in the Format Types section to perform the desired transformation.

Transform an image

The command below shows how to do a PDS4 to JPEG transformation:

%> transform ../examples/i943630r.xml -f jpg
      

Note that for image transformations, the tool performs normalization on the image for display purposes. This includes performing auto scaling in order to brighten or darken the image to an appropriate level.

Transform a table

The command below shows to do a PDS4 table to CSV transformation:

%> transform ../examples/Product_Table_Character.xml -f csv
      

Note that table transformations only applies to PDS4 data products.

Command-Line Options

The following table describes the command-line options available:

Command-Line OptionDescription
-t, --target <file>Explicitly specify a PDS3 or PDS4 product label that contains a reference to an image file to transform. The target can be specified implicitly as well. (example: transform i943630r.xml)
-o, --output <dir>Specify an output directory where the transformed images will reside. If this option is not specified on the command-line, the tool will default to placing the output files in the current working directory.
-f, --format-type <type>Specify the transformation format type to perform on the target(s). Please see the Format Types section for details on the valid format types available to the tool.
-n, --index <int value>Specify the index of the image or table to transform. The default is to transform the first one found. Please see the Advanced Use Cases section for details on using this flag option.
-d, --datafile <file>Specify the data file to transform. The default is to transform the first one found. Please see the Advanced Use Cases section for details on using this flag option.
-a, --allSpecify to transform all images or tables found within a given target PDS label. Please see the Advanced Use Cases section for details on using this flag option.
-O, --list-objectsList the table and image objects found within a given label that are currently supported by the tool.
-V, --versionDisplay the release number and copyright information.
-h, --helpDisplay tool usage.

Format Types

The following table lists the output formats available to both PDS3 and PDS4 target inputs:

Format TypeDescription
jpg or jpegSpecify to transform to an image in a Joint Photographic Experts Group format.
jp2Transform an image to a JPEG 2000 format.
gifTransform an image to a Graphics Interchange Format.
pngTransform an image to a Portable Network Graphics format.
tif or tiffTransform an image to a Tagged Image File Format.
pnmTransform an image to a Portable Any Map format.

The following table lists the output formats available to only PDS4 target inputs:

Format TypeDescription
pvlTransform a given PDS4 label into a Parameter Value Language file.
htmlTransform a given PDS4 label into an html representation of the label.
html-structure-onlyTransform a given PDS4 label into an html representation of the label structure. As a result, the element contents are stripped out in the resulting file.
csvTransform a given table into a comma-separated value file.

The following table lists the output formats available to only PDS3 target inputs:

Format TypeDescription
pdsTransform a VICAR image into a PDS4 Array 2D image. Additionally, the given PDS3 label will be transformed into a PDS4 label. Note that this transformation currently applies to target VICAR images only. Please see the PDS3 to PDS4 Label Transformation section for more details on how the tool performs a PDS3 to PDS4 label transformation.
pds4-labelTransform a given PDS3 data product label into a PDS4 Product Observational product label. Please see the PDS3 to PDS4 Label Transformation section for more details on how the tool performs a PDS3 to PDS4 label transformation.

Advanced Usage

This section describes more advanced ways to run the tool, as well as its behaviors and caveats.

Running the Transform Tool

The Transform Tool requires the passing in of a PDS3 or PDS4 product label that contains a reference to an image file. In addition, the tool assumes that the referenced image file is co-located with the product label. Other notable caveats with this version of the tool include:

  • For PDS3 to TIFF transformations, the output tile size defaults to 8 lines by the image width.

This section demonstrates some of the ways that the tool can be executed using the command-line option flags:

  • Transform a Single PDS4 Array 2D Image
  • Transform a Single PDS4 Label To Parameter Value Language (PVL) File
  • Transform a Single 8-Bit PDS3 Image
  • Transform a PDS3 Label to PDS4 Label
  • List the Supported Objects

The files used in the command-line examples below are packaged with the tool in the examples directory.

Transform a Single PDS4 Array 2D Image

The following command demonstrates transforming a PDS4 array 2D image referenced in the associated product label, i943630r.xml:

% transform ../examples/i943630r.xml -f jpg
        

The tool creates an output file on the current working directory using the target image file name with the user-specified format as the file extension. Using the example above, the tool creates the output file i943630r.jpg.

To write the output file in another directory location, use the -o flag option. The following command demonstrates writing the output to a location named /home/pds/images:

% transform ../examples/i943630r.xml -f jpg -o /home/pds/images          
        

Transform a Single PDS4 Label To Parameter Value Language (PVL) File

The following command demonstrates transforming a PDS4 label, i943630r.xml, to a PVL file, i943630r.pvl:

% transform ../examples/i943630r.xml -f pvl
        

Transform a Single 8-Bit PDS3 Image

The following command demonstrates transforming a PDS3, 8-bit image referenced in the associated attached label, FHA01118.LBL, into a gif image file named FHA01118.gif:

% transform ../examples/FHA01118.LBL -f gif
        

The following command demonstrates transforming a PDS3, 8-bit image referenced in the associated detached label, FF01.LBL, into a portable network graphics file named FF01.png:

% transform ../examples/FF01.LBL -f png
        

Transform a PDS3 Label to PDS4 Label

The following command demonstrates transforming a PDS3 data product label, ELE_MOM.LBL, into a PDS4 Product Observational label named ele_mom.xml:

% transform ../examples/ELE_MOM.LBL -f pds4-label
        

Advanced Usage

The section details some of the more advanced ways that the tool can be used.

Transforming Complex Labels

A common use case scenario of the Transform Tool is to transform a label that points to a single data file containing a single image or table. A more advanced use case scenario is handling a label that points to multiple data files where each data file contains multiple images or tables. This section details how to tell the Transform Tool to transform a specific image or table if given a more complex label.

Note: The flag options that allow control of which images to transform, -n, -d, -a, are not supported with PDS3 labels pointing to multiple, non-FITS, image transformations.

Transform a Specific Table

Use the -O,--list-objects flag option to first list the table objects being supported by the tool:

% transform ../examples/Product_Table_Multiple_Tables.xml -O
        

This will display the supported table objects:

Supported Images: 

  None Found

Supported Tables: 

  Data file: PDS4_ATM_TABLE_CHAR_MULTIPLE.TAB

    index = 1
    object type = Table_Character
    name = null
    local identifier = TABLE_CHAR_1
    records = 23
    record length = 88
    groups = 0
    fields = 10

    index = 2
    object type = Table_Character
    name = null
    local identifier = TABLE_CHAR_2
    records = 23
    record length = 88
    groups = 0
    fields = 10

        

To perform a table to CSV transformation of the 2nd table, run the following command:

% transform ../examples/Product_Table_Multiple_Tables.xml -f csv -n 2
        

Transform a Specific Data File

Use the -O,--list-objects flag option to first list the table objects being supported by the tool:

% transform ../examples/Product_Table_Multiple_Datafiles.xml -O
        

This will display the supported table objects:

Supported Images: 

  None Found

Supported Tables: 

  Data file: PDS4_ATM_TABLE_CHAR.TAB

    index = 1
    object type = Table_Character
    name = null
    local identifier = PHX-M-TT-5-WIND-VEL-DIR_TABLE_CHAR
    records = 23
    record length = 88
    groups = 0
    fields = 10

  Data file: PDS4_TABLE_DELIMITED.csv

    index = 1
    object type = Table_Delimited
    name = null
    local identifier = null
    records = 3
    groups = 0
    fields = 13

  Data file: 2d234493326edratf3d2537n0m1.dat

    index = 1
    object type = Table_Binary
    name = null
    local identifier = null
    records = 336
    record length = 96
    groups = 1
    fields = 20
    
        

The following command demonstrates transforming the table associated with the data file PDS4_TABLE_DELIMTED.csv:

% transform ../examples/Product_Table_Multiple_Datafiles.xml -f csv -d PDS4_TABLE_DELIMITED.csv
        

Transform Everything

Display the list of FITS images supported by the tool using the -O, --list-objects flag option:

% transform ../examples/20140804T205944Z_MAP_bias_V001.xml -O
        

This will display the following:

Supported Images: 

  Data file: 20140804T205944Z_MAP_bias_V001.fits

    index = 1
    object type = Array_2D_Image
    name = Primary Data Unit
    local identifier = PDU
    data type = IEEE754LSBSingle
    lines = 1024
    samples = 1024

    index = 2
    object type = Array_2D_Image
    name = Secondary Data Unit (Full Detector Array
    local identifier = SDU
    data type = IEEE754LSBSingle
    lines = 1044
    samples = 1112

Supported Tables: 

  None Found

        

The following command demonstrates transforming all FITS images found in the label:

% transform ../examples/20140804T205944Z_MAP_bias_V001.xml -f jpg -a
        

The label points to a single data file that contains multiple images. So the tool ends up creating the following output files:

  • 20140804T205944Z_MAP_bias_V001_1.jpg
  • 20140804T205944Z_MAP_bias_V001_2.jpg

Note that the tool appends an index value to the end of each output file name whenever the tool has to transform multiple images or tables contained within a single data file.

Improving Tool Performance

The Transform Tool uses the Java Advanced Imaging (JAI) library to perform image transformations. By default, the tool turns off JAI's acceleration layer since that requires installing platform-dependent libraries. The purpose of the acceleration layer is to improve image processing performance. This section details how to install the platform dependent libraries locally to enable the acceleration layer if needed.

Mac OS X Environment

For Mac OS X platforms, they automatically ship with these dependency libraries. They should be found at the following location: /System/Library/Java/Extensions and should contain the following files:

  • libmlib_jai.jnilib
  • mlibwrapper_jai.jar

So, to enable JAI's native acceleration feature, open the bin/transform shell script file and remove the -Dcom.sun.media.jai.disableMediaLib=true setting from the command-line call in the script. The resulting command-line call should look like the following:

"${JAVA_HOME}"/bin/java -Xms256m -Xmx1024m -Doverwrite.output=true -Dexternal.programs.home=${PARENT_DIR}/external-programs -jar ${TRANSFORM_JAR} "$@"    
        

To check that the native acceleration feature is working, run an image transformation:

% transform ../examples/i943630r.xml -f jpg          
        

Verify that an "Error: Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode." message does not appear during the transformation.

UNIX-Based Environment

For Unix and Linux installations, download the appropriate installer located in the Java Advanced Imaging Image I/O Tools 1.0_01 section from here. Once the installer has finished downloading, double-click the file and follow the installation instructions.

Once this installation step is complete, open the bin/transform shell script file and remove the -Dcom.sun.media.jai.disableMediaLib=true setting from the command-line call in the script. The resulting command-line call should look like the following:

"${JAVA_HOME}"/bin/java -Xms256m -Xmx1024m -Doverwrite.output=true -Dexternal.programs.home=${PARENT_DIR}/external-programs -jar ${TRANSFORM_JAR} "$@"    
        

To check that the native acceleration feature is working, run an image transformation:

% transform ../examples/i943630r.xml -f jpg          
        

Verify that an "Error: Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode." message does not appear during the transformation.

Windows Environment

Download the appropriate Windows installer located in the Java Advanced Imaging Image I/O Tools 1.0_01 section from here. Once the installer has finished downloading, double-click the file and follow the installation instructions.

Once this installation step is complete, open the bin/transform.bat batch file and remove the -Dcom.sun.media.jai.disableMediaLib=true setting from the command-line call in the script. The resulting command-line call should look like the following:

"%JAVA_HOME%"\bin\java -Xms256m -Xmx1024m -Doverwrite.output=true -Dexternal.programs.home=%PARENT_DIR%\external-programs -jar "%TRANSFORM_JAR%" %*       
        

To check that the native acceleration feature is working, run an image transformation:

% transform ../examples/i943630r.xml -f jpg          
        

Verify that an "Error: Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode." message does not appear during the transformation.

PDS3 to PDS4 Label Transformation

The intent of the PDS3 to PDS4 label transformation is to transform a PDS3 data product label into a PDS4 Product Observational product label with the minimum set of elements in order to be compliant with the PDS4 standards. Therefore, not all PDS3 keywords will map to the resulting label. This section details how the Transform tool populates some of the required elements of a PDS4 label. Please see the Known Issues section for details on the limitations with PDS3 to PDS4 label transformations.

Populating PDS4 Elements

This section describes how the tool leverages the PDS3 label in order to populate some of the PDS4 elements. Below is a list of PDS4 elements and a description of how the tool goes about populating that element.

logical_identifier

The tool will populate the logical_identifier element in the following mannner:

urn:nasa:pds:data:${DATA_SET_ID}:${PRODUCT_ID}
        

If PRODUCT_ID does not exist in the PDS3 label, then the tool will default to using the label filename instead:

urn:nasa:pds:data:${DATA_SET_ID}:${labelFilename}
        

title

The tool will populate the title element in the PDS4 label based on the existence of one or more of the following keywords:

  • OBSERVATION_NAME or
  • OBSERVATION_ID or
  • PRODUCT_NAME or
  • DATA_SET_ID + PRODUCT_ID or
  • DATA_SET_ID

So if OBSERVATION_NAME exists in the PDS3 label, it will use that to populate the title element. If it does not exist, it will use OBSERVATION_ID. If that does not exist, then it will use PRODUCT_NAME, and so forth.

Investigation_Area

For the Investigation_Area, the tool will use MISSION_NAME to populate this element in the following manner:

<name>${MISSION_NAME}</name>
<type>Mission</type>
<Internal_Reference>
  <lid_reference>urn:nasa:pds:investigation.${MISSION_NAME}</lid_reference>
  <reference_type>data_to_investigation</reference_type>
</Internal_Reference>
        

If MISSION_NAME does not exist, then the tool will use MISSION_PHASE_NAME instead.

Observing_System

For the Observing_System, the tool will populate this element with INSTRUMENT_HOST_NAME, INSTRUMENT_NAME, and INSTRUMENT_ID in the following manner:

<Observing_System_Component>
  <name>${INSTRUMENT_HOST_NAME}</name>
  <type>Spacecraft</type>
</Observing_System_Component>

<Observing_System_Component>
  <name>${INSTRUMENT_NAME}</name>
  <type>Instrument</type>
</Observing_System_Component>

<Observing_System_Component>
  <name>${INSTRUMENT_ID}</name>
  <type>Instrument</type>
</Observing_System_Component>
        

Known Issues

This section describes some of the known issues and limitations with the Transform Tool.

FITS Transformations

When performing FITS transformations, the Transform Tool may output the following warning message:

Warning: multiple occurrences of key:      
      

This message is coming out of the FITS dependency library. It does not appear to be of concern at this time as several FITS images were used in testing and a successful transformation has occurred each time in cases where this warning message has appeared.

PDS3 Image Transformations

  • MOC images are not supported.
  • The flag options that allow control of which images to transform, -n, -d, -a, are only supported with FITS image transformations. If the tool is given a PDS3 product label pointing to multiple, non-FITS images, the current implementation is limited to the transformation of the first image only.

PDS4 Image Transformations

The tool currently supports images that have the following data types:

  • SignedByte/UnsignedByte
  • SignedLSB2/UnsignedLSB2
  • SignedMSB2/UnsignedMSB2
  • UnsignedMSB4
  • UnsignedMSB8
  • IEEE754MSBSingle
  • IEEE754MSBDouble

If a data type is not supported, the tool will throw the following error:

Array data type is not valid, null, or unsupported
      

Due to lack of PDS4 image test data, it is unknown at this time what other data types are supported by the tool.

Array_3D_Specturm Transformations

For transformations of Array_3D_Specturm products, the tool currently displays the image using band 1. At this time, the tool does not provide the capability to display the image using another band.

PDS3 to PDS4 Label Transformations

At this time, the Transform tool will support PDS3 to PDS4 label transformations of the following object types:

  • TABLE
  • SERIES
  • COLUMN
  • BIT_COLUMN
  • CONTAINER
  • IMAGE
  • SPREADSHEET

Explicit FILE objects, QUBE objects, as well as labels containing pointers to label fragments (i.e. ^STRUCTURE) are not supported by the tool at this time.