Software Documentation
Download
Project Documentation
| Version | Date | Description |
|---|---|---|
| 1.15.0 | 2018-09-28 | This release of the Harvest Tool is a component of the integrated release 9.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.14.0 | 2018-03-30 | This release of the Harvest Tool is a component of the integrated release 8.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.13.0 | 2017-09-29 | This release of the Harvest Tool is a component of the integrated release 8.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.12.0 | 2017-03-31 | This release of the Harvest Tool is a component of the integrated release 7.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.11.0 | 2016-09-30 | This release of the Harvest Tool is a component of the integrated release 7.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.10.0 | 2016-03-31 | This release of the Harvest Tool is a component of the integrated release 6.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.9.0 | 2015-09-30 | This release of the Harvest Tool is a component of the integrated release 6.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.8.1 | 2015-08-05 | This release of the Harvest Tool is a component of the integrated release 5.1.1 of the PDS4 System. This is an operational release of the system components to date. |
| 1.8.0 | 2015-03-31 | This release of the Harvest Tool is a component of the integrated release 5.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.7.0 | 2014-09-30 | This release of the Harvest Tool is a component of the integrated release 5.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.6.1 | 2014-06-02 | This release of the Harvest Tool is a component of the integrated release 4.1.1 of the PDS4 System. This is an operational release of the system components to date. |
| 1.6.0 | 2014-03-31 | This release of the Harvest Tool is a component of the integrated release 4.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.5.0 | 2013-09-30 | This release of the Harvest Tool is a component of the integrated release 4.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.4.1 | 2013-06-21 | This release of the Harvest Tool is a component of the integrated release 3.1.1 of the PDS4 System. This is an operational release of the system components to date. |
| 1.4.0 | 2013-03-31 | This release of the Harvest Tool is a component of the integrated release 3.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.3.0 | 2012-09-30 | This release of the Harvest Tool is a component of the integrated release 3.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.2.0 | 2012-06-11 | This release of the Harvest Tool is a component of the integrated release 2.2.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.1.0 | 2012-01-31 | This release of the Harvest Tool is a component of the integrated release 2.1.0 of the PDS4 System. This is an operational release of the system components to date. |
| 1.0.0 | 2011-11-09 | This release of the Harvest Tool is a component of the integrated release 2.0.0 of the PDS4 System. This is an operational release of the system components to date. |
| 0.4.0 | 2011-08-29 | This release of the Harvest Tool is a component of the integrated release 1.2.0 of the PDS4 System. This release is intended as a prototype release in support of the assessment of the PDS4 standards and the system components to date. |
| 0.3.0 | 2011-03-25 | This release of the Harvest Tool is a component of the integrated release 1.1.0 of the PDS4 System. This release is intended as a prototype release in support of the assessment of the PDS4 standards and the system components to date. |
| 0.2.0 | 2010-10-31 | This release of the Registry Service is a component of the integrated release 1.0.0 of the PDS4 System. This release is intended as a prototype release in support of the assessment of the PDS4 standards. |
| 0.1.0 | 2010-08-25 | This release of the Harvest Tool is a component of the integrated release 0.1.0 of the PDS4 System. This release is intended as a prototype release in support of the demonstration at the Management Council Face-to-Face meeting in August 2010. |
| Type | Changes | By |
|---|---|---|
 | Updated to extract doi metadata from the Citation_Information area. Fixes SW-33. | mcayanan |
| Updated policy configuration example to extract Context_Area metadata from Product_Browse products. Fixes SW-34. | mcayanan |
| Updated reference type mapping to support additional facility relationships. Fixes SW-35. | mcayanan |
| Updated reference type mapping to support additional browse product relationships. Fixes SW-36. | mcayanan |
 | Added support for Source_Product_* classes. Fixes SW-38. | mcayanan |
| Type | Changes | By |
|---|---|---|
 | The tool needs to support collections with multiple collection product versions. Fixes PDS-471. | mcayanan |
| Improved resolution of relative file path references in a Checksum Manifest file. Fixes PDS-473. | mcayanan |
| Added support for Airborne and Telescope context products. | shardman |
| Type | Changes | By |
|---|---|---|
| Added support for registering unlabeled files in a PDS3 harvest run. Fixes PDS-288. | mcayanan |
| Updated Windows System Properties Install Guide. Fixes PDS-464. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Fixed to also recognize "Member Status" as a valid field name for the Field_Delimited element in the Inventory area of a collection. Fixes PDS-443. | mcayanan |
| Updated the JRE requirement to 1.7 in the Installation document. | shardman |
| Fixed support for Windows Style paths when creating the access urls. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Fixed to handle special characters in access URL specifications. Fixes PDS-388. | mcayanan |
| Strip extra whitespaces from the title metadata value in the label prior to ingestion. Fixes PDS-393. | mcayanan |
| Failing to detect files in persistent mode. Fixes PDS-397. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Improved registration performance by adding capability to perform batch registration on objects. Fixes PDS-361. | mcayanan |
| Fixed to create Product_File_Repository LIDs to end with an '_xml' instead of a '.xml' so that they can be properly retrieved by LID in the Registry. Fixes PDS-367. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added arguments in the shell scripts and Windows batch files to bump up the heap space memory allocation. | shardman, mcayanan |
| Added support for the Product_SIP_Deep_Archive product type. | shardman |
| Type | Changes | By |
|---|---|---|
| Added support for reading a checksum file for performing checksum validation during a PDS3 harvest run. Fixes PDS-289. | mcayanan |
| Delete harvest package if run results in no registrations. Fixes PDS-299. | mcayanan |
| Updated to set LIDVID members of a Collection Product in a slot instead of a hard association. | mcayanan |
| Fixed to create LIDs all in lowercase letters for Product_File_Repository products. | mcayanan |
| Fixed to report to the log if a target file or directory is not readable. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Modified to resolve environment variables specified within the Harvest policy file. Fixes PDS-270. | mcayanan |
| Added support for capturing the units designation from PDS4 labels and registering it with the registry as the slotType. For PDS3 registrations, allow the user to define the slotType in the ancillary metadata section for a given keyword. Fixes PDS-272. | mcayanan |
| Modified to not register a file product more than once if it is referenced multiple times in a label. Fixes PDS-274. | mcayanan |
| Modified the configuration files to extract External_Reference.reference_text from product labels. | shardman |
| Modified to set LIDVID-based references as slots for a Collection product when the reference type is not of type "member_ref". | mcayanan |
| Modified to replace white spaces with an underscore character for logical identifiers that are created during PDS3 registrations. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added support for labels utilizing xinclude statements. Label fragments will get registered as File Objects under the file type "Label Fragment". Fixes PDS-233. | mcayanan |
| Updated Unix shell script and Windows batch file to provide a more user friendly message when JAVA_HOME is not set in the environment. Fixes PDS-253. | mcayanan |
| Updated to return a more descriptive message when an error occurs while creating a registry package. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added the capability to detect a file object's mime type. This value will be added as a slot in the registry under the name 'mime_type'. | mcayanan |
| Added the capability to associate a specified file object in a product label to a file type. This value will be added as a slot in the registry under the name 'file_type'. | mcayanan |
| Modified to support v1.1 of the data model. | shardman |
| Fixed so that any errors that are found while reading in the global policy file will be reported in the log. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added a mechanism in the policy config for excluding sub-directories to traverse while crawling a target directory. | mcayanan |
| Added a '-D, --ignore-dir' command-line flag option to specify sub-directories to exclude while crawling a target directory. | mcayanan |
| Added a 'titleContents' element in the policy config to support the customizing of a PDS3 product's 'name' in the registry. | mcayanan |
| Updated policy config and command-line options to support the passing in of multiple PDS3 target directories. | mcayanan |
| Updated to no longer provide any default registration of metadata for PDS3 product registration. | mcayanan |
| Updated to support extraction of metadata within objects in a label for PDS3 product registrations. | mcayanan |
| Updated header in the report log with additional, user specified parameters: targets, file inclusions, directory exclusions. | mcayanan |
| Updated the policy config with a 'fileFilter' element instead of the 'filePattern' element to specify files to include while crawling a target directory. | mcayanan |
| Updated the mechanism for specifying PDS3 target directories on the command-line. A '-pds3, --is-pds3-dir' flag option is used instead of specifying '-D, --pds3-directory' to indicate that a target directory on the command-line is for PDS3 product registration. | mcayanan |
| Fixed to allow product_class to appear as a slot. | mcayanan |
| Fixed to support handling of windows paths as offset specifications when creating access urls. | mcayanan |
| Fixed to support checksum manifest files that contain a './' in the file specs. | mcayanan |
| Type | Changes | By |
|---|---|---|
| For PDS4 registration, updated to check that the logical identifier of a product does not contain URN reserved and excluded characters. If it does, issue an error message in the log and do not register the product. For PDS3 registration, if there are URN reserved and/or excluded characters, replace these with dash characters before registering the product. Fixes PDS-93. | mcayanan |
| Upgraded support for PDS3 data sets. This includes making changes to the policy config to allow the logical identifier to be configurable, allow static metadata to be set, and allow the slot name for ancillary metadata to be configurable. Fixes PDS-97. | mcayanan |
| Updated to support the v09 data model. Fixes PDS-98. | mcayanan |
| Updated to utilize a Checksum Manifest file for storing and verifying checksums. Also added a flag in the policy config to enable/disable checksum generation. Fixes PDS-99. | mcayanan |
| Fixed to key off of 'Product_Collection' and 'Product_Bundle' to determine the proper metadata extraction process. Fixes PDS-94. | mcayanan |
| Fixed to not create an empty slot in the Registry for empty or null element values. Fixes PDS-95. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Updated to compare tool generated checksums of Product_File_Repository products to its supplied value in either the inventory or associated product label (if provided). Fixes PDS-71. | mcayanan |
| Added the -v, --verbose flag option to change the severity level of the log messaging. Default is set to Info and above. | mcayanan |
| Updated to be able to provide an offset in the policy config to nix file paths of a registered product for an access url. Fixes PDS-74. | mcayanan |
| Updated to map reference types found in the products to a reference type that will be registered with the product. | mcayanan |
| Updated xpath expressions to support the latest changes in the PDS4 data model. | mcayanan |
| Updated the policy config to provide the capability to specify a registry package name and description. | mcayanan |
| Updated summary section of the tool report based on user feedback. | mcayanan |
| Updated to print to the log the XML Extrinsic and Association descriptions that will be registered when in debug mode. | mcayanan |
| Updated slot name that holds data objects specified in a product label from "dataClass" to "data_class". | mcayanan |
| Fixed to support registration of PDS3 labels containing multiple DATA_SET_ID values. Fixes PDS-63. | mcayanan |
| Fixed to support handling of metadata extraction where the desired metadata is found multiple times in a product label. Fixes PDS-73. | mcayanan |
| Fixed to rely on the field_location and field_length attributes instead of commas to parse an inventory table file. | mcayanan |
 | Updated to remove the validation functionality. | mcayanan |
| Removed the Release Notes document. This document now serves as the single source for release changes. | shardman |
| Type | Changes | By |
|---|---|---|
| Added an attribute called 'slotName' to the 'xPath' element tag in the policy configuration file to allow the capability to specify customized slot names in the registry for registered metadata. | mcayanan |
| Added the capability to look for the different object types specified in a data product label and stores them under the 'dataClass' slot name in the registry. | mcayanan |
| Added the capability to specify targets on the command-line. | mcayanan |
| Added a required -c, --config flag option to specify the harvest policy configuration file. | mcayanan |
| Added a -D, --pds3-directory flag option to specify a PDS3 directory for PDS3 data product registration. | mcayanan |
| Updated xpath expressions that look for specific metadata needed for product registration and tool behavior due to updates to the latest PDS4 model. | mcayanan |
| Updated to set the product type to "ProductFile" when ingesting products to the PDS Storage Service. | mcayanan |
| Cleaned up the documentation and removed the PDF generation feature. | shardman |
| Fixed a bug where the tool complained about the keystore password not being specified when attempting to register to an unsecured registry instance. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added support for ingestion to the PDS Storage Service. | mcayanan |
| Added support for providing access URLs with the registered products. | mcayanan |
| Added a new command-line flag option, -k, in order to be able to pass in a keystore password when accessing a secured instance of the registry. | mcayanan |
| Update to place LIDVID-based associations in the cross reference areas as slots. | mcayanan |
| Update to support validation of older versions of the PDS4 data model. | mcayanan |
| Fixed to look for the directory_path_name attribute when constructing the file specification for the document file. Fixes PDS-42. | mcayanan |
| Fixed HarvestController shell script and batch file to no longer throw a NoClassDefFoundError exception. Fixes PDS-49. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added the capability to register PDS3 data products. | mcayanan |
| Added the capability to perform file object registration. | mcayanan |
| Update to register associations at the end of a Harvest run. | mcayanan |
| Update to register LID-based associations as slots. | mcayanan |
| Update to register products to a registry package. | mcayanan |
| Update to register the version ID of a data product as a slot in the registry. | mcayanan |
| Update to support the latest version of the PDS Security Service. | mcayanan |
| Fixed a bug where namespaces being defined in the configuration policy file were not being recognized. | mcayanan |
| Fixed a bug to support crawling of multiple targets when running the tool in persistence mode. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added the capability to perform validation on a product label before being ingested into the registry. | mcayanan |
| Added a new tag in the policy file that gives the ability to turn off validation. | mcayanan |
| Added the capability to allow the Harvest Tool to run in continuous mode via a daemon. | mcayanan |
| Added a slot called 'verified' in the associations to indicate if the target reference exists in the registry at the time the association was registered. | mcayanan |
| Update to make sure that an association does not exist in the registry before attempting to register it. Fixes PDS-2. | mcayanan |
| Update to interface with the latest version of the Security Service. | mcayanan |
| Update to tie associations together with GUIDs instead of LIDVIDs. | mcayanan |
| Update to perform more robust checking and reporting of malformed association entries (example: an association entry missing the LID-VID tag). | mcayanan |
| Fixed to register the associations in the cross reference area of a collection. | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added the capability to register and crawl a PDS Bundle file. | mcayanan |
| Added the capability to support namespaces in the PDS4 product files. Harvest is initialized to look for expected metadata like logical_identifier, product_class, etc. within the pds namespace. | mcayanan |
| Updated the Harvest tool to look for the 'product_class' tag instead of the 'object_type' tag. | mcayanan |
| Updated the Harvest tool to support table-formatted collection files. | mcayanan |
| Updated the Harvest tool to register associations when processing a table-formatted collection file. | mcayanan |
| Updated the Harvest policy file to be able to specify namespaces and bundle files. Tag names in general were also updated. See the user guide for the latest and greatest description of the policy file. | mcayanan |
| Updated the Harvest report log to show header information and a summary. | mcayanan |
| Updated messaging to log the GUID upon a successful ingestion of a product into the registry. | mcayanan |
| Fix error handling in the associations for the case where a bad lidvid_reference is given (i.e. missing the version field). | mcayanan |
| Type | Changes | By |
|---|---|---|
| Added the capability to pass in an XML-based, policy file to the tool to find products and what metadata to register. | mcayanan |
| Added the capability to register data products with a target Registry Service. | mcayanan |
| Added the capability to register a PDS Inventory File first before crawling the file for products. | mcayanan |
| Added the capability to output log messages. | mcayanan |
| Added the capability to launch the tool with a shell script or batch file. | mcayanan |
| Added the capability to register associations that are in the "class" format. | mcayanan |
| Provide the initial documentation covering the design, installation and operation of the software. | shardman, mcayanan |