Plan Document

NASA Planetary Data System

PDS4 System

Build 8a Test Document

                                           

EG-0110-02-1

 

 

 

 

 

 

 

 

 

 

Change Log

 

Revision

Date

Description

Author

Draft

 

Initial draft release.

 

1c

May 16, 2011

Modified many tests to work with build 1c of Harvest and Registry. Other minor revisions.

Richard Chen

 

1d

Oct 24, 2011

Updated for build 1d

Richard Chen

2a

Nov 11, 2011

Updated for build 2a

Richard Chen

2b

Feb 28, 2012

Updated for build 2b

Richard Chen

2b.1

Mar 7, 2012

Re-added HVT.T3, expanded SRCH.T5

Richard Chen

2c

Jul 17, 2012

Updated for build 2c

Richard Chen

3a

Oct 26, 2012

Added BNDL.T1 to .T4

Richard Chen

3b

Apr 09, 2013

Added AAFUNCTION.*

Richard Chen

3b.1

Aug 30, 2013

Incorporated JIRA resolutions

Richard Chen

3b.2

Sep 15, 2013

Cleaned up and removed not applicable test procedures

Richard Chen, Emily Law

4a

Dec. 05, 2013

Incorporated PDS4ORR-RFA1’s reeendations by:

·   folding the test plan into this (test procedures and test results) document.

·   listing the requirements tested in each test case and their pass/fail status

·   adding version numbers in list of software components tested

·   indicating for each JIRA issue: the software build under which it was discovered, its severity, the test case demonstrating, and its description.

·   adding “pass”, “fail”, or “skip” to the requirements traceability matrix

Added tests TPRT.1, SCMA.1 to test transport service and schema.

Removed redundant tests AATESTME.*.

Richard Chen, Emily Law

4b’

Apr 14, 2014

Replaced generate0.7.0 with 0.7.1

Richard Chen

5a

Oct 27, 2014

Added PRV.3, TPRT.4. Changed HVT.4 to GEN.2. Changed TPRT.3 to test more of transport-proxy. Deleted TPRT.2 after folding its step into TPRT.1

Richard Chen

5b

Apr 19, 2015

Rewrote TPRT.4 to test transport-proxy correctly. Much work on formatting issues in Generate, Transform, and Transport. Added SRCH.8

Richard Chen

6a

Oct 19, 2015

Added HVT.7, REG.10. Modified HVT.5, PRG.1, PRT.1, PRV.1, PRV.3, PRV.5, PRV.6, REG.3, REG.9, SRCH.5, SRCH.8, TPRT.1, TPRT.3

Richard Chen

6b

Apr 29, 2016

Added PRP.1, PRT.2, PRT.3, PRV.7, TPRT.5

Richard Chen

7a

Oct 24, 2016

Added MOD.1, Changed DSV.1 to POR.1, added POR.2

Richard Chen

7b

Apr 17, 2017

Added HVT.8

Richard Chen

8a

Oct 23, 2017

Added harvest-search, search-analytics

Richard Chen

 

Contents

Change Log.................................................................................................................................................. ii

1 Introduction............................................................................................................................................. 1

1.1 Purpose............................................................................................................................................... 2

1.2 Scope................................................................................................................................................... 2

1.3 Document Revision.............................................................................................................................. 2

1.4 Test Approach..................................................................................................................................... 2

1.5 Applicable Documents......................................................................................................................... 3

2 Executive Summary.................................................................................................................................. 4

3 Test Procedures........................................................................................................................................ 6

3.1 Setup.................................................................................................................................................... 6

3.2 Testing of Bundle Processing.............................................................................................................. 8

3.3 Testing for Complete Coverage of PDS4 Level 5 Requirements....................................................... 21

4 Anomalies............................................................................................................................................. 100

5 Requirements Traceability.................................................................................................................. 102

6 Miscellaneous...................................................................................................................................... 105

6.1 Test Data......................................................................................................................................... 105

6.2 Test Environment............................................................................................................................ 105

6.3 Configuration Management............................................................................................................ 105

6.4 Acronyms........................................................................................................................................ 105


1 Introduction

 

For over twenty years, the Planetary Data System (PDS) has been NASA’s official data system for archiving and distribution of data from planetary exploration missions.  It has been a leader in defining data standards, working with missions and instrument teams, and developing data system technologies.  The PDS has been instrumental in changing the scientific culture by working with the planetary science community to publicly release and peer review the data it captures.  It has also been used as a model by other science data systems interested in establishing distributed scientific networks organized by independent discipline nodes at facilities that are doing leading-edge scientific research.

 

While PDS has been a leader in developing and exploiting new technologies and ideas, an increasing workload and substantial increases in the volume of delivered data are now threatening the system's ability to accomplish its primary missions of both archiving planetary science data and distributing it to working scientists. PDS identified these challenges in its Roadmap published in 2006.   In addition to these challenges, the ten year Roadmap outlined several goals including improving the PDS data standards, increasing user services by leveraging newer technologies and technical standards, and re-architecting PDS to ensure efficient operations of the system while supporting the increasing demands on PDS by both the data providers and end users.

 

In response to these challenges and goals, PDS has developed a plan for the next generation.  The vision, as defined by the PDS Management Council at its April 2008 meeting, includes:

 

·      Simplified, but rigorous, archiving standards that are consistent, easy to learn, and easy to use

·      Adaptable tools for designing archives, preparing data, and delivering the results efficiently to PDS

·      On-line services allowing users to access and transform data quickly from anywhere in the system

·      A highly reliable, scalable computing infrastructure that protects the integrity of data, links the nodes into an integrated data system, and provides the best service to both data providers and users

 

PDS previously maintained two separate documents:

·  the Integration and Test Plan

·  the Test Procedures and Report

Because the latter document grew to encompass most of the former, this document merges the two.

 

 

1.1 Purpose

 

This Test Document 1) defines specific tests that ensure that the new system and the new standards called “PDS4” comply with requirements and meet customers’ needs, and 2) reports results of the tests to verify and validate that the PDS4 system deployed for Build 8a is free of critical defects. This document describes the integration and test activities and contains test cases that demonstrate compliance to requirements. The test scenarios verify and validate the system components and data products in an integrated manner. A test traceability matrix in section 5 below traces these scenarios to the new PDS4 system design requirements, which in turn can be traced to high-level PDS requirements.

 

 

1.2 Scope

 

For PDS4 Build 8a, the following software will be deployed at the EN:

·      Ingest: Harvest 1.13.0, Harvest-pdap 1.7.0, Harvest-Search 0.1.0, Catalog 1.14.0

·      Model: LDDTool 8.0.0

·      Portal: Data Set View 2.10.0

·      Preparation: Design, Generate 0.13.0, Transform 1.6.0, Validate 1.12.0

·      Registry: Core 1.13.0, Service 1.13.0, UI 1.13.0

·      Report (Sawmill 8.5)

·      Search: Core 1.11.0, Service 1.11.0, Search-UI 1.11.0, Product-Search-UI 1.11.0, Analytics 1.11.0/

·      Security (OpenDS 2.2.0)

·      Storage-Service 1.2.0, Product-Service 1.2.0

·      Transport-Registry 1.5.0, Transport-OFSN 1.7.0, Transport-Proxy 1.6.0

 

The scope of this build is to support data providers and Discipline Nodes in developing and distributing PDS4 data products both for new missions and data migration.  Previous releases of PDS4 have been scoped to support the LADEE and MAVEN missions as early adopters as well as internal testing by PDS and the IPDA.  Future, incremental releases will target data users as PDS4 data is available within the PDS.  

 

 

1.3 Document Revision

 

Revisions of this document will be held in the PDS Engineering Node website through the use of its document history functionality.  Previous versions of this document can be accessed through the use of that tool.

 

 

1.4 Test Approach

 

The PDS4 build structure is organized such that the system can be tested and verified early on to ensure seamless transitions. The builds will ensure there is a coordinated testing and deployment of functionality coupled with upgrades of the data standards.

Build 8a Integration testing is the execution and management of tests by the Engineering Node to ensure that the release of Build 8a meets the intended functionality.  The process of verification testing includes the selection of verification items, integration, and regression testing. Any functionality that is added to the system is treated as a new verification item.

 

1.5 Applicable Documents

 

1.5.1 Controlling Documents

 

[1] Planetary Data System Strategic Roadmap 2006 - 2016, February 2006.

[2] Planetary Data System Level 1, 2 and 3 Requirements, March 26, 2010.

 

1.5.2 Referenced Documents

 

[3] PDS4 Project Plan, July 2013.

[4] PDS4 Operations Concept, September 2013.

[4] System Architecture Specification, September 2013.

[5] General System Requirements, September 2013.

[6] Software Requirements and Design, 2013.

[7] PDS4 Standards Documents, 2014.

 

 

2 Executive Summary

 

The testing documented herein substantiates that all tested tools and services meet Build 8a requirements as specified in their Software Requirements and Design documents.

 

# of tests performed

# of tests passed

# of tests failed

# of high priority anomalies

51

51

0

0

 

Build 8a confirms the closure of 2 JIRA issues reported in earlier testing: PDS-469 and PDS-496. It also confirms the closure of JIRA issues created elsewhere: PDS-495, PDS-501, PDS-505, PDS-506, PDS-511, PDS-515 Build 8a opens 6 JIRA issues, none major anomalies: PDS-518, PDS-519, PDS-520, PDS-521, PDS-522, PDS-523. PDS-521 and PDS-523 have already been closed. See the bottom of Section 4 below, Anomalies.

 

Section 3.2 has one sequence of four tests that represent the most likely operating scenario for PDS4 products: creation, validation, ingestion, search. Section 3.3 has one sequence of tests per software module. These tests cover level 4 and 5 requirements for completeness. The sequences:

·      CTLG.* tested the catalog tool and the storage service successfully.

·      GEN.* tested general functionality successfully. Testing of accessing a controlled registry was skipped, as no registries are run that way now or are availabe to test with. Test GEN.1 cleared PDS-515.

·      HVT.* tested the Ingest/Harvest tool successfully. HVT.6 previously opened PDS-469, a request for improvement, which was rejected. HVT.1 tested harvest-search for the first time.

·      MOD.1 tested LDDTool successfully and opened PDS-518, a request for improvement

·      POR.1 tested the Portal / Data Set View interface successfully, while POR.2 tested the service registry successfully.

·      PRG.1 tested the Preparation/Generate tool successfully and opened PDS-519, a request for improvement,  already resolved.

·      PRP.1 tested the tool extract-table successfully and cleared PDS-495.

·      PRT.* tested the Preparation/Transform tool. Request for improvement PDS-343 remains open. These tests closed PDS-506, PDS-505, and PDS-511 and opened minor bug PDS-523.

·      PRV.* tested the Preparation/Validation tool succesfully. PRV.1 previously opened PDS-496, a request for improvement, which was rejected. PRV.1 also opened PDS-520, a request for improvement, and PDS-521, a minor bug which has since been cleared. PRV.2 opened PDS-522, a request for improvement. PRV.7 cleared PDS-501.

·      REG.* tested the Registry successfully.

·      RPT.1 tested the Report service successfully.

·      SCMA.1 tested the PDS4 schema (v1.8.0.0) rather than software. These tests used the Validate tool and proceeded independently from the software build 8a.

·      SEC.1 tested the Security service successfully.

·      SRCH.* tested the Search service successfully. Minor issues PDS-258, PDS-319, PDS-357 remain open. SRCH.3 tested search-analytics for the first time.

·      TPRT.* tested the updated transport service successfully. PDS-322 remains open.

                                                                                                                                                   

Section 4 lists all issues and their info: status, JIRA tracking number, severity, relevant test case, and description.  Testing of Build 8a created 3 minor anomalies and 3 requests for improvement. Overall, 14 issues remain open: 5 minor anomalies, 9 requests for improvement.

 

Section 5 shows the traceability of the test cases to the level 5 requirements (and level 4 if no level 5 requirement applies). The table in this section shows that 5 test cases, each uniquely covering 1 level 5 requirement, have been skipped because appropriate PDS4 software has not yet been implemented.

 

3 Test Procedures

 

The following section defines the tests and their results.  All tests below have been run for build 8a (except those written for future builds) and will be run as necessary to re-test the system after software changes.

 

Section 3.2 below contains one sequence of tests that demonstrates how a bundle of products passes through the PDS4 software, especially the tools and services to support PDS4 data validation, registration, and search.

 

Section 3.3 contains tests that demonstrate the broader functionality of the PDS4 software.

 

3.1 Setup

The root URL for all software is https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0

The tests in section 3.2 require the installation of the following PDS4 software:

·      Harvest, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/ingest/harvest

·      Registry, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/registry

·      Search, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/search

·      Validate, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/preparation/validate

·      an XML editor, e.g. Oxygen. This can be skipped, though not recommended.

·      an XML-friendly web browser, e.g. firefox

The tests in Section 3.3 require the installation of the software above as well as:

·       Catalog, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/ingest/catalog

·       Extract-table, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/preparation/pds4-tools

·       Generate, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/preparation/generate

·       Model, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/model

·       Portal, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/portal

·       Storage, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/storage

·       Transform, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/preparation/transform

·       Transport, https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0/transport

·       curl, a command-line utility to access a URL, used here to manipulate a registry. The Registry Service Guide has more information. This is native to most versions of unix.

 

Please follow the installation instructions carefully. For more help, the file NOTES.txt, created during SETUP below, details one tester’s configuration experience.  Note that the tests are written for Unix, but running on other platforms requires simple changes.

 

In the tests in the rest of this document, replace

testDir

directory where input files are extracted

binDir

directory where the PDS4 software are installed

harvestpds3

 

If the registry is uncontrolled, do not replace. Else:

harvest –uusername –ppassword

Also add “‑k keystorePassword” depending on the registry configuration, especially if Harvest gives error “Keystore password must be specified”

curl

If the registry is uncontrolled, do not replace. Else:

curl ‑uusername:password –k

http://localhost:8080

Use the URL of the registry

 

The tests in Sections 3.2 and 3.3 require this:

Test Case ID

SETUP

Description

This is not a test. This sets up test data.

Test Steps

From https://pds-engineering.jpl.nasa.gov/content/build_8a_deliverables, download

PDS4test.build8a.tgz, then

·       mkdir testDir

·       cd testDir

·       tar xzf PDS4test.build8a.tgz

 

The registry is the central service. It can reside locally or remotely, controlled or uncontrolled. If remote, it need not be installed.  If local, testing is easier, but installation and configuration of it and of the required Apache Tomcat server can be difficult.

 

Many test sequences in this document assume a local, uncontrolled registry, which may get corrupted during testing. The following step resets the registry:

Test Case ID

RESETREGISTRY

Description

This is not a test. This wipes the database and the search indices clean.

dbDir is the directory for the database, set during the initialization of Tomcat.

Test Steps

$CATALINA_HOME/bin/shutdown.sh

rm binDir/search-service/../logs/*

rm $CATALINA_HOME/logs/*

rm -r binDir/search-service/pds/*data*

rm -r binDir/search-service/pds/solr-docs/*

rm -r binDir/search-service/pds/solr-docs_old/*

rm -f -r dbDir/registrypds*

cd binDir/registry-service

java -Djava.ext.dirs=lib/ org.apache.derby.tools.ij

  connect 'jdbc:derby:registrypds3;create=true;user=registry';

  run 'conf/derby-registry-schema.ddl';

  disconnect;

  connect 'jdbc:derby:registrypds4;create=true;user=registry';

  run 'conf/derby-registry-schema.ddl';

  exit;

mv registrypds3 registrypds4 dbDir/

rm derby.log

$CATALINA_HOME/bin/startup.sh

setenv REGISTRY_SERVICE http://localhost:8080/registry-pds3

cd binDir/registry-service/bin; ./registry-config  # wait for startup.sh

setenv REGISTRY_SERVICE http://localhost:8080/registry-pds4

cd binDir/registry-service/bin; ./registry-config

 

3.2 Testing of Bundle Processing

 

 

The AAFUNCTION sequence tests the PDS4 software’s ability to process a bundle of products.  All expected product types should be able to pass through the sequence.

 

Test Case ID

AAFUNCTION.1

Description

Create a PDS4 Product Label using a design tool based on PDS’s schema.

Requirements

PASS L5.PRP.DE.1: The tool shall initiate a design session as follows...

PASS L5.PRP.DE.2: The tool shall accept the following as input for specifying a schema file...

PASS L5.PRP.DE.3: The tool shall facilitate modification of a schema file as follows...

PASS L5.PRP.DE.4: The tool shall provide standard editing features as follows...

PASS L5.PRP.DE.5: The tool shall indicate when a schema is not valid.

PASS L5.PRP.DE.6: The tool shall generate an XML instance file from a schema.

PASS L5.PRP.DE.7: The tool shall export the schema for use outside the tool.

Success Criteria

Design tool produces a syntactically valid PDS Product Label else indicates where the label is invalid.

Test Steps

In general:

·       Consult the Data Providers’ Handbook (DPH),  Version 1.3.0, Appendix D.

Test Results

Creation of a label-template (xml) from the master-schema (xsd). 

Comments

Results met success criteria

Date of Testing

2017.10.16

Test Personnel

Richard Chen

 

Test Case ID

AAFUNCTION.2

Description

Validate PDS4 label

Requirements

PASS L5.PRP.VA.1: The tool shall accept the following as input for specifying the product(s) to be validated…

PASS L5.PRP.VA.2: The tool shall traverse a directory tree and validate products

PASS L5.PRP.VA.3: The tool shall validate aggregate products and all products referenced by such products.

PASS L5.PRP.VA.5: The tool shall verify that a product label is well-formed XML.

PASS L5.PRP.VA.6: The tool shall verify that a product label conforms to its associated schema file(s).

PASS L5.PRP.VA.7: The tool shall accept the following as input for specifying the associated schema file(s)…

PASS L5.PRP.VA.9: The tool shall indicate the schema(s) utilized during validation.

Success Criteria

Validation tool validates a file or all eligible products in a directory tree, indicates the schemas utilized during the validation, and ensures that a product label is well formed XML and conforms to its schemas. Also validate for content as well as syntax.

Test Steps

1.     cd testDir

2.     validate -m 1800 -t bundle_geo_ra -e "*.xml"

Test Results

Oct 16, 2017 5:58:47 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Directory' for location file:/Users/rchen/Desktop/test/bundle_geo_ra/

Oct 16, 2017 5:58:47 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundle_geo_ra/'

Oct 16, 2017 5:59:31 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundle_geo_ra/'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-17T00:58:46Z

   Core Schemas                  [PDS4_PDS_1800.xsd]

   Core Schematrons              [PDS4_PDS_1800.sch]

   Model Version                 1800

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_geo_ra/]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml]

   Force Mode                    off

Validation Details:

  PASS: file: testDir/bundle_geo_ra/bundle_1.xml

  PASS: file: testDir/bundle_geo_ra/context/context_collection_1.xml

  PASS: file: testDir/bundle_geo_ra/context/PDS4_host_PHX_1.0.xml

  PASS: file: testDir/bundle_geo_ra/context/PDS4_inst_RA__PHX.xml

...

  PASS: file: testDir/bundle_geo_ra/xml_schema/collection.xml

  PASS: file: testDir/bundle_geo_ra/xml_schema/PDS4_PDS_1800.xml

Summary:

  173 of 173 file(s) processed, 0 skipped

  173 of 173 file(s) passed validation

End of Report

Comments

Results met success criteria

Date of Testing

2017.10.16

Test Personnel

Richard Chen

 

Test Case ID

AAFUNCTION.3

Description

Harvest PDS4 labels. Harvest provides a command-line interface, accepts a configuration file, determines candidates for registration, captures metadata, and submits metadata to the Registry Service. Registry accepts the artifacts, assigns global unique IDs to the products. Registry relates artifacts via (LID-based) association

Requirements

PASS L5.HVT.1: The tool shall accept a configuration file specifying policy for tool behavior.

PASS L5.HVT.2: The tool shall provide a command-line interface for execution.

PASS L5.HVT.4: The tool shall recursively traverse the specified directory or directories…

PASS L5.HVT.5: The tool shall determine candidate products for registration through a combination of the following…

PASS L5.HVT.6: The tool shall capture metadata for a candidate product specified by the product type.

PASS L5.HVT.7: The tool shall submit the associated metadata for a candidate product to the [Registry].

PASS L5.HVT.8: The tool shall track each product registration.

PASS L5.REG.1: The service shall accept artifact registrations.

PASS L5.REG.2: The service shall provide a means for relating artifact registrations.

PASS L5.REG.4: The service shall accept metadata for a registered artifact in a defined format.

PASS L5.REG.6: The service shall assign a global unique identifier to a registered artifact.

PASS L5.REG.8: The service shall store metadata for a registered artifact in an underlying metadata store.

PASS L5.SEC.1: The service shall authenticate a user given identifying credentials for that user.

Success Criteria

Harvest tool, based on criteria given in a user-edited configuration file, executed from the command line, discovers all matching artifacts and for each submits metadata to the Registry service. Tools to view the registry show the metadata of the matching artifacts, with appropriate metadata, including the guid, which is assigned by the Registry.  Tools to view the registry show the associations.

Test Steps

1.     cd testDir

In the following commands, specify the absolute path, which must begin with harvest-policy-master.xml’s  policy/accessUrls/accessUrl/offset

2.     harvest testDir/contextPDS4onlyPHX -c harvest-policy-master.xml -l h1.out -e "*.xml"

3.     grep -v "SUCCESS\|INFO" h1.out | uniq

4.     harvest testDir/bundle_geo_ra -c harvest-policy-master.xml -l h2.out -e "*.xml"

5.     grep -v "SUCCESS\|INFO" h2.out | uniq

The following set up search-core in the next test case.

6.     (assuming binDir/harvest/bin/harvest points to registry.pds4, this has no data, as it defaults to registry-pds3) http://localhost:8080/registry-ui/

7.     set Registry Service(s) to registry-pds4

8.     Click tab “Packages”. Select the two “Harvest-Package_*”, set Status to “Approved”, click “Update Status”.

 






























 


















Step 3: Without grep, the output file is very large

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Tue, Oct 17 2017 at 11:30:39 AM

Target(s)                   [/Users/rchen/Desktop/test/contextPDS4onlyPHX]

File Inclusions             [*.xml]

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest-Package_20171017113039

Registration Package GUID   urn:uuid:24e506b8-92d5-45e1-8012-610cec865c5a

Summary:

26 of 26 file(s) processed, 0 other file(s) skipped

0 error(s), 0 warning(s)

26 of 26 products registered.

31 of 31 ancillary products registered.

Product Types Registered:

5 Product_Collection

31 Product_File_Repository

20 Product_Context

1 Product_Bundle

31 of 31 associations registered.

End of Log

Step 5:

PDS Harvest Tool Log                   

Version                     Version 1.13.0

Time                        Tue, Oct 17 2017 at 01:05:25 PM

Target(s)                   [/Users/rchen/Desktop/test/bundle_geo_ra]

File Inclusions             [*.xml]

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest-Package_20171017130525

Registration Package GUID   urn:uuid:0259258b-9c18-4a41-a851-f07f2c3124f4

WARNING:   [/Users/rchen/Desktop/test/bundle_geo_ra/context/PDS4_host_PHX_1.0.xml] Product already exists: urn:nasa:pds:context:instrument_host:spacecraft.phx::1.0

WARNING:   [/Users/rchen/Desktop/test/bundle_geo_ra/context/PDS4_inst_RA__PHX.xml] Product already exists: urn:nasa:pds:context:instrument:ra.phx::1.0

WARNING:   [/Users/rchen/Desktop/test/bundle_geo_ra/context/PDS4_mission_PHOENIX_1.0.xml] Product already exists: urn:nasa:pds:context:investigation:mission.phoenix::1.0

WARNING:   [/Users/rchen/Desktop/test/bundle_geo_ra/context/PDS4_target_MARS_1.0.xml] Product already exists: urn:nasa:pds:context:target:planet.mars::1.0

Summary:

173 of 173 file(s) processed, 0 other file(s) skipped

0 error(s), 4 warning(s)

169 of 173 products registered.

338 of 338 ancillary products registered.

Product Types Registered:

1 Product_XML_Schema

5 Product_Collection

120 Product_Observational

4 Product_Document

38 Product_Browse

338 Product_File_Repository

1 Product_Bundle

338 of 338 associations registered.

End of Log

Step 7: ../Screen%20Shot%202017-04-13%20at%2015.04.21.png

Step 8: ../Screen%20Shot%202017-10-17%20at%2013.10.27.png

Comments

Results met success criteria.

 

The warnings in step 5 are expected, as step 3 registered the context products already.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

AAFUNCTION.4

Description

Search for PDS4 data at the product level and the context level.

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results…

PASS L5.SCH.5: The service shall provide the capability to retrieve metadata associated with registered artifacts for the purpose of generating search indexes.
PASS L5.SCH.6: The service shall support searching by accepting criteria as a sequence of open text keywords.

PASS L5.SCH.8: The service shall support narrowing of additional index results based on specifications of terms and/or values on indexes.

PASS L5.SCH.10: The service shall provide results to a search as a sequence of matching URIs to resources that contain search desiderata.

PASS L5.SCH.11: The service shall annotate each URI of a result with metadata describing the URI.

PASS L5.SCH.12: The service shall support configuration on the kinds of indexes maintained on indexed data

Success Criteria

After configuration (e.g. regenerating search indices), Search returns the data harvested in the previous step.

Test Steps

 

 

                     

Build the search index

1.     In a browser, http://localhost:8080/product-search-ui

2.     Beneath “Data Search” in the middle of the page, type “phoenix”

3.     In a browser, http://localhost:8080/search-ui

4.     Beneath “Data Search” in the middle of the page, type “phoenix”

5.     search-core -H binDir/search-service/pds -p binDir/search-core/conf/defaults/pds/pds4/core.properties

6.     Repeat step 2

7.     Repeat step 4

Test Results

Step 2:../Screen%20Shot%202017-04-13%20at%2015.13.58.png

Step 4: ../Screen%20Shot%202017-04-13%20at%2015.13.06.png

Step 5:

Processing config: attribute.xml

Processing config: bundle.xml

Processing config: class.xml

Processing config: collection.xml

Processing config: context.xml

Processing config: document.xml

Processing config: observational.xml

PDS Search Core Run Log

Version                     Version 1.11.0

Time                        Tue, Oct 17 2017 at 01:49:01 PM

Severity Level              INFO

Search Home                 /PDS4tools/search-service/pds

Search Service URL          http://localhost:8080/search-service

Search Core Properties      /PDS4tools/search-core/conf/defaults/pds/pds4/core.properties

SUCCESS:   Completed extraction:  attribute.xml

INFO:   Completed: urn:nasa:pds:phx_ra

INFO:   Completed: urn:nasa:pds:context

SUCCESS:   Completed extraction:  bundle.xml

SUCCESS:   Completed extraction:  class.xml

INFO:   Completed: urn:nasa:pds:context:agency

INFO:   Completed: urn:nasa:pds:context:target

INFO:   Completed: urn:nasa:pds:context:investigation

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived

INFO:   Completed: urn:nasa:pds:phx_ra:data_test

INFO:   Completed: urn:nasa:pds:context:instrument_host

INFO:   Completed: urn:nasa:pds:context:instrument

INFO:   Completed: urn:nasa:pds:system_bundle:xml_schema

INFO:   Completed: urn:nasa:pds:phx_ra:document

INFO:   Completed: urn:nasa:pds:phx_ra:context

SUCCESS:   Completed extraction:  collection.xml

INFO:   Completed: urn:nasa:pds:context:instrument:meca_wcl.phx

INFO:   Completed: urn:nasa:pds:context:instrument:meca_tecp.phx

INFO:   Completed: urn:nasa:pds:context:node:node.psa

INFO:   Completed: urn:nasa:pds:context:node:node.pds-mgt

INFO:   Completed: urn:nasa:pds:context:target:satellite.phobos

INFO:   Completed: urn:nasa:pds:context:instrument:ra.phx

INFO:   Completed: urn:nasa:pds:context:instrument:ssi.phx

INFO:   Completed: urn:nasa:pds:context:instrument:meca_elec.phx

INFO:   Completed: urn:nasa:pds:context:target:satellite.deimos

INFO:   Completed: urn:nasa:pds:context:investigation:mission.phoenix

INFO:   Completed: urn:nasa:pds:context:instrument:rac.phx

INFO:   Completed: urn:nasa:pds:context:instrument:tt.phx

INFO:   Completed: urn:nasa:pds:context:instrument:ase.phx

INFO:   Completed: urn:nasa:pds:context:target:planet.mars

INFO:   Completed: urn:nasa:pds:context:instrument:lidar.phx

INFO:   Completed: urn:nasa:pds:context:instrument:tega.phx

INFO:   Completed: urn:nasa:pds:context:instrument:om.phx

INFO:   Completed: urn:nasa:pds:context:instrument:met.phx

INFO:   Completed: urn:nasa:pds:context:instrument:meca_afm.phx

INFO:   Completed: urn:nasa:pds:context:instrument_host:spacecraft.phx

SUCCESS:   Completed extraction:  context.xml

INFO:   Completed: urn:nasa:pds:phx_ra:document:readme

INFO:   Completed: urn:nasa:pds:phx_ra:document:ra_dataset

INFO:   Completed: urn:nasa:pds:phx_ra:document:ra_instrument

INFO:   Completed: urn:nasa:pds:phx_ra:document:activity_table_desc

SUCCESS:   Completed extraction:  document.xml

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol127a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol146a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol128a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol011

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol143

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol045a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol140

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol019a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol147a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol072

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol058a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol099b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol134a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol095

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol025

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol051

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol087a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol133b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol117

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol033c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol114

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol087b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol134c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol079b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol042c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol060

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol033b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol129

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol147b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol014

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol009

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_icy_soil_dig2

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol062

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_icy_soil_dig1

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol134d

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol146b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol098b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol130

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol032a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol148b

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_trench_wall_failure_bottom

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_trench_wall_failure_tip

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol057a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol090

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol093

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol073

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol149a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol071a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol126b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol085

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol126a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol071b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol031

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol066

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol125a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol116b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol058c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol141

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol089a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol042a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol115

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_duricrust

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol074b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol099a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol132b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol024

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol098a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol034

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol042b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol101a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol068a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol074a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol069

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol057b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol068b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol148c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol145b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol148a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol133c

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol083

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol049

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol042e

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol045b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol064

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol075

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol101b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol132a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol125b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol133d

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol077

INFO:   Completed: urnProcessing config: service.xml

:nasa:pds:phx_ra:data_derived:sol058b

INFO:   Completed: urn:nasa:pds:phx_ra:document:activity

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol134b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol128b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol148d

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol116a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol149b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol076

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol019b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol020a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol079a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol136

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol022b

INFO:   Completed: urn:nasa:pds:phx_ra:data_test:pit_test_scraping

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol105b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol013

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol033a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol067

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol133a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol127b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol020b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol007

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol022a

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol088

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol110

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol032b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol042d

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol089b

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol006

INFO:   Completed: urn:nasa:pds:phx_ra:data_derived:sol145a

SUCCESS:   Completed extraction:  observational.xml

SUCCESS:   Completed extraction:  service.xml

SUCCESS:   Completed extracting data from data source.

INFO:   Running Solr Indexer to create new solr documents for indexing ...

SUCCESS:   Completed transforming data into Solr Lucene index

INFO:   Running Solr Post to Post Data To Search Service ...

INFO:   Cleaning Search Service Index

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.0

INFO:   Posting: /PDS4tools/search-service/pds/index/search-tools.xml

INFO:   Optimizing Search Service index.

SUCCESS:   Completed posting data to the Search Service

Summary:

==================================================

The Numbers:

-- Number of Warnings: 0

-- Number of Errors: 0

-- Bad Registries: []

-- Number of Missing Associations: 0

-- Association Cache Hits: 0

-- Number of products: 156

==================================================

Processing Time:

-- collection.xml: 0 h, 0 m, 4 s

-- service.xml: 0 h, 0 m, 0 s

-- context.xml: 0 h, 0 m, 6 s

-- observational.xml: 0 h, 0 m, 53 s

-- attribute.xml: 0 h, 0 m, 0 s

-- class.xml: 0 h, 0 m, 0 s

-- document.xml: 0 h, 0 m, 1 s

-- bundle.xml: 0 h, 0 m, 3 s

==================================================

Total Processing Time: 0 h, 1 m, 9 s

End of Log

Step 6: ../Screen%20Shot%202017-04-13%20at%2015.19.47.png

Step 7:../Screen%20Shot%202017-10-17%20at%2014.06.07.png

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

3.3 Testing for Complete Coverage of PDS4 Level 5 Requirements

 

The following test cases test all Build 8a functions, including those not covered above. These tests ensure complete verification and validation of Build 8a level 5 requirements.

 

Test Case ID

CTLG.1

Description

Compare PDS3 data against other PDS3 data, both file to file and directory to directory

Requirements

PASS 4.2.4: PDS will provide a mechanism to upgrade products or data sets which do not meet usability requirements (e.g., data sets from old missions)

Success Criteria

Tool reports differences.

Test Steps

1.     cd testDir

2.     catalog -mcompare testCatalog/CORPWS_0164 testCatalog/CORPWS_0180

Compare two directories differing in only one file.

3.     catalog -c testCatalog/config

Test Results

Step 2:

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 02:49:16 PM

Parameters:

Mode                   compare

Target(s)               

  Source = file: testDir/testCatalog/CORPWS_0164/

  Target = file: testDir/testCatalog/CORPWS_0180/

Directory Recursion    true

Severity Level         WARNING

Compare Details:

  SAME: file: testDir/testCatalog/CORPWS_0180/INSTHOST.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/KEYDS.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/LRFULLDS.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/MISSION.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/PERSON.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/PROJREF.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/RAWDS.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/REF.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/RPWSINST.CAT

  DIFFERENT: file: testDir/testCatalog/CORPWS_0180/VOLDESC.CAT

      line 9: Element "VOLUME_NAME" has different value than source.

      Source: line 9 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      10c10

      < VOLUME 164: CASSINI RADIO AND PLASMA WAVE STANDARD PRODUCTS

      ----

      > VOLUME 180: CASSINI RADIO AND PLASMA WAVE STANDARD PRODUCTS

      line 16: Element "DESCRIPTION" has different value than source.

      Source: line 16 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      19c19

      < (SCET) dates 2011-05-27 (147) through 2011-06-12 (163).

      ----

      > (SCET) dates 2012-11-26 (331) through 2012-12-23 (358).

      line 23: Element "DATA_SET_ID" has different value than source.

      Source: line 23 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      23c23

      < CO-V/E/J/S/SS-RPWS-4-SUMM-KEY60S-V1.0

      ----

      > {CO-V/E/J/S/SS-RPWS-4-SUMM-KEY60S-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-ALL-V1.0, CO-V/E/J/S/SS-RPWS-3-RDR-LRFULL-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-WBRFULL-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-WFRFULL-V1.0}

      line 11: Element "VOLUME_ID" has different value than source.

      Source: line 11 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      11c11

      < CORPWS_0164

      ----

      > CORPWS_0180

      line 15: Element "PUBLICATION_DATE" has different value than source.

      Source: line 15 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      15c15

      < 2011-12-22

      ----

      > 2013-03-28

      line 74: Element "DATA_SET_ID" has different value than source.

      Source: line 70 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      70c74

      < CO-V/E/J/S/SS-RPWS-4-SUMM-KEY60S-V1.0

      ----

      > {CO-V/E/J/S/SS-RPWS-4-SUMM-KEY60S-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-ALL-V1.0, CO-V/E/J/S/SS-RPWS-3-RDR-LRFULL-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-WBRFULL-V1.0, CO-V/E/J/S/SS-RPWS-2-REFDR-WFRFULL-V1.0}

      line 86: Pointer "REFERENCE_CATALOG" has different value than source.

      Source: line 72 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      72c86

      < {REF.CAT}

      ----

      > {REF.CAT, PROJREF.CAT}

      line 80: Pointer "DATA_SET_CATALOG" has different value than source.

      Source: line 71 of file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

      71c80

      < KEYDS.CAT

      ----

      > {KEYDS.CAT, RAWDS.CAT, LRFULLDS.CAT, WBFULLDS.CAT, WFFULLDS.CAT}

  SAME: file: testDir/testCatalog/CORPWS_0180/WBFULLDS.CAT

  SAME: file: testDir/testCatalog/CORPWS_0180/WFFULLDS.CAT

Summary:

  12 of 12 validated, 0 skipped

  11 of 12 passed

End of Report

Step 3:                  

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 02:54:07 PM

Parameters:

Mode                   compare

Target(s)               

  Source = file:testDir/testCatalog/CORPWS_0164/RAWDS.CAT

  Target = file:testDir/testCatalog/CORPWSrawX.CAT

Directory Recursion    true

Severity Level         WARNING

Compare Details:

  DIFFERENT: file:testDir/testCatalog/CORPWSrawX.CAT

      line 56: Element "DATA_SET_DESC" has different value than source.

      Source: line 56 of file:testDir/testCatalog/CORPWS_0164/RAWDS.CAT

      126,130c126,130

      < kernels can be used with the SPICE toolkit to convert from the

      < spacecraft frame to virtually any frame which may be of use in

      < analyzing these data. However, for many purposes, the wave

      < amplitudes are extremely useful and may be entirely adequate with no

      < coordinate transformations at all.

      ----

      > kernels EXTRAWORDHERE can be used with the SPICE toolkit to convert

      > from the spacecraft frame to virtually any frame which may be of

      > use in analyzing these data. However, for many purposes, the

      > wave amplitudes are extremely useful and may be entirely adequate

      > with no coordinate transformations at all.

Summary:

  1 of 1 validated, 0 skipped

  0 of 1 passed

End of Report

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

CTLG.2

Description

Validate a submission of PDS3 data.

Requirements

PASS 4.2.4: PDS will provide a mechanism to upgrade products or data sets which do not meet usability requirements (e.g., data sets from old missions)

Success Criteria

Tool flags invalid language constructs.

Test Steps

1.     catalog -mvalidate -d testCatalog/pdsdd.full -t testCatalog/LRO_diviner

Test Results

Step 1:

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 02:59:30 PM

Parameters:

Mode                   validate

Target                 file:testDir/testCatalog/LRO_diviner/

Directory Recursion    true

Dictionary File(s)     [testCatalog/pdsdd.full]

Severity Level         WARNING

Aliasing Enabled       false

Validation Details:

  PASS: file:testDir/testCatalog/LRO_diviner/dsmap.cat

  PASS: file:testDir/testCatalog/LRO_diviner/dsmap_polar.cat

  PASS: file:testDir/testCatalog/LRO_diviner/gdrds.cat

  PASS: file:testDir/testCatalog/LRO_diviner/inst.cat

  PASS: file:testDir/testCatalog/LRO_diviner/insthost.cat

  PASS: file:testDir/testCatalog/LRO_diviner/mission.cat

  PASS: file:testDir/testCatalog/LRO_diviner/person.cat

  PASS: file:testDir/testCatalog/LRO_diviner/prpds.cat

  PASS: file:testDir/testCatalog/LRO_diviner/rdrds.cat

  PASS: file:testDir/testCatalog/LRO_diviner/ref.cat

  FAIL: file:testDir/testCatalog/LRO_diviner/voldesc.cat

Begin Fragment: file: testDir/testCatalog/LRO_diviner/REF.CAT

      WARNING  The label fragment, "REF.CAT", should not contain a PDS_VERSION_ID.

    End Fragment: file: testDir/testCatalog/LRO_diviner/REF.CAT

    Begin Fragment: file: testDir/testCatalog/LRO_diviner/INSTHOST.CAT

      WARNING  The label fragment, "INSTHOST.CAT", should not contain a PDS_VERSION_ID.

    End Fragment: file: testDir/testCatalog/LRO_diviner/INSTHOST.CAT

    Begin Fragment: file: testDir/testCatalog/LRO_diviner/PERSON.CAT

      WARNING  The label fragment, "PERSON.CAT", should not contain a PDS_VERSION_ID.

    End Fragment: file: testDir/testCatalog/LRO_diviner/PERSON.CAT

    Begin Fragment: file: testDir/testCatalog/LRO_diviner/GDRDS.CAT

      WARNING  The label fragment, "GDRDS.CAT", should not contain a PDS_VERSION_ID.

    End Fragment: file: testDir/testCatalog/LRO_diviner/GDRDS.CAT

    Begin Fragment: file: testDir/testCatalog/LRO_diviner/INST.CAT

      WARNING  The label fragment, "INST.CAT", should not contain a PDS_VERSION_ID.

    End Fragment: file: testDir/testCatalog/LRO_diviner/INST.CAT

    Begin Fragment: file: testDir/testCatalog/LRO_diviner/MISSION.CAT

      WARNING  The label fragment, "MISSION.CAT", should not contain a PDS_VERSION_ID.

      ERROR  line 40: Found a reference, "SAYLOR2006A", which is not defined in a REFERENCE_KEY_ID within the label.

      ERROR  line 40: Found a reference, "SAYLOR2006B", which is not defined in a REFERENCE_KEY_ID within the label.

    End Fragment: file: testDir/testCatalog/LRO_diviner/MISSION.CAT

Referential Integrity Details:

  PASS: Instrument Host

    Parent File(s): [insthost.cat]

    Begin checking children

    End checking children

  FAIL: Reference

    Parent File(s): [ref.cat]

    Begin checking children

      dsmap.cat: "REFERENCE_KEY_ID = SEIDELMANNETAL2002" is not found in a(n) "ref.cat".

      dsmap.cat: "REFERENCE_KEY_ID = SNYDER1987" is not found in a(n) "ref.cat".

      dsmap_polar.cat: "REFERENCE_KEY_ID = SEIDELMANNETAL2002" is not found in a(n) "ref.cat".

      dsmap_polar.cat: "REFERENCE_KEY_ID = SNYDER1987" is not found in a(n) "ref.cat".

    End checking children

  PASS: Mission

    Parent File(s): [mission.cat]

    Begin checking children

    End checking children

  PASS: Data Set

    Parent File(s): [gdrds.cat, prpds.cat, rdrds.cat]

    Begin checking children

    End checking children

  PASS: Instrument

    Parent File(s): [inst.cat]

    Begin checking children

    End checking children

  PASS: Personnel

    Parent File(s): [person.cat]

    Begin checking children

    End checking children

New Standard Values:

Referential Integrity Summary:

  6 of 6 referential integrity check(s) made, 0 skipped

  5 of 6 passed

New Standard Values Summary:

  0 new standard value(s) found

Summary:

  11 of 11 validated, 0 skipped

  10 of 11 passed

End of Report

Comments

Results met success criteria. All warnings and errors are either expected or carried over from PDS3 and do not affect meeting success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

CTLG.3

Description

Ingest valid PDS3 files into the PDS4 registry service

Requirements

PASS 4.2.4: PDS will provide a mechanism to upgrade products or data sets which do not meet usability requirements (e.g., data sets from old missions)

Success Criteria

Catalog successfully ingests the PDS3 files into the registry else indicates where the input is invalid.  Tools to view the registry show the metadata of the PDS3 files

Test Steps

The catalog ingest requires access to a storage service. In its own terminal window:

·       storage-service stop   # warning message if storage-service was not running

·       cd binDir/storage-service

·       rm -r archive/ catalog/ logs/ run/

·       storage-service start

·       storage-client --operation --ingestProduct --productName CatalogTest --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///PDS4tools/storage-service/test/test.txt.met" --refs file:///PDS4tools/storage-service/test/test.txt

·       storage-query --sql -query "SELECT * FROM CatalogFile WHERE CAS.ProductName == 'CatalogTest'" -outputFormat '$CAS.ProductId' | storage-delete --read

Also clean database as described in RESETREGISTRY in Section 3.1

 

Nominal case, including multiple REF.CATs:

1.     catalog testCatalog/CORPWS_0180 -m ingest -s http://localhost:9000 -T http://localhost:8080/product -v 1 -r c1.out

2.     In a browser: http://localhost:8080/registry-ui to see registrations. Then click on the row where Object Type is Product_Instrument_PDS3

3.     Test the product service: copy a productID from c1.out, e.g. the one for the inst:
curl -X GET -o x.cat -v 'http://localhost:8080/product/data?productID=productID'

4.     diff x.cat testCatalog/CORPWS_0180/RPWSINST.CAT

5.     Test the product service’s other function: curl -X GET -o v.zip -v "http://localhost:8080/product/dataset?typeID=urn:pds:CatalogFile"

6.     Expand and compare any one of the catalog files:
unzip v.zip CORPWS_0180:VOLDESC.CAT.zip ; unzip C*zip

7.     diff VOLDESC.CAT testCatalog/CORPWS_0180/VOLDESC.CAT

When >1 voldesc lists the same catalog files (e.g. mission.cat), do not re-register them.

8.     catalog testCatalog/CORPWS_0164 -m ingest -s http://localhost:9000 -T http://localhost:9999 -r c3.out

9.     See registration of only voldesc: http://localhost:8080/registry-ui

Give good error messages if file listed in voldesc is missing

10.  catalog testCatalog/MPC_review -m ingest -s http://localhost:9000 -T http://localhost:8080/product

Nicely ignore dsmap catalog files.

11.  catalog testCatalog/LRO_diviner -m ingest -s http://localhost:9000 -T http://localhost:8080/product -r c6.out

Quit and give a nice error message when the mode is not specified.

12.  catalog testCatalog/CORPWS_0180

Test Results

Step 1: Ignore brutal command line warnings. c1.out:

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 03:50:20 PM

Parameters:

Mode                   ingest

Target                 file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/

Directory Recursion    true

Severity Level         INFO

Report File            c1.out

Ingest Details:

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/INSTHOST.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:instrument_host:spacecraft.co::1.0

      INFO:  Product GUID - urn:uuid:73fced91-ba28-40db-9ff1-258c9139580c

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e085241-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:94bbf21b-3c55-4b81-9854-9287d3f166ef

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/KEYDS.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:data_set:data_set.co-v-e-j-s-ss-rpws-4-summ-key60s-v1.0::1.0

      INFO:  Product GUID - urn:uuid:7321deb2-7886-4fa6-9eda-383399ec0989

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e280f42-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:57620d4f-0ac0-40f6-bac2-9c2067fdb275

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/LRFULLDS.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:data_set:data_set.co-v-e-j-s-ss-rpws-3-rdr-lrfull-v1.0::1.0

      INFO:  Product GUID - urn:uuid:1302233f-0a75-43a3-b94e-b74fe0f337e6

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e37c6b3-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:37a480fd-9bc9-4318-9354-a39347c0ee85

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/MISSION.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens::1.0

      INFO:  Product GUID - urn:uuid:20e0af82-8075-4167-b337-f53937785fef

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e4693c4-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:5ae19ae3-a761-40fc-afd3-f95032d76d32

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/PERSON.CAT

      WARNING:  This file is not required to ingest into the registry.

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e525395-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:0d444eaf-ea1a-45f3-8b2a-6ec146254684

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/PROJREF.CAT

      WARNING:  This file is not required to ingest into the registry.

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e5c65b6-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:970d0759-64c2-4e10-98f7-c85dcbeb1eba

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/RAWDS.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:data_set:data_set.co-v-e-j-s-ss-rpws-2-refdr-all-v1.0::1.0

      INFO:  Product GUID - urn:uuid:a73e4659-8b25-4b7c-8b82-bf5fbdab70d8

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e6936f7-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:769840d8-db81-4777-b288-a19d771e791d

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/REF.CAT

      WARNING:  This file is not required to ingest into the registry.

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e734918-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:b984d5ee-65bc-43e3-a511-9a45e7051784

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/RPWSINST.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:instrument:instrument.rpws.co::1.0

      INFO:  Product GUID - urn:uuid:035c5b67-eb09-43cc-b160-7a414ceac09d

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e7dd069-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:cada3294-d316-4835-815c-2c9ade3d022f

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/VOLDESC.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:volume:volume.corpws_0180__usa_nasa_pds_corpws_0xxx::1.0

      INFO:  Product GUID - urn:uuid:3788072c-b914-490d-b54c-5eb99958c038

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e89692a-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:d8a57971-ed30-4e93-a41c-2b79dd1085f8

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/WBFULLDS.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:data_set:data_set.co-v-e-j-s-ss-rpws-2-refdr-wbrfull-v1.0::1.0

      INFO:  Product GUID - urn:uuid:1cf4a36a-c9f8-4e2f-b8df-30d8e095d775

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8e95500b-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:56539d60-6eb5-422e-bc6d-20915873318b

  PASS: file:/Users/rchen/Desktop/test/testCatalog/CORPWS_0180/WFFULLDS.CAT

      INFO:  Successfully registered a product. LIDVID - urn:nasa:pds:context_pds3:data_set:data_set.co-v-e-j-s-ss-rpws-2-refdr-wfrfull-v1.0::1.0

      INFO:  Product GUID - urn:uuid:f79298ae-eb5e-470a-9af1-74ae699c74e8

      INFO:  Successfully delivered a catalog file to the storage service. productID - 8ea2485c-b38d-11e7-8432-d70942fac8dd

      INFO:  Successfully ingested a file object. GUID - urn:uuid:eca0eda8-b6ec-496a-bd52-13e8074e8f43

End of Report

Step 2: ../Screen%20Shot%202017-10-17%20at%2015.53.11.png

 

Step 3:                                                                                                                                 

Note: Unnecessary use of -X or --request, GET is already inferred.

*   Trying ::1...

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to localhost (::1) port 8080 (#0)

> GET /product/data?productID=8e7dd069-b38d-11e7-8432-d70942fac8dd HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

< HTTP/1.1 200

< Content-Disposition: attachment; filename="RPWSINST.CAT"

< Content-Type: application/vnd.ms-pki.seccat

< Content-Length: 14202

< Date: Tue, 17 Oct 2017 23:04:23 GMT

{ [8012 bytes data]

100 14202  100 14202    0     0  81225      0 --:--:-- --:--:-- --:--:-- 82569

* Connection #0 to host localhost left intact

Step 4 shows no differences

Step 5:

Note: Unnecessary use of -X or --request, GET is already inferred.

*   Trying ::1...

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to localhost (::1) port 8080 (#0)

> GET /product/dataset?typeID=urn:pds:CatalogFile HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

>

< HTTP/1.1 200

< Content-Disposition: attachment; filename="CatalogFile.zip"

< Content-Length: 101529

< Date: Tue, 17 Oct 2017 23:06:28 GMT

<

{ [16245 bytes data]

100   99k  100   99k    0     0   517k      0 --:--:-- --:--:-- --:--:--  524k

* Connection #0 to host localhost left intact

 Step 6:

Archive:  v.zip

  inflating: CORPWS_0180:VOLDESC.CAT.zip 

Archive:  CORPWS_0180:VOLDESC.CAT.zip

  inflating: VOLDESC.CAT            

  inflating: CORPWS_0180:VOLDESC.CAT.met 

Step 7 shows no differences

Step 8: c3.out:                                                                                                                    

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 04:08:49 PM

Parameters:

Mode                   ingest

Target                 file: testDir/testCatalog/CORPWS_0164/

Directory Recursion    true

Severity Level         WARNING

Report File            c3.out

Ingest Details:

  PASS: file: testDir/testCatalog/CORPWS_0164/INSTHOST.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/KEYDS.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/LRFULLDS.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/MISSION.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/PERSON.CAT

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/CORPWS_0164/PROJREF.CAT

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/CORPWS_0164/RAWDS.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/REF.CAT

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/CORPWS_0164/RPWSINST.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/VOLDESC.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/WBFULLDS.CAT

  PASS: file: testDir/testCatalog/CORPWS_0164/WFFULLDS.CAT

Summary:

  4 of 12 file(s) ingested, 0 skipped

  Number of successful file object ingestion: 4

  Number of successful storage service ingestion: 4

  Number of successful registry ingestion: 1

  Name of the registry package: Catalog-Package_CORPWS_0164_20171017160850

End of Report

Step 9: 5 extra records: the voldesc file, the CORPWS_0164 product, and the three files not required to ingest into the registry.../Screen%20Shot%202017-04-14%20at%2012.46.51.png

Step 10:  exit nicely upon discovering a catalog file is missing:

WARNING: testDir/testCatalog/MPC_review/asteroid.cat is missing.

WARNING: testDir/testCatalog/MPC_review/comet.cat is missing.

WARNING: testDir/testCatalog/MPC_review/satellite.cat is missing.

Oct 17, 2017 4:11:28 PM org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer setFileManagerUrl

INFO: Remote Data Transfer to: [http://localhost:9000] enabled

Error: Failed to get a product by name. productName = SBN_0178:asteroid.cat

Error: Catalog file (asteroid.cat) is missing in the archive volume and can't get it from the storage service.

Step 11:  similar to step 1’s output. Used to die upon hitting dsmap.cat. c6.out::

PDS Catalog Ingest Tool Report

Configuration:

Version                Version 1.14.0

Date                   Tue, Oct 17 2017 at 04:11:41 PM

Parameters:

Mode                   ingest

Target                 file: testDir/testCatalog/LRO_diviner/

Directory Recursion    true

Severity Level         WARNING

Report File            c6.out

Ingest Details:

  PASS: file: testDir/testCatalog/LRO_diviner/dsmap.cat

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/LRO_diviner/dsmap_polar.cat

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/LRO_diviner/gdrds.cat

  PASS: file: testDir/testCatalog/LRO_diviner/inst.cat

  PASS: file: testDir/testCatalog/LRO_diviner/insthost.cat

  PASS: file: testDir/testCatalog/LRO_diviner/mission.cat

  PASS: file: testDir/testCatalog/LRO_diviner/person.cat

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/LRO_diviner/prpds.cat

  PASS: file: testDir/testCatalog/LRO_diviner/rdrds.cat

  PASS: file: testDir/testCatalog/LRO_diviner/ref.cat

      WARNING:  This file is not required to ingest into the registry.

  PASS: file: testDir/testCatalog/LRO_diviner/voldesc.cat

Summary:

  11 of 11 file(s) ingested, 0 skipped

  Number of successful file object ingestion: 11

  Number of successful storage service ingestion: 11

  Number of successful registry ingestion: 7

  Name of the registry package: Catalog-Package_LRODLR_1001_20171017161141

End of Report

Step 12:

Exception: No mode specified. 'm' flag must be specified.

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

 

Test Case ID

GEN.1

Description

Run components distributed over multiple machines on any PDS-supported platforms.

Requirements

PASS L5.GEN.1: The system shall operate in a distributed environment.

PASS L5.GEN.2: Components shall run on any PDS-supported platform.

Success Criteria

Services produce identical results independent of machine and platform.

Test Steps

This is similar to PRV.1 but with a URL instead of a local file

1.     validate -f http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml

Test Results

Step 1: Output is similar to PRV.1’s

Oct 17, 2017 4:15:47 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml

Oct 17, 2017 4:15:47 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml'

Oct 17, 2017 4:16:00 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-17T23:15:47Z

 Parameters:

   Targets                       [http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  PASS: http://pds-geosciences.wustl.edu/messenger/urn-nasa-pds-izenberg_pdart14_meap/data_eetable/ele_evt_8hr_orbit_2014-2015.xml

Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

Comments

Results met success criteria.

 

This also verifies PDS-515, validatig against schemas on PDS machines

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

GEN.2 *not run for build 8a

Description

Authorize only authenticated users access to a controlled capacity.

Requirements

PASS L5.GEN.10: Components shall control access to interfaces that alter content.

PASS L5.SEC.1: The service shall authenticate a user given identifying credentials for that user.

PASS L5.SEC.3: The service shall authorize an authenticated user for access to a controlled capability.

Success Criteria

Registration fails when given invalid credentials.

Test Steps

1.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.1b.xml http://pds-gamma.jpl.nasa.gov/services/registry-pds3

Test Results

Step 1:

* About to connect() to pds-gamma.jpl.nasa.gov port 80 (#0)

*   Trying 128.149.124.6...

* connected

* Connected to pds-gamma.jpl.nasa.gov (128.149.124.6) port 80 (#0)

> POST /services/registry-pds3 HTTP/1.1

> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8z zlib/1.2.5

> Host: pds-gamma.jpl.nasa.gov

> Accept: */*

> Content-type:application/xml

> Content-Length: 629

* upload completely sent off: 629 out of 629 bytes

< HTTP/1.1 403 Forbidden

< Date: Sat, 18 Oct 2014 19:19:49 GMT

< Content-Length: 224

< Connection: close

< Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<html><head>

<title>403 Forbidden</title>

</head><body>

<h1>Forbidden</h1>

<p>You don't have permission to access /services/registry-pds3

on this server.</p>

</body></html>

* Closing connection #0

Comments

Results met success criteria.

Date of Testing

 

Test Personnel

Richard Chen

 

Test Case ID

GEN.4 *not ready for build 8a. This is reserved for future testing

Description

Services provide an interface to enable monitoring of health.

Requirements

SKIP L5.GEN.8: Services shall provide an interface to enable monitoring of the service's health.

Success Criteria

The interface correctly reflects the services’ health.

Test Steps

 

Test Results

 

Comments

 

Date of Testing

 

Test Personnel

 

 

Test Case ID

GEN.7

Description

Document components’ capabilities, dependencies, interfaces, installation, operation

Requirements

PASS L5.GEN.11: Components shall provide documentation detailing their capabilities, dependencies, interfaces, installation and operation

Success Criteria

Documentation of components show capabilities, dependencies, interfaces, installation and operation.

Test Steps

Examine such documentation, currently accessible from https://pds-engineering.jpl.nasa.gov/development/pds4/8.0.0

Test Results

Documents were available and examined.

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

 

Test Case ID

HVT.1

Description

Provide a command-line interface, accept a configuration file, recursively traverse directories, determine candidates for registration, capture metadata, submit metadata to the Registry Service, track each artifact registration.

Requirements

PASS L5.HVT.1: The tool shall accept a configuration file specifying policy for tool behavior.

PASS L5.HVT.2: The tool shall provide a command-line interface for execution.

PASS L5.HVT.4: The tool shall recursively traverse the specified directory or directories…

PASS L5.HVT.5: The tool shall determine candidate products for registration through a combination of the following…

PASS L5.HVT.6: The tool shall capture metadata for a candidate product specified by the product type.

PASS L5.HVT.7: The tool shall submit the associated metadata for a candidate product to the [Registry].

PASS L5.HVT.8: The tool shall track each product registration.

PASS L5.GEN.7: Tools shall generate a report detailing results from a single execution of the tool.

Success Criteria

Harvest tool, executed from the command line, discovers all matching artifacts and for each submits metadata, based on both identifying and artifact-specific metadata, to the Registry service. A matching artifact resides in the directory tree of the target directory or is listed in a manifest file in the target directory, and it matches the criteria given in the user-edited configuration file and if previously registered, has been since modified. Tools to view the registry should show the matching artifacts, with appropriate metadata, and not show the non-matching artifacts.

Test Steps

The harvesting in this test is redundant to tests AAFUNCTION.*. The deleting (not a core function) is different, so if desired:

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     cd testDir; harvest testDir/contextPDS4onlyPHX -c harvest-policy-master.xml -l h.out -e "*.xml"

3.     grep -v "SUCCESS\|INFO" h.out | uniq

4.     At http://localhost:8080/registry-ui, select Registry Service registry-pds4

5.     Click “Packages”, select “Harvest-Package_...”. If verification desired, click on that line, and compare the GUID with Step 2’s output.

6.     harvest-search -e "*.xml" -c harvest-policy-master.xml testDir/contextPDS4onlyPHX | grep url: | wc

7.     Click “Delete”

8.     Click “Products”

Test Results

Step 3:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Tue, Oct 17 2017 at 04:19:32 PM

Target(s)                   [/Users/rchen/Desktop/test/contextPDS4onlyPHX]

File Inclusions             [*.xml]

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20171017161932

Registration Package GUID   urn:uuid:a1bb8b7a-53f7-4e34-8ecc-d436bbf9250d

Summary:

26 of 26 file(s) processed, 0 other file(s) skipped

0 error(s), 0 warning(s)

26 of 26 products registered.

31 of 31 ancillary products registered.

Product Types Registered:

5 Product_Collection

31 Product_File_Repository

20 Product_Context

1 Product_Bundle

31 of 31 associations registered.

End of Log

Step 4:../Screen%20Shot%202017-04-19%20at%2009.42.25.png

Step 5:../Screen%20Shot%202017-10-17%20at%2016.22.14.png

Step 5: # of access URLs that harvest-search creates == # of products + # of ancillary products that harvest registers.

57     342   12062

Step 7: “There is no data to display”

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

HVT.2

Description

Execute from a scheduler, accept a configuration file, recursively traverse directories, determine candidates for registration, capture metadata, submit metadata to the Registry Service.

Requirements

PASS L5.HVT.1: The tool shall accept a configuration file specifying policy for tool behavior.

PASS L5.HVT.3: The tool shall execute from a scheduler…

PASS L5.HVT.4: The tool shall recursively traverse the specified directory or directories…

PASS L5.HVT.5: The tool shall determine candidate products for registration through a combination of the following…

PASS L5.HVT.6: The tool shall capture metadata for a candidate product specified by the product type.

PASS L5.HVT.7: The tool shall submit the associated metadata for a candidate product to the [Registry].

PASS L5.HVT.8: The tool shall track each product registration.

Success Criteria

Harvest tool, executed from a scheduler, discovers all matching artifacts and for each submits metadata, based on both identifying and artifact-specific metadata, to the Registry service. A matching artifact resides in the directory tree of the target directory or is listed in a manifest file in the target directory, and it matches the criteria given in the user-edited configuration file and if previously registered, has been since modified. Tools to view the registry should show the matching artifacts, with appropriate metadata, and not show the non-matching artifacts.

Test Steps

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     cd testDir/; mkdir x; mv contextPDS4onlyPHX/* x

3.     harvest testDir/contextPDS4onlyPHX -c harvest-policy-master.xml -l log.txt -P 9001 -w 120

4.     In browser, http://localhost:8080/registry-ui/ shows no data

In a different terminal window                   

5.     harvest-ctrl --url http://localhost:9001/xmlrpc --operation --isRunning

6.     cd testDir; mv x/* contextPDS4onlyPHX; rmdir x

7.     In browser, after at most 120 seconds, note changing Num Records. http://localhost:8080/registry-ui/

After Num Records stops increasing                                                   

8.     harvest-ctrl --url http://localhost:9001/xmlrpc --operation --stop

9.     grep "products registered" log.txt

10.  On http://localhost:8080/registry-ui/#Packages, delete Harvest-Package_*; if verification desired, check GUID against Registration Package GUID in log.txt

11.  Check Num Records is original value: http://localhost:8080/registry-ui

Test Results

Step 5: Yes

Step 7: Same Num Records as in HVT.1, assuming both started with 0 records../Screen%20Shot%202016-10-13%20at%2002.53.03.png

Step 8:

Crawl Daemon: [http://localhost:9001/xmlrpc]: shutdown successful

Step 9:

26 of 26 new products registered.

31 of 31 new ancillary products registered.

Step 11: “There is no data to display” (same as HVT.1’s step 6)

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

HVT.3

Description

Harvest a large number of files.

Requirements

PASS. No specific functional requirement. This is a performance test case.

Success Criteria

Harvest completes in a reasonable amount of time per product.

Test Steps

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     cd testDir

3.     modify binDir/harvest/bin/harvest to use registry-pds3 instead of -pds4

4.     date; harvest testDir/contextPDS3 -c harvest-policy-master.xml -l h.out -e "*.xml"; date

5.     Check for harvested files. http://localhost:8080/registry-ui/

This lets SRCH.3 show Archive information

6.     harvest testDir/testHarv/archivePDS3 -c harvest-policy-master.xml -l h.out; grep -v "SUCCESS\|INFO" h.out

Test Results

Step 4: The time to harvest this bundle dropped from 214 minutes in build 5b.

Tue Oct 17 23:43:15 PDT 2017

Tue Oct 17 23:52:55 PDT 2017

The bottom of h.out has (the skipped .xml file is the harvest config file):

Summary:

27302 of 27302 file(s) processed, 0 other file(s) skipped

0 error(s), 8 warning(s)

27294 of 27302 products registered.

27307 of 27307 ancillary products registered.

Product Types Registered:

7161 Product_Context

2184 Product_Attribute_Definition

626 Product_Instrument_PDS3

5974 Product_Volume_PDS3

1541 Product_Subscription_PDS3

13 Product_Collection

4254 Product_Target_PDS3

199 Product_Instrument_Host_PDS3

2293 Product_Data_Set_PDS3

27307 Product_File_Repository

1 Product_Bundle

71 Product_Mission_PDS3

89 Product_Class_Definition

2888 Product_Volume_Set_PDS3

27307 of 27307 associations registered.

End of Log

Step 5:../Screen%20Shot%202017-10-17%20at%2021.45.51.png

Step 6:

PDS Harvest Tool Log

 

Version                     Version 1.13.0

Time                        Wed, Oct 18 2017 at 12:30:02 AM

Target(s)                   [/Users/rchen/Desktop/test/testHarv/archivePDS3]

File Inclusions             [*.xml]

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest-Package_20171018003002

Registration Package GUID   urn:uuid:1ee279db-33bb-4e0c-a2a8-afa8f5c2fe10

Summary:

1 of 1 file(s) processed, 0 other file(s) skipped

0 error(s), 0 warning(s)

1 of 1 products registered.

1 of 1 ancillary products registered.

Product Types Registered:

1 Product_Context

1 Product_File_Repository

1 of 1 associations registered.

End of Log

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

HVT.5

Description

Harvest skips candidate products not matching configuration file. Harvest also checks for previous registrations and skips those.

Requirements

PASS L5.HVT.1: The tool shall accept a configuration file specifying policy for tool behavior.

PASS L5.HVT.5: The tool shall determine candidate products for registration through a combination of the following…

PASS L5.HVT.8: The tool shall track each product registration.

Success Criteria

Tools to view the registry should show only matching products and not the others.

Test Steps

Run harvest with config file that does not accept Product_Document

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     harvest testDir/bundle_geo_ra -c harvestPolicyNoDoc.xml -l h.out -e "*.xml"

3.     grep -v "SUCCESS\|INFO" h.out | uniq

4.     In browser, http://localhost:8080/registry-ui, to see many products, then set “Object Type” to “Product_Document” and hit “Refresh”

Repeat to show nothing more gets registered.

5.     harvest testDir/bundle_geo_ra -c harvestPolicyNoDoc.xml -l h.out -e "*.xml"

6.     http://localhost:8080/registry-ui

Run harvest with config file that accepts Product_Document

7.     harvest testDir/bundle_geo_ra –c harvest-policy-master.xml –l h.out -e "*.xml"

8.     grep -v "SUCCESS\|INFO" h.out | uniq

9.     http://localhost:8080/registry-ui. Set “Object Type” to “Product_Document”. Hit “Refresh”

10.  Hit the “Packages” tab. Note the packages are unsorted.

11.  Click column headers such as “Name” to see sorting on that field

12.  Select both Harvest-*, hit “Delete”

13.  Hit the “Products” tab

Test Results

Step 3: Note the SKIPs of Product_Document:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Wed, Oct 18 2017 at 05:55:40 PM

Target(s)                   [/Users/rchen/Desktop/test/bundle_geo_ra]

File Inclusions             [*.xml]

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20170414153958

Registration Package GUID   urn:uuid:578cff4e-8bb4-44f4-9503-5d1a575ddfd6

SKIP:   [/Users/rchen/Desktop/test/bundle_geo_ra/document/activity_table_desc.xml] 'Product_Document' is not an object type found in the policy file.

SKIP:   [/Users/rchen/Desktop/test/bundle_geo_ra/document/ra_dataset.xml] 'Product_Document' is not an object type found in the policy file.

SKIP:   [/Users/rchen/Desktop/test/bundle_geo_ra/document/ra_instrument.xml] 'Product_Document' is not an object type found in the policy file.

SKIP:   [/Users/rchen/Desktop/test/bundle_geo_ra/document/readme.xml] 'Product_Document' is not an object type found in the policy file.

Summary:

169 of 169 file(s) processed, 4 other file(s) skipped

0 error(s), 0 warning(s)

169 of 169 products registered.

334 of 334 ancillary products registered.

Product Types Registered:

1 Product_XML_Schema

5 Product_Collection

120 Product_Observational

38 Product_Browse

334 Product_File_Repository

4 Product_Context

1 Product_Bundle

334 of 334 associations registered.

End of Log

Step 4:  Note that Total Records = 503 before filtering../Screen%20Shot%202016-10-13%20at%2011.58.29.png

Step 6: Note that still only 503 products are registered

Step 8: Many WARNINGs for “Product already exists”, and at the bottom:

Summary:

173 of 173 file(s) processed, 0 other file(s) skipped

0 error(s), 169 warning(s)

4 of 173 products registered.

8 of 8 ancillary products registered.

Product Types Registered:

4 Product_Document

8 Product_File_Repository

8 of 8 associations registered.

End of Log

Step 9: Note that 1) 515 products are registered and 2) there are 4 Product Documents

../Screen%20Shot%202016-10-14%20at%2012.43.56.png

Step 10:../Screen%20Shot%202016-10-14%20at%2015.52.28.png

Step 11 shows sorting within the Packages tab.../Screen%20Shot%202016-10-14%20at%2015.53.39.png

Step 12:../Screen%20Shot%202016-10-14%20at%2015.54.27.png

Step 13: “There is no data to display”

Comments

Results met success criteria.

 

PDS-166, created during testing of build 3b, requests an improvement: check if secondary members match primary members.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

HVT.6

Description

Harvest PDS3 products, not just catalog files.

Requirements

PASS 4.2.4: PDS will provide a mechanism to upgrade products or data sets which do not meet usability requirements (e.g., data sets from old missions)

Success Criteria

The registry shows the harvested PDS3 products.

Test Steps

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     In browser, http://localhost:8080/registry-ui/ shows no data

3.     cd testDir

Using the version of harvest that ingests into registry-pds3, get unlabeled files.

4.     harvest -c testHarv/harvestDawnNondata.xml

5.     http://localhost:8080/registry-ui/ shows such files.

Using that version of harvest again, harvest the 4 .LBLs and the files they point to.

6.     harvest -c testHarv/harvestDawnData.xml

7.     http://localhost:8080/registry-ui/ includes labelled files

Test Results

Step 2:Macintosh HD:Users:rchen:Desktop:Screen Shot 2014-10-18 at 11.44.18 AM.png

Step 4:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Wed, Oct 18 2017 at 11:35:28 PM

Target(s)                   [testDir/testHarv/DAWNGRAND1B]

Target Type                 PDS3 Files Only

File Exclusions             [*.cat, *.CAT]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest Package Example PDS3 Unlabeled Files

Registration Package GUID   urn:uuid:45d82aa0-9d5d-483b-abc4-ae928ffcc4b5

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/AAREADME.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/AAREADME.TXT] Successfully registered product: urn:nasa:pds:nondata:aareadme.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/AAREADME.TXT] Product has the following GUID: urn:uuid:7b77c577-3d92-4caf-b73e-19d1917b4150

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/ERRATA.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/ERRATA.TXT] Successfully registered product: urn:nasa:pds:nondata:errata.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/ERRATA.TXT] Product has the following GUID: urn:uuid:d18e0bf9-a115-45f9-bf18-82abb8b9c333

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/MD5_CHECKSUM.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/MD5_CHECKSUM.TXT] Successfully registered product: urn:nasa:pds:nondata:md5_checksum.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/MD5_CHECKSUM.TXT] Product has the following GUID: urn:uuid:bb03e109-7ae0-423e-9d94-9bf3399483ef

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.TAB] Begin processing.

SKIP:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.TAB] An associated label file exists '/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDXINFO.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDXINFO.TXT] Successfully registered product: urn:nasa:pds:nondata:index:indxinfo.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDXINFO.TXT] Product has the following GUID: urn:uuid:03d16a6b-a50c-4478-8c2d-84a6618241be

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/EXTRINFO.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/EXTRINFO.TXT] Successfully registered product: urn:nasa:pds:nondata:extras:extrinfo.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/EXTRINFO.TXT] Product has the following GUID: urn:uuid:b4b3b001-ece0-4b3c-9edb-79f329ef1201

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/GRaND_Acronyms_130711.pdf] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/GRaND_Acronyms_130711.pdf] Successfully registered product: urn:nasa:pds:nondata:extras:grand_acronyms_130711.pdf::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/GRaND_Acronyms_130711.pdf] Product has the following GUID: urn:uuid:f2628622-76a2-421e-9be9-4a3673d0f815

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/Extras_Guide_130711.pdf] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/Extras_Guide_130711.pdf] Successfully registered product: urn:nasa:pds:nondata:extras:software:extras_guide_130711.pdf::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/Extras_Guide_130711.pdf] Product has the following GUID: urn:uuid:cadb8400-40a0-41f4-aa66-3c9faf635cf3

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/plot_grand_L2.pro] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/plot_grand_L2.pro] Successfully registered product: urn:nasa:pds:nondata:extras:software:plot_grand_l2.pro::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/EXTRAS/SOFTWARE/plot_grand_L2.pro] Product has the following GUID: urn:uuid:97672f57-3344-4576-8882-3bf633db0d79

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG1.JPG] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG1.JPG] Successfully registered product: urn:nasa:pds:nondata:document:sis:dawn_grand_sis_fig1.jpg::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG1.JPG] Product has the following GUID: urn:uuid:78d12adb-5596-46f7-8bfb-4d2adff8245e

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG2.JPG] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG2.JPG] Successfully registered product: urn:nasa:pds:nondata:document:sis:dawn_grand_sis_fig2.jpg::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_FIG2.JPG] Product has the following GUID: urn:uuid:ecccb30a-b771-4e95-838a-e585a05003c9

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.HTM] Begin processing.

SKIP:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.HTM] An associated label file exists '/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.PDF] Begin processing.

SKIP:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.PDF] An associated label file exists '/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.TAB] Begin processing.

SKIP:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.TAB] An associated label file exists '/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.TAB] Begin processing.

SKIP:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.TAB] An associated label file exists '/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/CATALOG/CATINFO.TXT] Begin processing.

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/CATALOG/CATINFO.TXT] Successfully registered product: urn:nasa:pds:nondata:catalog:catinfo.txt::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/CATALOG/CATINFO.TXT] Product has the following GUID: urn:uuid:b4dc08f6-4b3c-406f-b6f9-85b42a774101

Summary:

11 of 11 file(s) processed, 5 other file(s) skipped

0 error(s), 0 warning(s)

0 of 0 products registered.

11 of 11 ancillary products registered.

Product Types Registered:

11 Product_File_Repository

0 of 0 associations registered.

End of Log

Step 5:../Screen%20Shot%202017-10-18%20at%2023.41.24.png

Step 6:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Wed, Oct 18 2017 at 11:42:06 PM

Target(s)                   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B]

Target Type                 PDS3

File Inclusions             [*.LBL, *.lbl]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest Package Example PDS3 Labeled Files

Registration Package GUID   urn:uuid:d31309e8-33a4-41a2-aa08-fbd234251bdd

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Begin processing.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Creating logical identifier.

WARNING:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] DATA_SET_ID not found.

WARNING:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] INSTRUMENT_ID not found.

WARNING:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] PRODUCT_ID not found.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Created the following logical identifier: urn:nasa:pds:data:index

WARNING:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] DATA_SET_ID not found.

WARNING:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] PRODUCT_ID not found.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Created title:

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Successfully registered product: urn:nasa:pds:data:index::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Product has the following GUID: urn:uuid:4b822483-b610-4244-b213-b307bdaafbba

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Capturing file object metadata for INDEX.LBL

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Capturing file object metadata for INDEX.TAB

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Successfully registered product: urn:nasa:pds:data:index:INDEX.LBL::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Product has the following GUID: urn:uuid:29b08aef-91d3-4b21-8372-0f9f27d5e07f

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Successfully registered product: urn:nasa:pds:data:index:INDEX.TAB::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Product has the following GUID: urn:uuid:5953eb7a-cc4c-4dcb-ad48-f1a454b7add0

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Successfully registered association to 'urn:uuid:29b08aef-91d3-4b21-8372-0f9f27d5e07f'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Association has the following GUID: urn:uuid:30badd9c-20de-49ad-8d39-10deaae7553a

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Successfully registered association to 'urn:uuid:5953eb7a-cc4c-4dcb-ad48-f1a454b7add0'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/INDEX/INDEX.LBL] Association has the following GUID: urn:uuid:0996c8f6-0328-4ce6-9f10-b4933e6c2214

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Begin processing.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Creating logical identifier.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Created the following logical identifier: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Created title: DAWN-M-GRAND-2-EDR-MARS-COUNTS-V1.0 DAWN_GRAND_SIS

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:95a8675b-178e-44f7-a328-4bf8a754013a

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Capturing file object metadata for DAWN_GRAND_SIS_R2_4.LBL

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Capturing file object metadata for DAWN_GRAND_SIS_R2_4.PDF

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Capturing file object metadata for DAWN_GRAND_SIS_R2_4.HTM

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Capturing file object metadata for DAWN_GRAND_SIS_FIG1.JPG

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Capturing file object metadata for DAWN_GRAND_SIS_FIG2.JPG

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis:DAWN_GRAND_SIS_R2_4.LBL::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:4ff6d887-171b-4f0f-95c9-299667f0d71f

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis:DAWN_GRAND_SIS_R2_4.PDF::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:5b574eae-3312-428c-b9e2-4242f3975e1c

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis:DAWN_GRAND_SIS_R2_4.HTM::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:fd24b467-edb2-4d38-aabe-35aa104fff0b

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis:DAWN_GRAND_SIS_FIG1.JPG::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:38b05f7b-5c90-45cf-a3b6-a0ca95222621

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered product: urn:nasa:pds:data:dawn-m-grand-2-edr-mars-counts-v1.0:grand:dawn_grand_sis:document:sis:DAWN_GRAND_SIS_FIG2.JPG::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Product has the following GUID: urn:uuid:3d34bfba-9294-4a96-92be-1e1160993afb

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered association to 'urn:uuid:4ff6d887-171b-4f0f-95c9-299667f0d71f'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Association has the following GUID: urn:uuid:2f3a6471-c8a3-4c47-bb84-87f536c5b291

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered association to 'urn:uuid:5b574eae-3312-428c-b9e2-4242f3975e1c'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Association has the following GUID: urn:uuid:8c278d52-f476-473f-bb4d-8cb6856afa81

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered association to 'urn:uuid:fd24b467-edb2-4d38-aabe-35aa104fff0b'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Association has the following GUID: urn:uuid:29aed47b-bf06-424d-9143-0473f3bf2037

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered association to 'urn:uuid:38b05f7b-5c90-45cf-a3b6-a0ca95222621'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Association has the following GUID: urn:uuid:178cc83c-4c54-434d-ac18-847b4ac3b417

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Successfully registered association to 'urn:uuid:3d34bfba-9294-4a96-92be-1e1160993afb'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DOCUMENT/SIS/DAWN_GRAND_SIS_R2_4.LBL] Association has the following GUID: urn:uuid:d30b21a0-efaf-456b-aa7a-22393514593d

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Begin processing.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Creating logical identifier.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Created the following logical identifier: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-ctl-bgoc:data

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Created title: DAWN-A-GRAND-3-RDR-CERES-COUNTS-V1.0 GRD-L1B-151023-151216_160816-CTL-BGOC

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-ctl-bgoc:data::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Product has the following GUID: urn:uuid:c0e1dbe4-d634-4c57-a685-5e70a0a0cfce

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Capturing file object metadata for GRD-L1B-151023-151216_160816-CTL-BGOC.LBL

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Capturing file object metadata for GRD-L1B-151023-151216_160816-CTL-BGOC.TAB

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-ctl-bgoc:data:GRD-L1B-151023-151216_160816-CTL-BGOC.LBL::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Product has the following GUID: urn:uuid:5d31e6ca-dda8-498d-afcf-782d5e55e740

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-ctl-bgoc:data:GRD-L1B-151023-151216_160816-CTL-BGOC.TAB::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Product has the following GUID: urn:uuid:538ccede-4710-4e9b-be24-46cecd52cef2

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Successfully registered association to 'urn:uuid:5d31e6ca-dda8-498d-afcf-782d5e55e740'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Association has the following GUID: urn:uuid:1230b546-37f8-4d0e-988c-0a72597aa7d8

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Successfully registered association to 'urn:uuid:538ccede-4710-4e9b-be24-46cecd52cef2'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-CTL-BGOC.LBL] Association has the following GUID: urn:uuid:4cda7d79-ed46-40fd-9c0e-fa81d3bb2b9d

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Begin processing.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Creating logical identifier.

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Created the following logical identifier: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-epg:data

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Created title: DAWN-A-GRAND-3-RDR-CERES-COUNTS-V1.0 GRD-L1B-151023-151216_160816-EPG

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-epg:data::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Product has the following GUID: urn:uuid:2f4c480d-b47c-466b-951e-fb88d15de8e5

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Capturing file object metadata for GRD-L1B-151023-151216_160816-EPG.LBL

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Capturing file object metadata for GRD-L1B-151023-151216_160816-EPG.TAB

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-epg:data:GRD-L1B-151023-151216_160816-EPG.LBL::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Product has the following GUID: urn:uuid:38cd5705-d1a1-4010-962a-e0bea4bf3d27

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Successfully registered product: urn:nasa:pds:data:dawn-a-grand-3-rdr-ceres-counts-v1.0:grand:grd-l1b-151023-151216_160816-epg:data:GRD-L1B-151023-151216_160816-EPG.TAB::1.0

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Product has the following GUID: urn:uuid:5987abca-033a-4e31-b9d4-79fcf3b28ad2

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Successfully registered association to 'urn:uuid:38cd5705-d1a1-4010-962a-e0bea4bf3d27'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Association has the following GUID: urn:uuid:be57b906-dce4-472b-913e-e1e2a29186d7

SUCCESS:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Successfully registered association to 'urn:uuid:5987abca-033a-4e31-b9d4-79fcf3b28ad2'

INFO:   [/Users/rchen/Desktop/test/testHarv/DAWNGRAND1B/DATA/GRD-L1B-151023-151216_160816-EPG.LBL] Association has the following GUID: urn:uuid:824d7ea0-e437-4171-b0a5-41f60c8ada6c

Summary:

4 of 4 file(s) processed, 0 other file(s) skipped

0 error(s), 5 warning(s)

4 of 4 products registered.

11 of 11 ancillary products registered.

Product Types Registered:

11 Product_File_Repository

4 Product_Proxy_PDS3

11 of 11 associations registered.

End of Log

Step 7:../Screen%20Shot%202017-10-18%20at%2023.47.12.png

Comments

Results met success criteria.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

HVT.7

Description

Test bug fixes to harvest.

Requirements

PASS No specific functional requirement.

Success Criteria

The registry shows the harvested products.

Test Steps

Test URL in harvest config file with ‘?’, LID ending in .xml, <title> with many spaces

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     cd testDir

3.     diff harvest-policy-master.xml harvest388.xml ; echo END_DIFF_11111 ; diff bundleLID/data_test/scraping/pit_test_scraping.xml bundle_geo_ra/data_test/scraping/pit_test_scraping.xml; echo END_DIFF_22222 ; diff bundleLID/data_test/scraping/pit_test_scraping_pic2.xml bundle_geo_ra/data_test/scraping/pit_test_scraping_pic2.xml

4.     harvest testDir/bundleLID/data_test/scraping -c harvest388.xml -e "*.xml" | grep -v INFO:

5.     On http://localhost:8080/registry-ui/, notice the automatically generated LIDs for label files have “_xml”

6.     Select any row, probably the different one, to see Access URL.

7.     Another view, more slot-based: http://localhost:8080/registry-pds4/extrinsics?lid=urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1 shows the two Product_Browse with the same LID

8.     On http://localhost:8080/registry-ui/, select the two with LID urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1 and set one to Approved and one to Deprecated.

9.     mkdir x; cp bundleLID/data_test/scraping/pit_test_scraping_pic1.* x; mv  x/pit_test_scraping_pic1.xml x/pit_test_scraping_pic44.xml
Then edit the .xml file to change version_id from 1.0 to 4.4

10.  harvest testDir/x -c harvest388.xml -e "*.xml"

11.  On http://localhost:8080/registry-ui/, see that third entry with LID urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1 has Status “Submitted”.

12.  rm –r x/

13.  Click “Associations”, Note the few with non-null “Source LID”. Click “Packages”. Delete both “Harvest-Package_…”

14.  Click “Associations”. The ones for this harvest should be gone.

For removing schemes and classifications via deleting their package

15.  Clean database as described in RESETREGISTRY in Section 3.1

16.  cd testDir

17.  On http://localhost:8080/registry-ui/, “Associations”, “Schemes”, “Packages”

18.  Delete all 4 packages

19.  Click “Associations” and “Schemes” again

Test Results

Step 3:

27c27

<       <baseUrl>http://localhost:8080</baseUrl>

---

>       <baseUrl>http://testWeirdChars.com/download?x=y&amp;a=b</baseUrl>

END_DIFF_11111

10c10

<         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test.xml</logical_identifier>

---

>         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test_scraping</logical_identifier>

10c10

<         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test.xml</logical_identifier>

---

>         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test_scraping</logical_identifier>

END_DIFF_22222

10,16c10,12

<         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1</logical_identifier>

<         <version_id>1.2</version_id>

<         <title>add uselesss       pit_test_scraping_pic2   

<    and arbitray     

< spaces

<

< </title>

---

>         <logical_identifier>urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic2</logical_identifier>

>         <version_id>1.0</version_id>

>         <title>pit_test_scraping_pic2</title>

 Step 4: harvest rejects LIDs ending in “.xml”.

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Thu, Oct 19 2017 at 01:54:31 PM

Target(s)                   [testDir/bundleLID/data_test/scraping]

File Inclusions             [*.xml]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20170420223351

Registration Package GUID   urn:uuid:0b3a91e0-68de-4c7b-a310-dad64a881e2f

ERROR:   [testDir/bundleLID/data_test/scraping/pit_test_scraping.xml] Lid cannot end in '.xml': urn:nasa:pds:phx_ra:data_test:pit_test.xml

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1::1.0

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic1_xml::1.0

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic1.jpg::1.0

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml] Successfully registered association to 'urn:uuid:84a13dc2-c2ca-46dd-97e6-b8f1f231cc10'

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml] Successfully registered association to 'urn:uuid:cc1a5c28-7819-492e-b1bb-81581a331001'

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1::1.2

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic2_xml::1.2

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic2.jpg::1.2

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml] Successfully registered association to 'urn:uuid:b23fe26c-605c-4ab1-bd21-13d2688c078a'

SUCCESS:   [testDir/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml] Successfully registered association to 'urn:uuid:c488e62f-1a66-4ec9-af0f-be355f67d7d4'

Summary:

3 of 3 file(s) processed, 0 other file(s) skipped

1 error(s), 0 warning(s)

2 of 2 products registered.

4 of 4 ancillary products registered.

Product Types Registered:

2 Product_Browse

4 Product_File_Repository

4 of 4 associations registered.

End of Log

Step 5: harvest (registry?) converts automatically generated products with LIDs ending in “*.xml” to “_xml”.  Field “Name” handles extra spaces.

../../../../../../Desktop/Screen%20Shot%202017-04-20%20at%2

Step 6: URLs with question marks allowed.

../../../../../../Desktop/Screen%20Shot%202017-04-20%20at%2

Step 7 shows numFound returns correctly.

../../../../../../Desktop/Screen%20Shot%202017-04-20%20at%2

Step 8:../../../../../../Desktop/Screen%20Shot%202017-04-20%20at%2

Step 10:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Thu, Oct 19 2017 at 02:25:26 PM

Target(s)                   [/Users/rchen/Desktop/test/x]

File Inclusions             [*.xml]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20171019142526

Registration Package GUID   urn:uuid:0b81781c-3c64-41c2-ada1-904a97b0a18f

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Begin processing.

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] line 26: Mapping reference type 'browse_to_data' to 'data_ref'.

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Setting LID-based association, 'urn:nasa:pds:phx_ra:document:RA_dataset', under slot name 'data_ref'.

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Created access url: http://testWeirdChars.com/download?x=y&a=b/x/pit_test_scraping_pic44.xml

SUCCESS:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1::4.4

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Product has the following GUID: urn:uuid:be9c4b61-e87a-4f31-94d2-1028bcb2a2da

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Capturing file information for pit_test_scraping_pic44.xml

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Capturing file object metadata for pit_test_scraping_pic1.jpg

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] line 30: Setting file type for the file object 'pit_test_scraping_pic1.jpg' to 'Browse'

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Created access url: http://testWeirdChars.com/download?x=y&a=b/x/pit_test_scraping_pic44.xml

SUCCESS:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic44_xml::4.4

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Product has the following GUID: urn:uuid:f79e3f71-e889-42e4-83e6-309c0d762ad3

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Created access url: http://testWeirdChars.com/download?x=y&a=b/x/pit_test_scraping_pic1.jpg

SUCCESS:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Successfully registered product: urn:nasa:pds:phx_ra:data_test:pit_test_scraping_pic1:pit_test_scraping_pic1.jpg::4.4

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Product has the following GUID: urn:uuid:e7e00b22-4af7-4b5f-b57f-934203e7b763

SUCCESS:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Successfully registered association to 'urn:uuid:f79e3f71-e889-42e4-83e6-309c0d762ad3'

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Association has the following GUID: urn:uuid:010ac4b2-fd7d-4f61-8e42-bfc33f755042

SUCCESS:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Successfully registered association to 'urn:uuid:e7e00b22-4af7-4b5f-b57f-934203e7b763'

INFO:   [/Users/rchen/Desktop/test/x/pit_test_scraping_pic44.xml] Association has the following GUID: urn:uuid:c80ef461-2724-4b87-b94c-e5cb5a9f7b8c

Summary:

1 of 1 file(s) processed, 0 other file(s) skipped

0 error(s), 0 warning(s)

1 of 1 products registered.

2 of 2 ancillary products registered.

Product Types Registered:

1 Product_Browse

2 Product_File_Repository

2 of 2 associations registered.

End of Log

Step 11: In early versions, the Status was not “Submitted”../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Step 13:../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Step 14 should show no associations from any harvest

../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Step 17: ../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011

Step 19: Both Schemes and Associations should be empty.

../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011

../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011

Comments

Results met success criteria.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

Test Case ID

HVT.8

Description

Test new capabilities of harvest.

Requirements

PASS No specific functional requirement.

Success Criteria

The registry shows the harvested products.

Test Steps

Test URL in harvest config file with ‘?’, LID ending in .xml, <title> with many spaces

1.     Clean database as described in RESETREGISTRY in Section 3.1

2.     cd testDir

harvestAtmV1/ and …V2/ show NAIF’s versioning of a growing ATM dataset.

3.     diff harvestAtmV1/checksum.bad harvestAtmV2/checksum.works

4.     validate -f -M harvestAtmV1/checksum.bad -t harvestAtmV1

5.     harvest testDir/harvestAtmV1 -c harvestAtmV1.xml -e "*.xml" -l h1.out

6.     At http://localhost:8080/registry-ui/, select registry-pds4/, then “Object Type” to “Product_Observational”, then hit “Refresh”

7.     validate -f -M harvestAtmV2/checksum.works -t harvestAtmV2

8.     harvest testDir/harvestAtmV2 -c harvestAtmV2.xml -e "*.xml" -l h2.out

9.     repeat step 5

Test Results

Step 3: line 1 has the debateable leading \. One line has an all CAPS checksum

1c1

< a9e18e8e7bc034f69c77e7aeee114fa0  \bundle_v001.xml

---

> 3d20c6cecfb18f6f2421d72a031b1e1d  bundle_v002.xml

3a4,5

> 9209d2f4b6b55b9391686c2c9567f04e  data_mer1\collection_v002.csv

> e8d92e26de80b508be15833a1dc1f25c  data_mer1\collection_v002.xml

10,13c12,17

< 7b69a8dc6136ddd98b4da378441dbc3c  data_mer2\sol0013\a0013_p236402_01_01v04.qub

< 194601535dca45f7b587b7786790d7eb  data_mer2\sol0013\a0013_p236402_01_01v04.xml

< ed1398a702ae0dae28ed13a98ceb832e  data_mer2\sol1245\a1245_p276309_02_02v01.qub

< 5d013088de380339795b8627b50d97b1  data_mer2\sol1245\a1245_p276309_02_02v01.xml

---

> 1c095f0a7cfc33908e29d8860c6cbd7a  data_mer2\collection_v002.csv

> 5dea8f9e6af0adfc2d3bfcd0bdb351a9  data_mer2\collection_v002.xml

> 7b69a8dc6136ddd98b4da378441dbc3c  data_mer2\sol0013/a0013_p236402_01_01v04.qub

> 194601535dca45f7b587b7786790d7eb  data_mer2\sol0013/a0013_p236402_01_01v04.xml

> ed1398a702ae0dae28ed13a98ceb832e  data_mer2\sol1245/a1245_p276309_02_02v01.qub

> 5d013088de380339795b8627b50d97b1  data_mer2\sol1245/a1245_p276309_02_02v01.xml

Step 4:

Oct 19, 2017 3:05:56 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Directory' for location file:/Users/rchen/Desktop/test/harvestAtmV1/

Oct 19, 2017 3:05:56 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/harvestAtmV1/'

Oct 19, 2017 3:05:59 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/harvestAtmV1/'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-19T22:05:56Z

   Core Schemas                  [PDS4_PDS_1800.xsd]

   Core Schematrons              [PDS4_PDS_1800.sch]

   Model Version                 1800

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/harvestAtmV1/]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

   Checksum Manifest File        file:/Users/rchen/Desktop/test/harvestAtmV1/checksum.bad

   Manifest File Base Path       file:/Users/rchen/Desktop/test/harvestAtmV1/

Validation Details:

  FAIL: file:/Users/rchen/Desktop/test/harvestAtmV1/bundle_v001.xml

      ERROR  No checksum found in the manifest for 'file:/Users/rchen/Desktop/test/harvestAtmV1/bundle_v001.xml'

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/data_mer1/collection_v001.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/data_mer1/sol0011/b0011_p237201_01_01v02.xml

  FAIL: file:/Users/rchen/Desktop/test/harvestAtmV1/data_mer2/collection_v001.xml

      ERROR  Generated checksum '0d0ddeeaa2036ebb69cc4aeb30e11203' does not match supplied checksum '0D0DDEEAA2036EBB69CC4AEB30E11203' in the manifest for 'file:/Users/rchen/Desktop/test/harvestAtmV1/data_mer2/collection_v001.xml'

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/data_mer2/sol0013/a0013_p236402_01_01v04.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/document/collection_document_inventory.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/document/pancam_instrument_description.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/miscellaneous/collection_misc_inventory.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/miscellaneous/mer1_cube_list.xml

  PASS: file:/Users/rchen/Desktop/test/harvestAtmV1/miscellaneous/mer2_cube_list.xml

Summary:

  10 of 10 file(s) processed, 0 skipped

  8 of 10 file(s) passed validation

End of Report

Step 5: The interesting parts of h1.out:

PDS Harvest Tool Log

Version                     Version 1.12.0

Time                        Mon, Apr 17 2017 at 03:58:11 PM

Target(s)                   [/Users/rchen/Desktop/test/harvestAtmV1]

File Inclusions             [*.xml]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20170417155811

Registration Package GUID   urn:uuid:1d9eabb5-5162-44e8-a0e9-c6335a781690

Checksum Manifest File      /Users/rchen/Desktop/test/harvestAtmV1/checksum.bad

Manifest File Base Path     /Users/rchen/Desktop/test/harvestAtmV1

 

 

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/harvestAtmV1/checksum.bad] Processing checksum manifest.

INFO:   [/Users/rchen/Desktop/test/harvestAtmV1/bundle_v001.xml] Begin processing.

INFO:   [/Users/rchen/Desktop/test/harvestAtmV1/bundle_v001.xml] line 54: Mapping reference type 'bundle_to_investigation' to 'investigation_ref'.

[snip…]

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV1/data_mer2/collection_v001.xml] Generated checksum '0d0ddeeaa2036ebb69cc4aeb30e11203' does not match supplied checksum '0D0DDEEAA2036EBB69CC4AEB30E11203' in the manifest for file object '/Users/rchen/Desktop/test/harvestAtmV1/data_mer2/collection_v001.xml'.

[snip…]

SUCCESS:   [/Users/rchen/Desktop/test/harvestAtmV1/data_mer1/sol0011/b0011_p237201_01_01v02.xml] Successfully registered association to 'urn:uuid:3901be3c-891f-4591-bd81-8afd4ab31091'

INFO:   [/Users/rchen/Desktop/test/harvestAtmV1/data_mer1/sol0011/b0011_p237201_01_01v02.xml] Association has the following GUID: urn:uuid:9cbfa248-8af6-4d21-ba4d-58b5cacc4c0f

Summary:

10 of 10 file(s) processed, 0 other file(s) skipped

0 error(s), 3 warning(s)

10 of 10 products registered.

19 of 19 ancillary products registered.

Product Types Registered:

4 Product_Collection

1 Product_Document

2 Product_Observational

2 Product_Ancillary

19 Product_File_Repository

1 Product_Bundle

18 of 19 generated checksums matched their supplied value in the manifest, 0 value(s) not checked.

0 of 0 generated checksums matched the supplied value in their product label, 19 value(s) not checked.

19 of 19 associations registered.

End of Log

Step 6: (snipped from bottom: “Total Records: 29”)../Screen%20Shot%202017-04-17%20at%2016.00.23.png

Step 7: very similar to the output of step 4

Step 8:The interesting parts of h2.out:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Tue, Oct 17 2017 at 02:01:03 PM

Target(s)                   [/Users/rchen/Desktop/test/harvestAtmV2]

File Inclusions             [*.xml]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds4

Registry Package Name       Harvest-Package_20170417161145

Registration Package GUID   urn:uuid:774d1e95-3391-45cc-bc3c-6d075365b027

Checksum Manifest File      /Users/rchen/Desktop/test/harvestAtmV2/checksum.works

Manifest File Base Path     /Users/rchen/Desktop/test/harvestAtmV2

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/harvestAtmV2/checksum.works] Processing checksum manifest.

INFO:   [/Users/rchen/Desktop/test/harvestAtmV2/bundle_v001.xml] Begin processing.

[snip…]

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/bundle_v001.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/miscellaneous/collection_misc_inventory.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:misc::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/miscellaneous/mer1_cube_list.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:misc:mer1_cube_list::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/miscellaneous/mer2_cube_list.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:misc:mer2_cube_list::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/document/collection_document_inventory.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:document::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/document/pancam_instrument_description.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:document:pancam_inst::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/data_mer2/collection_v001.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:mer2::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/data_mer2/sol0013/a0013_p236402_01_01v04.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:mer2:a0013_p236402_01_01v04::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/data_mer1/collection_v001.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:mer1::1.0

WARNING:   [/Users/rchen/Desktop/test/harvestAtmV2/data_mer1/sol0011/b0011_p237201_01_01v02.xml] Product already exists: urn:nasa:pds:mer_pancam_photometry:mer1:b0011_p237201_01_01v02::1.0

[snip…]

Summary:

15 of 15 file(s) processed, 0 other file(s) skipped

0 error(s), 10 warning(s)

5 of 15 products registered.

9 of 9 ancillary products registered.

Product Types Registered:

2 Product_Collection

2 Product_Observational

9 Product_File_Repository

1 Product_Bundle

9 of 9 generated checksums matched their supplied value in the manifest, 0 value(s) not checked.

0 of 0 generated checksums matched the supplied value in their product label, 9 value(s) not checked.

9 of 9 associations registered.

End of Log

Step 9: (snipped from bottom: “Total Records: 43”)../Screen%20Shot%202017-04-17%20at%2016.15.58.png

 

Comments

Results met success criteria.

 

Biuld 7b opens PDS-497. The first checksum manifest has file “\bundle.xml” (leading ‘\’), which validate and harvest handle differently. It also has a checksum in all caps, which both validate and harvest fail to recognize as a good value.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

 

Test Case ID

MOD.1

Description

Use LDDTool to create local data dictionaries

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results

 

Success Criteria

Validate uses the created local data dictionary to validate a label.

Test Steps

1.     cd testDir/testLDD

2.     validate -x ../PDS4_PDS_1800.xsd -S ../PDS4_PDS_1800.sch -t nsyt_maro.xml

3.     lddtool -pl ingest_ldd.xml

At line 25 of ingest_ldd_INSIGHT_0510.xsd (i.e. before <xs:complexType name="Observation_Information">), insert:

<xs:element name="Observation_Information" type="insight:Observation_Information"/>

4.     validate -x ../PDS4_PDS_1800.xsd ingest_ldd_INSIGHT_0510.xsd -S ../PDS4_PDS_1800.sch ingest_ldd_INSIGHT_0510.sch -t nsyt_maro.xml

5.     rm ingest_ldd_INSIGHT*

Test Results

Step 2:

Oct 18, 2017 1:23:55 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml

Oct 18, 2017 1:23:55 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml'

Oct 18, 2017 1:23:57 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-18T20:23:54Z

Parameters:

   Targets                       [file:testDir/testLDD/nsyt_maro.xml]

   User Specified Schemas        [file:testDir/PDS4_PDS_1800.xsd]

   User Specified Schematrons    [file:testDir/PDS4_PDS_1800.sch]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  FAIL: file:testDir/testLDD/nsyt_maro.xml

      ERROR  line 113, 46: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'insight:Observation_Information'.

Summary:

  1 of 1 file(s) processed, 0 skipped

  0 of 1 file(s) passed validation

End of Report

Step 3:

>>info    - LDDTOOL Version: 0.2.1.3

>>info    - Date: Wed Oct 18 13:33:10 PDT 2017

>>info    - JAVAHOME: /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home

>>info    - PARENT_DIR: /PDS4tools/lddtool

>>info    - SCRIPT_DIR: /PDS4tools/lddtool/bin

>>info    - LIB_DIR: /PDS4tools/lddtool/lib

>>info    - Found input file: ingest_ldd.xml

>>info    - Found required file: /PDS4tools/lddtool/Data/MDPTNConfigClassDisp.xml

>>info    - Configured Steward/NameSpaceId Pairs

>>info    - cart - cart

>>info    - cart - pds

>>info    - geo - geom

>>info    - img - disp

>>info    - img - img

>>info    - img - pds

>>info    - msn - msn

>>info    - ops - pds

>>info    - pds - pds

>>info    - rings - rings

finding identifier: ---->rings

finding identifier: ---->cart

finding identifier: ---->sbn

finding identifier: ---->msn

finding identifier: ---->ppi

finding identifier: ---->geom

finding identifier: ---->naif

finding identifier: ---->disp

finding identifier: ---->atm

finding identifier: ---->pds

ADDING master:pds

>>info    - Found required file: /PDS4tools/lddtool/Data/dd11179.pins

>>info    - Found required file: /PDS4tools/lddtool/Data/DMDocument.pins

>>info    - Found required file: /PDS4tools/lddtool/Data/UpperModel.pont

>>info    - Found required file: /PDS4tools/lddtool/Data/Glossary.pins

>>info    - getMasterObjectDict - Found LDDToolSingletonClass - DMDocument.LDDToolSingletonClass.title:Discipline_Area

>>info    - Found required file: /PDS4tools/lddtool/Data/UpperModel.pins

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Equirectangular - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Lambert_Conformal_Conic - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Oblique_Mercator - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Polar_Stereographic - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Polyconic - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Sinusoidal - lClass.subClassOfTitle:Map_Projection_Base

***error*** - missing superClass in master while trying to set subClassOf - lClass.identifier:0001_NASA_PDS_1.cart.Transverse_Mercator - lClass.subClassOfTitle:Map_Projection_Base

>>info    - Generate Schematron Rules - Custom Rule Counts - Before generation

>>info    - Rule count for Arr: 59

>>info    - Rule count for Map: 59

>>info    - Rule count for Id Map: 59

   WARNING  Header:  - New namespace id has been specified:insight

   INFO     Attribute: <mission_phase_name> - This local attribute has a duplicate in the PDS4 data dictionary.

   INFO     Attribute: <release_number> - The default minimum value provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <software_name> - The default minimum characters provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <software_name> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <software_version_id> - The default minimum characters provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <software_version_id> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <software_version_id> - This local attribute has a duplicate in the PDS4 data dictionary.

   INFO     Attribute: <sol_number> - The default minimum value provided by the attribute's data type is being overridden with 0

   INFO     Attribute: <spacecraft_clock_start_count> - The default minimum characters provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <spacecraft_clock_start_count> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <spacecraft_clock_start_count> - This local attribute has a duplicate in the PDS4 data dictionary.

   INFO     Attribute: <spacecraft_clock_stop_count> - The default minimum characters provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <spacecraft_clock_stop_count> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <spacecraft_clock_stop_count> - This local attribute has a duplicate in the PDS4 data dictionary.

   INFO     Attribute: <start_local_mean_solar_time> - The default minimum characters provided by the attribute's data type is being overridden with 8

   INFO     Attribute: <start_local_mean_solar_time> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <start_local_true_solar_time> - The default minimum characters provided by the attribute's data type is being overridden with 8

   INFO     Attribute: <start_local_true_solar_time> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <start_sol_number> - The default minimum value provided by the attribute's data type is being overridden with 0

   INFO     Attribute: <start_solar_longitude> - The default minimum value provided by the attribute's data type is being overridden with 0

   INFO     Attribute: <start_solar_longitude> - The default maximum value provided by the attribute's data type is being overridden with 360

   INFO     Attribute: <stop_local_mean_solar_time> - The default minimum characters provided by the attribute's data type is being overridden with 8

   INFO     Attribute: <stop_local_mean_solar_time> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <stop_local_true_solar_time> - The default minimum characters provided by the attribute's data type is being overridden with 8

   INFO     Attribute: <stop_local_true_solar_time> - The default maximum characters provided by the attribute's data type is being overridden with 255

   INFO     Attribute: <stop_sol_number> - The default minimum value provided by the attribute's data type is being overridden with 0

   INFO     Attribute: <stop_solar_longitude> - The default minimum value provided by the attribute's data type is being overridden with 0

   INFO     Attribute: <stop_solar_longitude> - The default maximum value provided by the attribute's data type is being overridden with 360

   INFO     Attribute: <instrument_clock_start_count> - The default minimum characters provided by the attribute's data type is being overridden with 1

   INFO     Attribute: <instrument_clock_start_count> - The default maximum characters provided by the attribute's data type is being overridden with 255

>>info    - Checking for attribute consistency - checkSameNameOverRide

>>info    - Counts

>>info    - Classes: 377

>>info    - Attributes: 1421

>>info    - Rules: 268

>>info    - LDDTOOL Exit

 Step 4:

Oct 18, 2017 3:00:00 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml

Oct 18, 2017 3:00:00 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml'

Oct 18, 2017 3:00:02 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-18T21:59:59Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testLDD/nsyt_maro.xml]

   User Specified Schemas        [file:/Users/rchen/Desktop/test/PDS4_PDS_1800.xsd, file:/Users/rchen/Desktop/test/testLDD/ingest_ldd_INSIGHT_0510.xsd]

   User Specified Schematrons    [file:/Users/rchen/Desktop/test/PDS4_PDS_1800.sch, file:/Users/rchen/Desktop/test/testLDD/ingest_ldd_INSIGHT_0510.sch]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  PASS: file:testDir/testLDD/nsyt_maro.xml

Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

Comments

Results met success criteria.

 

https://oodt.jpl.nasa.gov/jira/browse/PDS-518, created during testing of build 8a, asks for an explanation of the ***error*** output messages.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

 

Test Case ID

POR.1

Description

Use Data Set View (not Search) to browse products

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results

 

Success Criteria

Access any registered, individual PDS3 data set (a context product)

Test Steps

The registry must have data, and Harvest must have gotten absolute paths as inputs.

Test cases SRCH.3, SRCH.5, or SRCH.6 (typically run after HVT.3) leave PDS3 context products in the registry.

1.     In a browser,
http://localhost:8080/ds-view/query?identifier=MSL-M-CHEMCAM-LIBS-4/5-RDR-V1.0&resclass=data.dataset

Test Results

Step 1:../Screen%20Shot%202016-10-11%20at%2016.13.28.png

Comments

Results met success criteria.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

POR.2

Description

Submit and browse tools and services.

Requirements

PASS 4.2.2: Develop and implement procedures for periodically monitoring the user community interests and practices and verifying the usability of the products in the archive

Success Criteria

Upload and download non-core tools

Test Steps

Use the web site for EN development

1.     ssh pds-gamma

2.     ls /home/pds4/staging

3.     In a browser, http://pds-gamma.jpl.nasa.gov/tools/tool-registry/. Browse.

4.     Choose tab “Submit a Tool”. Fill in various false info, e.g. “espn.com”

5.     repeat step 2

6.     inspect newly created .xml file, e.g. search for a value entered, e.g.
grep -r -a1 espn.com /home/pds4/staging

Test Results

Step 6:

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml-        <version_id>1.0</version_id>

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml:        <url>http://espn.com</url>

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml-        <release_date>2017-04-19</release_date>

--

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml-        <category>Search</category>

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml:        <description>espn.com</description>

/home/pds4/staging/20170419-104442/testing_-_please_delete_1.0.xml-    </Service>

Comments

Results met success criteria.

Date of Testing

2017.04.19

Test Personnel

Richard Chen

 

 

Test Case ID

PRG.1

Description

Generate a PDS4 label from a PDS3 label or a PDS-specific DOM object.

Requirements

PASS L4.PRP.2 : The system shall provide a tool that assists users in the generation of PDS product labels.

Success Criteria

Generate produces a syntactically valid PDS Product Label else indicates where the input is invalid.

Test Steps

Some files in testDir/testHarvest/ come from PDS3 labels. Generate automatically and compare. Step 3 would be better with an xml diff.

1.     cd testDir/testPrep

2.     generate -p gendoc.lbl -t gendoc.vm

3.     diff -w gendoc.xml gendoc.baseline.xml

4.     rm gendoc.xml

5.     generate -p gen_ELE_MOM1.LBL gen_ELE_MOM2.LBL gen_ELE_MOM3.LBL -t gen_data.vm

6.     diff -w gen_ELE_MOM1.xml gen_ele_baseline.xml

7.     diff gen_ELE_MOM[13].LBL

8.     diff gen_ELE_MOM[13].xml

9.     rm gen_ELE_MOM?.xml

Test new functionality from build 6a

10.  cd genmpf

11.  generate -p i455934l.drk -t mpf_imp_raw_template_1400.xml

12.  generate -p i646954r.img -t mpf_imp_raw_template_1400.xml

13.  diff i455934l.xml i455934l.baseline.xml; diff i646954r.xml i646954r.baseline.xml

14.  rm i455934l.xml i646954r.xml

Test Results

Step 2:

New PDS4 Label: testDir/testPrep/gendoc.xml

Step 3: Differences only in spacing

Step 5: Note that generate used to (erroneously) require –d or –o

New PDS4 Label: testDir/testPrep/gen_ELE_MOM1.xml

New PDS4 Label: testDir/testPrep/gen_ELE_MOM2.xml

New PDS4 Label: testDir/testPrep/gen_ELE_MOM3.xml

Step 6: Differences only in spacing.

Step 7: diff the inputs

12c12

< PRODUCT_ID                    = "ELE_MOM1.TAB"

---

> PRODUCT_ID                    = "ELE_MOM3.TAB"

21c21

< ^TABLE                        = "ELE_MOM1.TAB"

---

> ^TABLE                        = "ELE_MOM3.TAB"

Step 8: diff the outputs

83,84c83,84

<             <file_name>ELE_MOM1.TAB</file_name>

<             <local_identifier>ELE_MOM1.TAB</local_identifier>

---

>             <file_name>ELE_MOM3.TAB</file_name>

>             <local_identifier>ELE_MOM3.TAB</local_identifier>

Step 11

New PDS4 Label: testDir/testPrep/genmpf/i455934l.xml

Step 12:

New PDS4 Label: testDir/testPrep/genmpf/i646954r.xml

Step 13: Differences only in spacing except for modification_date (for each pair).

42c28

<         <modification_date>2017-10-19</modification_date>

---

>                 <modification_date>2015-10-22</modification_date>

Comments

Generate converts most constructs in a PDS3 label into a PDS4 label.

 

Results met success criteria.

 

PDS-113, created during testing of build 2c, requests handling carets in PDS3 labels used for detached labels.

PDS-398, created during testing of build 6a, requests that `generate -o .` write output to a different directory.

PDS-519, created during testing of build 8a, requests documentation for handling the WARNings of step 2. Already resolved.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

 

Test Case ID

PRP.1

Description

Extract tabular data from a PDS4 product.

Requirements

PASS L4?.???.?

Success Criteria

Produce equivalent data in fixed width or csv.

Test Steps

1.     cd testDir/testPrep

2.     java -version

3.     extract-table extr_hp3.xml

4.     Operate on label pointing to multiple data files:
extract-table -l extr3.xml

5.     extract-table -t^ maven_events_20140801.xml

6.     extract-table https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml

Test Results

Step 2:

java version "1.8.0_101"

Java(TM) SE Runtime Environment (build 1.8.0_101-b13)

Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

Step 3:

Spacecraft Clock Time   Tilt X   Tilt Y   Tilt A   Tilt B Sol Number

      527817604.00000    90823  -215633   372377  1118264       0029

      527817614.00000    90210  -215075   372241  1118067       0029

      527817624.00000    91265  -215667   373688  1118125       0029

Step 4:

file: extr3.TAB

  table 1: fixed-width character table

    field 1: SOL (ASCII_Integer)

    field 2: LTST (ASCII_Real)

    field 3: LMST (ASCII_Real)

    field 4: V (ASCII_Real)

    field 5: DV+ (ASCII_Real)

    field 6: DV- (ASCII_Real)

    field 7: DIR (ASCII_Real)

    field 8: DDIR (ASCII_Real)

    field 9: EXPOSURE TIME (ASCII_Real)

    field 10: FILE NAME (ASCII_File_Name)

 

file: extr3.csv

  table 1: delimited table

    field 1: CHANNEL_NUMBER (ASCII_Integer)

    field 2: SPECTRA_01 (ASCII_Integer)

    field 3: SPECTRA_02 (ASCII_Integer)

    field 4: SPECTRA_03 (ASCII_Integer)

    field 5: SPECTRA_04 (ASCII_Integer)

    field 6: SPECTRA_05 (ASCII_Integer)

    field 7: SPECTRA_06 (ASCII_Integer)

    field 8: SPECTRA_07 (ASCII_Integer)

    field 9: SPECTRA_08 (ASCII_Integer)

    field 10: SPECTRA_09 (ASCII_Integer)

    field 11: SPECTRA_10 (ASCII_Integer)

    field 12: SPECTRA_11 (ASCII_Integer)

    field 13: SPECTRA_12 (ASCII_Integer)

 

file: extr3.dat

  table 1: fixed-width binary table

    field 1: SCLK_SECONDS (UnsignedMSB4)

    field 2: SCLK_SUBSECONDS (UnsignedMSB2)

    field 3: SPARE (UnsignedMSB2)

    field 4: ROTATION_MOTOR_POSITION (IEEE754MSBDouble)

    field 5: ROTATION_MOTOR_CURRENT_SENSOR (IEEE754MSBDouble)

    field 6: REVOLUTION_MOTOR_POSITION (IEEE754MSBDouble)

    field 7: REVOLUTION_MOTOR_CURRENT_SENSOR (IEEE754MSBDouble)

    field 8: Z_MOTOR_POSITION (IEEE754MSBDouble)

    field 9: Z_MOTOR_CURRENT_SENSOR (IEEE754MSBDouble)

    field 10: TEMPERATURE_SENSOR (IEEE754MSBDouble)

    field 11: BUTTERFLY_SWITCH_1 (UnsignedMSB4)

    field 12: BUTTERFLY_SWITCH_2 (UnsignedMSB4)

    field 13: RAT_OVER_CURRENT_ALARM (UnsignedMSB4)

    field 14: Z_AXIS_MOTOR_CONTROLLER_STATUS (UnsignedByte)

    field 15: REVOLVE_MOTOR_CONTROLLER_STATUS (UnsignedByte)

    field 16: GRIND_MOTOR_CONTROLLER_STATUS (UnsignedByte)

    field 17: SPARE (UnsignedByte)

    field 18: ROVER_BUS_VOLTAGE (IEEE754MSBDouble)

    field 19: ALGORITHM_STATE (UnsignedMSB4)

field 20: ANOMALY_FLAG (UnsignedMSB4)

Step 5:         

Event ID^event_type_id^start_time^end_time^source^description^discussion^modified_time^mission_event_id

   13642^          122^2014-11-23T18:15:39-07:00^2014-11-23T18:15:39-07:00^Integrated Report^Start of outbound side orbit segment.^['VM_10', 'VM_GV_SET_STR', 'vm_gv=GV_ORBIT_SEGMENT', '(152),vm_string=OB_SIDE'] FILENAME:report_integrated.MVN.txt^2014-12-19T11:31:41.566628^          140997

   13646^          121^2014-11-18T04:26:15-07:00^2014-11-18T04:26:15-07:00^Integrated Report^Start of periapse orbit segment.^['VM_10', 'VM_GV_SET_STR', 'vm_gv=GV_ORBIT_SEGMENT', '(152),vm_string=PERIAPSE'] FILENAME:report_integrated.MVN.txt^2014-12-19T11:31:41.597141^          122415

[snip…]

   24062^          131^2015-02-14T21:34:37-07:00^2015-02-14T21:34:37-07:00^SPICE ^Spacecraft entered eclipse^Generated using mvn_spk_2015_050_20_00_02.bsp^2015-02-22T02:30:13.515326^          594473

   24063^          130^2015-02-14T17:27:26-07:00^2015-02-14T17:27:26-07:00^SPICE ^Spacecraft exited eclipse^Generated using mvn_spk_2015_050_20_00_02.bsp^2015-02-22T02:30:13.916244^          594472

Step 6: demonstrate URL support

AST_NUMBER AST_NAME          PROV_DESIG REF_ID                         TYPE FLAG         PERIOD1 AMP1       PERIOD2 AMP2          PORB DS_DP_RATIO A_DP_RATIO

        22 Kalliope          -          summary                        -    -             4.1483 0.53 -999.99999999 -.99 -999.99999999        -.99     -9.999

        22 Kalliope          -          Descamps 2008a                 B    A    -99999.99999999 -.99 -999.99999999 -.99 -999.99999999        -.99     -9.999

[snip ...]

         0 -                 2016 BU13  summary                        -    -               39.5 0.24 -999.99999999 -.99 -999.99999999        -.99     -9.999

         0 -                 2016 BU13  Warner 2016m                   ?    A               39.5 0.24        2.4499 0.11 -999.99999999        -.99     -9.999

Comments

Results met success criteria.

 

Step 6 verifies https://oodt.jpl.nasa.gov/jira/browse/PDS-495.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

 

Test Case ID

PRT.1

Description

Transform PDS4 images into other formats. Transform is built upon a Java API.

Requirements

PASS L4.PRP.4: The system shall provide a tool for transforming PDS products as follows…

PASS L5.GEN.4: Tools shall have an application programming interface.

Success Criteria

Input and output images look the same.

Test Steps

1.     cd testDir/

2.     transform testPrep/i943630r.xml -o ./ -f jpg

3.     transform testPrep/tfm_FF01.LBL -o ./ -f jp2

4.     transform testPrep/tfm_ELE_MOM.LBL -f pds4-label

5.     grep -H NAME testPrep/tfm_ELE_MOM.LBL | grep , ; grep -H name testPrep/tfm_ELE_MOM.xml | grep ,

6.     diff -w tfm_ele_mom.xml testPrep/tfm_ELE_MOM.xml

7.     transform testPrep/tfm_414.xml -f jpg

8.     transform testPrep/tfm_grand1A.lbl -f csv

9.     Edit output file tfm_grand1A.csv to replace all commas with spaces

10.  diff -w tfm_grand1A.csv testPrep/tfm_grand1A.tab

Also check that the generated .xml file is good, and that transform to pds3 works

11.  transform tfm_grand1a.xml -f pds3-label

12.  ln -s testPrep/tfm_grand1a.tab .

13.  Run NASAView to show TFM_GRAND1A.LBL == testPrep/tfmgrand1A.lbl

14.  transform -f csv https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml

Test Results

Step 2:

PDS Transform Tool Log

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 10:37:06 PM

Target                      [testDir/testPrep/i943630r.xml]

Output Directory            .

Index                       1

Format Type                 jpg

INFO:   [testDir/testPrep/i943630r.xml] Transforming image '1' of file 'i943630r.raw'

INFO:   [TwoDImageExporter:setImageStatistics] No display settings found for identifier 'MPFL-M-IMP_IMG_GRAYSCALE'.

INFO:   [testDir/testPrep/i943630r.xml] Successfully transformed image '1' of file 'i943630r.raw' to the following output: ./i943630r.jpg

 i943630r.jpg:                                 

i943630r.jpg

Step 3: transform VICAR IO

Image write Done

JConvertIIO

0) INP = testPrep/tfm_FF01.LBL

1) OUT = ../TFM_FF01.bmp

2) FORMAT = bmp

3) RI = true

4) OFORM = BYTE

PDS Transform Tool Log

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 10:39:18 PM

Target                      [testDir/testPrep/tfm_FF01.LBL]

Output Directory            ..

Index                       1

Format Type                 jp2

INFO:   [testPrep/tfm_FF01.LBL] Transforming image file: testPrep/TFM_FF01.IMG

INFO:   [testPrep/tfm_FF01.LBL] Successfully transformed image file 'testPrep/TFM_FF01.IMG' to the following output: ../TFM_FF01.jp2

 TFM_FF01.jp2:

Step 4:

PDS Transform Tool Log

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 10:42:56 PM

Target                      [testDir/testPrep/tfm_ELE_MOM.LBL]

Output Directory            testDir

Index                       1

Format Type                 pds4-label

INFO:   [testPrep/tfm_ELE_MOM.LBL] Transforming label file: testPrep/tfm_ELE_MOM.LBL

INFO:   [testPrep/tfm_ELE_MOM.LBL] Successfully transformed PDS3 label 'testPrep/tfm_ELE_MOM.LBL' to a PDS4 label ' testDir/tfm_ele_mom.xml'

Step 5 shows proper handling of commas in column name.

testPrep/tfm_ELE_MOM.LBL:    NAME                      = "ELE_TEMP,OK,COMMA"

testPrep/tfm_ELE_MOM.xml:          <name>ele_temp,ok,comma</name>

Step 6: Differences not related to spacing:

1,9c1,2

< <?xml version="1.0" encoding="UTF-8" standalone="no"?>

<

< <?xml-model href="https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1900.sch"

<   schematypens="http://purl.oclc.org/dsdl/schematron"?>

<

< <Product_Observational xmlns="http://pds.nasa.gov/pds4/pds/v1"

<     xmlns:pds="http://pds.nasa.gov/pds4/pds/v1"

<     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

<     xsi:schemaLocation="http://pds.nasa.gov/pds4/pds/v1 https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1900.xsd">

---

> <?xml version="1.0" encoding="UTF-8"?>

> <Product_Observational xmlns="http://pds.nasa.gov/pds4/pds/v1" xmlns:pds="http://pds.nasa.gov/pds4/pds/v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="">

14c7

<     <information_model_version>1.9.0.0</information_model_version>

---

>     <information_model_version>1.2.0.1</information_model_version>

18c11

<         <modification_date>2017-10-24</modification_date>

---

>         <modification_date>2014-04-07</modification_date>

 Step 7:

 testPrep/tfm_414.jpg                               

Step 8:

New PDS4 Label: /Users/rchen/Desktop/test/tfm_grand1a.xml

outputs = [/Users/rchen/Desktop/test/tfm_grand1A.csv]

PDS Transform Tool Log

Version                     Version 1.6.0

Time                        Sun, Oct 22 2017 at 12:25:15 AM

Target                      [file:/Users/rchen/Desktop/test/testPrep/tfm_grand1A.lbl]

Output Directory            /Users/rchen/Desktop/test

Index                       1

Format Type                 csv

INFO:   [/Users/rchen/Desktop/test/testPrep/tfm_grand1A.lbl] Transforming label file: /Users/rchen/Desktop/test/testPrep/tfm_grand1A.lbl

INFO:   [/Users/rchen/Desktop/test/testPrep/tfm_grand1A.lbl] Finished transforming PDS3 label '/Users/rchen/Desktop/test/testPrep/tfm_grand1A.lbl' to a PDS4 label '/Users/rchen/Desktop/test/tfm_grand1a.xml'

INFO:   [/Users/rchen/Desktop/test/tfm_grand1a.xml] Transforming table '1' of file: /Users/rchen/Desktop/test/testPrep/tfm_grand1A.tab

INFO:   [/Users/rchen/Desktop/test/tfm_grand1a.xml] Successfully transformed table '1' to the following output: /Users/rchen/Desktop/test/tfm_grand1A.csv

Step 10:

1d0

< scet_utc state_index delta_sclk sclk telreadout telsoh mode hvps1_set hvps1 hvps2_set hvps2 hvps3_set hvps3 hvps4_set hvps4 hvps5_set hvps5 hvps6_set hvps6 pm5_lvps p12_lvps czt_pm5_lvps czt_enables nemg_tot_evts nemg_czt_evts nemn_tot_evts l_bgo_cw h_bgo_cw l_bgo_roi h_bgo_roi l_blp_my_cw h_blp_my_cw l_blp_my_roi h_blp_my_roi l_blp_py_cw h_blp_py_cw l_blp_py_roi h_blp_py_roi l_blp_mz_cw h_blp_mz_cw l_blp_mz_roi h_blp_mz_roi l_blp_pz_cw h_blp_pz_cw l_blp_pz_roi h_blp_pz_roi

Step 11:                                                                                                                               

outputs = [/PDS/PDS4test.build8a/TFM_GRAND1A.LBL]

PDS Transform Tool Log

 

Version                     Version 1.7.0-dev

Time                        Tue, Oct 24 2017 at 10:04:17 PM

Target                      [file:/PDS/PDS4test.build8a/tfm_grand1a.xml]

Output Directory            /PDS/PDS4test.build8a

Index                       1

Format Type                 pds3-label

 

INFO:   [/PDS/PDS4test.build8a/tfm_grand1a.xml] Transforming label file: /PDS/PDS4test.build8a/tfm_grand1a.xml

INFO:   [/PDS/PDS4test.build8a/tfm_grand1a.xml] Successfully transformed target label to a PDS3 label: /PDS/PDS4test.build8a/TFM_GRAND1A.LBL

 Step 14: URL as input. Then inspect output file lc_binary.csv

outputs = [/Users/rchen/Desktop/test/lc_binary.csv]

PDS Transform Tool Log

 

Version                     Version 1.6.0

Time                        Sun, Oct 22 2017 at 06:20:19 PM

Target                      [https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml]

Output Directory            /Users/rchen/Desktop/test

Index                       1

Format Type                 csv

 

FINEST:   [HttpURLConnection:plainConnect0] ProxySelector Request for https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml

FINEST:   [HttpsClient:New] Looking for HttpClient for URL https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml and proxy value of DIRECT

FINEST:   [HttpsClient:<init>] Creating new HttpsClient with url:https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml and proxy:DIRECT with connect timeout:-1

FINEST:   [HttpURLConnection:plainConnect0] Proxy used: DIRECT

FINE:   [HttpURLConnection:writeRequests] sun.net.www.MessageHeader@5ddeb7cb5 pairs: {GET /archive/bundles/ast_lightcurve_derived/data/lc_binary.xml HTTP/1.1: null}{User-Agent: Java/1.8.0_101}{Host: sbn.psi.edu}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}

FINEST:   [HttpClient:logFinest] KeepAlive stream used: https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml

FINE:   [HttpURLConnection:getInputStream0] sun.net.www.MessageHeader@70ed52de10 pairs: {null: HTTP/1.1 200 OK}{Date: Mon, 23 Oct 2017 01:20:11 GMT}{Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips}{Last-Modified: Mon, 18 Sep 2017 16:55:29 GMT}{ETag: "2c1c-559799c22e240"}{Accept-Ranges: bytes}{Content-Length: 11292}{Keep-Alive: timeout=5, max=100}{Connection: Keep-Alive}{Content-Type: text/xml}

INFO:   [/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml] Transforming table '1' of file: /archive/bundles/ast_lightcurve_derived/data/lc_binary.xml/lc_binary.tab

FINEST:   [HttpURLConnection:plainConnect0] ProxySelector Request for https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.tab

FINEST:   [HttpsClient:New] Looking for HttpClient for URL https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.tab and proxy value of DIRECT

FINEST:   [HttpsClient:<init>] Creating new HttpsClient with url:https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.tab and proxy:DIRECT with connect timeout:-1

FINEST:   [HttpURLConnection:plainConnect0] Proxy used: DIRECT

FINE:   [HttpURLConnection:writeRequests] sun.net.www.MessageHeader@3fa247d15 pairs: {GET /archive/bundles/ast_lightcurve_derived/data/lc_binary.tab HTTP/1.1: null}{User-Agent: Java/1.8.0_101}{Host: sbn.psi.edu}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}

FINEST:   [HttpClient:logFinest] KeepAlive stream used: https://sbn.psi.edu/archive/bundles/ast_lightcurve_derived/data/lc_binary.tab

FINE:   [HttpURLConnection:getInputStream0] sun.net.www.MessageHeader@2cb2fc209 pairs: {null: HTTP/1.1 200 OK}{Date: Mon, 23 Oct 2017 01:20:24 GMT}{Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips}{Last-Modified: Fri, 01 Sep 2017 15:59:02 GMT}{ETag: "1f99e-55822d6f3fd80"}{Accept-Ranges: bytes}{Content-Length: 129438}{Keep-Alive: timeout=5, max=100}{Connection: Keep-Alive}

INFO:   [/archive/bundles/ast_lightcurve_derived/data/lc_binary.xml] Successfully transformed table '1' to the following output: /Users/rchen/Desktop/test/lc_binary.csv

 

Comments

Results met success criteria.

 

https://oodt.jpl.nasa.gov/jira/browse/PDS-343 (formerly PDS-349), created during testing of build 5b, requests friendlier warning messages

 

Steps 8-10 verify https://oodt.jpl.nasa.gov/jira/browse/PDS-506, PDS3 to csv, and https://oodt.jpl.nasa.gov/jira/browse/PDS-505, output .xml with latest (1900) model. Step 13 verifies https://oodt.jpl.nasa.gov/jira/browse/PDS-511, input from URL.

 

https://oodt.jpl.nasa.gov/jira/browse/PDS-523, created and resolved during testing of build 8a, step 12, notes transform’s inability to create a pds3 label. Now fixed, this verifies https://oodt.jpl.nasa.gov/jira/browse/PDS-507.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

 

Test Case ID

PRT.2

Description

Transform PDS4 Array_3D products into other formats.

Requirements

PASS L4.PRP.4: The system shall provide a tool for transforming PDS products as follows…

PASS L5.GEN.4: Tools shall have an application programming interface.

Success Criteria

If browse product exists, compare output against it.

Test Steps

1.     cd testDir/testPrep

Files tfm_d00.* were d000m5240t494053574edr_f0000_0501m1.* from an Insight review

2.     transform tfm_d00.xml -f jpg

3.     rm tfm_d00.jpg

Transform an unsignedLSB2 and a signedLSB2.

4.     transform tfm_434u.xml -f png

5.     transform tfm_434s.xml -f gif

6.     rm tfm_434.png tfm_434.gif

Test Results

Step 2:

PDS Transform Tool Log                                                  

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 11:28:29 PM

Target                      [file:/Users/rchen/Desktop/test/testPrep/tfm_d00.xml]

Output Directory            testDir/testPrep

Index                       1

Format Type                 jpg

INFO:   [tfm_d00.xml] Transforming image '1' of file 'tfm_d00.vic'

INFO:   [ThreeDImageExporter:setImageStatistics] No display settings found for identifier 'IMAGE DATA'.

INFO:   [tfm_d00.xml] Successfully transformed image '1' of file 'tfm_d00.vic' to the following output: testDir/testPrep/tfm_d00.jpg

 Output looks good and matches browse product tfm_d00.png../../Users/rchen/Desktop/d000m5240t494053574edr_f0000_0501m

Step 4:testPrep/tfm_434.png

Step 5:testPrep/tfm_434.gif

Comments

Results met success criteria.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

 

Test Case ID

PRT.3

Description

Test various capabilities described in release documents:

·       Transform PDS4 Array_3D_Spectrum into other formats.

·       Use scaling_factor, value_offset, and orientation values.

·       Transform label with multiple images

Requirements

PASS L4.PRP.4: The system shall provide a tool for transforming PDS products as follows…

Success Criteria

Visually inspect output product

Test Steps

1.     cd testDir/testPrep

A. Transform PDS4 Array_3D_Spectrum into other formats. Note, tfm_spec.xml is tfm_d00.xml with “…_Spectrum” replacing “…_Image”

2.     transform tfm_spec.xml -f jpg

3.     rm tfm_d00.jpg

B. Use scaling_factor, value_offset, and orientation values. Note, the input files tfm_433*.xml, are all variations of tfm_d00.xml

4.     diff tfm_433a.xml tfm_433b.xml

5.     diff tfm_433a.xml tfm_433c.xml

6.     transform tfm_433a.xml -f gif

7.     transform tfm_433b.xml -f jpg

8.     transform tfm_433c.xml -f png

9.     rm tfm_433.gif tfm_433.jpg tfm_433.png

C. File pancam.qub has many images and uses <disp:*display_direction>

10.  mkdir xA ; transform pancamA.xml -f jpg -a -o xA

11.  mkdir xB ; transform pancamB.xml -f jpg -a -o xB

12.  ls -l xA

13.  diff pancamA.xml pancamB.xml

14.  diff -r xA xB

15.  view the two first files (and any other so desired)

16.  rm -r xA xB

17.  transform -O pancamA.xml

Test Results

Step 2:

PDS Transform Tool Log                     

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 11:31:00 PM

Target                      [file:/Users/rchen/Desktop/test/testPrep/tfm_spec.xml]

Output Directory            testDir/testPrep

Index                       1

Format Type                 jpg

INFO:   [tfm_spec.xml] Transforming image '1' of file 'tfm_d00.vic'

INFO:   [ThreeDSpectrumExporter:setImageStatistics] No display settings found for identifier 'IMAGE DATA'.

INFO:   [tfm_spec.xml] Successfully transformed image '1' of file 'tfm_d00.vic' to the following output: testDir/testPrep/tfm_d00.jpg

../../Users/rchen/Desktop/tfm_d0

Step 4:

118,119c118,119

<                 <line_display_direction>Down</line_display_direction>

<                 <sample_display_direction>Right</sample_display_direction>

---

>                 <line_display_direction>Up</line_display_direction>

>                 <sample_display_direction>Left</sample_display_direction>

 Step 5:

105c105

<                 <scaling_factor>1</scaling_factor>

---

>                 <scaling_factor>0</scaling_factor>

Steps 6,7,8 (images have been shrunk to fit 3-wide on this page)

../../Users/rchen/Desktop/../../Users/rchen/Desktop/../../Users/rchen/Desktop/

Step 10 (Step 11 is very similar):

PDS Transform Tool Log

Version                     Version 1.6.0

Time                        Thu, Oct 19 2017 at 11:35:05 PM

Target                      [file:/Users/rchen/Desktop/test/testPrep/pancamA.xml]

Output Directory            xA

Transform All               true

Format Type                 jpg

INFO:   [/Users/rchen/Desktop/test/testPrep/pancamA.xml] Transforming image '1' of file 'pancam.qub'

INFO:   [/Users/rchen/Desktop/test/testPrep/pancamA.xml] Successfully transformed image '1' of file 'pancam.qub' to the following output: xA/pancam_1.jpg

INFO:   [/Users/rchen/Desktop/test/testPrep/pancamA.xml] Transforming image '2' of file 'pancam.qub'

[snip...]                                                                                                 

INFO:   [/Users/rchen/Desktop/test/testPrep/pancamA.xml] Transforming image '20' of file 'pancam.qub'

INFO:   [TwoDImageExporter:setImageStatistics] No display settings found. Missing local_identifier element in the Array_2D_Image area.

INFO:   [/Users/rchen/Desktop/test/testPrep/pancamA.xml] Successfully transformed image '20' of file 'pancam.qub' to the following output: xA/pancam_20.jpg

Step 12:

total 2928

-rw-rw-r--  1 rchen  703763885   96617 Oct 19 23:35 pancam_1.jpg

-rw-rw-r--  1 rchen  703763885   38511 Oct 19 23:35 pancam_10.jpg

-rw-rw-r--  1 rchen  703763885   51693 Oct 19 23:35 pancam_11.jpg

-rw-rw-r--  1 rchen  703763885   63297 Oct 19 23:35 pancam_12.jpg

-rw-rw-r--  1 rchen  703763885   72366 Oct 19 23:35 pancam_13.jpg

-rw-rw-r--  1 rchen  703763885   98680 Oct 19 23:35 pancam_14.jpg

-rw-rw-r--  1 rchen  703763885   73826 Oct 19 23:35 pancam_15.jpg

-rw-rw-r--  1 rchen  703763885  115606 Oct 19 23:35 pancam_16.jpg

-rw-rw-r--  1 rchen  703763885  113894 Oct 19 23:35 pancam_17.jpg

-rw-rw-r--  1 rchen  703763885   68505 Oct 19 23:35 pancam_18.jpg

-rw-rw-r--  1 rchen  703763885   80052 Oct 19 23:35 pancam_19.jpg

-rw-rw-r--  1 rchen  703763885   89209 Oct 19 23:35 pancam_2.jpg

-rw-rw-r--  1 rchen  703763885   64422 Oct 19 23:35 pancam_20.jpg

-rw-rw-r--  1 rchen  703763885   76261 Oct 19 23:35 pancam_3.jpg

-rw-rw-r--  1 rchen  703763885   71655 Oct 19 23:35 pancam_4.jpg

-rw-rw-r--  1 rchen  703763885   68849 Oct 19 23:35 pancam_5.jpg

-rw-rw-r--  1 rchen  703763885   71295 Oct 19 23:35 pancam_6.jpg

-rw-rw-r--  1 rchen  703763885   63583 Oct 19 23:35 pancam_7.jpg

-rw-rw-r--  1 rchen  703763885   29191 Oct 19 23:35 pancam_8.jpg

-rw-rw-r--  1 rchen  703763885   52468 Oct 19 23:35 pancam_9.jpg

 Step 13:

71c71

<           <disp:horizontal_display_direction>Left to Right</disp:horizontal_display_direction>

---

>           <disp:horizontal_display_direction>Right to Left</disp:horizontal_display_direction>

73c73

<           <disp:vertical_display_direction>Top to Bottom</disp:vertical_display_direction>

---

>           <disp:vertical_display_direction>Bottom to Top</disp:vertical_display_direction>

Step 14: The two images with <local_identifier>Spectral_Qube_Object</…>  differ:

Binary files xA/pancam_1.jpg and xB/pancam_1.jpg differ

Binary files xA/pancam_3.jpg and xB/pancam_3.jpg differ

Step 15: (images have been shrunk to fit 2-wide on this page)

../../../../../../Desktop/pancam_ ../../../../../../Desktop/pancam_

Step 17:

Supported Images:

  Data file: pancam.qub

    index = 1

    object type = Array_3D_Spectrum

    name = null

    local identifier = Spectral_Qube_Object

    data type = SignedMSB2

    lines = 1024

    samples = 1024

    index = 2

    object type = Array_2D_Image

    name = iof_r2

    local identifier = null

    data type = SignedMSB2

    lines = 1024

    samples = 1024

[snip…]

    index = 20

    object type = Array_2D_Image

    name = approx_phase_angle

    local identifier = null

    data type = SignedMSB2

    lines = 1024

    samples = 1024

Supported Tables:

  None Found

Comments

Results met success criteria.

Date of Testing

2017.10.19

Test Personnel

Richard Chen

                                                                                                                                                          

 

Test Case ID

PRV.1

Description

Accept a file or a directory name for product(s) to be validated. If directory, be able to traverse the tree to find products. Indicate the schemas utilized during validation. Validate is built upon a Java API.

Requirements

PASS L5.PRP.VA.1: The tool shall accept the following as input for specifying the product(s) to be validated…

PASS L5.PRP.VA.2: The tool shall traverse a directory tree and validate products discovered within that tree.

PASS L5.PRP.VA.5: The tool shall verify that a product label is well-formed XML.

PASS L5.PRP.VA.6: The tool shall verify that a product label conforms to its associated schema file(s).

PASS L5.PRP.VA.9: The tool shall indicate the schema(s) utilized during validation.

PASS L5.GEN.4: Tools shall have an application programming interface.

PASS L5.GEN.7: Tools shall generate a report detailing results from a single execution of the tool.

Success Criteria

Validation tool validates a file or all eligible products in a directory tree. When validating a product, a label, or a schema, indicates which schemas it utilized during the validation. Ensures that a product label is well-formed XML and conforms to its schemas.

Test Steps

1.     cd testDir/bundle_psi

2.     validate -e "*xml" -x xml_schema/*xsd -S xml_schema/*sch -R pds4.bundle -t .

Some files need local data dictionary img. Note they must follow PDS4_PDS* on the command line in step 4, hence the name ZZ.*

3.     cd xml_schema; ln -s PDS4_IMG_1100.xsd.later ZZ.xsd; ln -s PDS4_IMG_1100.sch.later ZZ.sch; cd ..

4.     repeat step 2 

5.     rm xml_schema/ZZ*

Test Results

Step 2:

Oct 20, 2017 2:36:59 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Directory' for location file:/Users/rchen/Desktop/test/bundle_psi/

Oct 20, 2017 2:36:59 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundle_psi/'

Oct 20, 2017 2:37:01 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundle_psi/'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-20T21:36:59Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_psi/]

   User Specified Schemas        [file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.xsd]

   User Specified Schematrons    [file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.sch]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*xml]

   Force Mode                    off

Validation Details:

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/bundle_mars_spectra_reddy.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/collection_mars_spectra_reddy_data.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/2001/1916_boreas.xml

      ERROR  line 69, 42: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'img:Imaging_Instrument_Parameters'.

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/163000_2001sw169.xml

      ERROR  line 69, 42: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'img:Imaging_Instrument_Parameters'.

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/2008rw24.xml

      ERROR  line 69, 42: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'img:Imaging_Instrument_Parameters'.

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/39572_1993dq1.xml

      ERROR  line 69, 42: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'img:Imaging_Instrument_Parameters'.

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/446791_1998sj70.xml

      ERROR  line 69, 42: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'img:Imaging_Instrument_Parameters'.

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/AA_Document_Index.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/allspectra.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/bundle_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/collection_mars_spectra_reddy_document.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/host_OBS376T3_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/instrument_I0046_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/ref.xml

Summary:

  15 of 15 file(s) processed, 0 skipped

  10 of 15 file(s) passed validation

End of Report

Step 4:

Oct 20, 2017 4:14:27 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Bundle' for location file:/Users/rchen/Desktop/test/bundle_psi/

Oct 20, 2017 4:14:27 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundle_psi/'

Oct 20, 2017 4:14:29 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundle_psi/'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-20T23:14:26Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_psi/]

   Rule Type                     pds4.bundle

   User Specified Schemas        [file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.xsd, file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/ZZ.xsd]

   User Specified Schematrons    [file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.sch, file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/ZZ.sch]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml]

   Force Mode                    off

Validation Details:

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/.DS_Store

      ERROR  File is not valid in bundle root directory

      ERROR  File name uses invalid character

      ERROR  File is not referenced by any label

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/bundle_mars_spectra_reddy.xml

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/.DS_Store

      ERROR  File name uses invalid character

      ERROR  File is not referenced by any label

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/collection_mars_spectra_reddy_data.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/2001/1916_boreas.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/163000_2001sw169.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/2008rw24.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/39572_1993dq1.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/2008/446791_1998sj70.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/AA_Document_Index.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/allspectra.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/bundle_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/collection_mars_spectra_reddy_document.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/host_OBS376T3_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/instrument_I0046_description.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/document/ref.xml

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_IMG_1100.sch.later

      ERROR  File is not referenced by any label

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_IMG_1100.xsd.later

      ERROR  File is not referenced by any label

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.sch

      ERROR  File is not referenced by any label

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/PDS4_PDS_1600.xsd

      ERROR  File is not referenced by any label

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/ZZ.sch

      ERROR  File is not referenced by any label

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/xml_schema/ZZ.xsd

      ERROR  File is not referenced by any label

Summary:

  23 of 23 file(s) processed, 0 skipped

  15 of 23 file(s) passed validation

End of Report

Comments

Results met success criteria.

 

Steps 4 and 5 reopened PDS-486, for validate put out an odd error message possibly related to the -e option.

Build 8a opens PDS-520, where validate does not catch a ‘+’ in a text field format.

Build 8a opens PDS-521, where option -R pds4.bundle negates -e "*.xml". This has already been closed as being expected behavior, simply undocumented at the time.

Date of Testing

2017.10.20

Test Personnel

Richard Chen

 

Test Case ID

PRV.2

Description

Verify that a referenced file exists.

Requirements

PASS L5.PRP.VA.10: The tool shall verify that a file exists when referenced from a product label.

Success Criteria

Validation tool succeeds if referenced file exists, throws an error if not

Test Steps

1.     mv bundle_psi/data/science_index.tab .

2.     validate -f bundle_psi/data/science_index.xml

3.     mv science_index.tab bundle_psi/data/

4.     repeat step 2

References also consider <directory_path_name>, which can be absolute, relative, and with or without a trailing ‘/’.

5.     diff testPrep/product_document/Product_Doc_bad.xml testPrep/product_document/Product_Doc_good.xml

6.     validate -t testPrep/product_document/Product_Doc_bad.xml -f

7.     validate -t testPrep/product_document/Product_Doc_good.xml -f

Test Results

Step 2: the referenced file was removed in step 1

Oct 20, 2017 5:47:46 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

Oct 20, 2017 5:47:46 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml'

Oct 20, 2017 5:47:48 PM gov.nasa.pds.objectAccess.ByteWiseFileAccessor <init>

SEVERE: I/O error.

java.io.FileNotFoundException: /Users/rchen/Desktop/test/bundle_psi/data/science_index.tab (No such file or directory)

                        at java.io.FileInputStream.open0(Native Method)

                        at java.io.FileInputStream.open(FileInputStream.java:195)

                        at java.io.FileInputStream.<init>(FileInputStream.java:138)

                        at java.io.FileInputStream.<init>(FileInputStream.java:93)

                        at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)

                        at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)

                        at gov.nasa.pds.objectAccess.utility.Utility.openConnection(Utility.java:88)

                        at gov.nasa.pds.objectAccess.ByteWiseFileAccessor.<init>(ByteWiseFileAccessor.java:149)

                        at gov.nasa.pds.objectAccess.TableReader.<init>(TableReader.java:106)

                        at gov.nasa.pds.tools.validate.content.table.RawTableReader.<init>(RawTableReader.java:61)

                        at gov.nasa.pds.tools.validate.rule.pds4.TableDataContentValidationRule.validateTableDataContents(TableDataContentValidationRule.java:166)

                        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

                        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                        at java.lang.reflect.Method.invoke(Method.java:498)

                        at gov.nasa.pds.tools.validate.rule.AbstractValidationRule.execute(AbstractValidationRule.java:61)

                        at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:191)

                        at gov.nasa.pds.tools.validate.task.ValidationTask.execute(ValidationTask.java:129)

                        at gov.nasa.pds.tools.validate.task.BlockingTaskManager.submit(BlockingTaskManager.java:27)

                        at gov.nasa.pds.tools.label.LocationValidator.validate(LocationValidator.java:141)

                        at gov.nasa.pds.validate.ValidateLauncher.doValidation(ValidateLauncher.java:729)

                        at gov.nasa.pds.validate.ValidateLauncher.processMain(ValidateLauncher.java:912)

                        at gov.nasa.pds.validate.ValidateLauncher.main(ValidateLauncher.java:936)

Oct 20, 2017 5:47:48 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T00:47:45Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  FAIL: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

      ERROR  URI reference does not exist: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.tab

    Begin Content Validation: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.tab

      ERROR  table 1: Error occurred while trying to read table: /Users/rchen/Desktop/test/bundle_psi/data/science_index.tab (No such file or directory)

    End Content Validation: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.tab

Summary:

  1 of 1 file(s) processed, 0 skipped

  0 of 1 file(s) passed validation

End of Report

Step 4: the referenced file was restored in step 3

Oct 20, 2017 5:53:22 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

Oct 20, 2017 5:53:22 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml'

Oct 20, 2017 5:53:24 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T00:53:22Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  PASS: file:/Users/rchen/Desktop/test/bundle_psi/data/science_index.xml

Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

 Step 5: the first difference (trailing /) doesn’t matter. The second does.

94c94

<          <directory_path_name>meca_rdr_sis_files/</directory_path_name>

---

>          <directory_path_name>meca_rdr_sis_files</directory_path_name>

103c103

<          <directory_path_name>/replaceWithFullPath</directory_path_name>

---

>          <directory_path_name> testDir/testPrep/product_document/meca_rdr_sis_files</directory_path_name>

Step 6: _bad.xml should fail because of a non-existent path.

Oct 20, 2017 5:57:18 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_bad.xml

Oct 20, 2017 5:57:18 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_bad.xml'

Oct 20, 2017 5:57:20 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_bad.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T00:57:17Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_bad.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  FAIL: file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_bad.xml

      ERROR  URI reference does not exist: file:/replaceWithFullPath/image002.gif

 Summary:

  1 of 1 file(s) processed, 0 skipped

  0 of 1 file(s) passed validation

End of Report

Step 7: non-existent path has been replaced by a real path

Oct 20, 2017 5:58:16 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml

Oct 20, 2017 5:58:16 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml'

Oct 20, 2017 5:58:18 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T00:58:16Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  PASS: file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml

 Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

Comments

Results met success criteria.

 

Build 8a opened PDS-522, which requests a less brutal error message in step 2

Date of Testing

2017.10.20

Test Personnel

Richard Chen

 

Test Case ID

PRV.3

Description

Validate aggregate products’ integrity of LID references. This differs from validating a directory by 1) hierarchically validating based on bundle.xml’s lid_reference and/or collection.tab’s inventory, and 2) validating only the references, not other syntax.

Requirements

PASS L5.PRP.VA.3: The tool shall validate aggregate products and all products referenced by such products.

 

Success Criteria

Find no bugs in bundle_geo_ra/.  Find some bugs in bundleLID/.

Test Steps

1.     cd testDir/

2.     validate -R pds4.bundle -t bundle_geo_ra -f

3.     validate -R pds4.bundle -t bundleLID -f
where bundleLID == bundle_geo_ra with some files removed except:

a)    data_test/data_test_collection_1.xml has another collection’s logical_identifier

b)    context/context_collection_1.xml has a typo in its logical_identifier

c)     data_derived/sol149b.xml has another product’s LIDVID

d)    data_test/scraping/pit_test_scraping_pic[12].xml have the same LID but different VIDs, which should not be flagged.

e)     data_test/scraping/pit_test_scraping.xml’s LID changed to “<xxx>.xml”

f)     data_test/scraping/pit_test_scraping.xml’s <title> added various spaces

4.     validate -R pds4.bundle -t bundle_geo_ra/bundle_1.xml
Integrity checking only 1 file is illogical since references point to other files

Test Results

Step 2:

Oct 20, 2017 9:21:09 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml

Oct 20, 2017 9:21:09 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml'

Oct 20, 2017 9:21:11 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_good.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T04:21:08Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_geo_ra/]

   Rule Type                     pds4.bundle

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  PASS: file:/Users/rchen/Desktop/test/bundle_geo_ra/bundle_1.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_geo_ra/context/context_collection_1.xml

[snip…]

  PASS: file:/Users/rchen/Desktop/test/bundle_geo_ra/xml_schema/collection.xml

  PASS: file:/Users/rchen/Desktop/test/bundle_geo_ra/xml_schema/PDS4_PDS_1800.xml

Summary:

  173 of 173 file(s) processed, 0 skipped

  173 of 173 file(s) passed validation

End of Report

Step 3: flagging duplicate u:n:p:phx_ra:data_derived:sol149a::1.0 and not flagging pit_test_scraping_pic[23].xml are correct.

Oct 20, 2017 11:47:31 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Bundle' for location file:/Users/rchen/Desktop/test/bundleLID/

Oct 20, 2017 11:47:31 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/bundleLID/'

Oct 20, 2017 11:47:37 PM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/bundleLID/'

PDS Validate Tool Report

 

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T06:47:31Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundleLID/]

   Rule Type                     pds4.bundle

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  PASS: file:/Users/rchen/Desktop/test/bundleLID/bundle_1.xml

      WARNING  The member 'urn:nasa:pds:phx_ra:data_test' could not be found in any product within the given target.

      WARNING  The member 'urn:nasa:pds:phx_ra:context' could not be found in any product within the given target.

  PASS: file:/Users/rchen/Desktop/test/bundleLID/context/context_collection_1.xml

      WARNING  Identifier 'urn:nasa:pds:phx_ra:context_typo::1.0' is not a member of any collection within the given target

  PASS: file:/Users/rchen/Desktop/test/bundleLID/context/PDS4_host_PHX_1.0.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/context/PDS4_inst_RA__PHX.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/context/PDS4_mission_PHOENIX_1.0.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/context/PDS4_target_MARS_1.0.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_derived/data_derived_collection_1.xml

      WARNING  The member 'urn:nasa:pds:phx_ra:data_derived:sol149b::1.0' could not be found in any product within the given target.

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_derived/sol006.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_derived/sol149a.xml

  FAIL: file:/Users/rchen/Desktop/test/bundleLID/data_derived/sol149b.xml

      ERROR  Identifier urn:nasa:pds:phx_ra:data_derived:sol149a::1.0 already defined (old location: file:/Users/rchen/Desktop/test/bundleLID/data_derived/sol149a.xml)

  FAIL: file:/Users/rchen/Desktop/test/bundleLID/data_test/data_test_collection_1.xml

      ERROR  Identifier urn:nasa:pds:phx_ra:data_derived::1.0 already defined (old location: file:/Users/rchen/Desktop/test/bundleLID/data_derived/data_derived_collection_1.xml)

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/duricrust/pit_test_duricrust.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/duricrust/pit_test_duricrust_dig1_pic1.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/duricrust/pit_test_duricrust_dig2_pic13.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/scraping/pit_test_scraping.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/scraping/pit_test_scraping_pic1.xml

  PASS: file:/Users/rchen/Desktop/test/bundleLID/data_test/scraping/pit_test_scraping_pic2.xml

 Summary:

  17 of 17 file(s) processed, 0 skipped

  15 of 17 file(s) passed validation

End of Report

Step 4:

Oct 20, 2017 11:49:47 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Bundle' for location file:/Users/rchen/Desktop/test/bundle_geo_ra/bundle_1.xml

Oct 20, 2017 11:49:47 PM gov.nasa.pds.tools.label.LocationValidator validate

SEVERE: 'PDS4 Bundle' validation style is not applicable for location file:/Users/rchen/Desktop/test/bundle_geo_ra/bundle_1.xml

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T06:49:47Z

   Core Schemas                  [PDS4_PDS_1900.xsd]

   Core Schematrons              [PDS4_PDS_1900.sch]

   Model Version                 1900

 Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/bundle_geo_ra/bundle_1.xml]

   Rule Type                     pds4.bundle

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

Summary:

  0 of 0 file(s) processed, 0 skipped

  0 of 0 file(s) passed validation

End of Report

Comments

Results met success criteria.

Date of Testing

2017.10.20

Test Personnel

Richard Chen

 

Test Case ID

PRV.4

Description

Merge label fragments

Requirements

PASS L5.PRP.VA.4: The tool shall merge the contents of label fragments referenced by include elements with the contents of the parent label when validating a product.

Success Criteria

As of build 7b, this functionality (merging via xinculde) is gone, i.e. this should fail. The requirement above no longer applies, for PDS-488, removes support for xinclude.

Test Steps

1.     cd testDir/testPrep/product_document

2.     validate -t Product_Doc_part1.xml -f

Test Results

Step 2:

Oct 21, 2017 12:01:45 AM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Label' for location file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_part1.xml

Oct 21, 2017 12:01:45 AM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_part1.xml'

Oct 21, 2017 12:01:47 AM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_part1.xml'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T07:01:44Z

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_part1.xml]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

Validation Details:

  FAIL: file:/Users/rchen/Desktop/test/testPrep/product_document/Product_Doc_part1.xml

      ERROR  line 96, 56: cvc-complex-type.2.4.a: Invalid content was found starting with element 'xi:include'. One of '{"http://pds.nasa.gov/pds4/pds/v1":comment, "http://pds.nasa.gov/pds4/pds/v1":directory_path_name, "http://pds.nasa.gov/pds4/pds/v1":document_standard_id}' is expected.

      ERROR  URI reference does not exist: file:/Users/rchen/Desktop/test/testPrep/product_document/image001.gif

Summary:

  1 of 1 file(s) processed, 0 skipped

  0 of 1 file(s) passed validation

End of Report

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

PRV.5

Description

Validate schemas

Requirements

PASS L5.PRP.VA.8: The tool shall verify that a schema file is valid.

Success Criteria

Validation tool verifies whether a schema is well formed.

Test Steps

The validate tool does not accept a schema as its target, i.e. this does not work:

validate PDS4_PDS_1800.xsd

However, validate, when validating a label file, does complain when the schema is bad

1.     validate bundle_geo_ra/bundle_1.xml -x PDS4_PDS_1800.xsd -S PDS4_PDS_1800.sch

2.     diff PDS4_PDS_1800.xsd testPrep/PDS4_PDS_1800.bad.xsd

3.     validate bundle_geo_ra/bundle_1.xml -x testPrep/PDS4_PDS_1800.bad.xsd  -S PDS4_PDS_1800.sch

Validate also threw a null pointer exception when given a non-existent schema

4.     validate bundle_geo_ra/bundle_1.xml -x xxx.xsd -S PDS4_PDS_1700.sch

Test Results

Step 1: a normal validation with a schema specified on the command line

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T07:09:42Z

Parameters:

   Targets                       [file:testDir/bundle_geo_ra/bundle_1.xml]

   User Specified Schemas        [file:testDir/PDS4_PDS_1800.xsd]

   User Specified Schematrons    [file:testDir/PDS4_PDS_1800.sch]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  PASS: file:testDir/bundle_geo_ra/bundle_1.xml

Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

Step 2: the first schema is good; the second has an important line commented out

15c15

<   </xs:annotation>

---

>   <!--/xs:annotation-->

Step 3: validate fails because of the schema’s syntactic problems

Error while getting targetNamespace of schema 'file: testDir/testPrep/PDS4_PDS_1700.bad.xsd': org.xml.sax.SAXParseException; systemId: file: testDir/testPrep/PDS4_PDS_1700.bad.xsd; lineNumber: 7887; columnNumber: 3; The element type "xs:annotation" must be terminated by the matching end-tag "</xs:annotation>".

Step 4: validate throws a normal java exception. Weird copy/paste below into Word.

java.io.FileNotFoundException: /Users/rchen/Desktop/test/xxx.xsd (No such file or directory)

................................................................................................... at java.io.FileInputStream.open0(Native Method)

..................................................................................... at java.io.FileInputStream.open(FileInputStream.java:195)

................................................................................... at java.io.FileInputStream.<init>(FileInputStream.java:138)

..................................................................................... at java.io.FileInputStream.<init>(FileInputStream.java:93)

.................................. at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)

.................. at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)

.......................................................................... at gov.nasa.pds.tools.util.Utility.openConnection(Utility.java:74)

......................................................................................................................................................................................... at gov.nasa.pds.tools.label.CachedLSResourceResolver.resolveResource(CachedLSResourceResolver.java:175)

.............................. at gov.nasa.pds.validate.ValidateLauncher.validateSchemas(ValidateLauncher.java:805)

..................................... at gov.nasa.pds.validate.ValidateLauncher.processMain(ValidateLauncher.java:890)

................................................... at gov.nasa.pds.validate.ValidateLauncher.main(ValidateLauncher.java:932)

null

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

PRV.6

Description

Accept schema file specified by file or directory

Requirements

PASS L5.PRP.VA.7: The tool shall accept the following as input for specifying the associated schema file(s)…

Success Criteria

Label file validates against the schema specified.

Test Steps

Besides PRV.1 and PRV.5:

1.     cd testDir/testPrep

2.     validate 376i750800r.xml -x 376PDS4_PDS_0300a.xsd 376imaging_dictionary.xsd 376mpf_dictionary.xsd

Test Results

Step 2:

PDS Validate Tool Report

Configuration:

      Version                       1.12.0

   Date                          2017-10-21T07:13:56Z

   Core Schematrons              [PDS4_PDS_1900.sch]

   Model Version                 1900

 Parameters:

   Targets                       [file: testDir/testPrep/376i750800r.xml]

   User Specified Schemas        [file: testDir/testPrep/376PDS4_PDS_0300a.xsd, file: testDir/testPrep/376imaging_dictionary.xsd, file: testDir/testPrep/376mpf_dictionary.xsd]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

  PASS: file: testDir/testPrep/376i750800r.xml

Summary:

  1 of 1 file(s) processed, 0 skipped

  1 of 1 file(s) passed validation

End of Report

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

PRV.7

Description

Test various capabilities described in release documents:

·       Validate multiple files via a checksum manifest.

·       Do not duplicate warning messages.

·       -R pds4.collection and pds4.bundle give appropriate error messages

Requirements

PASS L5.PRP.VA.3: The tool shall validate aggregate products and all products referenced by such products.

 

Success Criteria

Besides validating w.r.t. the schema, catch the 1 bad checksum

Test Steps

1.     cd testDir/testPrep

Validate multiple files via a checksum manifest

2.     validate -m1400 -t cara_data -M cara_data/checksum.tab

3.     cp cara_data/checksum.tab /tmp

4.     validate -m1400 -t cara_data -M /tmp/checksum.tab

Do not duplicate warning messages.          

5.     validate -f -t vld_415

-R pds4.bundle and pds4.collection

6.     validate -m1400 -t cara_data -R pds4.collection

7.     validate -m1400 -t cara_data/cara_documents -R pds4.bundle

Test Results

Step 2: The one error is deliberate to demonstrate that the rest are working

Oct 21, 2017 1:02:07 AM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Directory' for location file:/Users/rchen/Desktop/test/testPrep/cara_data/

Oct 21, 2017 1:02:07 AM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Starting validation task for location 'file:/Users/rchen/Desktop/test/testPrep/cara_data/'

Oct 21, 2017 1:02:10 AM gov.nasa.pds.tools.validate.task.ValidationTask execute

INFO: Validation complete for location 'file:/Users/rchen/Desktop/test/testPrep/cara_data/'

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T08:02:07Z

   Core Schemas                  [PDS4_PDS_1400.xsd]

   Core Schematrons              [PDS4_PDS_1400.sch]

   Model Version                 1400

 Parameters:

   Targets                       [file:/PDS/PDS4test.build6b/testPrep/cara_data/]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

   Checksum Manifest File        file:/Users/rchen/Desktop/test/testPrep/cara_data/checksum.tab

   Manifest File Base Path       file:/Users/rchen/Desktop/test/testPrep/cara_data/

 Validation Details:

  PASS: file:testDir/testPrep/cara_data/bundle.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/collection.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/116P.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/174P.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/29P.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/67P.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/C2004Q2.xml

  PASS: file:testDir/testPrep/cara_data/cara_data/data/C2013A1.xml

  FAIL: file:testDir/testPrep/cara_data/cara_documents/cara_observer_code.xml

      ERROR  Generated checksum '4cba3db065570c5c85fa0907574e6f04' does not match supplied checksum '00000000000000000000000000000000' in the manifest for 'file:testDir/testPrep/cara_data/cara_documents/cara_observer_code.xml'.

  PASS: file:testDir/testPrep/cara_data/cara_documents/collection.xml

Summary:

  10 of 10 file(s) processed, 0 skipped

  9 of 10 file(s) passed validation

End of Report

Step 4: This failed before 6b. Works now, i.e. same output as step 2.

Step 5: At one time, the 2nd warning would have 2 lines; the 3rd, three.

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-21T08:51:02Z

Parameters:

   Targets                       [file: testDir/testPrep/vld_415/]

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    on

   Referential Integrity Check   off

Validation Details:

  PASS: file:testDir/testPrep/vld_415/bundle.xml

  PASS: file:testDir/testPrep/vld_415/cara_data/collection.xml

  PASS: file:testDir/testPrep/vld_415/cara_data/data/116P.xml

      WARNING  File reference'/Users/rchen/Desktop/test/testPrep/vld_415/cara_data/data/116p.tab' exists but the case doesn't match

  PASS: file:testDir/testPrep/vld_415/cara_data/data/174P.xml

      WARNING  File reference'testDir/testPrep/vld_415/cara_data/data/174p.tab' exists but the case doesn't match.

  PASS: file:testDir/testPrep/vld_415/cara_data/data/C2004Q2.xml

      WARNING  File reference'testDir/testPrep/vld_415/cara_data/data/C2004q2.tab' exists but the case doesn't match.

Summary:

  5 of 5 file(s) processed, 0 skipped

  5 of 5 file(s) passed validation

End of Report

Step 6: can’t use -R pds4.bundle on a collection

Oct 22, 2017 11:08:33 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Bundle' for location file:/Users/rchen/Desktop/test/testPrep/cara_data/cara_documents/

Oct 22, 2017 11:08:33 PM gov.nasa.pds.tools.label.LocationValidator validate

SEVERE: 'PDS4 Bundle' validation style is not applicable for location file:/Users/rchen/Desktop/test/testPrep/cara_data/cara_documents/

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-23T06:08:33Z

   Core Schemas                  [PDS4_PDS_1400.xsd]

   Core Schematrons              [PDS4_PDS_1400.sch]

   Model Version                 1400

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testPrep/cara_data/cara_documents/]

   Rule Type                     pds4.bundle

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

Summary:

  0 of 0 file(s) processed, 0 skipped

  0 of 0 file(s) passed validation

End of Report

Step 7: can’t use -R pds4.collection on a bundle

Oct 22, 2017 11:08:08 PM gov.nasa.pds.tools.label.LocationValidator validate

INFO: Using validation style 'PDS4 Collection' for location file:/Users/rchen/Desktop/test/testPrep/cara_data/

Oct 22, 2017 11:08:08 PM gov.nasa.pds.tools.label.LocationValidator validate

SEVERE: 'PDS4 Collection' validation style is not applicable for location file:/Users/rchen/Desktop/test/testPrep/cara_data/

PDS Validate Tool Report

Configuration:

   Version                       1.12.0

   Date                          2017-10-23T06:08:08Z

   Core Schemas                  [PDS4_PDS_1400.xsd]

   Core Schematrons              [PDS4_PDS_1400.sch]

   Model Version                 1400

Parameters:

   Targets                       [file:/Users/rchen/Desktop/test/testPrep/cara_data/]

   Rule Type                     pds4.collection

   Severity Level                WARNING

   Recurse Directories           true

   File Filters Used             [*.xml, *.XML]

   Force Mode                    off

Validation Details:

Summary:

  0 of 0 file(s) processed, 0 skipped

  0 of 0 file(s) passed validation

End of Report

Comments

Results met success criteria.

 

Steps 6 and 7 clear https://oodt.jpl.nasa.gov/jira/browse/PDS-501.

Date of Testing

2017.10.20

Test Personnel

Richard Chen

 

 

Test Case ID

REG.1

Description

Validate and accept metadata to register an artifact or modify an artifact’s registration, query for a registered artifact, delete a registered artifact. Use the REST-based API.

Requirements

PASS L5.REG.1: The service shall accept artifact registrations.

PASS L5.REG.4: The service shall accept metadata for a registered artifact in a defined format.

PASS L5.REG.5: The service shall validate metadata for a registered artifact.

PASS L5.REG.13: The service shall allow deletion of registered artifacts.

PASS L5.REG.14: The service shall allow queries for registered artifacts.

PASS L5.GEN.3: The system shall generate metrics regarding performance and activity.

Success Criteria

Registry service validates and accepts metadata for an artifact in a defined format, consistent with the appropriate schema for the artifact. Registering an Inventory artifact should allow locating and auditing the artifact. Registering a Dictionary artifact should be reflected in the Information Model. Registering a Document artifact, e.g. a schema, should store the file and make the document available. Registering a Service artifact should document and promote the service.  Query and delete artifacts or provide error messages for unrecognized artifacts.

Test Steps

Clean database as described in RESETREGISTRY in Section 3.1

1.     cd testDir

2.     http://localhost:8080/registry-pds3/extrinsics/logicals/testing.REG.1

in a browser shows no current product has lid “testing.REG.1”, which input files test.REG.1[abc].xml have.

3.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.1a.xml http://localhost:8080/registry-pds3/extrinsics

attempts to register the bad input file

4.     Repeat step 2 to ensure lid still does not exist.

5.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.1b.xml http://localhost:8080/registry-pds3/extrinsics

registers a good input file

6.     Repeat step 2 to see the lid

7.     At http://localhost:8080/registry-ui/, select “Product 1234 v1”, Delete, OK

8.     Repeat step 2 to ensure lid no longer exists

Test Results

Step 2: The error message should be (if lid does exist, run step 7):Macintosh HD:Users:rchen:Desktop:Screen Shot 2014-10-19 at 12.00.52 AM.png

Step 3:

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/extrinsics HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 653

* upload completely sent off: 653 out of 653 bytes

< HTTP/1.1 400

< Content-Length: 0

< Date: Tue, 17 Oct 2017 16:18:51 GMT

< Connection: close                             

* Closing connection 0

Step 4: Same as step 2

Step 5:

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/extrinsics HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 629

* upload completely sent off: 629 out of 629 bytes

< HTTP/1.1 201

< Location: http://localhost:8080/registry-pds3/extrinsics/testing.REG.1.v1.0

< Content-Type: application/xml

< Transfer-Encoding: chunked

< Date: Tue, 17 Oct 2017 16:19:30 GMT

testing.REG.1.v1.0

Step 6: Upon success, the registry service returns good xml. In firefox: Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.08.09 AM.png

Step 7: Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-04-18 at 10.13.01 PM.png

Step 8: Same as step 2

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

REG.2

Description

Relate artifact registrations. Query and delete such associations.

Requirements

PASS L5.REG.2: The service shall provide a means for relating artifact registrations.

PASS L5.REG.13: The service shall allow deletion of registered artifacts.
PASS L5.REG.14: The service shall allow queries for registered artifacts.

Success Criteria

Registry service relates together multiple artifacts during their registrations, whether as a batch or as individual registrations.  The associations are removed from the registry after deletion.

Test Steps

 

 

                     

1.     http://localhost:8080/registry-pds3/associations/linkREG.1toREG.2a

shows no such associations                                                                                   

2.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.2.xml http://localhost:8080/registry-pds3/associations

adds 1 association (from a nonexistent sourceLid) to desired targetLid

3.     Repeat step 1 to see the association.

4.     curl -X DELETE -v http://localhost:8080/registry-pds3/associations/linkREG.1toREG.2a

5.     Repeat step 1 to see no association

Test Results

Step 1: The error should look likeMacintosh HD:Users:rchen:Desktop:Screen Shot 2014-10-19 at 12.07.24 AM.png

If not (i.e. if output looks like step 3’s below), delete as is Test Step 4.

Step 2: Benign output messages without “ERROR”

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/associations HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 225

* upload completely sent off: 225 out of 225 bytes

< HTTP/1.1 201

< Location: http://localhost:8080/registry-pds3/associations/linkREG.1toREG.2a

< Content-Type: text/plain

< Transfer-Encoding: chunked

< Date: Tue, 17 Oct 2017 17:51:51 GMT

 * Connection #0 to host localhost left intact

linkREG.1toREG.2a

Step 3:../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2010

Step 4:

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> DELETE /registry-pds3/associations/linkREG.1toREG.2a HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

< HTTP/1.1 200

< Content-Length: 0

< Date: Tue, 17 Oct 2017 17:53:02 GMT

* Connection #0 to host localhost left intact

Step 5: same as step 1

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

REG.3

Description

Maintain policies for classes of artifacts, i.e. all classes of artifacts capture a base set of metadata, in the form of XML attributes: objectType, guid as well as metadata specific to each artifact class.

Requirements

PASS L5.REG.3: The system shall register products of a data delivery into an instance of the registry.

Success Criteria

Registry service defines separate policies for each class of artifact. Changes to the policies of a class can reflect in the validation of a registered artifact in that class.

Test Steps

In a browser,

1.     http://localhost:8080/registry-pds3/extrinsics (e.g. REG.1’s step 5)

2.     http://localhost:8080/registry-pds3/associations (e.g. REG.2’s step 2)

3.     http://localhost:8080/registry-pds3/services

4.     http://localhost:8080/registry-pds3/schemes

5.     http://localhost:8080/registry-pds3/events

6.     http://localhost:8080/registry-pds3/packages

7.     http://localhost:8080/registry-pds3/docs/

From a command line (to show the registry version),

8.     curl -X GET -H "Accept:application/xml" -v http://localhost:8080/registry-pds3/report

Test Results

Specific results will differ, but every item in every class has XML attributes objectType, guid. Each class may have others attributes such as lid, name, home.

Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.25.23 AM.png

Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.27.51 AM.png

Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.26.11 AM.png

Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.27.06 AM.png

../Screen%20Shot%202017-04-20%20at%2014.05.21.png

Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.29.55 AM.png

Step 7 shows correct output formattingMacintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-21 at 11.43.34 PM.png

Step 8:

Note: Unnecessary use of -X or --request, GET is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> GET /registry-pds3/report HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept:application/xml

< HTTP/1.1 200

< Content-Type: application/xml

< Content-Length: 314

< Date: Tue, 17 Oct 2017 18:16:52 GMT

* Connection #0 to host localhost left intact

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:report xmlns:ns2="http://registry.pds.nasa.gov" status="OK" serverStarted="2017-10-17T09:06:20.399-07:00" associations="71" extrinsics="0" services="0" classificationSchemes="2" classificationNodes="69" packages="4" events="79" registryVersion="1.12.0"/>

Comments

Results met success criteria.

Date of Testing

2017.10.17

Test Personnel

Richard Chen

 

Test Case ID

REG.4

Description

Assign a global unique identifier to a registered artifact with no global unique identifier, query for the registered artifact, delete the registered artifact.

Requirements

PASS L5.REG.6: The service shall assign a global unique identifier to a registered artifact.

PASS L5.REG.13: The service shall allow deletion of registered artifacts.

PASS L5.REG.14: The service shall allow queries for registered artifacts.

Success Criteria

Registry service assigns each registered artifact, including multiple versions of an artifact, a global unique identifier.

Test Steps

1.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.4.xml http://localhost:8080/registry-pds3/extrinsics

From the output, copy the GUID (assigned by the registry) from the last line               

2.     http://localhost:8080/registry-pds3/extrinsics/guid

3.     To be nice, delete via http://localhost:8080/registry-ui

Test Results

Step 1: Note the value (an assigned LID) of “Location:” in the positive message:

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/extrinsics HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 668

* upload completely sent off: 668 out of 668 bytes

< HTTP/1.1 201

< Location: http://localhost:8080/registry-pds3/extrinsics/urn:uuid:8f26bc46-9149-4dec-a0a4-f8c9af570a78

< Content-Type: application/xml

< Transfer-Encoding: chunked

< Date: Sat, 21 Oct 2017 08:57:24 GMT

 * Connection #0 to host localhost left intact

urn:uuid:8f26bc46-9149-4dec-a0a4-f8c9af570a78

Step 2:../Screen%20Shot%202017-10-21%20at%2002.03.25.png

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

REG.5

Description

Assign a version to a registered artifact based on its unique identifier

Requirements

PASS L5.REG.7: The service shall assign a version to a registered artifact based on its logical identifier.

Success Criteria

Registry service assigns each registered artifact, especially multiple versions of an artifact, a version identifier, derivable from its logical identifier.

Test Steps

1.     http://localhost:8080/registry-pds3/extrinsics/logicals/testing.REG.2b

shows no current product with lid “testing.REG.2b”

2.     Register a product with no versionId attribute
curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.5a.xml http://localhost:8080/registry-pds3/extrinsics

3.     Repeat step 1. Note that versionName is 1.0

Note versionName is independent of extrinsicObject’s versionId, name, guid.

Test Results

Step 1: Macintosh HD:Users:rchen:Desktop:Screen Shot 2014-10-19 at 12.52.53 AM.png

Step 2:

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/extrinsics HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 641

* upload completely sent off: 641 out of 641 bytes

< HTTP/1.1 201

< Location: http://localhost:8080/registry-pds3/extrinsics/urn:uuid:6a773b96-1b73-48bd-88a3-515367fb3607

< Content-Type: application/xml

< Transfer-Encoding: chunked

< Date: Sat, 21 Oct 2017 09:07:23 GMT

 * Connection #0 to host localhost left intact

urn:uuid:6a773b96-1b73-48bd-88a3-515367fb3607

 Step 3: Note that versionName=1.0 even though input file had no versionId attribute

../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2010

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

REG.6

Description

Allow replacement, approval, deprecation, undeprecation, and verification of registered artifacts. The Tomcat server access log lists the search.

Requirements

PASS L5.REG.9: The service shall allow updates to registered artifacts.

PASS L5.REG.10: The service shall allow approval of registered artifacts.

PASS L5.REG.11: The service shall allow deprecation of registered artifacts.

PASS L5.REG.12: The service shall allow undeprecation of registered artifacts.

PASS L5.GEN.6: Applications shall generate metrics in a format suitable for ingestion by the Report Service.

Success Criteria

Registry service provides these standard functions with expected results. Initial registration results in an artifact being in an unapproved state. Also, the Tomcat server access log lists the actions.

Test Steps

1.     Show that no current product has LID “testing.REG.2a”:
http://localhost:8080/registry-pds3/extrinsics/logicals/testing.REG.2a

2.     curl -X POST -H "Content-type:application/xml" -v -d @testRegistry/test.REG.6a.xml http://localhost:8080/registry-pds3/extrinsics

3.     Repeat step 1 to see the lid

An alternative to the steps below is to modify via GUID

curl -X POST -H "Content-type:application/xml" -v http://localhost:8080/registry-pds3/extrinsics/testing.REG.2a.v1.0/{approve,deprecate,undeprecate}

4.     In a browser,  http://localhost:8080/registry-ui/

Under “LID”, enter “testing.REG.2a”. Click “Refresh”

5.     Select row, set “Status” to “Approved”, hit “Update Status”

6.     Select row, set “Status” to “Deprecated”, hit “Update Status”

7.     Select row, set “Status” to “Submitted”, hit “Update Status”

The above actions get into the Tomcat server log, which the report service can process.

8.     grep testing.REG.2a $CATALINA_HOME/logs/localhost_access_log.yyyy-mm-dd.txt

Test Results

Step 1: If browser isn’t as below, delete using http://localhost:8080/registry-ui

Macintosh HD:Users:rchen:Desktop:Screen Shot 2014-10-19 at 12.57.51 AM.png

Step 2: Benign output messages without “ERROR”

Note: Unnecessary use of -X or --request, POST is already inferred.

*   Trying ::1...

* Connected to localhost (::1) port 8080 (#0)

> POST /registry-pds3/extrinsics HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

> Content-type:application/xml

> Content-Length: 645

* upload completely sent off: 645 out of 645 bytes

< HTTP/1.1 201

< Location: http://localhost:8080/registry-pds3/extrinsics/testing.REG.2a.v1.0

< Content-Type: application/xml

< Transfer-Encoding: chunked

< Date: Sat, 21 Oct 2017 09:27:52 GMT

 * Connection #0 to host localhost left intact

Step 3: ../../Users/rchen/Desktop/Screen%20Shot%202016-04-21%20at%2011

Step 4: Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.55.22 AM.png

Step 5: Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.55.48 AM.png

Step 6: Macintosh HD:Users:rchen:Desktop:Screen Shot 2015-10-16 at 1.57.56 AM.png

Step 7: same as Step 4

Step 8:

0:0:0:0:0:0:0:1 - - [21/Oct/2017:02:27:37 -0700] "GET /registry-pds3/extrinsics/logicals/testing.REG.2a HTTP/1.1" 404 68

0:0:0:0:0:0:0:1 - - [21/Oct/2017:02:27:55 -0700] "GET /registry-pds3/extrinsics/logicals/testing.REG.2a HTTP/1.1" 200 776

127.0.0.1 - - [21/Oct/2017:02:30:22 -0700] "GET /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 720

127.0.0.1 - - [21/Oct/2017:02:30:22 -0700] "POST /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 -

127.0.0.1 - - [21/Oct/2017:02:30:36 -0700] "GET /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 719

127.0.0.1 - - [21/Oct/2017:02:30:36 -0700] "POST /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 -

127.0.0.1 - - [21/Oct/2017:02:31:04 -0700] "GET /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 721

127.0.0.1 - - [21/Oct/2017:02:31:04 -0700] "POST /registry-pds3/extrinsics/testing.REG.2a.v1.0 HTTP/1.1" 200 -

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

REG.7 *not ready for build 8a. This is reserved for future testing

Description

Enable replication of registry contents.

Requirements

SKIP L5.REG.15: The service shall enable replication of registry contents with another instance of the service.

Success Criteria

Contents of the registry are duplicated on a separate machine.

Test Steps

 

Test Results

 

Comments

 

Date of Testing

 

Test Personnel

 

 

Test Case ID

REG.8  *not ready for build 5a. This is reserved for future testing

Description

Verify registry contents.

Requirements

SKIP L5.REG.16: The service shall enable verification of registry contents.

Success Criteria

Contents and checksums of the registry artifacts match what have been ingested.

Test Steps

 

Test Results

 

Comments

 

Date of Testing

 

Test Personnel

 

 

Test Case ID

REG.9

Description

Test scalability of registry.

Requirements

PASS No specific functional requirement. This is a performance test case.

Success Criteria

Performance of registry in ingesting and viewing artifacts remains acceptable under logarithmic increases in volume.

Test Steps

This requires python to be installed on the local machine and possible editing of stressTest.py to change the location of the python executable. 

Step 3 is configurable. 5M registrations for build 6a took 16 hours.

1.     cd testDir/bin

2.     ./stressTest.py

3.     ./stressTest.py -v -n500000 > ../out.txt

4.     grep Time ../out.txt

Test Results

Step 2:

stressTesting.T000000.v1.0

stressTesting.T000001.v1.0

stressTesting.T000002.v1.0

REGSTR      3 good. Time(sec): avg=0.011 median=0.008 stdDev=0.00571 sum=0.0

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:extrinsicObject xmlns:ns2="http://registry.pds.nasa.gov" lid="stressTesting.T000000" name="Stress The Registry " objectType="Product" status="Submitted" description="Stolen from http://pdscm/2010/registry/registry-service/operate/index.html with attribute lid above changed for uniqueness" versionName="1.0" guid="stressTesting.T000000.v1.0" home="http://localhost:8080/registry"><ns2:slot id="147" name="last-name"><ns2:value>Doe</ns2:value></ns2:slot><ns2:slot id="148" name="cannotPossibleBeAnExistingSlot"><ns2:value>cannot possibly be an existing value</ns2:value></ns2:slot><ns2:slot id="149" name="phone"><ns2:value>(818)777-7777</ns2:value><ns2:value>(818)888-8888</ns2:value></ns2:slot><ns2:slot id="150" name="first-name"><ns2:value>John</ns2:value></ns2:slot></ns2:extrinsicObject>

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:extrinsicObject xmlns:ns2="http://registry.pds.nasa.gov" lid="stressTesting.T000001" name="Stress The Registry " objectType="Product" status="Submitted" description="Stolen from http://pdscm/2010/registry/registry-service/operate/index.html with attribute lid above changed for uniqueness" versionName="1.0" guid="stressTesting.T000001.v1.0" home="http://localhost:8080/registry"><ns2:slot id="152" name="last-name"><ns2:value>Doe</ns2:value></ns2:slot><ns2:slot id="153" name="cannotPossibleBeAnExistingSlot"><ns2:value>cannot possibly be an existing value</ns2:value></ns2:slot><ns2:slot id="154" name="phone"><ns2:value>(818)777-7777</ns2:value><ns2:value>(818)888-8888</ns2:value></ns2:slot><ns2:slot id="155" name="first-name"><ns2:value>John</ns2:value></ns2:slot></ns2:extrinsicObject>

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:extrinsicObject xmlns:ns2="http://registry.pds.nasa.gov" lid="stressTesting.T000002" name="Stress The Registry " objectType="Product" status="Submitted" description="Stolen from http://pdscm/2010/registry/registry-service/operate/index.html with attribute lid above changed for uniqueness" versionName="1.0" guid="stressTesting.T000002.v1.0" home="http://localhost:8080/registry"><ns2:slot id="157" name="last-name"><ns2:value>Doe</ns2:value></ns2:slot><ns2:slot id="158" name="cannotPossibleBeAnExistingSlot"><ns2:value>cannot possibly be an existing value</ns2:value></ns2:slot><ns2:slot id="159" name="phone"><ns2:value>(818)777-7777</ns2:value><ns2:value>(818)888-8888</ns2:value></ns2:slot><ns2:slot id="160" name="first-name"><ns2:value>John</ns2:value></ns2:slot></ns2:extrinsicObject>

VIEW        3 good. Time(sec): avg=0.006 median=0.006 stdDev=0.00015 sum=0.0

stressTesting.T000000.v1.0 deleted

stressTesting.T000001.v1.0 deleted

stressTesting.T000002.v1.0 deleted

DELETE      3 good. Time(sec): avg=0.008 median=0.007 stdDev=0.00138 sum=0.0

 Step 4: In 5b, the sums for 500,000 were 2348.7, 10706.9, 2476.9.

REGSTR 500000 good. Time(sec): avg=0.006 median=0.005 stdDev=0.00679 sum=2774.9

VIEW   500000 good. Time(sec): avg=0.005 median=0.004 stdDev=0.00895 sum=2297.7

DELETE 500000 good. Time(sec): avg=0.005 median=0.005 stdDev=0.00670 sum=2740.1

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

Test Case ID

REG.10

Description

Access registry via the registry client API (pds.registry)

Requirements

PASS L5.GEN.3: Services shall have an application programming interface.

PASS L5.REG.14: The service shall allow queries for registered artifacts.

Success Criteria

Contents and checksums of the registry artifacts match what have been ingested.

Test Steps

To setup the registry client, which only needs to be done once,

cd <wherever the .tar.gz file was expanded>

sudo python setup.py install

Clean the database as described in RESETREGISTRY in Section 3.1. Then harvest two files and see their associations listed via the registry client

1.     cd testDir

2.     bin/registry-file-list.py http://localhost:8080/registry-pds4

3.     harvest testDir/bundle_geo_ra/data_test/scraping -c harvest-policy-master.xml -e "*.xml" -l h.out

4.     bin/registry-file-list.py http://localhost:8080/registry-pds4

5.     grep <GUIDs from previous step> h.out

6.     bin/registry-package.py http://localhost:8080/registry-pds4 | grep Harvest

7.     grep <GUID from previous step> h.out

8.     bin/registry-association.py http://localhost:8080/registry-pds4

Test Results

Step 2 shows nothing, correctly

Step 4:                                                                                              

Association Type: file_ref, Target: urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35

Association Type: file_ref, Target: urn:uuid: urn:uuid:d647ce17-fae6-49a2-a9cf-cc48d1e754fa

 Step 5:                                                                              

% grep urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35 h.out

INFO:   [/Users/rchen/Desktop/test/bundle_geo_ra/data_test/scraping/pit_test_scraping.xml] Product has the following GUID: urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35

SUCCESS:   [/Users/rchen/Desktop/test/bundle_geo_ra/data_test/scraping/pit_test_scraping.xml] Successfully registered association to 'urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35'

Step 6:                                                 

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 http://localhost:8080/registry set([]) Harvest-Package_20171023011137

urn:uuid:e51afe4c-f2d1-4652-9c19-e28d95a7d4c8 http://localhost:8080/registry set([]) Harvest-Package_20171023013016

Step 7:

% grep urn:uuid:1dbdf222-83de-4d6c-8cd0-9bfdf25868b4 h.out

Registration Package GUID   urn:uuid:e51afe4c-f2d1-4652-9c19-e28d95a7d4c8

Step 8:                                    

urn:uuid:ef021136-4862-429c-a1b6-c09dea5ad20a urn:uuid:e50888c5-7c79-4801-9302-90a766ba83e2

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:a4b3fb0f-3ae6-4ee6-9f25-6395e31263fb

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:1ffac21c-ac0d-4566-95f2-ec8613934fa3

urn:uuid:7bccdeb0-985f-45bd-b4d7-65ed42968c02 urn:nasa:pds:profile:regrep:AssociationType:Product_Bundle.bundle_has_calibration_collection

urn:uuid:7bccdeb0-985f-45bd-b4d7-65ed42968c02 urn:nasa:pds:profile:regrep:AssociationType:Product_Class_Definition.component_of

urn:uuid:e51afe4c-f2d1-4652-9c19-e28d95a7d4c8 urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35

urn:uuid:7ca0b5d0-dc99-4683-b307-108c22183c77 urn:uuid:a8958c6a-72d3-4583-9c07-496b6fb2fdde

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:59c37ebc-bbcb-402a-b50e-69b64b11f6d1

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:86df0db5-7572-423c-9827-3b94dd86e48f

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:bb07826a-2b2d-487c-ab78-ff718a632adc

urn:uuid:1bc1d772-d3c0-428e-a6a2-d4f0051ced1e urn:nasa:pds:profile:regrep:ObjectType:Product_File_Text

urn:uuid:7bccdeb0-985f-45bd-b4d7-65ed42968c02 urn:nasa:pds:profile:regrep:AssociationType:Product_Bundle.bundle_has_spice_kernel_collection

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:7ca0b5d0-dc99-4683-b307-108c22183c77

urn:uuid:086c3ed1-7dac-4717-9dfb-63b95cdc1380 urn:registry:classificationScheme:ObjectType

urn:uuid:a4b3fb0f-3ae6-4ee6-9f25-6395e31263fb urn:uuid:1fa013c3-395c-4948-980f-13ce691dde5b

urn:uuid:7a6cba04-1d15-47c0-8236-d3792473077f urn:uuid:cff5962f-119d-4570-891f-df9a35d82c41

urn:uuid:58b0779a-75bc-421f-85fc-318f01fdf5ae urn:uuid:1a39429d-e093-4e1d-b109-b38359c7ec35

urn:uuid:1bc1d772-d3c0-428e-a6a2-d4f0051ced1e urn:nasa:pds:profile:regrep:ObjectType:Product_Zipped

urn:uuid:adb65d49-b371-47b9-bb4e-d55a7e3240f4 urn:uuid:791e194a-ddf6-48a9-8327-ac6f370ffdb8

urn:uuid:31c16d34-7e55-4f33-827a-3859e0745ea4 urn:uuid:01367582-68eb-4c7c-b703-1d9e7b55298e

Comments

Results met success criteria

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

 

 

Test Case ID

RPT.1

Description

Various requirements regarding reporting

Requirements

PASS L5.RPT.1: The service shall support periodic submission of metrics.

PASS L5.RPT.2: The service shall allow the submission of metrics in the form of a log file.

PASS L5.RPT.3: The service shall utilize a secure transfer protocol for transferring log files across the Internet.

PASS L5.RPT.4: The service shall support log files from the following sources…

PASS L5.RPT.5: The service shall discover product-related information by querying the Registry service.

PASS L5.RPT.6: The service shall aggregate and store the metrics in a repository.

PASS L5.RPT.7: The service shall control access to the user interface and metrics repository.

PASS L5.RPT.8: The service shall allow users to tailor reports and report templates as follows...

PASS L5.RPT.9: The service shall allow users to save report templates for reuse.

PASS L5.RPT.10: The service shall allow periodic generation of reports from saved templates.

PASS L5.RPT.11: The service shall export reports in the following formats...

Success Criteria

Following operator configuration of content, representation, filter, and scope of reports and report templates, Report Service receives metrics periodically in log files generated by web and FTP servers, PDS4 services, and node-specific services. Tools can view the repository to compare against log. Report Service queries Registry Service for metrics regarding products instead of transfers or views. Tools can view the repository to compare against the registry. Report Service authenticates for proper access, and reports unsuccessful attempts. Generated reports, even when generated from saved templates, should match configuration and meet the export format specified.

Test Steps

Commercial applications, in particular Sawmill, provide the functionality required. Verification of installation suffices.

Test Results

Report Service was installed

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

 

Test Case ID

SCMA.1

Description

Verify various change requests made to the Information Model schema and schemarons.

Requirements

PASS 1.3.3: PDS will provide criteria for validating archival products

Success Criteria

Validate tool accepts (or rejects) constructs deemed as valid (or invalid), primarily through software change requests.

Test Steps

Document testScma.docx describes the testing of the PDS4 schema and schematron.

Test Results

Document testScma.docx includes the test results of testing the PDS4 schema and schematron.

Comments

Results met success criteria.

Date of Testing

2017.10.03

Test Personnel

Richard Chen

 

 

Test Case ID

SEC.1

Description

 Various requirements regarding security

Requirements

PASS L5.SEC.1: The service shall authenticate a user given identifying credentials for that user.

PASS L5.SEC.2: The service shall encrypt the transmission of identifying credentials across the network.

PASS L5.SEC.3: The service shall authorize an authenticated user for access to a controlled capability.

PASS L5.SEC.4: The service shall allow an operator of the system to create, update or delete a user identity.

PASS L5.SEC.5: The service shall capture identifying information associated with a user identity.

PASS L5.SEC.6: The service shall allow an operator of the system to create, update or delete a group identity.

PASS L5.SEC.7: The service shall allow an operator of the system to add or remove a user from a group.

Success Criteria

Security service provides standard functions. Tools to view identities verify each activity. Security service allows an operator of the system to add or remove a user from a group. The user should subsequently be able or unable to access capabilities specific to the group. Security service captures identifying information. Tools to view identities show the information. Captured network packets show encryption, or trust that security service’s protocol encrypts. Security service authorizes valid users, denies invalid users.

Test Steps

(From https://pds-engineering.jpl.nasa.gov/development/pds4/5.0.0/security/)

The Security Service provides the authentication and authorization functions for the PDS4 system. The intent of this service is to control access to interfaces and services that require authentication and authorization (e.g., Monitor, Report, Registry interfaces, etc.).

Test Results

The functionality for this service is satisfied by the open source software package OpenDS, which is a directory service supporting the Lightweight Directory Access Protocol (LDAP).

Comments

Results met success criteria.

Date of Testing

2017.10.21

Test Personnel

Richard Chen

 

 

Test Case ID

SRCH.1 *not ready for build 5a. This is reserved for future testing

Description

Degrade gracefully on archaic browsers.

Requirements

SKIP L5.SCH.2: The service shall degrade gracefully on browsers that lack modern features and not depend on them for operation.

Success Criteria

Using an archaic browser to search does not freeze the browser.

Test Steps

 

Test Results

 

Comments

 

Date of Testing

 

Test Personnel

 

 

Test Case ID

SRCH.2

Description

Comply with Section 508 and adhere to WCAG level A

Requirements

PASS L5.SCH.3: The service's browser-based user interface shall be Section 508 compliant and adhere to WCAG …

PASS L5.GEN.9: Applications shall meet Section 508 compliance guidelines.

Success Criteria

PDS home page successfully passes through JPL website release process.

Test Steps

Submit PDS portal to JPL document review office to get approval for release

Test Results

JPL approved the release result PDS portal made available to general public

Comments

Results met success criteria.

Date of Testing

2017.10

Test Personnel

Paul Zimdars

 

Test Case ID

SRCH.3

Description

Provide HTTP-based API to enter queries and return results. The browser utilizes the REST-based API. The Tomcat server access log lists the search.

Requirements

PASS L5.SCH.4: The service shall provide a programmatic interface for entering of queries and return of search results that communicates over HTTP for use by client applications developed by PDS, PDS nodes, and others.

PASS L5.GEN.3: Services shall have an application programming interface.

PASS L5.GEN.5: Services shall generate metrics in a format suitable for ingestion by the Report Service.

Success Criteria

Receives correct search results after using HTTP-based API.

Test Steps

This test is best run on an operational machine or after harvesting a large number of files including context products, e.g. after HVT.3 (assume database is registry-pds3):

1.     http://localhost:8080/registry-ui/

2.     set Registry Service(s) to registry-pds3

3.     Click tab “Packages”. Select one of the “Harvest-Package_*”, set Status to “Approved”, click “Update Status”.

4.     date; search-core -H binDir/search-service/pds -p binDir/search-core/conf/defaults/pds/pds3/core.properties; date

The search-core above may take an hour.                           

5.     http://localhost:8080/search-service/pds/search?q=cassini cda

6.     In the center of http://localhost:8080/search-ui, type “cassini cda” (without quotes) and hit the “Search” button

The search-ui gets into the Tomcat server log, which the report service can process.

7.     grep cassini $CATALINA_HOME/logs/localhost_access_log.yyyy-mm-dd.txt

8.     http://localhost:8080/search-analytics

Test Results

Step 3: this took ~30 minutes ../Screen%20Shot%202016-10-11%20at%2014.32.53.png

Step 4:

Wed Oct 18 06:15:06 PDT 2017

Processing config: archiveinfo.xml

Processing config: attribute.xml

PDS Search Core Run Log

Version                     Version 1.11.0

Time                        Wed, Oct 18 2017 at 06:15:06 AM

Severity Level              INFO

Search Home                 /PDS4tools/search-service/pds

Search Service URL          http://localhost:8080/search-service

Search Core Properties      /PDS4tools/search-core/conf/defaults/pds/pds3/core.properties

INFO:   Completed: urn:nasa:pds:context_pds3:resource:resource.cassini_archive_information

SUCCESS:   Completed extraction:  archiveinfo.xmlINFO:   Completed: urn:nasa:pds:context_pds3:attribute:attribute.dawn__fsa_shoot_enable_flag

INFO:   Completed: urn:nasa:pds:context_pds3:attribute:attribute.focal_plane_temperature

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.9

INFO:   Posting: /PDS4tools/search-service/pds/index/search-tools.xml

INFO:   Optimizing Search Service index.

SUCCESS:   Completed posting data to the Search Service

Summary:

==================================================

The Numbers:

-- Number of Warnings: 10

-- Number of Errors: 0

-- Bad Registries: []

-- Number of Missing Associations: 0

-- Association Cache Hits: 0

-- Number of products: 16879

==================================================

Processing Time:

-- service.xml: 0 h, 0 m, 0 s

-- instrument.xml: 0 h, 1 m, 6 s

-- context.xml: 0 h, 40 m, 32 s

-- target.xml: 0 h, 6 m, 39 s

-- archiveinfo.xml: 0 h, 0 m, 6 s

-- instrumenthost.xml: 0 h, 0 m, 18 s

-- attribute.xml: 0 h, 3 m, 52 s

-- class.xml: 0 h, 0 m, 9 s

-- dataset.xml: 0 h, 13 m, 41 s

-- investigation.xml: 0 h, 0 m, 7 s

==================================================

Total Processing Time: 1 h, 6 m, 35 s

End of Log

Wed Oct 18 07:22:35 PDT 2017

Step 5:../Screen%20Shot%202017-10-18%20at%2009.49.34.png

Step 6: Note that 1) search-ui filters deprecated data sets, e.g. HRD < 16.0, and 2) Archive Information is there thanks to harvest of such information.

../Screen%20Shot%202017-10-18%20at%2009.51.39.png

Step 7:                                                                                                                     

127.0.0.1 - - [18/Oct/2017:07:13:25 -0700] "GET /registry-pds3/extrinsics?queryOp=AND&lid=urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens&start=1&sort=guid&rows=100 HTTP/1.1" 200 128901

127.0.0.1 - - [18/Oct/2017:07:14:56 -0700] "GET /registry-pds3/extrinsics/logicals/urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens/latest HTTP/1.1" 200 128864

0:0:0:0:0:0:0:1 - - [18/Oct/2017:09:48:53 -0700] "GET /search-ui/search.jsp?q=cassini+cda HTTP/1.1" 200 6809

0:0:0:0:0:0:0:1 - - [18/Oct/2017:09:48:53 -0700] "GET /search-service/pds/archive-filter?q=cassini+cda& HTTP/1.1" 200 38544

0:0:0:0:0:0:0:1 - - [18/Oct/2017:09:49:03 -0700] "GET /search-service/pds/search?q=cassini%20cda HTTP/1.1" 200 32665

Step 8: Since this was almost the last test run, I cheated and harvested only contextPDS4onlyPHX instead of HVT.3. Next time this will have better info:

../Screen%20Shot%202017-10-23%20at%2001.17.02.png

Comments

Results met success criteria.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

SRCH.5

Description

Search based on a sequence of open text keywords. Do so in a browser. 

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results

PASS L5.SCH.6: The service shall support searching by accepting criteria as a sequence of open text keywords.

Success Criteria

Receives reasonable results based on text such as “Cassini”.  Also, the Tomcat server access log lists the searched da ta.

Test Steps

This test is best run on an operational machine or after harvesting a large number of files including context products, e.g. HVT.3. If running after HVT.3, first:

search-core -H binDir/search-service/pds -p binDir/search-core/conf/pds/pds3/core.properties

The search-core above may take an hour.

 

In http://localhost:8080/search-ui, type

1.     mro spice

2.     voyager plasma wave

3.     (continued) In Refine Your Search, click “Comet SL9/Jupiter Collision (88)”

4.     mars digital elevation maps

The “More” and “Hide” link failed for a while on firefox and chrome, so test that on various platforms for Neptune

5.     Neptune

6.     jupiter images

7.     corona

8.     NEAR-A-SPICE-6-V1.0 (a specific data set ID)

Test Results

Step 1: ../Screen%20Shot%202017-04-17%20at%2000.41.08.png

Step 2: ../Screen%20Shot%202017-04-17%20at%2000.41.35.png

Step 3: ../Screen%20Shot%202017-04-17%20at%2000.41.57.png

Step 4: ../Screen%20Shot%202017-04-17%20at%2000.43.33.png../Screen%20Shot%202017-04-17%20at%2000.43.33.png

Step 5:../Screen%20Shot%202017-04-17%20at%2000.44.19.png

Step 6: ../Screen%20Shot%202017-04-17%20at%2000.44.41.png

Step 7:  ../Screen%20Shot%202017-04-17%20at%2000.44.59.png

Step 8:../Screen%20Shot%202017-04-17%20at%2000.45.24.png

 

Comments

Results met success criteria.

 

https://oodt.jpl.nasa.gov/jira/browse/PDS-258, created during testing of build 4b, suggests merging 2 related facets into 1

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

SRCH.6

Description

Search based on constraints on specific indexes, and narrow results based on more constraints. Support ordering of results based on specified criteria. Results returned as clickable URIs with metadata describing each URI.

Requirements

PASS L5.SCH.7: The service shall accept criteria as a series of values for constraints on specified indexes.

PASS L5.SCH.8: The service shall support narrowing of additional index results…

PASS L5.SCH.9: The service shall support the ordering of results based on specified criteria...

PASS L5.SCH.10: The service shall provide results to a search as a sequence of matching URIs…

PASS L5.SCH.11: The service shall annotate each URI of a result with metadata describing the URI.

Success Criteria

Results match constraint criteria and consist of clickable links with text describing each link.

Test Steps

Also best run after HVT.3. In http://localhost:8080/search-ui:

1.     mission: mars global surveyor

2.     target:mercury

3.     target: mercury

Test Results

Step 1: ../Screen%20Shot%202017-04-17%20at%2000.54.43.png

 

Step 2: ../Screen%20Shot%202017-04-17%20at%2000.55.06.png

Step 3: inserting a space bfore “mercury” gives significantly different results

../Screen%20Shot%202017-04-17%20at%2000.55.19.png

Comments

Results met success criteria.

 

https://oodt.jpl.nasa.gov/jira/browse/PDS-319, created during build 5a, notes differences in search results between “target:mercury” and “target: mercury”

Date of Testing

2017.10.18

Test Personnel

Richard Chen                                                                                    

 

Test Case ID

SRCH.7 *not ready for build 8a. This is reserved for future testing

Description

Capture metrics on search index usage and contents

Requirements

SKIP L5.SCH .13: The service shall capture metrics pertaining to its search indexes usage and contents.

Success Criteria

A log shows metrics pertaining to usage of search indexes.

Test Steps

 

Test Results

 

Comments

 

Date of Testing

 

Test Personnel

 

 

Test Case ID

SRCH.8

Description

Query for the data dictionary’s PDS3 elements and objects and PDS4 attributes and classes.

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results…

 

Success Criteria

Results match parameters and consist of clickable links with text describing each link.

Test Steps

Find PDS3 values from pdsdd.full at https://pds.nasa.gov/tools/dd-search:

1.     pdsdd.full’s *_OBJECT_DEFINITION vs. dd-search’s Class,PDS3

2.     pdsdd.full’s ELEMENT_DEFINITION vs. dd-search’s Attribute,PDS3

3.     pdsdd.full’s APERTURE_TYPE element vs. dd-search’s attribute aperture_type

Find PDS4 values from PDS4_PDS_1800.xsd in dd-search, which is most useful in presenting elements in a format

4.     dd-search’s “Observation_Area” vs `grep -A19 '<xs:complexType name="Observation_Area">' PDS4_PDS_1800.xsd`

Test Results

Step 1: dd-search and pdsdd.full have the same 89 results, e.g. ALIAS, ARRAY, …

Step 2: dd-search has 2184 results; pdsdd.full, 2192. All diffs were recently added to the pds3 data dictionary. They are:

  GEO:BAUD

  GEO:CENTROID_LOCATION

  GEO:CODE_LENGTH

  GEO:DELAY_OFFSET

  GEO:MODE

  GEO:PARALLACTIC_ANGLE_CORRECTION

  GEO:POINTING

  GEO:TRANSFORM_LENGTH

 Step 3: pdsdd.full:

OBJECT = ELEMENT_DEFINITION

  NAME = APERTURE_TYPE

  STATUS_TYPE = APPROVED

  GENERAL_DATA_TYPE = IDENTIFIER

  UNIT_ID = NONE

  STANDARD_VALUE_TYPE = DYNAMIC

  MAXIMUM_LENGTH = 6

  DESCRIPTION = "

     The APERTURE_TYPE element describes a short string of free-format text

     which provides a distinguishing name or abbreviation for one (or more)

     of a set of apertures used during data collection.

     Note:  For the International Ultraviolet Explorer (IUE) spacecraft,

     the spectrographs have small and large apertures, and can operate with

     either or both open."

  STANDARD_VALUE_SET = {

     "BOTH",

     "LARGE",

     "SMALL"}

END_OBJECT = ELEMENT_DEFINITION

END

dd-search covers all those pairs../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Step 4:../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

  <xs:complexType name="Observation_Area">

    <xs:annotation>

      <xs:documentation>The observation area consists of attributes that

        provide information about the circumstances under which the data

        were collected.</xs:documentation>

    </xs:annotation>

    <xs:complexContent>

      <xs:restriction base="pds:Context_Area">

        <xs:sequence>

          <xs:element name="comment" type="pds:comment" minOccurs="0" maxOccurs="1"> </xs:element>

          <xs:element name="Time_Coordinates" type="pds:Time_Coordinates" minOccurs="1" maxOccurs="1"> </xs:element>

          <xs:element name="Primary_Result_Summary" type="pds:Primary_Result_Summary" minOccurs="0" maxOccurs="1"> </xs:element>

          <xs:element name="Investigation_Area" type="pds:Investigation_Area" minOccurs="1" maxOccurs="unbounded"> </xs:element>

          <xs:element name="Observing_System" type="pds:Observing_System" minOccurs="1" maxOccurs="unbounded"> </xs:element>

          <xs:element name="Target_Identification" type="pds:Target_Identification" minOccurs="1" maxOccurs="unbounded"> </xs:element>

          <xs:element name="Mission_Area" type="pds:Mission_Area" minOccurs="0" maxOccurs="1"> </xs:element>

          <xs:element name="Discipline_Area" type="pds:Discipline_Area" minOccurs="0" maxOccurs="1"> </xs:element>

        </xs:sequence>

      </xs:restriction>

    </xs:complexContent>

Comments

Results met success criteria.

 

PDS-357, created during testing of 5b, handles the questions raised below.

 

Maybe it would help the user to see what schemas and versions of schemas are used.

 

Is it ok for ddsearch to lack:

Steps 1 and 2: is it all right for ddsearch not to have the equivalent attributes listed in the output? It may be because EN did not produce the corresponding Product_Class_Definition or _Attribute_. There are very likely more missing Product_Class_Attribute, but full lists (especially for dd-search) are hard to create.

 

Step 4: Does dd-search’s list correspond to the .xsd’s complexType + simpleType="[A-Z]*"  but not simpleType="[a-z]*"? Where did dd-search’s Unsigned* classes come from? There are probably more mismatches, but full lists are hard to create.  Do the PDS4 attributes come from all (including node) released schemas? That would explain "along_track_timing_offset". "abstract_flag" is listed twice because dd search shows one attribute per Class x Attribute? Where do "alias" and "alias_list" come from? dd-search:"Class" has a capitalized version of each. Where does character_constraint come from?

Date of Testing

2017.04.21

Test Personnel

Richard Chen

 

Test Case ID

SRCH.9

Description

Test various capabilities described in release documents:

A.   View datasets despite duplicate identifiers.

Requirements

PASS L5.SCH.1: The service shall provide a user interface for entering of queries and display of search results…

 

Success Criteria

All datasets with the same IDs but different registries (namely –pds3 and psa) are clickable

Test Steps

View datasets despite duplicate identifiers

1.     cd testDir/; mkdir x

2.     ln -s testDir/contextPDS3/*/Product/*VEX-E-V-SPICE*.xml x/

Using the version of harvest that writest to registry-pds3

3.     harvest -c harvest-policy-master.xml testDir/x -e "*.xml"

4.     http://localhost:8080/registry-ui >> Packages >> Approved >> Update Status

5.     search-core -H binDir/search-service/pds -p binDir/search-core/conf/defaults/psa/pds3/core.properties binDir/search-core/conf/defaults/pds/pds3/core.properties
[I added “/pds” to -H. Maybe that’s why the ESA output is weird?]

6.     http://localhost:8080/search-ui >> vex spice

7.     Click the two separate Data Sets; one from PSA, one from PDS

Test Results

Step 3:

PDS Harvest Tool Log

Version                     Version 1.13.0

Time                        Sat, Oct 21 2017 at 10:32:54 AM

Target(s)                   [testDir/x]

File Inclusions             [*.xml]

Severity Level              INFO

Registry Location           http://localhost:8080/registry-pds3

Registry Package Name       Harvest-Package_20171021103254

Registration Package GUID   urn:uuid:e7210407-5e6f-476f-b2b7-56544c572e7f

INFO:   XML extractor set to the following default namespace: http://pds.nasa.gov/pds4/pds/v1

INFO:   [/Users/rchen/Desktop/test/x/data_set_VEX-E-V-SPICE-6-V1.0_1.0.xml] Begin processing.

INFO:   [/Users/rchen/Desktop/test/x/data_set_VEX-E-V-SPICE-6-V1.0_1.0.xml] line 28: Mapping reference type 'has_mission' to 'investigation_ref'.

[snip…]

INFO:   [/Users/rchen/Desktop/test/x/resource_VEX-E-V-SPICE-6-V1.0__BROWSERP_VEX-E-V-SPICE-6-V1.0_1.0.xml] Association has the following GUID: urn:uuid:c12f2f1f-aa25-4a86-99c8-86397aa836a0

Summary:                                                                                                                         

5 of 5 file(s) processed, 0 other file(s) skipped

0 error(s), 0 warning(s)

5 of 5 products registered.

5 of 5 ancillary products registered.

Product Types Registered:

1 Product_Data_Set_PDS3

5 Product_File_Repository

4 Product_Context

5 of 5 associations registered.

End of Log

Step 5:

Processing config: archiveinfo.xml

Processing config: attribute.xml

Processing config: class.xml

Processing config: context.xml

Processing config: dataset.xml

Processing config: instrument.xml

Processing config: instrumenthost.xml

Processing config: investigation.xml

Processing config: service.xml

Processing config: target.xml

Processing config: archiveinfo.xml

Processing config: attribute.xml

Processing config: class.xml

Processing config: context.xml

Processing config: dataset.xml

Processing config: instrument.xml

Processing config: instrumenthost.xml

Processing config: investigation.xml

Processing config: service.xml

Processing config: target.xml

PDS Search Core Run Log

Version                     Version 1.11.0

Time                        Sat, Oct 21 2017 at 10:38:41 AM

Severity Level              INFO

Search Home                 /PDS4tools/search-service/pds

Search Service URL          http://localhost:8080/search-service

Search Core Properties      /PDS4tools/search-core/conf/defaults/psa/pds3/core.properties

Search Core Properties      /PDS4tools/search-core/conf/defaults/pds/pds3/core.properties

INFO:   Completed: urn:esa:psa:context_pds3:data_set:data_set.ro-cal-alice-3-cr4b-v1.0

[snip…]

SUCCESS:   Completed extraction:  context.xml

INFO:   Completed: urn:nasa:pds:context_pds3:data_set:data_set.vex-e-v-spice-6-v1.0

SUCCESS:   Completed extraction:  dataset.xml

SUCCESS:   Completed extraction:  instrument.xml

SUCCESS:   Completed extraction:  instrumenthost.xml

SUCCESS:   Completed extraction:  investigation.xml

SUCCESS:   Completed extraction:  target.xml

SUCCESS:   Completed extracting data from data source.

INFO:   Running Solr Indexer to create new solr documents for indexing ...

SUCCESS:   Completed transforming data into Solr Lucene index

INFO:   Running Solr Post to Post Data To Search Service ...

INFO:   Cleaning Search Service Index

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.0

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.1

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.2

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.3

INFO:   Posting: /PDS4tools/search-service/pds/index/solr_index.xml.4

INFO:   Posting: /PDS4tools/search-service/pds/index/search-tools.xml

INFO:   Optimizing Search Service index.

SUCCESS:   Completed posting data to the Search Service

Summary:

==================================================

The Numbers:

-- Number of Warnings: 0

-- Number of Errors: 0

-- Bad Registries: []

-- Number of Missing Associations: 0

-- Association Cache Hits: 0

-- Number of products: 4111

==================================================

Processing Time:

-- instrument.xml: 0 h, 0 m, 0 s

-- context.xml: 0 h, 0 m, 2 s

-- target.xml: 0 h, 0 m, 0 s

-- archiveinfo.xml: 0 h, 0 m, 0 s

-- instrumenthost.xml: 0 h, 0 m, 0 s

-- attribute.xml: 0 h, 0 m, 0 s

-- class.xml: 0 h, 0 m, 0 s

-- dataset.xml: 0 h, 0 m, 0 s

-- investigation.xml: 0 h, 0 m, 0 s

==================================================

Total Processing Time: 0 h, 0 m, 5 s

End of Log

Step 6:../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Step 7:../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

 

../../../../../../Desktop/Screen%20Shot%202017-04-21%20at%2

Comments

Results met success criteria.

Date of Testing

2017.04.21

Test Personnel

Richard Chen

 

 

Test Case ID

TPRT.1

Description

Request data from transport-registry by specifying a LID

Requirements

PASS L5.TRS.1: The service shall accept requests for download of PDS products.

PASS L5.TRS.2: The service shall accept requests for download of an individual file.

PASS L5.TRS.4: The service shall package the requested product(s) or file into the specified format.

PASS L5.TRS.5: The service shall include a checksum manifest … along with their associated MD5 checksums.

PASS L5.TRS.6: The service shall transfer the result of a request via HTTP to the calling application.

PASS L5.GEN.3: Services shall have an application programming interface.

PASS L5.GEN.5: Services shall generate metrics in a format suitable for ingestion by the Report Service.

Success Criteria

The transport service returns the requested data. Also, the Tomcat server access log lists the transport.

Test Steps

Harvest must have run (e.g. HVT.3) and gotten absolute paths as inputs. Get some of the harvested LIDs; see http://localhost:8080/registry-ui if needed. This test uses

·       urn:nasa:pds:context_pds3:target:satellite.titan

·       urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens

·       urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr17-v1.0

Ensure CATALINA_HOME/webapps/transport-registry/WEB-INF/config.xml has transport.RegistryProductHandler.registryUrl = http://localhost:8080/registry-pds3
Check transport-registry’s many downloading options by getting the same three files:

1.     curl -X GET -o x1.zip -v "http://localhost:8080/transport-registry/prod?q=identifier+EQ+urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens+AND+identifier+EQ+urn:nasa:pds:context_pds3:target:satellite.titan+AND+identifier+EQ+urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0"

2.     curl -X GET -o x2.tgz -v "http://localhost:8080/transport-registry/prod?q=identifier+EQ+urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens+AND+identifier+EQ+urn:nasa:pds:context_pds3:target:satellite.titan+AND+identifier+EQ+urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0+AND+package+EQ+TGZ"

3.     curl -X GET -o x3.zip "http://localhost:8080/transport-registry/prod?identifier=urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens&identifier=urn:nasa:pds:context_pds3:target:satellite.titan&identifier=urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0"

4.     Same thing but in a browser, http://localhost:8080/transport-registry/prod?identifier=urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens&identifier=urn:nasa:pds:context_pds3:target:satellite.titan&identifier=urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0&package=TGZ
which creates file pds-package-<yyyymmddhhmmss>.tar.gz

5.     Extract the four into their own directories.

6.     diff -r x1 x2

7.     diff -r x1 x3

8.     diff -r x1 pds-package-<yyyymmddhhmmss>

9.     diff x1/data_set_CO-SSA-RSS-1-TIGR19-V1.0_1.0.xml testDir/contextPDS3/dataset/Product/data_set_CO-SSA-RSS-1-TIGR19-V1.0_1.0.xml

10.  diff x1/mission_CASSINI-HUYGENS_1.0.xml testDir/contextPDS3/investigation/Product/mission_CASSINI-HUYGENS_1.0.xml

11.  diff x1/target_TITAN_1.0.xml testDir/contextPDS3/target/Product/target_TITAN_1.0.xml

12.  Does x1/md5_checksum.txt have correct checksums for all 3 files? On a mac:
openssl md5 x1/*.xml

To get size, replace step 4’s package=TGZ with package=TGZ_SIZE. In a browser:

13.  http://localhost:8080/transport-registry/prod?identifier=urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens&identifier=urn:nasa:pds:context_pds3:target:satellite.titan&identifier=urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0&package=SIZE

14.  date; ls –l pds-package-20160420233723 |awk '{ total += $5 }; END { print total }'

Test Results

Step 1 (Steps 2 and 3 are very similar):

Note: Unnecessary use of -X or --request, GET is already inferred.

*   Trying ::1...

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to localhost (::1) port 8080 (#0)

> GET /transport-registry/prod?q=identifier+EQ+urn:nasa:pds:context_pds3:investigation:mission.cassini-huygens+AND+identifier+EQ+urn:nasa:pds:context_pds3:target:satellite.titan+AND+identifier+EQ+urn:nasa:pds:context_pds3:dataset:dataset.co-ssa-rss-1-tigr19-v1.0 HTTP/1.1

> Host: localhost:8080

> User-Agent: curl/7.48.0

> Accept: */*

>

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< HTTP/1.1 200

< Content-disposition: attachment; filename="products_pds-package-20170417115455.zip.zip"

< Content-Type: application/zip

< Content-Length: 49109

< Date: Wed, 18 Oct 2017 17:14:12 GMT

<

{ [7995 bytes data]

100 49109  100 49109    0     0  44189      0  0:00:01  0:00:01 --:--:-- 44282

* Connection #0 to host localhost left intact

Step 4:../Screen%20Shot%202017-10-18%20at%2010.32.34.png

Steps 6-11 show no differences

Step 12: Compare the values below with those in x1/md5_checksum.txt

MD5(x1/data_set_CO-SSA-RSS-1-TIGR19-V1.0_1.0.xml)= 1f95f35a45f736ce04f9af4df4095c5b

MD5(x1/mission_CASSINI-HUYGENS_1.0.xml)= 722ad7f09b6e61573ee1b1083183384e

MD5(x1/target_TITAN_1.0.xml)= cd366b9be257359b7c6d6cc53c23b6a6

Step 13: ../Screen%20Shot%202017-04-17%20at%2012.07.46.png

Step 14:

Wed Oct 18 10:47:43 PDT 2017

170041

Comments

Results met success criteria.

Date of Testing

2017.10.18

Test Personnel

Richard Chen

 

Test Case ID

TPRT.3

Description

Request data from transport-ofsn by specifying a path. Transport-ofsn can also transform the requested product.

Requirements

L5.TRS.3: The service shall transform the requested product(s) or file into the specified format.

Success Criteria

The original and the transformed products have the same data.

Test Steps

Besides $CATALINA_HOME/webapps/transport-ofsn/WEB-INF/config.xml, modify $CATALINA_HOME/webapps/transport-ofsn/WEB-INF/ofsn-ps.xml so that

productRoot=testDir

To activate that change:

shutdown.sh; startup.sh

Test every Return Type listed in the middle of https://pds-engineering.jpl.nasa.gov/development/pds4/7.1.0/transport/transport-ofsn/operate. First, some ls information:

1.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/&RT=DIRFILELIST"

2.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/&RT=DIRFILELIST1"

3.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/&RT=DIRLIST"

4.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/&RT=DIRLIST1"

5.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.tab&RT=FILELIST"

6.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.tab&RT=FILE_LIST_ZIP"

A get, then some ls information about that file

7.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.tab&RT=RAW" > x.xml

8.     diff x.xml testDir/testTprt/ladee_ldex.tab

9.     curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.tab&RT=RAW_SIZE"

10.  ls -ol testDir/testTprt/ladee_ldex.tab

11.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.tab&RT=MD5"

12.  openssl md5 testdir/testTprt/ladee_ldex.tab

Some image transformations (get and transform)

13.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_JPG" > x.jpg

14.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_JP2" > x.jp2

15.  nothing; PDS_TO_BMP is no longer supported.

16.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_GIF" > x.gif

17.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_PNG" > x.png

18.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_TIFF" > x.tiff

19.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_TIF" > x.tif

20.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testPrep/i943630r.xml&RT=PDS_TO_PNM" > x.pnm

Some label transformations                         

21.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.xml&RT=PDS4_TO_PVL" > x.pvl

22.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.xml&RT=PDS4_TO_HTML" > x.html

23.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.xml&RT=PDS4_TO_HTML_STRUCTURE_ONLY" > xSO.html

24.  Using editor or browser, visually compare x.* with testDir/testTprt/ladee_ldex.xml

Data transformation

25.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/ladee_ldex.xml&RT=PDS4_TO_CSV"

26.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/msl_chemcam.lbl&RT=PDS3_TO_PDS4_LABEL"

27.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/msl_chemcam.lbl&RT=PDS_LABEL"

More “Return Type”s

28.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/msl_chemcam.lbl&RT=PDS_ZIP" > x.zip

29.  unzip x.zip

30.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/msl_chemcam.lbl&RT=PDS_ZIP_SIZE"

31.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt&RT=PDS_ZIPD" > x.zip

32.  unzip x.zip

33.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt&RT=PDS_ZIPD_SIZE"

Currently unsupported capabilities: FITS file, CDF file

34.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/mvn_lpw.xml&RT=PDS_TO_JPG"

35.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/mvn_iuv.xml&RT=PDS_TO_JPG"

Check proper handling of inaccessible directories

36.  chmod 000 testTprt/dirLevel1a/dirLevel2

37.  curl "http://localhost:8080/transport-ofsn/prod?OFSN=/testTprt/dirLevel1a/&RT=DIRFILELIST"

38.  cp testTprt/dirLevel1a/fileLevel2b.txt testTprt/dirLevel1a/.ignoreME

39.  chmod 777 testTprt/dirLevel1a/dirLevel2

40.  repeat step 37

41.  rm testTprt/dirLevel1a/.ignoreME

Test Results

Step 1:

<?xml version="1.0" encoding="UTF-8"?>

<dirResult xmlns="http://oodt.jpl.nasa.gov/xml/namespaces/dirlist/1.0">

   <dirEntry xmlns="">

      <OFSN>/testTprt/a0013.qub</OFSN>

      <fileSize>50339840</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/a0013.xml</OFSN>

      <fileSize>22618</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/ladee_ldex.tab</OFSN>

      <fileSize>500</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/ladee_ldex.xml</OFSN>

      <fileSize>7031</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/msl_chemcam.csv</OFSN>

      <fileSize>2241087</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/msl_chemcam.lbl</OFSN>

      <fileSize>28086</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_iuv.fits</OFSN>

      <fileSize>449280</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_iuv.xml</OFSN>

      <fileSize>75445</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_lpw.cdf</OFSN>

      <fileSize>230296</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_lpw.xml</OFSN>

      <fileSize>8423</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/dirLevel1b/tcshrc</OFSN>

      <fileSize>29</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/dirLevel1a/fileLevel2a.txt</OFSN>

      <fileSize>72</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/dirLevel1a/fileLevel2b.txt</OFSN>

      <fileSize>307</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/dirLevel1a/dirLevel2/fileLevel3.txt</OFSN>

      <fileSize>1550</fileSize>

   </dirEntry>

 </dirResult>

Step 2:

<?xml version="1.0" encoding="UTF-8"?>

<dirResult xmlns="http://oodt.jpl.nasa.gov/xml/namespaces/dirlist/1.0">

   <dirEntry xmlns="">

      <OFSN>/testTprt/a0013.qub</OFSN>

      <fileSize>50339840</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/a0013.xml</OFSN>

      <fileSize>22618</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/ladee_ldex.tab</OFSN>

      <fileSize>500</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/ladee_ldex.xml</OFSN>

      <fileSize>7031</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/msl_chemcam.csv</OFSN>

      <fileSize>2241087</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/msl_chemcam.lbl</OFSN>

      <fileSize>28086</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_iuv.fits</OFSN>

      <fileSize>449280</fileSize>

   </dirEntry>

   <dirEntry xmlns="">

      <OFSN>/testTprt/mvn_iuv.xml</OFSN>

      <fileSize>75445</fileSize>