Operation

The Registry is an Apache Solr adaptation that includes the Solr Admin interface along with the pre-installed collections that come with the PDS Registry and Search. Once it is deployed, operations include modifying the configuration and leveraging the PDS and PDAP protocols to query data and metadata that has been extracted from PDS4 data products. The following topics will be covered below:

Common Operations

Here are some common operations you may wish to perform to interact with the Registry and Search. A tool to simplify these actions is in the plan, but in the meantime, here are the various raw commands you can do to modify data in your registry and search. For additional information for interacting with the Registry, please also take a look at the Solr Collections API and other info on interacting with Solr documents.

Add Data To Registry and Search

After you have run Harvest, you should now see files like solr_doc_0.xml under $REGISTRY_HOME/../registry-data/solr-docs (or on Windows C:\\%REGISTRY_HOME%\..\registry-data\solr-docs). To ingest these into the Search index (http://localhost:8983/solr/search), run the following command:

Docker Installation: 

docker exec -it registry post -c pds -params "tr=add-hierarchy.xsl" /data/solr-docs

Standalone Installation: 

$SOLR_HOME/bin/post -c pds -params "tr=add-hierarchy.xsl" $REGISTRY_HOME/../registry-data/solr-docs

Delete Data From Registry and Search

Once you have data in the Registry and/or Search, the following cUrl commands can modify the data you have ingested.

Docker Installation: 

# Clean out all data ingested into Registry and/or Search
docker exec -it registry post -c .system -d "<delete><query>-type:metrics_rrd</query></delete>"
docker exec -it registry post -c pds -d "<delete><query>*:*</query></delete>"
docker exec -it registry post -c xpath -d "<delete>=<query>*:*</query></delete>"

# Remove collection with LIDVID = urn:nasa:pds:insight_rad:data_raw::1.0
docker exec -it registry post -c .system -d '<delete><query>-type:metrics_rrd AND lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'
docker exec -it registry post -c pds -d '<delete><query>lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'
docker exec -it registry post -c xpath -d '<delete><query>lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'

Standalone Installation: 

# Clean out all data ingested into Registry and/or Search
$SOLR_HOME/bin/post -c .system -d "<delete><query>-type:metrics_rrd</query></delete>"
$SOLR_HOME/bin/post -c pds -d "<delete><query>*:*</query></delete>"
$SOLR_HOME/bin/post -c xpath -d "<delete><query>*:*</query></delete>"

# Remove collection with LIDVID = urn:nasa:pds:insight_rad:data_raw::1.0
$SOLR_HOME/bin/post -c .system -d '<delete><query>-type:metrics_rrd AND lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'
$SOLR_HOME/bin/post -c pds -d '<delete><query>lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'
$SOLR_HOME/bin/post -c xpath -d '<delete><query<lidvid:"urn:nasa:pds:insight_rad:data_raw::1.0"</query></delete>'

Re-Ingest Data To Registry and Search

  • Delete the data you would like to remove from Registry and Search.
  • If you removed data from both the Registry and Search, you will need to re-run Harvest and then re-ingest the data into the Search index.
  • If you only removed data from the Search index, you can then just re-ingest the data into the Search index.

Administration Interface

The Solr Administration Interface is available for advanced configuration and debugging of the Solr instance. The page can be viewed at http://localhost:8983/solr or if you have setup the proxy pass as directed in the installation guide, https://<hostname>:8983/services/registry. The following resources are available detailing the how to use this interface:

From the query page, you can perform various queries against the available end points supported by the service. Details regarding the supported query parameters can be found on the Common Query Parameters page of the Solr wiki. The most useful of these being the wt parameter. When this parameter is specified as wt=xslt, you can view the output as it is queried from the Search UI and Product Search UI.

Configuration

The Registry comes preconfigured for supporting common PDS search terms and facets. That said, it can be tailored to support Discipline Node specific search requirements. See the Apache Solr website and wiki for more information on configuring Apache Solr. The sub-sections that follow detail the steps for performing some common configuration changes.

Add A Facet

The following steps will allow a new facet to be added to the Registry output. The facets are the categories in the left sidebar of the Search UI that provide a means for browsing through the data. The name for the facet should not yet be a field in the schema, because a new field specifically dedicated to the faceting will be created. For instance, if you wanted to facet on agency_name, we will create a facet named facet_agency. These steps assume that the Registry was installed at $REGISTRY_HOME. If not, substitute the installation directory as needed. For the following example, we will be creating a facet on the node_id field.

  1. Add the following to the $REGISTRY_HOME/conf/xslt/facets.xml file:

    <facet name="facet node id" caption="Node" />

  2. Update the $REGISTRY_HOME/conf/xslt/add-hierarchy.xsl file:

    <xsl:template match="field[@name = 'node_id']">
  <xsl:copy-of select="." >
  <field name="facet_node_id"><xsl:value-of select="concat('1,',.)" /></field>
</xsl:template>

  3. Update the $REGISTRY_HOME/conf/solrconfig.xml file:

    <str name="facet.field">facet_node_id</str>

<str name="f.facet_node_id.facet.prefix">1,</str>

  4. Update the $REGISTRY_HOME/conf/managed-schema file by finding the area where facet fields are being declared (search for facet_) and add the following:

    <field name="facet_node" type="string_lc" indexed="true" stored="false" \
  multiValued="false" />
  5. Execute the appropriate registry_uninstall and registry_install scripts (dependent upon operating system) to re-install the Registry with the new facets.

Update Output Format

The user interface components of the Registry may also be modified to tailor the look-and-feel of the search results.

  • To update the output queried through the Search UI (archive-filter end point), make updates to the $REGISTRY_HOME/conf/xslt/data-search.xsl file.
  • To update the output queried through the Product Search UI (product-search end point), make updates to the $REGISTRY_HOME/conf/xslt/product-search.xsl file.

Add a Field to the Schema

In order to add a field to the schema, modify the $REGISTRY_HOME/conf/schema.xml file. The following example will add the field my_field: 

<field name="my_field" type="text_general" indexed="true" stored="false" \
multiValued="false" />

See the Schema XML page of the Solr wiki for more information on update the Solr schema.

Search Protocols

The Registry provides two protocols for searching the contents of the service.

PDS Search Protocol

See the PDS Search Protocol document for more information on search terms and result formatting provided by this protocol. If viewing this document from the registry package, view the PDS Search Protocol document from the Engineering Node site.

PDAP Protocol

See the PDAP Search Protocol document for more information on search terms and result formatting provided by this protocol. If viewing this document from the registry package, view the PDAP Search Protocol document from the Engineering Node site.

Common Errors

TBD