gov.nasa.pds.harvest.search.crawler

Class ProductCrawler

java.lang.Object

gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean

gov.nasa.pds.harvest.search.crawler.ProductCrawler

All Implemented Interfaces:

gov.nasa.jpl.oodt.cas.commons.spring.SpringSetIdInjectionType, gov.nasa.jpl.oodt.cas.filemgr.metadata.CoreMetKeys

Direct Known Subclasses:

PDSProductCrawler

public abstract class ProductCrawler
extends gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean

Author:

mcayanan

An abstract base class for Product Crawling. This class provides methods to communicate with the file manager and parse met files that show how to ingest a particular Product into the File Manager.

Field Summary

Fields

Modifier and Type	Field and Description
protected FileFilter	DIR_FILTER
protected FileFilter	FILE_FILTER
protected static Logger	LOG

Fields inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean

MIME_TYPES_HIERARCHY

Fields inherited from interface gov.nasa.jpl.oodt.cas.filemgr.metadata.CoreMetKeys

FILE_LOCATION, FILENAME, MIME_TYPE, PRODUCT_ID, PRODUCT_NAME, PRODUCT_RECEVIED_TIME, PRODUCT_STRUCTURE, PRODUCT_TYPE

Constructor Summary

Constructors

Constructor and Description
ProductCrawler()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addKnownMetadata(File product, gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
void	clearIngestStatus()
void	crawl()
void	crawl(File dirRoot)
protected ExtrinsicObject	createProduct(gov.nasa.jpl.oodt.cas.metadata.Metadata metadata, File prodFile) Create the Product object.
protected boolean	generateDoc(File product, gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
List<gov.nasa.jpl.oodt.cas.crawl.status.IngestStatus>	getIngestStatus()
protected abstract gov.nasa.jpl.oodt.cas.metadata.Metadata	getMetadataForProduct(File product)
SearchDocGenerator	getSearchDocGenerator()
protected void	handleFile(File product)
protected abstract boolean	passesPreconditions(File product)
void	setActionRepo(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerActionRepo repo)
void	setSearchDocGenerator(SearchDocGenerator generator)

Methods inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean

addRequiredMetadata, getActionIds, getApplicationContext, getDaemonPort, getDaemonWait, getFilemgrUrl, getGlobalMetadata, getId, getIngester, getProductPath, getRequiredMetadata, isCrawlForDirs, isNoRecur, isSkipIngest, setActionIds, setApplicationContext, setCrawlForDirs, setDaemonPort, setDaemonWait, setFilemgrUrl, setGlobalMetadata, setId, setIngester, setNoRecur, setProductPath, setRequiredMetadata, setSkipIngest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static Logger LOG

DIR_FILTER

protected FileFilter DIR_FILTER

FILE_FILTER

protected FileFilter FILE_FILTER

Constructor Detail

ProductCrawler

public ProductCrawler()

Method Detail

crawl

public void crawl()

crawl

public void crawl(File dirRoot)

getIngestStatus

public List<gov.nasa.jpl.oodt.cas.crawl.status.IngestStatus> getIngestStatus()

clearIngestStatus

public void clearIngestStatus()

addKnownMetadata

protected void addKnownMetadata(File product,
                    gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)

handleFile

protected void handleFile(File product)

createProduct

protected ExtrinsicObject createProduct(gov.nasa.jpl.oodt.cas.metadata.Metadata metadata,
                            File prodFile)

Create the Product object.

Parameters:

metadata - A class representation of the metdata.

Returns:

A Product object.

generateDoc

protected boolean generateDoc(File product,
                  gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)

passesPreconditions

protected abstract boolean passesPreconditions(File product)

getMetadataForProduct

protected abstract gov.nasa.jpl.oodt.cas.metadata.Metadata getMetadataForProduct(File product)

setActionRepo

public void setActionRepo(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerActionRepo repo)

getSearchDocGenerator

public SearchDocGenerator getSearchDocGenerator()

setSearchDocGenerator

public void setSearchDocGenerator(SearchDocGenerator generator)