SCR 3-1003 – PDS Engineering

SCR 3-1003: JPEG 2000 as PDS Archive Format

Priority: urgent

Problem Summary

The MRO HiRISE (High Resolution Imaging Science Experiment) team is currently predicting the production of 11,600 GB of RDR products. The maximum sizes of individual RDR products are expected to be about 5 GB. This large volume of data and the large size of the individual products raises several data management issues involving data storage and the distribution of products over the internet.

The MRO HiRISE team is suggesting that the JPEG 2000 compression scheme be used to compress the HiRISE products for use in both online data repositories and for the PDS archive. The JPEG 2000 compression scheme provides a good compression ratio (~3:1 to 4:1 lossless) and uses a wavelet encoding scheme to allow easy viewing of extremely large images. The use of the JPEG 2000 compression scheme will (in the words of the team) result in lower data volume requirements, smaller storage needs, lower media costs, more easily managed data sets, lower per-unit count of hard media, and faster transfer times. It will also preclude the need for separate product tiling.

Note that the MRO HiRISE team is proposing JPEG 2000 for HiRISE RDRs only.

Working Group

E. Rye, S. LaVoie (leads) B. Castalia, P. Garcia, S. Hughes, C. Isbell, R. Joyner, M. McAuley, B. Sword

Originator: E. Eliason

Standards Change Request

Version	Date	Author(s)	Format	Note
1	2003-08-22	S. Hughes	TEXT
2	2004-04-19	J. Wilf	TEXT
3	2005-09-27	E. Rye	PDF
4	2005-09-28	E. Rye	PDF
5	2005-09-30	R. Simpson & E. Rye	PDF PDF	clean with tracked changes
6	2005-10-10	E. Rye	PDF PDF	clean with tracked changes

Discussion

Discussion Blog
Working Group Meetings: 2005-08-01 (TEXT)
Telecons: 2005-08-31 (HTML), 2005-09-28 (WAV)

Supplementary Material

ISO/IEC 15444, “Information technology — JPEG 2000 image coding system” (Official specification for JPEG 2000 format.)
JPEG2000: Standard for Interactive Imaging (Taubman and Marcellin – Jul 31, 2002)

JPEG 2000 Overview (Bradford Castalia – Sep 8, 2003)
JPEG 2000: A PDS Perspective (PDF, PPT) (Elizabeth Rye – May 23, 2005)
JPEG 2000: Performance / Status Presentation (PDF, PPT) (Myche McAuley – Aug 31, 2005)
JPEG 2000 as a PDS Archive Format: Supplementary Material (PDF, PPT) (Elizabeth Rye – Sep 28, 2005)
JPEG 2000 as a PDS Archive Format: Introduction and Status (PDF, PPT) (Presentation to PDS Management Council) (Myche McAuley and Elizabeth Rye – Oct 6, 2005)

Status

Status	Date	Task	Responsible Party	Response
SUBMITTED	2003-08-22
IN_PROGRESS	2004-04-19
DRAFT	2005-09-27
APPROVED	2005-10-17
IN_IMPLEMENTATION	2005-10-24	Update queue to Implementation.	EN-SE	Queue updated by E. Rye on 10/17/05.
		Develop tool requirements.	EN-SE	JPEG 2000 decompression code needs to be added to tools and tools need to be able to find the UNCOMPRESSED FILE object after decompressing the file.
		Make updates to tools.	EN-DEV	Assigned to M. Cayanan 03/29/06.
		Spoke to S. Hardman and M. Cayanan on 07/18/06 asking about progress. M. Cayanan said this had been looked into but wasn’t going to be an easy task. They said they’d bump it up the priority list.
		Sent an email to S. Hardman and M. Cayanan on 04/20/07 asking about status.
		M. Cayanan responded on 04/20/07 that when he and D. Schultz looked into this in March of 2006, they couldn’t find open source decompression software, but have since found it via the JasPer project. M. Cayanan requested sample labels to test against. E. Rye responded the same day directing him to contact M. McAuley for help in locating both software and labels.
		Submit element definition template(s) to EN-OPS team.	EN-SE	Element defs. for ENCODING_TYPE_VERSION_NAME and ENCODING_TYPE submitted to S. Adams by E. Rye on 01/12/06.
		Ingest element definition template(s) into PSDD.	EN-OPS	S. Adams submitted templates to J. Ho 04/18/06.
		New keywords ingested into online database by J. Ho on 4/26/06.
		Make updates to Standards Reference.	EN-SE	Updates to StdRef completed by E. Rye 01/11/06.
		Submit revised StdRef to EN-OPS team.	EN-SE	Updated StdRef sent to E. Law by E. Rye on 03/21/06.

Votes

Click on nodes for comments.

Node	Representative	Version 4
ATM	Lyle Huber	yes
EN	Steve Hughes	yes
GEO	Susie Slavney	yes
IMG	Myche McAuley	yes
NAIF	Lee Elson	abstain
PPI	Todd King	no
RINGS	Mitch Gordon	yes
RS	Dick Simpson	no
SBN	Anne Raugh	no

The Tech group voted to RECOMMEND version 5 of the SCR with a final tally of 6-0-3. The vote took place by email Sept. 29 – 30, 2005.

Node	Representative	Version 5
ATM	Lyle Huber	yes
EN	Steve Hughes	yes
GEO	Susie Slavney	yes
IMG	Myche McAuley	yes
NAIF	Lee Elson
PPI	Todd King
RINGS	Mitch Gordon	yes
RS	Dick Simpson	yes
SBN	Anne Raugh

The Management Council voted to APPROVE the SCR with a final vote of 8(5)-1(1)-0. (The numbers in parentheses are the number of science node votes.) The vote took place by email Oct. 13 – 27, 2005 on version 6 of the SCR.

Node	Representative	Version 6
ATM	Reta Beebe	yes
GEO	Ed Guinness	yes
IMG	Lisa Gaddis	yes
PPI	Ray Walker	no
RINGS	Mark Showalter	yes
SBN	Mike A’Hearn	yes
EN	Dan Crichton	yes
NAIF	Chuck Acton	yes
RS	Dick Simpson	yes