Planetary Data System
Engineering Node

PDS3 Standards: Individual Issue

SCR 3-1003: JPEG 2000 as PDS Archive Format

Priority: urgent

Problem Summary: The MRO HiRISE (High Resolution Imaging Science Experiment) team is currently predicting the production of 11,600 GB of RDR products. The maximum sizes of individual RDR products are expected to be about 5 GB. This large volume of data and the large size of the individual products raises several data management issues involving data storage and the distribution of products over the internet.

The MRO HiRISE team is suggesting that the JPEG 2000 compression scheme be used to compress the HiRISE products for use in both online data repositories and for the PDS archive. The JPEG 2000 compression scheme provides a good compression ratio (~3:1 to 4:1 lossless) and uses a wavelet encoding scheme to allow easy viewing of extremely large images. The use of the JPEG 2000 compression scheme will (in the words of the team) result in lower data volume requirements, smaller storage needs, lower media costs, more easily managed data sets, lower per-unit count of hard media, and faster transfer times. It will also preclude the need for separate product tiling.

Note that the MRO HiRISE team is proposing JPEG 2000 for HiRISE RDRs only.

Working group: E. Rye, S. LaVoie (leads) B. Castalia, P. Garcia, S. Hughes, C. Isbell, R. Joyner, M. McAuley, B. Sword

Originator: E. Eliason

Standards Change Request:

Version Date Author(s) Format Note
1 2003-08-22 S. Hughes TEXT
2 2004-04-19 J. Wilf TEXT
3 2005-09-27 E. Rye PDF
4 2005-09-28 E. Rye PDF
5 2005-09-30 R. Simpson & E. Rye PDF clean
PDF with tracked changes
6 2005-10-10 E. Rye PDF clean
PDF with tracked changes

Discussion:

Supplementary Material:

  • JPEG 2000 Overview (Bradford Castalia - Sep 8, 2003)
  • JPEG 2000: A PDS Perspective (PDF, PPT) (Elizabeth Rye - May 23, 2005)
  • JPEG 2000: Performance / Status Presentation (PDF, PPT) (Myche McAuley - Aug 31, 2005)
  • JPEG 2000 as a PDS Archive Format: Supplementary Material (PDF, PPT) (Elizabeth Rye - Sep 28, 2005)
  • JPEG 2000 as a PDS Archive Format: Introduction and Status (PDF, PPT) (Presentation to PDS Management Council) (Myche McAuley and Elizabeth Rye - Oct 6, 2005)

Status:

Status Task ResponsibleParty Response
SUBMITTED 2003-08-22
IN_PROGRESS 2004-04-19
DRAFT 2005-09-27
APPROVED 2005-10-17
IN_IMPLEMENTATION 2005-10-24 Update queue to Implementation. EN-SE Queue updated by E. Rye on 10/17/05.
Develop tool requirements. EN-SE JPEG 2000 decompression code needs to be added to tools and tools need to be able to find the UNCOMPRESSED FILE object after decompressing the file.
Make updates to tools. EN-DEV Assigned to M. Cayanan 03/29/06.
Spoke to S. Hardman and M. Cayanan on 07/18/06 asking about progress. M. Cayanan said this had been looked into but wasn't going to be an easy task. They said they'd bump it up the priority list.
Sent an email to S. Hardman and M. Cayanan on 04/20/07 asking about status.
M. Cayanan responded on 04/20/07 that when he and D. Schultz looked into this in March of 2006, they couldn't find open source decompression software, but have since found it via the JasPer project. M. Cayanan requested sample labels to test against. E. Rye responded the same day directing him to contact M. McAuley for help in locating both software and labels.
Submit element definition template(s) to EN-OPS team. EN-SE Element defs. for ENCODING_TYPE_VERSION_NAME and ENCODING_TYPE submitted to S. Adams by E. Rye on 01/12/06.
Ingest element definition template(s) into PSDD. EN-OPS S. Adams submitted templates to J. Ho 04/18/06.
New keywords ingested into online database by J. Ho on 4/26/06.
Make updates to Standards Reference. EN-SE Updates to StdRef completed by E. Rye 01/11/06.
Submit revised StdRef to EN-OPS team. EN-SE Updated StdRef sent to E. Law by E. Rye on 03/21/06.

Votes:

Node Representative Version 4
ATM Lyle Huber yes
EN Steve Hughes yes
GEO Susie Slavney yes
IMG Myche McAuley yes
NAIF Lee Elson abstain
PPI Todd King no
RINGS Mitch Gordon yes
RS Dick Simpson no
SBN Anne Raugh no

The Tech group voted to RECOMMEND version 5 of the SCR with a final tally of 6-0-3. The vote took place by email Sept. 29 - 30, 2005. Click on nodes below for comments.

Node Representative Version 5
ATM Lyle Huber yes
EN Steve Hughes yes
GEO Susie Slavney yes
IMG Myche McAuley yes
NAIF Lee Elson
PPI Todd King
RINGS Mitch Gordon yes
RS Dick Simpson yes
SBN Anne Raugh

The Management Council voted to APPROVE the SCR with a final vote of 8(5)-1(1)-0. (The numbers in parentheses are the number of science node votes.) The vote took place by email Oct. 13 - 27, 2005 on version 6 of the SCR.

Node Representative Version 6
Science Nodes ATM Reta Beebe yes
GEO Ed Guinness yes
IMG Lisa Gaddis yes
PPI Ray Walker no
RINGS Mark Showalter yes
SBN Mike A'Hearn yes
Support Nodes EN Dan Crichton yes
NAIF Chuck Acton yes
RS Dick Simpson yes