Minutes from Aug. 1, 2005 JPEG2000 Meeting with Imaging Node Attendees: Alice Betty Elizabeth Myche Patty Ron Sue Proposed list of issues from E. Rye prior to meeting: When using an externally maintained format as a core part of a PDS archive, we need to: 1) verify that the data product is compliant with the externally maintained format 2) include in the product meta-data an indication of the version of the external format with which the product complies 3) verify that the archive includes decompression software (including source code) 3a) also include software that can convert the product to a standard PDS decompressed product (in a manner similar to vdcomp, clemdcmp, and readmoc) 4) verify that the product can be successfully decompressed using the archived decompression algorithm 5) verify that the PDS has detailed documentation regarding the external format at the version level of any products in its holdings 6) verify that all science related meta-data is stored in the PDS header, not just in format specific headers Specifically regarding the JPEG2000 / HiRISE products, we'll need to work out the following additional issues: 7) which of the JPEG2000 data compression capabilities (progressive decompression, image tiling, etc.) are we going to take advantage of? 8) which parameters need to be captured in the meta-data to indicate the presence of these features and how they are utilized? 9) come up with keywords for the above parameters 10) come up with a standard value for the JPEG2000 compression scheme (trivial) 11) does the HiRISE team intend to archive JPEG2000 compressed image data with attached PDS headers (and .IMQ extension) or JP2 formatted images (.JP2 extension) and detached PDS headers? 12) if using the JP2 format, does the HiRISE team intend to store science meta-data in the JP2 header? If so, which meta-data and what is the meta-data language that they are going to use? (To what extent can we build on work that has already been done by other organizations?) Also, how will they verify that the meta-data in the JP2 header matches that in the PDS header? Finally, separate from the whole standards issue, work will need to be done to determine how to take advantage of JPEG2000 capabilities in searching for and delivering products (or portions of products) over the internet through our various data interfaces and browsers. (I consider my involvement in this area to be only peripheral.) ------------------------------------------------ Patty believes Dick Simpson needs to be involved in working group. need to explain why the alternatives are not good enough: Huffman first difference, LZW issues with bit errors random access (Myche and I to work this issue) Under impact - what if worst comes to worst and somebody successfully proves licensing rights to format after we've produced an archive? Address this issue as how we'd respond (re-process to other format, or leave in JP2 on our servers but only allow public access in other formats) Decisions we're going to make and see how HiRISE reacts: JP2 format with detached labels lossless compression only must stick to the core standard all meta-data must be in at least the PDS header and PDS header must contain the latest/greatest info Concern: documentation - ability to "own" the document describing the standard in detail; ie., since this is an ISO document, we can't distribute it freely with our archives. Can we negotiate with ISO to acquire freely distributable copies of documentation when and if current version is superceded? As an alternative, make sure that archive includes well documented decompression software, including source code. According to Patty, this is a problem since Valerie sent her an email (April 28) stating that JPL document review no longer permits the distribution of any software. The technical steps: - determining which capabilities we can take advantage of (and understanding the details of what that does to the data) (Myche and Elizabeth) - what parameters need to be tracked; come up with keywords (Myche, Elizabeth, and Betty) - what work has already been done on defining meta-data formats for JP2 headers (Chris) Validation Requirements we propose: (as always) validate meta-data for all products validate process for producing compliant JP2 images spot check actual images Action Items: - Patty will send me contact info for Eric. - find out if there is such a thing as an uncompressed RDR (Elizabeth) - Patty will check on MRO freeze date for GDS for RDRs - Elizabeth send to Sue decisions that we've already made Tasks / Roles: Elizabeth, Myche work on specification, come up with reasons why other compression schemes won't work Patty interfaces with HiRISE team - GDS freeze date, what they're planning to do, how do they react to our work Chris - JP2 meta data in headers Betty - helping to write up element object definitions for new keywords Valerie - get her to persue issues with not being to include decompression software in an archive Schedule: January 2007 - first delivery? published deadline for standard is April 1, 2006 our internal deadline for the standard is January 1, 2006 solid draft ready for tech group discussion November 1, 2006 first draft to working group October 1, 2006 (then Dick becomes involved) software demo of compression, decompression (HiRISE team, when?) Deliverables: updated / new chapter of StdRef updated SCR, including relevant (IMAGE OBJECT) component of the label Working group: Elizabeth, Myche, Patty, Chris, Brad, and Steve Hughes, Ron Joyner, Dick, Sue, Betty management council mtg is first week of October