From: Michael McAuley To: Elizabeth D Rye Date: Mon, 6 Mar 2006 11:29:00 -0800 Subject: Re: question about MD5 checksums On Mar 6, 2006, at 9:53 AM, Elizabeth D Rye wrote: > Hi Myche, > > Do I correctly recall you once telling me that it is possible > (albeit unlikely) to have to distinct files with the same MD5 > checksum? I have a vague recollection of you telling me that in the > case of two adjacent bytes within the same record being swapped, the > checksum might be the same. Is this correct, or am I > mis-remembering? What I commented on earlier was the fact that MD5 has been shown to have collisions. That is, two different inputs to a hash function produces identical outputs. The case where just two adjacent bytes are swapped typically does not result in such a collision. This isn't as bad as it sounds for our use, though. If you think about it, any practical hash algorithm must accept an infinite number of inputs. Since the output is not infinite, there _must_ be collisions. The question becomes: are the collisions obtainable and exploitable in a feasible manner? I only mentioned this collision issue so that we would all be aware of it. I don't think it, necessarily, disqualifies MD5 for use within PDS. There are better algorithms that we may wish to avail ourselves of (e.g. SHA-1). However, as I've stated above, _no_ algorithm can be perfect if it must accept all possible input sequences. As we would use such a hash to perform file integrity checks, the only malicious use of the collisions that I see are for someone to intentionally place a bogus file within our collection that has the same sum as the original. This requires more than being able to create a bogus file with the same checksum. It also requires access to our servers and the permissions to change the file system. Other attacks seem even less likely (spoofing file transfers, etc...) Here's some references for further reading: http://en.wikipedia.org/wiki/MD5 and from this page: http://en.wikipedia.org/wiki/Hash_collision http://www.rtfm.com/movabletype/archives/2004_08.html#001059 also on same page: http://www.rtfm.com/movabletype/archives/2004_08.html#001056 http://www.rtfm.com/movabletype/archives/2004_08.html#001055 M ----------------------------------------------------------------------- From: Elizabeth D Rye To: pds_tech@schema.jpl.nasa.gov Date: Mon, 6 Mar 2006 11:33:09 -0800 Subject: Checksum SCR for Wednesday's telecon Hello All, As promised, attached is the Checksum SCR for Wednesday's Standards telecon. Elizabeth ("scr3-1034.v3.pdf" attached) ----------------------------------------------------------------------- From: Dick Simpson 650-723-3525 To: Elizabeth.D.Rye@jpl.nasa.gov Date: Mon, 6 Mar 2006 13:11:40 -0800 (PST) Subject: Re: Checksum SCR for Wednesday's telecon >As promised ... Except you promised "within the hour" and mine arrived 61 minutes later. But you're forgiven. Is there a Word version of the SCR? It would be easier to mark up a Word version than submit a page of free-form text where 75% of the text is used to locate the points for discussion. You may have hoped to preserve the integrity of the SCR by sending it out as PDF; but for those of us who refuse to be silenced, the comments will still bubble up. I can actually see this one happening now; but it's not ready yet. Rather than a radio active curmudgeon, think of me as ... Your friend, Dick ----------------------------------------------------------------------- From: "J. Steven Hughes" To: Elizabeth D Rye Date: Mon, 06 Mar 2006 13:37:15 -0800 Subject: Re: Standards Telecon this Wednesday, March 8 I would send the question to Sean. He can have Mike look into it. steve ----------------------------------------------------------------------- From: Elizabeth D Rye To: Sean Hardman Date: Mon, 6 Mar 2006 13:52:13 -0800 Subject: Fwd: Standards Telecon this Wednesday, March 8 Hi Sean, We've run across a problem on a standards issue that we need your input on (or perhaps you could assign someone to look into it for us - Steve suggested Mike). We're trying to put together a standard for calculating and storing checksums, and are particularly interested at the moment in using MD5 checksums. These are typically represented as 32-character hexadecimal strings. Strictly speaking, we should be representing these as the hexadecimal numbers that they are (in ODL notation; see 12.3.1.2 in the Standards Reference): MD5_CHECKSUM = 16#737F5DA689CD2846E17DB3C290AFF21C3E7F# However, it was pointed out by folks at our last meeting on this topic that our computers and software may not have the capacity to accurately handle and compare two 32-character hexadecimal numbers. If this is the case, then we'd be forced to compare them as character strings instead (which would be a headache from the standards point of view, but may be necessary nonetheless). Could you find out for me if these can be handled as numbers, or whether we need to do the comparison as strings? If at all possible, I'd really appreciate an answer by Wednesday morning. Thanks! Elizabeth ----------------------------------------------------------------------- From: Sean Hardman To: Elizabeth D Rye Date: Mon, 6 Mar 2006 14:25:39 -0800 Subject: Re: Fwd: Standards Telecon this Wednesday, March 8 Elizabeth, Yes, I will have Mike look at it, but he is out sick today. I will ask him to look at first thing tomorrow morning. Sean ----------------------------------------------------------------------- From: "Todd King" To: "'Elizabeth D Rye'" , Cc: Subject: RE: Checksum SCR for Wednesday's telecon Date: Mon, 6 Mar 2006 16:03:24 -0800 Hi Elizabeth - I'll share my concerns with the new Checksum SCR ahead of the telecon in the hope that we can resolve them more quickly. 1) It states that the intent to "create and maintain a catalog of checksum values". That is a very big task which will be prone to discrepancies. PPI alone has over 2 million products with each product consisting of at least 2 files and as many as 5. So that means tracking about 5-7 million files. The current catalog has trouble tracking a few thousand datasets. I think we need a more self-contained and incremental solution. 2) The parenthetical comment on checksums in labels is a very important feature. The original reason for the checksum SCR was to provide a self-documented completeness check. A checksum in a label for each object is far better than an external catalog of checksums. I think we need to consider that a corruption of a label will be far more obvious then a corruption of a binary data file. So, the need to have a checksum for the label is less important. Besides the archive will outlive any services we can provide. It must stand alone. 3) The "CHECKSUM.TAB" and "CHECKSUM.LBL" are marked as "optional". If we adopt this approach they should be required for all future volumes. 4) The "Appendix D, Section D.2" reference in the text identifies examples for the "CUMINDEX.TAB" and "CUMINDEX.LBL". Shouldn't these be "CHECKSUM.TAB" and "CHECKUM.LBL". 5) Changes to tool suites - Why should we write a PDS tool to do what existing tools already do? The example CHECKSUM.TAB seems to be designed so you can't use existing tools. More on this later, but here the question is "Why?" 6) Impact assessment: "(1) It remains to be determined if PDS computers and software are capable of handling 32 character hexadecimal." Huh? I can run MD5 software on a 16 bit PC. Of course they can. The definition of the MD5 checksum as a "128-bit number, represented as a 32 character string of hexadecimal digits" tells us all we need to know. The MD5 checksum is a *string*. So the MD5_CHECKSUM value should be of type string. No waivers for using strings are needed. 7) Example CHECKSUM.TAB and CHECKSUM.LBL - The MD5 checksum example should use the well established format used by the md5deep program. This would allow an existing program to use the data without modification. It would also allow an existing program to generate the file. No need for some special PDS tool to work with some special PDS format that no one else uses. I won't even touch whether a fixed length of 77 bytes is enough to hold checksum+filepath in all situations. 8) CHECKSUM_TYPE: The way that CHECKSUM_TYPE is used it would be more appropriate to make it a DATA_TYPE. Just like a DATA_TYPE of DATE means a character string in PDS date format, a DATA_TYPE of MD5_CHECKSUM would mean a 32 character hexadecimal string. 9) The text in the first paragraph of the element definition for MD5_CHECKSUM is useful. The rest of the text is not. I think this version of the checksum SCR has strayed way off from the original intent and will make it more difficult to implement then is necessary. We don't need to create new data formats and new tools to do something I can already do with existing tools. For an MD5 checksum all we need to do is preserve the MD5 algorithm (RFC 1321) in the archive so that the values can be properly interpreted when a tool no longer exists. -Todd- ----------------------------------------------------------------------- From: Elizabeth D Rye To: "Todd King" Cc: , Date: Mon, 6 Mar 2006 16:36:49 -0800 Subject: Re: Checksum SCR for Wednesday's telecon Hi Todd, > I'll share my concerns with the new Checksum SCR ahead of the > telecon in the hope that we can resolve them more quickly. > 1) It > states that the intent to "create and maintain a catalog of checksum > values". That is a very big task which will be prone to > discrepancies. PPI alone has over 2 million products with each > product consisting of at least 2 files and as many as 5. So that > means tracking about 5-7 million files. The current catalog has > trouble tracking a few thousand datasets. I think we need a more > self-contained and incremental solution. I didn't mean catalog in the sense that you're taking it. I simply meant it as in "cataloging" the checksums for a specific archive. They can be stored in the "checksums.tab" file included in the archive, in a file external to the archive, in a database, or all of the above. The extent to which any particular node decides to "catalog" their entire archive is up to them (or up to overall PDS policy, which is not the subject of this SCR). > 2) The parenthetical comment on checksums in labels is a very > important feature. The original reason for the checksum SCR was to > provide a self-documented completeness check. A checksum in a label > for each object is far better than an external catalog of > checksums. I think we need to consider that a corruption of a label > will be far more obvious then a corruption of a binary data > file. So, the need to have a checksum for the label is less > important. Besides the archive will outlive any services we can > provide. It must stand alone. I grant that it's important, but we already have (and have long had) this capability. (And let's keep in mind that both are important; checksums in individual labels are most useful to end recipients of single data files whereas external catalogs of all the checksums in an archive are most useful for periodically validating the integrity of the whole archive.) The CHECKSUM keyword has been around for a while and has been used in data product labels in numerous archived data sets. And I disagree that the original SCR focused on checksums in labels over an external catalog - version 2 of the SCR focused fairly significantly on the md5deep utility and explaining the format of the file it produced. This version simply updates the treatment of that file (now called "checksum.tab") in response to the discussion held by the standards group. As an aside, I so strongly believe in the importance of checksums in labels because I still believe there is an important need for checksums of individual data objects, not just files. However, I intentionally left that out of this SCR so as not to complicate matters; those issues can be resolved by resurrecting SCR 3-1035. Finally, I'm not sure what you're getting at with the final sentence about archives outliving services, unless it's related to the confusion above about what is meant by a "catalog". Again, the "catalog" I'm referring to could simply be the checksums.tab file included in the archive. > 3) The "CHECKSUM.TAB" and "CHECKSUM.LBL" are marked as > "optional". If we adopt this approach they should be required for > all future volumes. We argued about this extensively during the telecon. At least two nodes indicated they would not vote for the SCR if checksums were required. In the end, we decided to make the SCR about the standards used to implement checksums and allow the policy debate about whether or not they're required to take place elsewhere. > 4) The "Appendix D, Section D.2" reference in the text identifies > examples for the "CUMINDEX.TAB" and "CUMINDEX.LBL". Shouldn't these > be "CHECKSUM.TAB" and "CHECKUM.LBL". Yup. Needs to be fixed. > 5) Changes to tool suites - Why should we write a PDS tool to do > what existing tools already do? The example CHECKSUM.TAB seems to be > designed so you can't use existing tools. More on this later, but > here the question is "Why?" Because objections were raised to tying ourselves to a specific type of checksum and even more to a specific utility. The consensus was that we should come up with a general approach that would be flexible with future improved types of checksums. > 6) Impact assessment: "(1) It remains to be determined if PDS > computers and software are capable of handling 32 character > hexadecimal." Huh? I can run MD5 software on a 16 bit PC. Of course > they can. The definition of the MD5 checksum as a "128-bit number, > represented as a 32 character string of hexadecimal digits" tells us > all we need to know. The MD5 checksum is a *string*. So the > MD5_CHECKSUM value should be of type string. No waivers for using > strings are needed. It's technically not a string, it's a number, albeit represented in hexadecimal. That's still a number. (In ODL-speak, an "integer number in based notation"; see 12.3.1.2 in the Standards Reference.) We debated this for a long time and the issue still isn't comfortably settled. > 7)Example CHECKSUM.TAB and CHECKSUM.LBL - The MD5 checksum example > should use the well established format used by the md5deep > program. This would allow an existing program to use the data > without modification. It would also allow an existing program to > generate the file. No need for some special PDS tool to work with > some special PDS format that no one else uses. I won't even touch > whether a fixed length of 77 bytes is enough to hold > checksum+filepath in all situations. See above comment in response to 5. > 8) CHECKSUM_TYPE: The way that CHECKSUM_TYPE is used it would be > more appropriate to make it a DATA_TYPE. Just like a DATA_TYPE of > DATE means a character string in PDS date format, a DATA_TYPE of > MD5_CHECKSUM would mean a 32 character hexadecimal string. DATA_TYPE wouldn't necessarily capture differences in the way the checksum was calculated, just the way it was represented. You could have multiple different algorithms that all represented their results as a 32 character hexadecimal number. > 9) The text in the first paragraph of the element definition for > MD5_CHECKSUM is useful. The rest of the text is not. I don't have strong feelings one way or the other on this. I'd be willing to truncate it if others agree with you. > I think this version of the checksum SCR has strayed way off from > the original intent and will make it more difficult to implement > then is necessary. We don't need to create new data formats and new > tools to do something I can already do with existing tools. For an > MD5 checksum all we need to do is preserve the MD5 algorithm (RFC > 1321) in the archive so that the values can be properly interpreted > when a tool no longer exists. While slightly more difficult to implement in the near term, this more generic approach is designed to prevent us from having to re-invent the wheel when the next latest, greatest algorithm for calculating checksums comes along. (And it's frankly not that difficult; most of us could put a script together in less than an hour that would get the job done. Granted, a formal PDS tool would require more of a time investment, but this is not a difficult task.) This is the approach the group recommended taking during our discussion in the standards telecon. Elizabeth ----------------------------------------------------------------------- From: Dick Simpson 650-723-3525 To: Elizabeth.D.Rye@jpl.nasa.gov Date: Mon, 6 Mar 2006 17:28:32 -0800 (PST) Subject: Re: Checksum SCR for Wednesday's telecon >Maybe I should have just sent out the Word version, rather than >converting it to PDF - then it would have been on time. ;-) The value-added stuff always seems to take a lot more time than you ever imagine. >Word version attached. Thanks. In glancing over Todd's comments and your response, I'm surprised there's so little overlap. But I'll get this annotated and back to you and him (and maybe McAuley) before I leave. ----------------------------------------------------------------------- From: Dick Simpson 650-723-3525 To: elizabeth.rye@jpl.nasa.gov, myche.mcauley@jpl.nasa.gov, tking@igpp.ucla.edu, rsimpson@magellan.stanford.edu Date: Mon, 6 Mar 2006 18:12:10 -0800 (PST) Subject: Re: checksum SCR Elizabeth, Todd, Myche: Like Todd, I had some quick reactions to the checksum SCR. My comments parallel Todd's in some areas but at least have a different emphasis. For example, I can see bundling lots of things together (different granularities) and I can see alternatives to MD5. I have marked up the original SCR. If anyone wants to distribute this to the full list, feel free; I'm sending it only to the three "principals". Dick --- ("scr3-1034.v3_ras.doc" attached) ----------------------------------------------------------------------- From: "Bill Harris" To: Date: Mon, 6 Mar 2006 19:08:52 -0800 Subject: RE: Checksum SCR for Wednesday's telecon Afraid I'll have to add to the noise on this one: 3) -- I'm concerned that two nodes are not interested in requiring checksums for future products. Isn't ensuring the integrity of data one of our top priorities? Is this a workload problem? 5) -- I agree with Todd on our need to avoid adding programming work when a wide variety of tools already exists for crunching checksums. Adding a script to the verification toolbox would be useful but I don't think it should go beyond using or calling an already existing tool, such as in Java. This would decrease the work for us and ensure that the code used to produce and verify our checksums has been widely and tested by others. Using the more popular programs for handling checksums also makes it easier for data providers and end users. In the case of MD5, a standard format for its usage already exists and is supported by the most popular software. When it comes to adopting new checksums, I think we would benefit the most by using the most common tools and methods. As for using other algorithms, now or in the future, would we even want to consider relying on checksums that aren't as thoroughly tested and supported as MD5 and SHA-1 are today? 6) -- Since an MD5 checksum would be written to an ASCII file for later use, it would be easiest if any tool we use for generating or checking MD5s could handle character strings. 8) -- I believe what Todd means by using DATA_TYPE is that the type would be "MD5_CHECKSUM", and from that we know how to use the checksum; not that any 32-byte hex should be considered to be MD5. I think we can safely assume that if we mistake a checksum for an MD5 when it is not, we'll know something is wrong when we get the verification error. - B Harris ----------------------------------------------------------------------- From: raugh@astro.umd.edu (Anne Raugh) To: pds_tech@schema.jpl.nasa.gov, bharris@igpp.ucla.edu Date: Tue, 7 Mar 2006 06:51:58 -0500 (EST) Subject: RE: Checksum SCR for Wednesday's telecon > 3) -- I'm concerned that two nodes are not interested in requiring > checksums for future products. Isn't ensuring the integrity of data > one of our top priorities? Is this a workload problem? Well, if one of those nodes was the SBN, then let me just say this: A funny thing happened on the road to Damascus... -Anne. ----------------------------------------------------------------------- From: Mitch Gordon To: pds_tech@schema.jpl.nasa.gov CC: bharris@igpp.ucla.edu Date: Tue, 07 Mar 2006 09:46:19 -0500 Subject: Re: Checksum SCR for Wednesday's telecon Good Morning, >3) -- I'm concerned that two nodes are not interested in requiring >checksums for future products. Isn't ensuring the integrity of >data one of our top priorities? Is this a workload problem? Actually, the concern was changing requirements for active mission such as Cassini. Here is the suggestion I put forward: 1. We've all been concerned about increasing the requirements on data providers. In the SCR, as we discussed on the telecon, use "strongly recommend (encourage) data providers to provide a checksum". Plan to make that mandatory in PDS 4. 2. Recommend to the management council a new policy that requires the use of checksums within the PDS, and implement that policy immediately. Can we accomplish this by making checksums required in PDS 3 and then granting waivers to the missions which already have established pipelines? The curating node for such a mission's data sets would be required to generate and incorporate the checksum file when they archive the data. This also would provide the requirement for internal PDS use at each stage. We'd need to make it clear that the waivers were only for current missions and would not be granted in the future. >5) -- I think most of us agree that it makes sense to start with MD5. But I really think that we need to keep our options open for the future, and I don't think it costs us much to do so. I'm a little uncertain about '2. Establish a reserved file, "CHECKSUM.TAB", which contains checksum values for all files in an archive, to be optionally included in the INDEX directory of the archive.' While this makes sense from a PDS perspective, didn't I hear on the telecon that MD5 requires that the file be in the root directory? Mitch ----------------------------------------------------------------------- From: raugh@astro.umd.edu (Anne Raugh) To: pds_tech@schema.jpl.nasa.gov, mgordon@seti.org Cc: bharris@igpp.ucla.edu Date: Tue, 7 Mar 2006 10:05:08 -0500 (EST) Subject: Re: Checksum SCR for Wednesday's telecon > '2. Establish a reserved file, "CHECKSUM.TAB", which contains checksum > values for all files in an archive, to be optionally included in the > INDEX directory of the archive.' > > While this makes sense from a PDS perspective, didn't I hear on the > telecon that MD5 requires that the file be in the root directory? No, what I believe you heard was that a specific checksum utility required it (followed by arguments that the PDS SR should not be written to accommodate a particular contemporary bit of software). The checksum utility I wrote, for example, takes the location of the checksum file as an argument, but makes other environment-specific assumptions. -A. ----------------------------------------------------------------------- From: "Todd King" To: Date: Tue, 7 Mar 2006 10:19:12 -0800 Subject: RE: Checksum SCR for Wednesday's telecon Hi all - After distilling all the comments buzzing around the checksum SCR I think the current SCR is really trying to address a very simple question. If we have a file containing checksums how do we label it? A listing of checksums is quite similar to an index so using the model of the INDEX tables seems appropriate. With INDEX tables there are required fields and optional fields. No order for the required fields is specified. With this mind the CHECKSUM table could have the required fields of PATH_NAME and CHECKSUM. The type of checksum would be identified either in the label (for those with a uniform type of checksum or in the CHECKSUM table as an additional column (CHECKSUM_TYPE). As with the INDEX tables the order of the required fields would not be specified so that format of the CHECKSUM would be flexible enough to allow different approaches. For example, the checksum could be the first field and path name the second or vice-a-versa. The SCR points out, rather indirectly, that the current data dictionary lacks a way to identify the algorithm used to calculate a checksum value that is stored as a COLUMN in a table. The SCR proposes we create an CHECKSUM_TYPE element to identify the algorithm, but it leaves the value of the required DATA_TYPE unspecified since there is no existing data type which supports a 128 bit number. Alternatively we could introduce a new DATA_TYPE of MD5_CHECKSUM to have a very concise and specific meaning. That is, "it is a 26 byte hexadecimal string representing a 128 bit number generated using the MD5 algorithm". Other checksum data types would have unique DATA_TYPE. For example, the "simple" would have a DATA_TYPE of SIMPLE_CHECKSUM or 32BIT_CHECKSUM or PDS_CHECKSUM. The SHA1 checksum, which is a 160 bit number, would be SHA1_CHECKSUM. Representing checksums as strings does give us the flexibility to add other checksums. Checksums do vary in bit length, MD5 is 128 bits, SHA1 is 160 bits and other checksum have longer bit lengths. -Todd- ----------------------------------------------------------------------- From: "Todd King" To: "'Dick Simpson 650-723-3525'" , , Date: Tue, 7 Mar 2006 10:28:31 -0800 Subject: RE: checksum SCR Hi Elizabeth - On the SCR I'm identified as the "lead" for the checksum working group. I'd like to re-assert that role. If I'm not to have that role I don't wish to be identified as the lead. Also, as a member of the working group I would have liked to have participated in the writing of the SCR and at the very least been given an opportunity to review it before it was circulated to the entire the tech group. I could have raised my concerns within the working group rather than in an open forum. We did not have any working group meetings (or telecons) since you took on the role of SCR lead. -Todd- ----------------------------------------------------------------------- From: "Susan Slavney" To: Date: Tue, 7 Mar 2006 12:41:00 -0600 Subject: RE: Checksum SCR for Wednesday's telecon Dear PDS Techies, In the interest of smoothing the way for tomorrow's checksum discussion, let's break down the problem into pieces. There are several issues involved, and I think it may help us to consider them separately. In the following breakdown I'll also give the opinion of Geosciences at each step. 1. Should data providers be required to deliver checksums along with their data? Geo answers YES. I think most of this group would also answer yes. A yes answer leads to a corollary question: 1a. WHEN should PDS start requiring data providers to deliver checksums? I propose we do NOT try to answer this question during tomorrow's telecon. It will just distract us from the business at hand. It needs a separate discussion. 2. Should checksums be stored on the archive volume, or separate from the archive volume (e.g. in a database), or both? Geo believes checksums should be stored separately from the archive, because if they are stored on the archive, they are subject to corruption just like the archive itself, and therefore useless for validating it. This is the reason we do not support making a checksum table a required component of the archive. 3. In what format should checksums be stored? If checksums are stored separately from the archive, Geo believes the format should be agreed upon between the node and the data provider. There is no point in developing a PDS-specific format for a checksum file if the file is not stored on the archive. If checksums are stored on the archive, Geo still believes the format should not be PDS-specific, but rather should be one of the commonly-used formats with readily available tools, to be agreed upon between the node and data provider. Our reasons are: a) A PDS-specific format imposes a burden on the data provider to obtain and use a PDS tool to generate the checksum file. b) A PDS-specific format does not add any value to the checksum file; a checksum file in a commonly-used format is just as good for validation. c) A PDS-specific format imposes a burden on the PDS node receiving the checksum file to obtain and use a PDS tool to validate the archive. d) A PDS-specific format imposes a burden on the user who downloads all or part of an archive to obtain and use a PDS tool to validate the archive. (You just KNOW those users are gonna holler if they can't use their favorite checksum tool.) e) It seems to go against common sense to spend our limited tool-making resources to make checksum tools, when there are existing tools already in common use that do a perfectly good job. That's it. I hope this helps us focus on where we really disagree, so we can spend our phone time productively. Susie ----------------------------------------------------------------------- From: Elizabeth D Rye To: pds_tech@schema.jpl.nasa.gov Date: Tue, 7 Mar 2006 11:32:50 -0800 Subject: Fwd: checksum SCR Hi Folks, Dick's comments on the SCR are attached. Elizabeth Begin forwarded message: > Elizabeth, Todd, Myche: Like Todd, I had some quick reactions to the > checksum SCR. My comments parallel Todd's in some areas but at > least have a different emphasis. For example, I can see bundling > lots of things together (different granularities) and I can see > alternatives to MD5. I have marked up the original SCR. If anyone > wants to distribute this to the full list, feel free; I'm sending > it only to the three "principals". Dick ----------------------------------------------------------------------- From: Sean Hardman To: Elizabeth Rye Cc: Michael Cayanan , Steve Hughes , Sean Hardman Date: Tue, 7 Mar 2006 11:43:02 -0800 Subject: Re: Fwd: Standards Telecon this Wednesday, March 8 Elizabeth, Well, Mike more or less confirmed what I was thinking. The 32 character hex number represents a 128-bit integer which far exceeds the capabilities of our 32-bit and even 64-bit operating systems. The maximum signed integer is listed in the lvtool error below. The md5 hex value converts to the following integer: 1.5352254067491483e+38 So, it looks like character string comparisons are in order. Sean ----------------------------------------------------------------------- From: Elizabeth D Rye To: Sean Hardman Date: Tue, 7 Mar 2006 14:46:29 -0800 Subject: Re: Standards Telecon this Wednesday, March 8 Thanks Sean, much appreciated. Elizabeth ----------------------------------------------------------------------- ----------------------------------------------------------------------- From: Elizabeth D Rye To: Sean Hardman Date: Mon, 6 Mar 2006 15:42:08 -0800 Subject: Re: Standards Telecon this Wednesday, March 8 Thanks Sean - really appreciate it! Elizabeth ----------------------------------------------------------------------- From: Elizabeth D Rye To: Michael McAuley Date: Mon, 6 Mar 2006 11:34:30 -0800 Subject: Re: question about MD5 checksums Thanks Myche! Much appreciated. Elizabeth ----------------------------------------------------------------------- From: Elizabeth D Rye To: Michael McAuley Date: Mon, 6 Mar 2006 09:53:44 -0800 Subject: question about MD5 checksums Hi Myche, Do I correctly recall you once telling me that it is possible (albeit unlikely) to have to distinct files with the same MD5 checksum? I have a vague recollection of you telling me that in the case of two adjacent bytes within the same record being swapped, the checksum might be the same. Is this correct, or am I mis-remembering? Elizabeth ----------------------------------------------------------------------- From: Elizabeth D Rye To: Dick Simpson 650-723-3525 Date: Wed, 8 Mar 2006 12:07:00 -0800 Subject: Re: Call-in information for standards telecon Thanks Dick - I appreciate that. Elizabeth ----------------------------------------------------------------------- From: Elizabeth D Rye To: "J. Steven Hughes" Date: Mon, 6 Mar 2006 11:08:03 -0800 Subject: Re: Standards Telecon this Wednesday, March 8 Glad to hear it! By the way, how do we go about figuring out if our machines and software can handle 32 character hexadecimal numbers? Who should I talk to about this? Elizabeth ----------------------------------------------------------------------- From: "Ronald Joyner" To: "'Elizabeth D Rye'" Cc: "'Steve Hughes'" Date: Mon, 20 Mar 2006 07:15:21 -0800 Subject: RE: how to solve problem with element definition object template Howdy, You can use SQLAdvantage to update the rows requiring double quotes by using the following notation: update ddcold set cold =3D "test "" test" where blname =3D "mro:1" The use of 2 double quotes produces a single quote in the string. RJ ----------------------------------------------------------------------- From: Elizabeth D Rye [mailto:Elizabeth.D.Rye@jpl.nasa.gov] To: Ron Joyner Cc: 'Steve Hughes' Date: Monday, March 20, 2006 2:26 AM Subject: how to solve problem with element definition object template Hi Ron, When completing the element definition object for a new (or revised) keyword, one has to convert all double quotes inside a description to single quotes because the DESCRIPTION value itself is double quote delimited. This raises a problem in the description of the MD5_CHECKSUM keyword, part of which reads: ------------------ In order to comply with other standards relating to the use of lowercase letters in strings, the value must be quoted using double quotes. =A0=A0Example:=A0MD5_CHECKSUM =3D '0ff0a5dd0f3ea4e104b0eae98c87f36c' ------------------ Note that the double quotes in the example have been replaced with single quotes, precisely contrary to the specified usage. How can we get around this problem? Elizabeth ----------------------------------------------------------------------- From: Dick Simpson 650-723-3525 Date: Fri, 7 Apr 2006 09:36:54 -0700 (PDT) To: Elizabeth.D.Rye@jpl.nasa.gov, arvidson@rsmail.wustl.edu, ma@astro.umd.edu, rbeebe@nmsu.edu Cc: Dan.Crichton@jpl.nasa.gov, Steve.Hughes@jpl.nasa.gov, guinness@rsmail.wustl.edu, lhuber@nmsu.edu, raugh@astro.umd.edu, slavney@rsmail.wustl.edu Subject: Re: feedback requested on NO vote on 3-1034 Sent list of issues earlier. #1 is that the checksum would be required on future PDS3 volume designs. I'm OK with PDS4, just don't want the requirement in PDS3. Other problems were mostly errors in the SCR - particularly inconsistencies between the proposed new text for documents and what the body of the SCR said was going to be done. > Dick, are your concerns primarily regarding the upper/lower case > issue? The case issue has been resolved; the example needs to be corrected. > Please let me know your reasons so I can determine whether we should > try to modify this SCR and seek another vote. I would not object to a reconsideration; but let's get the proposal cleaner before it goes to MC. And let's ease into this rather than dropping a hard requirement onto designers of new volumes now. Or ... let's get moving on PDS4. ----------------------------------------------------------------------- From: Elizabeth D Rye To: Dick Simpson 650-723-3525 Date: Fri, 7 Apr 2006 10:35:29 -0700 Subject: Re: feedback requested on NO vote on 3-1034 Hi, > Over the long haul, I think coffee CAUSES brain death. > Depends on whether SCR deadlines or celebrating your > 100th birthday has higher priority. Or, maybe you'd > like a third choice? You're probably right, but it definitely helps with my migraines. > Sorry not to have been on the telecon when the decision > was made to send these to MC. I could at least have > voiced my concerns at that point. But I didn't even > receive the telecon materials until the following week, > by which point Reta was calling for votes. I should have run it by you before sending it off to the MC for a vote; we definitely always emerge with a better product after you've reviewed it. Thank you for all the times you have reviewed stuff and the very constructive criticisms you provide - I really appreciate the time you put into these things. > I think CHECKSUM can be salvaged. It sounds from GEO > and ATM (as well as myself) that deferring the > REQUIRED use of checksums to PDS4 would get it through > (though GEO also had some issues about placement of the > file(s), as I recall. I think you're right - we can salvage this one. > This is where sifting the > feedback to discover the common ground can lead to > progress. Once you've mastered that, you run for > governor. Better get my American citizenship first! ;-) > On that happy note, have a nice rest-of-Friday and > weekend. Thanks - you too! Elizabeth ----------------------------------------------------------------------- From: Elizabeth D Rye To: lhuber@nmsu.edu Cc: Reta Beebe Date: Fri, 7 Apr 2006 10:21:01 -0700 Subject: Re: feedback requested on NO vote on 3-1034 Hi Lyle, Just a point of clarification - I understand that you don't believe checksums should be required to be included on the archive volumes until PDS4; however, do you believe they should be required in some form (to be negotiated between the node and the data provider) before then or would you rather not see them required at all until PDS4? Elizabeth On Apr 7, 2006, at 9:54 AM, lhuber@nmsu.edu wrote: > Elizabeth, > > The ATM No vote was for essentially the same reasons as GEO. Requiring > checksums to be included on archive volumes is an unnecessary new > requirement, particularly if it is implemented under the auspices > of "PDS3". In "PDS4", if it ever exists, there is a clean slate for > something like this to be required. The chances for validation software > to break, albeit unintentionally, are too great. > > I like the idea of requiring providers to have checksums but it seems > to me that it's really a delivery issue to PDS and that most users > won't care about it unless/until they encounter a problem, in which > case their first response will be to e-mail us anyway. > > Lyle Huber > PDS Atmospheres Node > --Apple-Mail-8-767020373 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hi Lyle, Just a point of clarification - I understand that you don't believe checksums should be required to be included on the archive volumes until PDS4; however, do you believe they should be required in some form (to be negotiated between the node and the data provider) before then or would you rather not see them required at all until PDS4? Elizabeth On Apr 7, 2006, at 9:54 AM, lhuber@nmsu.edu wrote: Elizabeth, The ATM No vote was for essentially the same reasons as GEO. Requiring checksums to be included on archive volumes is an unnecessary new requirement, particularly if it is implemented under the auspices of "PDS3". In "PDS4", if it ever exists, there is a clean slate for something like this to be required. The chances for validation software to break, albeit unintentionally, are too great. I like the idea of requiring providers to have checksums but it seems to me that it's really a delivery issue to PDS and that most users won't care about it unless/until they encounter a problem, in which case their first response will be to e-mail us anyway. Lyle Huber PDS Atmospheres Node From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:31 MDT 2006 Hi, I would like to hear a few more details about your "NO" votes on SCR 3-1034 (File Checksums). Specifically, do you approve of the principle in general but need some of the details changed, or do you oppose the principle completely? If the former, what about the SCR in its current form do you object to? I believe I know (please correct me if I'm wrong) the reasons for GEO's vote - you would prefer that the presence of the checksums within an archive be optional rather than required, and that each node should be free to negotiate with its data providers how the checksums are provided. Dick, are your concerns primarily regarding the upper/lower case issue? Please let me know your reasons so I can determine whether we should try to modify this SCR and seek another vote. Thank you all! Elizabeth -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov --Apple-Mail-8-767020373-- From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:05:48 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200604071657.k37Gv0D20358@magellan.stanford.edu> References: <200604071657.k37Gv0D20358@magellan.stanford.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <24e75b0e46e46fff6784e1dd73052982@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: feedback requested on NO vote on 3-1034 Date: Fri, 7 Apr 2006 10:05:48 -0700 To: Dick Simpson 650-723-3525 You're right - I have it. Sorry for the brain death this morning. Not enough coffee yet, I guess. ;-) Elizabeth On Apr 7, 2006, at 9:57 AM, Dick Simpson 650-723-3525 wrote: > >> Sorry if this is the second time you've had to write all of this - I >> never saw any of the feedback on the vote, just the final results. >> Maybe I need to pester Reta first. ;-) > > You must have received it; that's what prompted the discussion > on upper/lower case in the example. > > I can send the message again; but I think you have it. > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 09:45:51 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200604071636.k37GasM20086@magellan.stanford.edu> References: <200604071636.k37GasM20086@magellan.stanford.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <9d082c500b6459b52154067647951745@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: feedback requested on NO vote on 3-1034 Date: Fri, 7 Apr 2006 09:45:51 -0700 To: Dick Simpson 650-723-3525 Thanks for the feedback Dick. Sorry if this is the second time you've had to write all of this - I never saw any of the feedback on the vote, just the final results. Maybe I need to pester Reta first. ;-) Elizabeth On Apr 7, 2006, at 9:36 AM, Dick Simpson 650-723-3525 wrote: > Sent list of issues earlier. #1 is that the checksum > would be required on future PDS3 volume designs. I'm > OK with PDS4, just don't want the requirement in PDS3. > > Other problems were mostly errors in the SCR - particularly > inconsistencies between the proposed new text for documents > and what the body of the SCR said was going to be done. > >> Dick, are your concerns primarily regarding the upper/lower case >> issue? > > The case issue has been resolved; the example needs to be > corrected. > >> Please let me know your reasons so I can determine whether we should >> try to modify this SCR and seek another vote. > > I would not object to a reconsideration; but let's get the > proposal cleaner before it goes to MC. And let's ease into > this rather than dropping a hard requirement onto designers > of new volumes now. Or ... let's get moving on PDS4. > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 09:27:31 2006 Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit Cc: Anne Raugh , Dan Crichton , Susan Slavney , Lyle Huber , 'Steve Hughes' , Ed Guinness From: Elizabeth D Rye Subject: feedback requested on NO vote on 3-1034 Date: Fri, 7 Apr 2006 09:27:30 -0700 To: Dick Simpson , Mike A'Hearn , Ray Arvidson , Reta Beebe Hi, I would like to hear a few more details about your "NO" votes on SCR 3-1034 (File Checksums). Specifically, do you approve of the principle in general but need some of the details changed, or do you oppose the principle completely? If the former, what about the SCR in its current form do you object to? I believe I know (please correct me if I'm wrong) the reasons for GEO's vote - you would prefer that the presence of the checksums within an archive be optional rather than required, and that each node should be free to negotiate with its data providers how the checksums are provided. Dick, are your concerns primarily regarding the upper/lower case issue? Please let me know your reasons so I can determine whether we should try to modify this SCR and seek another vote. Thank you all! Elizabeth -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From lhuber@nmsu.edu Fri Apr 7 10:34:15 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 07 Apr 2006 10:33:56 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 7 Apr 2006 10:33:56 -0700 Received: from xmta3.jpl.nasa.gov (xmta3.jpl.nasa.gov [137.78.160.111]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37HXuHW013763 for ; Fri, 7 Apr 2006 10:33:56 -0700 Received: from ccserver1.nmsu.edu (ccserver1.nmsu.edu [128.123.34.19]) by xmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37HXtc9023446 for ; Fri, 7 Apr 2006 10:33:55 -0700 Received: from ccserver1.nmsu.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id B13416DC2B8 for ; Fri, 7 Apr 2006 11:33:54 -0600 (MDT) Received: from ccserver4.nmsu.edu (ccserver4.nmsu.edu [128.123.34.102]) by ccserver1.nmsu.edu (Postfix) with ESMTP id A99576DC29D for ; Fri, 7 Apr 2006 11:33:54 -0600 (MDT) Received: from ccserver4.nmsu.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 9CF79B4242; Fri, 7 Apr 2006 11:33:54 -0600 (MDT) Received: from mariner.nmsu.edu (mariner.nmsu.edu [128.123.5.202]) by ccserver4.nmsu.edu (Postfix) with ESMTP id 9121DB4120; Fri, 7 Apr 2006 11:33:50 -0600 (MDT) Received: (from lhuber@localhost) by mariner.nmsu.edu (8.11.7p1+Sun/8.9.0) id k37HXnp23007; Fri, 7 Apr 2006 11:33:49 -0600 (MDT) Date: Fri, 7 Apr 2006 11:33:49 -0600 (MDT) From: lhuber@nmsu.edu Message-Id: <200604071733.k37HXnp23007@mariner.nmsu.edu> To: lhuber@nmsu.edu, Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: feedback requested on NO vote on 3-1034 Cc: rbeebe@nmsu.edu X-Sun-Charset: US-ASCII X-PMX-Version: 5.1.2.240295, Antispam-Engine: 2.3.0.0, Antispam-Data: 2006.4.7.101606 X-PerlMx-Spam: Gauge=IIIIIII, Probability=7%, Report='NO_REAL_NAME 0, __C230066_P5 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __SANE_MSGID 0, __cbl.abuseat.org_TIMEOUT ' X-Source-IP: ccserver1.nmsu.edu [128.123.34.19] X-Source-Sender: lhuber@nmsu.edu X-JPL-spam-score: 0.00% Yes, required to be delivered to PDS (negotiated between node and provider) but not required as a formal part of the archive. Lyle > From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 11:31 MDT 2006 > Hi Lyle, > > Just a point of clarification - I understand that you don't believe > checksums should be required to be included on the archive volumes > until PDS4; however, do you believe they should be required in some > form (to be negotiated between the node and the data provider) before > then or would you rather not see them required at all until PDS4? > > Elizabeth > > > On Apr 7, 2006, at 9:54 AM, lhuber@nmsu.edu wrote: > > > Elizabeth, > > > > The ATM No vote was for essentially the same reasons as GEO. Requiring > > checksums to be included on archive volumes is an unnecessary new > > requirement, particularly if it is implemented under the auspices > > of "PDS3". In "PDS4", if it ever exists, there is a clean slate for > > something like this to be required. The chances for validation software > > to break, albeit unintentionally, are too great. > > > > I like the idea of requiring providers to have checksums but it seems > > to me that it's really a delivery issue to PDS and that most users > > won't care about it unless/until they encounter a problem, in which > > case their first response will be to e-mail us anyway. > > > > Lyle Huber > > PDS Atmospheres Node > > > > > >> From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:31 MDT 2006 > >> > >> Hi, > >> > >> I would like to hear a few more details about your "NO" votes on SCR > >> 3-1034 (File Checksums). Specifically, do you approve of the > >> principle > >> in general but need some of the details changed, or do you oppose the > >> principle completely? If the former, what about the SCR in its > >> current > >> form do you object to? > >> > >> I believe I know (please correct me if I'm wrong) the reasons for > >> GEO's > >> vote - you would prefer that the presence of the checksums within an > >> archive be optional rather than required, and that each node should be > >> free to negotiate with its data providers how the checksums are > >> provided. > >> > >> Dick, are your concerns primarily regarding the upper/lower case > >> issue? > >> > >> Please let me know your reasons so I can determine whether we should > >> try to modify this SCR and seek another vote. > >> > >> Thank you all! > >> > >> Elizabeth > >> -- > >> Elizabeth D. Rye > >> Phone: (818) 354-6135 > >> Email: Elizabeth.D.Rye@jpl.nasa.gov > >> > >> > > > > > -- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov > From rsimpson@magellan.stanford.edu Fri Apr 7 10:29:13 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 07 Apr 2006 10:26:18 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 7 Apr 2006 10:26:18 -0700 Received: from xmta3.jpl.nasa.gov (xmta3.jpl.nasa.gov [137.78.160.111]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37HQIcn010780 for ; Fri, 7 Apr 2006 10:26:18 -0700 Received: from smtp3.Stanford.EDU (smtp3.Stanford.EDU [171.67.16.138]) by xmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37HQHkv017475 for ; Fri, 7 Apr 2006 10:26:17 -0700 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp3.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k37HQHR9022390 for ; Fri, 7 Apr 2006 10:26:17 -0700 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k37HQGQ21890 for Elizabeth.D.Rye@jpl.nasa.gov; Fri, 7 Apr 2006 10:26:16 -0700 (PDT) Date: Fri, 7 Apr 2006 10:26:16 -0700 (PDT) From: Dick Simpson 650-723-3525 Message-Id: <200604071726.k37HQGQ21890@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: feedback requested on NO vote on 3-1034 X-Source-IP: smtp3.Stanford.EDU [171.67.16.138] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.00% >You're right - I have it. Sorry for the brain death this morning. >Not enough coffee yet, I guess. Over the long haul, I think coffee CAUSES brain death. Depends on whether SCR deadlines or celebrating your 100th birthday has higher priority. Or, maybe you'd like a third choice? Sorry not to have been on the telecon when the decision was made to send these to MC. I could at least have voiced my concerns at that point. But I didn't even receive the telecon materials until the following week, by which point Reta was calling for votes. I think CHECKSUM can be salvaged. It sounds from GEO and ATM (as well as myself) that deferring the REQUIRED use of checksums to PDS4 would get it through (though GEO also had some issues about placement of the file(s), as I recall. This is where sifting the feedback to discover the common ground can lead to progress. Once you've mastered that, you run for governor. On that happy note, have a nice rest-of-Friday and weekend. From rsimpson@magellan.stanford.edu Fri Apr 7 09:59:14 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 07 Apr 2006 09:57:01 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 7 Apr 2006 09:57:01 -0700 Received: from xmta1.jpl.nasa.gov (xmta1.jpl.nasa.gov [137.78.160.144]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37Gv11I028704 for ; Fri, 7 Apr 2006 09:57:01 -0700 Received: from smtp1.Stanford.EDU (smtp1.Stanford.EDU [171.67.16.123]) by xmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37Gv1wU024824 for ; Fri, 7 Apr 2006 09:57:01 -0700 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp1.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k37Gv0oR008635 for ; Fri, 7 Apr 2006 09:57:00 -0700 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k37Gv0D20358 for Elizabeth.D.Rye@jpl.nasa.gov; Fri, 7 Apr 2006 09:57:00 -0700 (PDT) Date: Fri, 7 Apr 2006 09:57:00 -0700 (PDT) From: Dick Simpson 650-723-3525 Message-Id: <200604071657.k37Gv0D20358@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: feedback requested on NO vote on 3-1034 X-Source-IP: smtp1.Stanford.EDU [171.67.16.123] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.00% >Sorry if this is the second time you've had to write all of this - I >never saw any of the feedback on the vote, just the final results. >Maybe I need to pester Reta first. ;-) You must have received it; that's what prompted the discussion on upper/lower case in the example. I can send the message again; but I think you have it. From lhuber@nmsu.edu Fri Apr 7 09:59:14 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP; Fri, 07 Apr 2006 09:55:01 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942); Fri, 7 Apr 2006 09:55:01 -0700 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37Gt0KB027880; Fri, 7 Apr 2006 09:55:01 -0700 Received: from ccserver1.nmsu.edu (ccserver1.NMSU.Edu [128.123.34.19]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37Gt0vZ023167; Fri, 7 Apr 2006 09:55:00 -0700 Received: from ccserver1.nmsu.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id D84C76DC2BD; Fri, 7 Apr 2006 10:54:59 -0600 (MDT) Received: from ccserver4.nmsu.edu (ccserver4.nmsu.edu [128.123.34.102]) by ccserver1.nmsu.edu (Postfix) with ESMTP id CA4E46DC296; Fri, 7 Apr 2006 10:54:59 -0600 (MDT) Received: from ccserver4.nmsu.edu (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id BC136B41FF; Fri, 7 Apr 2006 10:54:59 -0600 (MDT) Received: from mariner.nmsu.edu (mariner.nmsu.edu [128.123.5.202]) by ccserver4.nmsu.edu (Postfix) with ESMTP id 9E07FB424B; Fri, 7 Apr 2006 10:54:57 -0600 (MDT) Received: (from lhuber@localhost) by mariner.nmsu.edu (8.11.7p1+Sun/8.9.0) id k37Gsv322997; Fri, 7 Apr 2006 10:54:57 -0600 (MDT) Date: Fri, 7 Apr 2006 10:54:57 -0600 (MDT) From: lhuber@nmsu.edu Message-Id: <200604071654.k37Gsv322997@mariner.nmsu.edu> To: rbeebe@nmsu.edu, ma@astro.umd.edu, rsimpson@magellan.stanford.edu, arvidson@rsmail.wustl.edu, Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: feedback requested on NO vote on 3-1034 Cc: raugh@astro.umd.edu, Dan.Crichton@jpl.nasa.gov, lhuber@nmsu.edu, slavney@rsmail.wustl.edu, Steve.Hughes@jpl.nasa.gov, guinness@rsmail.wustl.edu X-Sun-Charset: US-ASCII X-PMX-Version: 5.1.2.240295, Antispam-Engine: 2.3.0.0, Antispam-Data: 2006.4.7.93606 X-PerlMx-Spam: Gauge=IIIIIII, Probability=7%, Report='NO_REAL_NAME 0, __C230066_P5 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __SANE_MSGID 0' X-Source-IP: ccserver1.NMSU.Edu [128.123.34.19] X-Source-Sender: lhuber@nmsu.edu X-JPL-spam-score: 0.00% Elizabeth, The ATM No vote was for essentially the same reasons as GEO. Requiring checksums to be included on archive volumes is an unnecessary new requirement, particularly if it is implemented under the auspices of "PDS3". In "PDS4", if it ever exists, there is a clean slate for something like this to be required. The chances for validation software to break, albeit unintentionally, are too great. I like the idea of requiring providers to have checksums but it seems to me that it's really a delivery issue to PDS and that most users won't care about it unless/until they encounter a problem, in which case their first response will be to e-mail us anyway. Lyle Huber PDS Atmospheres Node > From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:31 MDT 2006 > > Hi, > > I would like to hear a few more details about your "NO" votes on SCR > 3-1034 (File Checksums). Specifically, do you approve of the principle > in general but need some of the details changed, or do you oppose the > principle completely? If the former, what about the SCR in its current > form do you object to? > > I believe I know (please correct me if I'm wrong) the reasons for GEO's > vote - you would prefer that the presence of the checksums within an > archive be optional rather than required, and that each node should be > free to negotiate with its data providers how the checksums are > provided. > > Dick, are your concerns primarily regarding the upper/lower case issue? > > Please let me know your reasons so I can determine whether we should > try to modify this SCR and seek another vote. > > Thank you all! > > Elizabeth > -- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov > > From guinness@rsmail.wustl.edu Fri Apr 7 12:02:52 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP; Fri, 07 Apr 2006 12:02:12 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942); Fri, 7 Apr 2006 12:02:12 -0700 Received: from xmta1.jpl.nasa.gov (xmta1.jpl.nasa.gov [137.78.160.144]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37J2CO5004057; Fri, 7 Apr 2006 12:02:12 -0700 Received: from rsmail.wustl.edu (rsmail.wustl.edu [128.252.144.20]) by xmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37J2BJg016083; Fri, 7 Apr 2006 12:02:11 -0700 Received: from mercury.wunder.wustl.edu (wormhole.eprsl.wustl.edu [128.252.144.154] (may be forged)) (authenticated bits=0) by rsmail.wustl.edu (8.13.1/8.13.1) with ESMTP id k37J0K6B004708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 7 Apr 2006 14:00:20 -0500 Message-Id: <6.2.3.4.2.20060407135959.029deb50@wunder.wustl.edu> X-Mailer: QUALCOMM Windows Eudora Version 6.2.3.4 Date: Fri, 07 Apr 2006 14:02:11 -0500 To: Elizabeth D Rye , Reta Beebe , "Mike A'Hearn" , Dick Simpson , Ray Arvidson From: "Edward A. Guinness" Subject: Re: feedback requested on NO vote on 3-1034 Cc: Anne Raugh , Dan Crichton , Lyle Huber , Susan Slavney , "'Steve Hughes'" In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Source-IP: rsmail.wustl.edu [128.252.144.20] X-Source-Sender: guinness@wunder.wustl.edu X-JPL-spam-score: 0.00% Elizabeth et al., Here is feedback from the Geosciences Node on the checksum SCR. For the Geosciences Node the main objectives of using checksums are to: A) verify the transfer of data from data providers; B) have nodes use checksums of their holdings to make sure their copy does not deviate from what was delivered; and C) allow nodes to provide checksums to users if the user wants the information. All of these objectives can be met without the need of another required file on an archive volume. Nodes can work with data providers through writing ICD's during the planning for archiving to develop the procedures for delivering checksums with data deliveries. Requiring the checksum file on the archive volume does not fully meet the objectives. For example, there is nothing in the current SCR about the source of the checksums. A node could generate the file after the products are delivered and be in compliance with the standard. We also agree with Lyle in that users probably do not want to download the entire checksum file as the method of checking the transfer of a few data products. Ensuring integrity of data archives, of which checksum usage is a part, is really a set of requirements and procedures that are needed for the archiving process and is not a standard's issue. -Ed At 11:27 AM 4/7/2006, Elizabeth D Rye wrote: >Hi, > >I would like to hear a few more details about your "NO" votes on SCR >3-1034 (File Checksums). Specifically, do you approve of the >principle in general but need some of the details changed, or do you >oppose the principle completely? If the former, what about the SCR >in its current form do you object to? > >I believe I know (please correct me if I'm wrong) the reasons for >GEO's vote - you would prefer that the presence of the checksums >within an archive be optional rather than required, and that each >node should be free to negotiate with its data providers how the >checksums are provided. > >Dick, are your concerns primarily regarding the upper/lower case issue? > >Please let me know your reasons so I can determine whether we should >try to modify this SCR and seek another vote. > >Thank you all! > >Elizabeth >-- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov From ma@astro.umd.edu Fri Apr 7 15:03:18 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP; Fri, 07 Apr 2006 15:01:41 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942); Fri, 7 Apr 2006 15:01:41 -0700 Received: from xmta1.jpl.nasa.gov (xmta1.jpl.nasa.gov [137.78.160.144]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37M1enw031525; Fri, 7 Apr 2006 15:01:40 -0700 Received: from halley.astro.umd.edu (halley.astro.umd.edu [129.2.14.38]) by xmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37M1enS032232; Fri, 7 Apr 2006 15:01:40 -0700 Received: from [192.168.0.2] (lap111.astro.umd.edu [129.2.15.111]) by halley.astro.umd.edu (Postfix) with ESMTP id A04C81FB44; Fri, 7 Apr 2006 18:01:39 -0400 (EDT) Mime-Version: 1.0 X-Sender: ma@halley.astro.umd.edu Message-Id: In-Reply-To: References: Date: Fri, 7 Apr 2006 18:01:39 -0400 To: Elizabeth D Rye , Reta Beebe , "Mike A'Hearn" , Dick Simpson , Ray Arvidson From: "Mike A'Hearn" Subject: Re: feedback requested on NO vote on 3-1034 Cc: Anne Raugh , Dan Crichton , Lyle Huber , Susan Slavney , "'Steve Hughes'" , Ed Guinness Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Source-IP: halley.astro.umd.edu [129.2.14.38] X-Source-Sender: ma@astro.umd.edu X-JPL-spam-score: 0.00% Elizabeth, My reasons were primarily based on what I viewed as effectively an invitation to projects to define their own checksums rather than making it absolutely clear that we are using a single type of checksum across all datasets. I have sympathy with many of the concerns expressed by other nodes, particularly the issue of generating a table of checksums that might end up being irrelevant to a "typical" download of data from a node. It seems to me that the checksum is mainly a data transfer issue rather than a PDS object issue and thus might better be handled in some other manner not yet apparent to me. Mike At 9:27 AM -0700 7 4 2006, Elizabeth D Rye wrote: >Hi, > >I would like to hear a few more details about your "NO" votes on SCR >3-1034 (File Checksums). Specifically, do you approve of the >principle in general but need some of the details changed, or do you >oppose the principle completely? If the former, what about the SCR >in its current form do you object to? > >I believe I know (please correct me if I'm wrong) the reasons for >GEO's vote - you would prefer that the presence of the checksums >within an archive be optional rather than required, and that each >node should be free to negotiate with its data providers how the >checksums are provided. > >Dick, are your concerns primarily regarding the upper/lower case issue? > >Please let me know your reasons so I can determine whether we should >try to modify this SCR and seek another vote. > >Thank you all! > >Elizabeth >-- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov -- Michael F. A'Hearn Tel: 301 405 6076 Department of Astronomy FAX: 301 405 3538 University of Maryland College Park MD 20742-2421 From Elizabeth.D.Rye@jpl.nasa.gov Fri Mar 31 12:15:53 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200603312010.k2VKA3U11983@magellan.stanford.edu> References: <200603312010.k2VKA3U11983@magellan.stanford.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <84ecac64c444cf7e6b03246891db12ea@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: CHECKSUM SCR Date: Fri, 31 Mar 2006 12:15:53 -0800 To: Dick Simpson 650-723-3525 To be honest Dick, I don't know. Do you want me to pursue this and find out for you? I could try to contact someone who has it installed. Elizabeth On Mar 31, 2006, at 12:10 PM, Dick Simpson 650-723-3525 wrote: > >> Again, I was instructed by a majority of those present at the telcons >> to use the file format produced by common checksum utilities. These >> utilities generally output their results using lower case file names. > > So, if I run md5deep on a volume with upper case directory and file > names, it will give me output that will be in lower case? > > Or is this a case where the directories and files were lower case to > begin with? > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Mar 31 12:17:56 2006 Mime-Version: 1.0 (Apple Message framework v623) Content-Transfer-Encoding: 7bit Message-Id: <4f5fb31f81dc2a10b74805e5dcc9c72f@jpl.nasa.gov> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Todd King , Anne Raugh From: Elizabeth D Rye Subject: question about md5deep Date: Fri, 31 Mar 2006 12:17:55 -0800 Hi, I have the impression that both of you have been running the md5deep utility. Could you tell me if it output a file with uppercase filenames if you run it on a volume where the filenames are all uppercase? Elizabeth -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Mar 31 12:18:32 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200603312010.k2VKA3U11983@magellan.stanford.edu> References: <200603312010.k2VKA3U11983@magellan.stanford.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <5384b716684ca4ab70e046892112d9a8@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: CHECKSUM SCR Date: Fri, 31 Mar 2006 12:18:32 -0800 To: Dick Simpson 650-723-3525 Hi Dick, I've sent a question off to Anne and Todd asking if either of them can answer your question. Elizabeth On Mar 31, 2006, at 12:10 PM, Dick Simpson 650-723-3525 wrote: > >> Again, I was instructed by a majority of those present at the telcons >> to use the file format produced by common checksum utilities. These >> utilities generally output their results using lower case file names. > > So, if I run md5deep on a volume with upper case directory and file > names, it will give me output that will be in lower case? > > Or is this a case where the directories and files were lower case to > begin with? > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Mar 31 12:20:59 2006 In-Reply-To: <6.2.3.4.2.20060331141347.02a88f78@wunder.wustl.edu> References: <200603311648.k2VGm8p02394@magellan.stanford.edu> <85af05cb0908797a08d9990d6cda36d8@jpl.nasa.gov> <6.2.3.4.2.20060331141347.02a88f78@wunder.wustl.edu> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <2dda64464e7fc6462b868bcd2037a0f6@jpl.nasa.gov> Content-Transfer-Encoding: 7bit Cc: Dick Simpson 650-723-3525 From: Elizabeth D Rye Subject: Re: CHECKSUM SCR Date: Fri, 31 Mar 2006 12:20:58 -0800 To: "Edward A. Guinness" Hi Ed, > Just for the record Geo was the only no vote at the tech session, but > Atmospheres, NAIF, and Radio Science were not present. That's absolutely true. However, at the 03/08 telecon GEO voted against required tables, RS voted for them, and NAIF said "either is fine". Therefore, they all sort of cancelled each other out. Elizabeth > Ed > > >>> Section D.2: >>> >>> a. I thought we agreed during the last discussion I joined >>> that checksum tables would not be required until PDS4; the >>> SCR says each archive volume MUST have one, which means >>> they will be required now. I understand that pipelines >>> already designed without the checksum are exempt; but I'm >>> not ready to REQUIRE checksums in future PDS3 designs. >> >> At the 03/08 telecon (the last one prior to the 03/22 one you >> missed), we were split on whether or not the checksum tables should >> be required. We had also agreed to postpone the decision on when >> they should be required. In order to resolve the split, I chose to >> write the SCR as if checksum tables were required, then asked for a >> vote on the SCR in that form. I told everyone present that if the >> vote failed on the SCR in that form, we would change the wording to >> make the checksum tables optional and then conduct the vote again. >> However, the SCR was approved in the required form, with only the GEO >> node dissenting. The "when" question was addressed by our >> accompanying recommendation that only missions archiving to v3.8 and >> higher of the standards should be required to comply with the SCR, if >> approved. >> > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Mar 31 12:28:32 2006 In-Reply-To: <6.2.3.4.2.20060331141347.02a88f78@wunder.wustl.edu> References: <200603311648.k2VGm8p02394@magellan.stanford.edu> <85af05cb0908797a08d9990d6cda36d8@jpl.nasa.gov> <6.2.3.4.2.20060331141347.02a88f78@wunder.wustl.edu> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: multipart/alternative; boundary=Apple-Mail-11-173471092 Message-Id: <6e09c5c3883d5f1f8a89698db423deb8@jpl.nasa.gov> Cc: Dick Simpson 650-723-3525 From: Elizabeth D Rye Subject: Re: CHECKSUM SCR Date: Fri, 31 Mar 2006 12:28:31 -0800 To: "Edward A. Guinness" --Apple-Mail-11-173471092 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Sorry, I should have said: ATM voted against required tables, RS voted for them, and NAIF said "either is fine". Basically, if I'd assumed the votes that these three nodes made at the previous telecon, we still would have ended up with 6 for, 2 against, and 1 abstention. It would still have been recommended. Elizabeth On Mar 31, 2006, at 12:16 PM, Edward A. Guinness wrote: > Elizabeth and Dick, > > Just for the record Geo was the only no vote at the tech session, but > Atmospheres, NAIF, and Radio Science were not present. > > Ed > > >>> Section D.2: >>> >>> a. I thought we agreed during the last discussion I joined >>> that checksum tables would not be required until PDS4; the >>> SCR says each archive volume MUST have one, which means >>> they will be required now. I understand that pipelines >>> already designed without the checksum are exempt; but I'm >>> not ready to REQUIRE checksums in future PDS3 designs. >> >> At the 03/08 telecon (the last one prior to the 03/22 one you >> missed), we were split on whether or not the checksum tables should >> be required. We had also agreed to postpone the decision on when >> they should be required. In order to resolve the split, I chose to >> write the SCR as if checksum tables were required, then asked for a >> vote on the SCR in that form. I told everyone present that if the >> vote failed on the SCR in that form, we would change the wording to >> make the checksum tables optional and then conduct the vote again. >> However, the SCR was approved in the required form, with only the GEO >> node dissenting. The "when" question was addressed by our >> accompanying recommendation that only missions archiving to v3.8 and >> higher of the standards should be required to comply with the SCR, if >> approved. >> > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov --Apple-Mail-11-173471092 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Sorry, I should have said: ATM voted against required tables, RS voted for them, and NAIF said "either is fine". Basically, if I'd assumed the votes that these three nodes made at the previous telecon, we still would have ended up with 6 for, 2 against, and 1 abstention. It would still have been recommended. Elizabeth On Mar 31, 2006, at 12:16 PM, Edward A. Guinness wrote: Elizabeth and Dick, Just for the record Geo was the only no vote at the tech session, but Atmospheres, NAIF, and Radio Science were not present. Ed Section D.2: a. I thought we agreed during the last discussion I joined that checksum tables would not be required until PDS4; the SCR says each archive volume MUST have one, which means they will be required now. I understand that pipelines already designed without the checksum are exempt; but I'm not ready to REQUIRE checksums in future PDS3 designs. At the 03/08 telecon (the last one prior to the 03/22 one you missed), we were split on whether or not the checksum tables should be required. We had also agreed to postpone the decision on when they should be required. In order to resolve the split, I chose to write the SCR as if checksum tables were required, then asked for a vote on the SCR in that form. I told everyone present that if the vote failed on the SCR in that form, we would change the wording to make the checksum tables optional and then conduct the vote again. However, the SCR was approved in the required form, with only the GEO node dissenting. The "when" question was addressed by our accompanying recommendation that only missions archiving to v3.8 and higher of the standards should be required to comply with the SCR, if approved. -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov --Apple-Mail-11-173471092-- From Elizabeth.D.Rye@jpl.nasa.gov Mon Apr 3 09:18:53 2006 In-Reply-To: <01b601c65737$973ef060$27446180@kingpc> References: <01b601c65737$973ef060$27446180@kingpc> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit Cc: "'Anne Raugh'" From: Elizabeth D Rye Subject: Re: question about md5deep Date: Mon, 3 Apr 2006 09:18:52 -0700 To: "Todd King" Thanks to both of you! Elizabeth On Apr 3, 2006, at 8:59 AM, Todd King wrote: > Hi Elizabeth - > > The output preserves the case of the original file name. If it is > uppercase > then the output will be in uppercase, mixed-case it will be > mixed-case. The > md5deep utility also supports relative paths. > > -Todd- > >> -----Original Message----- >> From: Elizabeth D Rye [mailto:Elizabeth.D.Rye@jpl.nasa.gov] >> Sent: Friday, March 31, 2006 12:18 PM >> To: Anne Raugh; Todd King >> Subject: question about md5deep >> >> >> Hi, >> >> I have the impression that both of you have been running the md5deep >> utility. Could you tell me if it output a file with uppercase >> filenames if you run it on a volume where the filenames are all >> uppercase? >> >> Elizabeth >> -- >> Elizabeth D. Rye >> Phone: (818) 354-6135 >> Email: Elizabeth.D.Rye@jpl.nasa.gov >> >> > > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Wed Apr 5 10:35:44 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200603312035.k2VKZas12187@magellan.stanford.edu> References: <200603312035.k2VKZas12187@magellan.stanford.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: CHECKSUM SCR Date: Wed, 5 Apr 2006 10:35:44 -0700 To: Dick Simpson 650-723-3525 Hi Dick, > I'm curious ... I think I voted with the majority in saying that > we should pick a format that is consistent with what the available > utilities put out. But, if md5deep lists paths as entirely lower > case when they are actually mixed or upper, then md5deep has a > defect that I wasn't expecting. I'm guessing that the listing that > was pasted into the SCR was from a run on paths that were not PDS > compliant. > > Let's see whether Anne or Todd has anything to help. Both Anne and Todd have now gotten back to me. The md5deep utility outputs filenames in the same format that it reads them in. According to Anne, the problem is how a particular file system reads a PDS disk. Even if the disk was recorded with all uppercase filenames, sometimes the Unix systems will display them as all lower case. She said it depends on the options used on the CD pre-mastering software. Anyway, sounds like it's time to resurrect the uppercase/lowercase SCR. (sigh) Elizabeth -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 12:10:23 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <6.2.3.4.2.20060407135959.029deb50@wunder.wustl.edu> References: <6.2.3.4.2.20060407135959.029deb50@wunder.wustl.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <2b0a0737fcf7ca55e23502c9d29180e9@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: feedback requested on NO vote on 3-1034 Date: Fri, 7 Apr 2006 12:10:23 -0700 To: "Edward A. Guinness" Thanks Ed - much appreciated! Elizabeth On Apr 7, 2006, at 12:02 PM, Edward A. Guinness wrote: > Elizabeth et al., > > Here is feedback from the Geosciences Node on the checksum SCR. > > For the Geosciences Node the main objectives of using checksums are > to: A) verify the transfer of data from data providers; B) have nodes > use checksums of their holdings to make sure their copy does not > deviate from what was delivered; and C) allow nodes to provide > checksums to users if the user wants the information. All of these > objectives can be met without the need of another required file on an > archive volume. Nodes can work with data providers through writing > ICD's during the planning for archiving to develop the procedures for > delivering checksums with data deliveries. Requiring the checksum > file on the archive volume does not fully meet the objectives. For > example, there is nothing in the current SCR about the source of the > checksums. A node could generate the file after the products are > delivered and be in compliance with the standard. We also agree with > Lyle in that users probably do not want to download the entire > checksum file as the method of checking the transfer of a few data > products. > > Ensuring integrity of data archives, of which checksum usage is a > part, is really a set of requirements and procedures that are needed > for the archiving process and is not a standard's issue. > > -Ed > > > At 11:27 AM 4/7/2006, Elizabeth D Rye wrote: >> Hi, >> >> I would like to hear a few more details about your "NO" votes on SCR >> 3-1034 (File Checksums). Specifically, do you approve of the >> principle in general but need some of the details changed, or do you >> oppose the principle completely? If the former, what about the SCR >> in its current form do you object to? >> >> I believe I know (please correct me if I'm wrong) the reasons for >> GEO's vote - you would prefer that the presence of the checksums >> within an archive be optional rather than required, and that each >> node should be free to negotiate with its data providers how the >> checksums are provided. >> >> Dick, are your concerns primarily regarding the upper/lower case >> issue? >> >> Please let me know your reasons so I can determine whether we should >> try to modify this SCR and seek another vote. >> >> Thank you all! >> >> Elizabeth >> -- >> Elizabeth D. Rye >> Phone: (818) 354-6135 >> Email: Elizabeth.D.Rye@jpl.nasa.gov > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:41:39 2006 Mime-Version: 1.0 (Apple Message framework v623) In-Reply-To: <200604071733.k37HXnp23007@mariner.nmsu.edu> References: <200604071733.k37HXnp23007@mariner.nmsu.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <17490fe36ea004bc3ea3b5ef7fc1b180@jpl.nasa.gov> Content-Transfer-Encoding: 7bit From: Elizabeth D Rye Subject: Re: feedback requested on NO vote on 3-1034 Date: Fri, 7 Apr 2006 10:41:39 -0700 To: lhuber@nmsu.edu Thanks Lyle. Elizabeth On Apr 7, 2006, at 10:33 AM, lhuber@nmsu.edu wrote: > Yes, required to be delivered to PDS (negotiated between node and > provider) but not required as a formal part of the archive. > > Lyle > > >> From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 11:31 MDT 2006 >> Hi Lyle, >> >> Just a point of clarification - I understand that you don't believe >> checksums should be required to be included on the archive volumes >> until PDS4; however, do you believe they should be required in some >> form (to be negotiated between the node and the data provider) before >> then or would you rather not see them required at all until PDS4? >> >> Elizabeth >> >> >> On Apr 7, 2006, at 9:54 AM, lhuber@nmsu.edu wrote: >> >>> Elizabeth, >>> >>> The ATM No vote was for essentially the same reasons as GEO. >>> Requiring >>> checksums to be included on archive volumes is an unnecessary new >>> requirement, particularly if it is implemented under the auspices >>> of "PDS3". In "PDS4", if it ever exists, there is a clean slate for >>> something like this to be required. The chances for validation >>> software >>> to break, albeit unintentionally, are too great. >>> >>> I like the idea of requiring providers to have checksums but it seems >>> to me that it's really a delivery issue to PDS and that most users >>> won't care about it unless/until they encounter a problem, in which >>> case their first response will be to e-mail us anyway. >>> >>> Lyle Huber >>> PDS Atmospheres Node >>> >>> >>>> From Elizabeth.D.Rye@jpl.nasa.gov Fri Apr 7 10:31 MDT 2006 >>>> >>>> Hi, >>>> >>>> I would like to hear a few more details about your "NO" votes on SCR >>>> 3-1034 (File Checksums). Specifically, do you approve of the >>>> principle >>>> in general but need some of the details changed, or do you oppose >>>> the >>>> principle completely? If the former, what about the SCR in its >>>> current >>>> form do you object to? >>>> >>>> I believe I know (please correct me if I'm wrong) the reasons for >>>> GEO's >>>> vote - you would prefer that the presence of the checksums within an >>>> archive be optional rather than required, and that each node should >>>> be >>>> free to negotiate with its data providers how the checksums are >>>> provided. >>>> >>>> Dick, are your concerns primarily regarding the upper/lower case >>>> issue? >>>> >>>> Please let me know your reasons so I can determine whether we should >>>> try to modify this SCR and seek another vote. >>>> >>>> Thank you all! >>>> >>>> Elizabeth >>>> -- >>>> Elizabeth D. Rye >>>> Phone: (818) 354-6135 >>>> Email: Elizabeth.D.Rye@jpl.nasa.gov >>>> >>>> >>> >>> >> -- >> Elizabeth D. Rye >> Phone: (818) 354-6135 >> Email: Elizabeth.D.Rye@jpl.nasa.gov >> > > -- Elizabeth D. Rye Phone: (818) 354-6135 Email: Elizabeth.D.Rye@jpl.nasa.gov From J.Steven.Hughes@jpl.nasa.gov Sat Apr 8 09:50:49 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Sat, 08 Apr 2006 09:46:21 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Sat, 8 Apr 2006 09:46:21 -0700 Received: from xmta3.jpl.nasa.gov (xmta3.jpl.nasa.gov [137.78.160.111]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k38GkLNt021956 for ; Sat, 8 Apr 2006 09:46:21 -0700 Received: from jhughes-xp.jpl.nasa.gov (pool-71-108-220-129.lsanca.dsl-w.verizon.net [71.108.220.129]) by xmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k38GkJ5r028975 for ; Sat, 8 Apr 2006 09:46:20 -0700 Message-Id: <6.1.1.1.2.20060408094356.03d878f0@mail.jpl.nasa.gov> X-Sender: jshughes@mail.jpl.nasa.gov X-Mailer: QUALCOMM Windows Eudora Version 6.1.1.1 Date: Sat, 08 Apr 2006 09:46:18 -0700 To: Elizabeth.D.Rye@jpl.nasa.gov From: "J. Steven Hughes" Subject: Re: feedback requested on NO vote on 3-1034 In-Reply-To: <6.2.3.4.2.20060407135959.029deb50@wunder.wustl.edu> References: <6.2.3.4.2.20060407135959.029deb50@wunder.wustl.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Source-IP: pool-71-108-220-129.lsanca.dsl-w.verizon.net [71.108.220.129] X-Source-Sender: J.Steven.Hughes@jpl.nasa.gov X-JPL-spam-score: 0.00% Hi Elizabeth, I am not sure how you want to manage this issue however if you don't mind, I would like to respond in general about the need for requiring the checksum file on an archive volume. Are you ok with me doing this? thanks, steve At 12:02 PM 4/7/2006, you wrote: >Elizabeth et al., > >Here is feedback from the Geosciences Node on the checksum SCR. > >For the Geosciences Node the main objectives of using checksums are to: A) >verify the transfer of data from data providers; B) have nodes use >checksums of their holdings to make sure their copy does not deviate from >what was delivered; and C) allow nodes to provide checksums to users if >the user wants the information. All of these objectives can be met >without the need of another required file on an archive volume. Nodes can >work with data providers through writing ICD's during the planning for >archiving to develop the procedures for delivering checksums with data >deliveries. Requiring the checksum file on the archive volume does not >fully meet the objectives. For example, there is nothing in the current >SCR about the source of the checksums. A node could generate the file >after the products are delivered and be in compliance with the >standard. We also agree with Lyle in that users probably do not want to >download the entire checksum file as the method of checking the transfer >of a few data products. > >Ensuring integrity of data archives, of which checksum usage is a part, is >really a set of requirements and procedures that are needed for the >archiving process and is not a standard's issue. > >-Ed > > >At 11:27 AM 4/7/2006, Elizabeth D Rye wrote: >>Hi, >> >>I would like to hear a few more details about your "NO" votes on SCR >>3-1034 (File Checksums). Specifically, do you approve of the principle >>in general but need some of the details changed, or do you oppose the >>principle completely? If the former, what about the SCR in its current >>form do you object to? >> >>I believe I know (please correct me if I'm wrong) the reasons for GEO's >>vote - you would prefer that the presence of the checksums within an >>archive be optional rather than required, and that each node should be >>free to negotiate with its data providers how the checksums are provided. >> >>Dick, are your concerns primarily regarding the upper/lower case issue? >> >>Please let me know your reasons so I can determine whether we should try >>to modify this SCR and seek another vote. >> >>Thank you all! >> >>Elizabeth >>-- >> Elizabeth D. Rye >> Phone: (818) 354-6135 >> Email: Elizabeth.D.Rye@jpl.nasa.gov From J.Steven.Hughes@jpl.nasa.gov Mon Apr 10 12:49:53 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Mon, 10 Apr 2006 12:45:40 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Mon, 10 Apr 2006 12:45:40 -0700 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k3AJjeWa011079 for ; Mon, 10 Apr 2006 12:45:40 -0700 Received: from jhughes-xp.jpl.nasa.gov (dhcp-78-33-253.jpl.nasa.gov [137.78.33.253]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k3AJjcpL006469 for ; Mon, 10 Apr 2006 12:45:39 -0700 Message-Id: <6.1.1.1.2.20060410120152.03d04d18@mail.jpl.nasa.gov> X-Sender: jshughes@mail.jpl.nasa.gov X-Mailer: QUALCOMM Windows Eudora Version 6.1.1.1 Date: Mon, 10 Apr 2006 12:45:39 -0700 To: Elizabeth D Rye From: "J. Steven Hughes" Subject: Re: MD5 SCR - Where to from here? In-Reply-To: References: <6.1.1.1.2.20060410114558.03bca0e8@mail.jpl.nasa.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Source-IP: dhcp-78-33-253.jpl.nasa.gov [137.78.33.253] X-Source-Sender: J.Steven.Hughes@jpl.nasa.gov X-AUTH: Internal IP Hi Elizabeth, At 11:59 AM 4/10/2006, Elizabeth D Rye wrote: >Hi Steve, > >I'm a little confused about one of your points: > >>>Given the current status of this SCR, it seems that that if we continue >>>we will have simply defined a new specific object with a recommendation >>>that it be used as resource during node data delivery and distribution. >>>If this is the case, why is there the need for a formally defined >>>object and yet another recommendation that will be ignored when convient. > >I'm not sure what you mean by "defined a new specific object". Aren't we >simply using the existing TABLE object, albeit in its non-recommended >format? Are you saying that in a sense it's a specific instance of the >TABLE object, ie., it must have two columns and they must be these two >columns in this specific format? Yes. > But we haven't really done that, have we, because we specifically left > open to the implementer the issue of how to format the table, and even > the order of the columns? Actually you are more correct. I should have said "essentially" a specific object. > Anyway, I'm still a little fuzzy on the issue of specific versus > general objects, so please help me understand this. If the definition was more strict, then a specific object could be formally defined. It has a type of "specific" with required elements and required subobjects. No optional elements or subobjects. It just allows a more strict validation. For example, all catalog objects are specific. This would be a good topic for one of our standards meetings. steve >Elizabeth >-- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov From J.Steven.Hughes@jpl.nasa.gov Mon Apr 10 11:55:27 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP; Mon, 10 Apr 2006 11:51:28 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942); Mon, 10 Apr 2006 11:51:28 -0700 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k3AIpRDr026011; Mon, 10 Apr 2006 11:51:28 -0700 Received: from jhughes-xp.jpl.nasa.gov (dhcp-78-33-253.jpl.nasa.gov [137.78.33.253]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k3AIpQJ7014796; Mon, 10 Apr 2006 11:51:27 -0700 Message-Id: <6.1.1.1.2.20060410114558.03bca0e8@mail.jpl.nasa.gov> X-Sender: jshughes@mail.jpl.nasa.gov X-Mailer: QUALCOMM Windows Eudora Version 6.1.1.1 Date: Mon, 10 Apr 2006 11:50:18 -0700 To: Elizabeth.D.Rye@jpl.nasa.gov From: "J. Steven Hughes" Subject: Fwd: MD5 SCR - Where to from here? Cc: Dan.Crichton@jpl.nasa.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Source-IP: dhcp-78-33-253.jpl.nasa.gov [137.78.33.253] X-Source-Sender: J.Steven.Hughes@jpl.nasa.gov X-AUTH: Internal IP >Hi Elizabeth, I will not be sending the following out. However, I understand that Susie, you and Dan might be talking tomorrow about the SCR. I am forwarding my input since I will be on travel till Thu. thanks, steve >Date: Mon, 10 Apr 2006 11:17:33 -0700 >To: Dan.Crichton@jpl.nasa.gov >From: "J. Steven Hughes" >Subject: MD5 SCR - Where to from here? > >Hi all, > >I guess I am confused about the requirements for SCR 3-1034 >(File Checksums)? We can easily derive a requirement like "ascertaining >the integrity of the archive" from our current level 1 and 2 >requirements. (This is an excerpt from the problem statement for SCR >3-1034.) However a data distribution checksum can be dervied only from >the proposed level 3 requirements and it is less clear that a data >delivery (to the nodes) requirement can be derived at all. > > >I suggest that there at least four issues for continued discussion. > >1) Should file checksums be required for node data delivery and distribution? > >2) Will file checksums for archive volumes start to address the current L1 >and L2 preservation requirements? > 3) If file checksums are required for archive volumes, how are the > checksum files to be maintained. > >4) When should these requirements be made effective? > >Given the current status of this SCR, it seems that that if we continue >we will have simply defined a new specific object with a recommendation >that it be used as resource during node data delivery and distribution. >If this is the case, why is there the need for a formally defined object >and yet another recommendation that will be ignored when convient. > >thanks, Steve > >Wikipedia - A checksum is a form of redundancy check, a very simple >measure for protecting the integrity of data by detecting errors in data >that is sent through space (telecommunications) or time (storage). From rsimpson@magellan.stanford.edu Fri Apr 7 11:57:54 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 07 Apr 2006 11:54:34 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 7 Apr 2006 11:54:33 -0700 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37IsXRK005462 for ; Fri, 7 Apr 2006 11:54:33 -0700 Received: from smtp2.Stanford.EDU (smtp2.Stanford.EDU [171.67.16.125]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k37IsXr6009357 for ; Fri, 7 Apr 2006 11:54:33 -0700 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp2.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k37IsW5Y032452 for ; Fri, 7 Apr 2006 11:54:32 -0700 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k37IsW823335 for Elizabeth.D.Rye@jpl.nasa.gov; Fri, 7 Apr 2006 11:54:32 -0700 (PDT) Date: Fri, 7 Apr 2006 11:54:32 -0700 (PDT) From: Dick Simpson 650-723-3525 Message-Id: <200604071854.k37IsW823335@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: feedback requested on NO vote on 3-1034 X-Source-IP: smtp2.Stanford.EDU [171.67.16.125] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.01% >Better get my American citizenship first! details ... From rsimpson@magellan.stanford.edu Wed Apr 5 10:52:33 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Wed, 05 Apr 2006 10:49:41 -0700 Received: from nmta2.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Wed, 5 Apr 2006 10:49:41 -0700 Received: from xmta3.jpl.nasa.gov (xmta3.jpl.nasa.gov [137.78.160.111]) by nmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k35Hnfmv026355 for ; Wed, 5 Apr 2006 10:49:41 -0700 Received: from smtp2.Stanford.EDU (smtp2.Stanford.EDU [171.67.16.125]) by xmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k35Hne9D017727 for ; Wed, 5 Apr 2006 10:49:40 -0700 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp2.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k35Hne1I001810 for ; Wed, 5 Apr 2006 10:49:40 -0700 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k35Hnes03342 for Elizabeth.D.Rye@jpl.nasa.gov; Wed, 5 Apr 2006 10:49:40 -0700 (PDT) Date: Wed, 5 Apr 2006 10:49:40 -0700 (PDT) From: Dick Simpson 650-723-3525 Message-Id: <200604051749.k35Hnes03342@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: CHECKSUM SCR X-Source-IP: smtp2.Stanford.EDU [171.67.16.125] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.00% >Both Anne and Todd have now gotten back to me. The md5deep utility >outputs filenames in the same format that it reads them in ... OK; this makes sense. The example file in the SCR would have been made from a CD mounted on a Sun. Since the files that go onto the CD must be upper case, any CHECKSUM.TAB file that appears on the CD should show upper case directory/file names. >Anyway, sounds like it's time to resurrect the uppercase/lowercase >SCR (sigh). That's not necessary; just change the example! From raugh@astro.umd.edu Mon Apr 3 07:02:32 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Mon, 03 Apr 2006 07:02:03 -0700 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Mon, 3 Apr 2006 07:02:03 -0700 Received: from xmta1.jpl.nasa.gov (xmta1.jpl.nasa.gov [137.78.160.144]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k33E22R3003724 for ; Mon, 3 Apr 2006 07:02:02 -0700 Received: from icarus.astro.umd.edu (icarus.astro.umd.edu [129.2.14.42]) by xmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k33E2230022664 for ; Mon, 3 Apr 2006 07:02:02 -0700 Received: by icarus.astro.umd.edu (Postfix, from userid 5257) id 9D76791F8; Mon, 3 Apr 2006 07:36:58 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by icarus.astro.umd.edu (Postfix) with ESMTP id 84E1391F4; Mon, 3 Apr 2006 07:36:58 -0400 (EDT) Date: Mon, 3 Apr 2006 07:36:58 -0400 (EDT) From: Anne Raugh To: Elizabeth D Rye Cc: Todd King Subject: Re: question about md5deep In-Reply-To: <4f5fb31f81dc2a10b74805e5dcc9c72f@jpl.nasa.gov> Message-ID: References: <4f5fb31f81dc2a10b74805e5dcc9c72f@jpl.nasa.gov> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Source-IP: icarus.astro.umd.edu [129.2.14.42] X-Source-Sender: raugh@astro.umd.edu X-JPL-spam-score: 0.00% > I have the impression that both of you have been running the md5deep utility. > Could you tell me if it output a file with uppercase filenames if you run it > on a volume where the filenames are all uppercase? Nope - I've been using "md5sum". I don't keep uppercase file names, but it's easy enough to check. Hang on... Md5sum writes the output file names in the same case it finds them, which is what I would have expected. Now, whether it sees upper or lowercase file names on a CD or DVD probably depends on how the disk was written and what options are set on the driver or volume manager... -Anne. From tking@igpp.ucla.edu Mon Apr 3 09:17:24 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Mon, 03 Apr 2006 08:59:38 -0700 Received: from nmta1.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Mon, 3 Apr 2006 08:59:38 -0700 Received: from xmta3.jpl.nasa.gov (xmta3.jpl.nasa.gov [137.78.160.111]) by nmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k33Fxcet026249 for ; Mon, 3 Apr 2006 08:59:38 -0700 Received: from igpp.ucla.edu (terra.igpp.ucla.edu [128.97.94.1]) by xmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k33FxbMI018158 for ; Mon, 3 Apr 2006 08:59:38 -0700 Received: from mail.igpp.ucla.edu (mail.igpp.ucla.edu [128.97.68.126]) by igpp.ucla.edu (8.12.10+Sun/8.12.10) with ESMTP id k33FsRiD005534; Mon, 3 Apr 2006 08:54:27 -0700 (PDT) Received: from kingpc ([128.97.68.39]) by mail.igpp.ucla.edu (8.12.8/8.12.5) with ESMTP id k33FxPM3008027; Mon, 3 Apr 2006 08:59:34 -0700 From: "Todd King" To: "'Elizabeth D Rye'" , "'Anne Raugh'" Subject: RE: question about md5deep Date: Mon, 3 Apr 2006 08:59:24 -0700 Message-ID: <01b601c65737$973ef060$27446180@kingpc> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 Importance: Normal In-Reply-To: <4f5fb31f81dc2a10b74805e5dcc9c72f@jpl.nasa.gov> X-Source-IP: terra.igpp.ucla.edu [128.97.94.1] X-Source-Sender: tking@igpp.ucla.edu X-JPL-spam-score: 0.00% Hi Elizabeth - The output preserves the case of the original file name. If it is uppercase then the output will be in uppercase, mixed-case it will be mixed-case. The md5deep utility also supports relative paths. -Todd- > -----Original Message----- > From: Elizabeth D Rye [mailto:Elizabeth.D.Rye@jpl.nasa.gov] > Sent: Friday, March 31, 2006 12:18 PM > To: Anne Raugh; Todd King > Subject: question about md5deep > > > Hi, > > I have the impression that both of you have been running the md5deep > utility. Could you tell me if it output a file with uppercase > filenames if you run it on a volume where the filenames are all > uppercase? > > Elizabeth > -- > Elizabeth D. Rye > Phone: (818) 354-6135 > Email: Elizabeth.D.Rye@jpl.nasa.gov > > From rsimpson@magellan.stanford.edu Fri Mar 31 12:39:01 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 31 Mar 2006 12:35:40 -0800 Received: from nmta2.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 31 Mar 2006 12:35:40 -0800 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKZeNW021521 for ; Fri, 31 Mar 2006 12:35:40 -0800 Received: from smtp2.Stanford.EDU (smtp2.Stanford.EDU [171.67.16.125]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKZded017992 for ; Fri, 31 Mar 2006 12:35:39 -0800 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp2.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k2VKZatB032045 for ; Fri, 31 Mar 2006 12:35:37 -0800 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k2VKZas12187; Fri, 31 Mar 2006 12:35:36 -0800 (PST) Date: Fri, 31 Mar 2006 12:35:36 -0800 (PST) From: Dick Simpson 650-723-3525 Message-Id: <200603312035.k2VKZas12187@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: CHECKSUM SCR Cc: rsimpson@magellan.stanford.edu X-Source-IP: smtp2.Stanford.EDU [171.67.16.125] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.00% >To be honest Dick, I don't know. Do you want me to pursue this and >find out for you? I could try to contact someone who has it installed. I'm curious ... I think I voted with the majority in saying that we should pick a format that is consistent with what the available utilities put out. But, if md5deep lists paths as entirely lower case when they are actually mixed or upper, then md5deep has a defect that I wasn't expecting. I'm guessing that the listing that was pasted into the SCR was from a run on paths that were not PDS compliant. Let's see whether Anne or Todd has anything to help. Thanks, Dick From rsimpson@magellan.stanford.edu Fri Mar 31 12:14:01 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 31 Mar 2006 12:10:05 -0800 Received: from nmta3.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 31 Mar 2006 12:10:05 -0800 Received: from xmta2.jpl.nasa.gov (xmta2.jpl.nasa.gov [137.78.160.56]) by nmta3.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKA4pt002281 for ; Fri, 31 Mar 2006 12:10:04 -0800 Received: from smtp1.Stanford.EDU (smtp1.Stanford.EDU [171.67.16.123]) by xmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKA4dD026259 for ; Fri, 31 Mar 2006 12:10:04 -0800 Received: from magellan.stanford.edu (magellan.Stanford.EDU [171.64.90.216]) by smtp1.Stanford.EDU (8.12.11/8.12.11) with ESMTP id k2VKA3W1015603 for ; Fri, 31 Mar 2006 12:10:03 -0800 Received: (from rsimpson@localhost) by magellan.stanford.edu (8.11.7p1+Sun/8.11.7) id k2VKA3U11983; Fri, 31 Mar 2006 12:10:03 -0800 (PST) Date: Fri, 31 Mar 2006 12:10:03 -0800 (PST) From: Dick Simpson 650-723-3525 Message-Id: <200603312010.k2VKA3U11983@magellan.stanford.edu> To: Elizabeth.D.Rye@jpl.nasa.gov Subject: Re: CHECKSUM SCR Cc: rsimpson@magellan.stanford.edu X-Source-IP: smtp1.Stanford.EDU [171.67.16.123] X-Source-Sender: rsimpson@magellan.stanford.edu X-JPL-spam-score: 0.00% >Again, I was instructed by a majority of those present at the telcons >to use the file format produced by common checksum utilities. These >utilities generally output their results using lower case file names. So, if I run md5deep on a volume with upper case directory and file names, it will give me output that will be in lower case? Or is this a case where the directories and files were lower case to begin with? From guinness@rsmail.wustl.edu Fri Mar 31 12:19:01 2006 Return-Path: Received: from localhost by mbox1.jpl.nasa.gov with LMTP for ; Fri, 31 Mar 2006 12:16:31 -0800 Received: from nmta2.jpl.nasa.gov with LMTP by mbox1.jpl.nasa.gov (3.0.2/sieved-3-0-build-942) for ; Fri, 31 Mar 2006 12:16:31 -0800 Received: from xmta1.jpl.nasa.gov (xmta1.jpl.nasa.gov [137.78.160.144]) by nmta2.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKGVFv006319 for ; Fri, 31 Mar 2006 12:16:31 -0800 Received: from rsmail.wustl.edu (rsmail.wustl.edu [128.252.144.20]) by xmta1.jpl.nasa.gov (Switch-3.1.8/Switch-3.1.7) with ESMTP id k2VKGUrM017183 for ; Fri, 31 Mar 2006 12:16:30 -0800 Received: from mercury.wunder.wustl.edu (wormhole.eprsl.wustl.edu [128.252.144.154] (may be forged)) (authenticated bits=0) by rsmail.wustl.edu (8.13.1/8.13.1) with ESMTP id k2VKG5BN006845 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 31 Mar 2006 14:16:05 -0600 Message-Id: <6.2.3.4.2.20060331141347.02a88f78@wunder.wustl.edu> X-Mailer: QUALCOMM Windows Eudora Version 6.2.3.4 Date: Fri, 31 Mar 2006 14:16:30 -0600 To: Dick Simpson 650-723-3525 , elizabeth D Rye From: "Edward A. Guinness" Subject: Re: CHECKSUM SCR In-Reply-To: <85af05cb0908797a08d9990d6cda36d8@jpl.nasa.gov> References: <200603311648.k2VGm8p02394@magellan.stanford.edu> <85af05cb0908797a08d9990d6cda36d8@jpl.nasa.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Source-IP: rsmail.wustl.edu [128.252.144.20] X-Source-Sender: guinness@wunder.wustl.edu X-JPL-spam-score: 0.00% Elizabeth and Dick, Just for the record Geo was the only no vote at the tech session, but Atmospheres, NAIF, and Radio Science were not present. Ed >>Section D.2: >> >>a. I thought we agreed during the last discussion I joined >>that checksum tables would not be required until PDS4; the >>SCR says each archive volume MUST have one, which means >>they will be required now. I understand that pipelines >>already designed without the checksum are exempt; but I'm >>not ready to REQUIRE checksums in future PDS3 designs. > >At the 03/08 telecon (the last one prior to the 03/22 one you >missed), we were split on whether or not the checksum tables should >be required. We had also agreed to postpone the decision on when >they should be required. In order to resolve the split, I chose to >write the SCR as if checksum tables were required, then asked for a >vote on the SCR in that form. I told everyone present that if the >vote failed on the SCR in that form, we would change the wording to >make the checksum tables optional and then conduct the vote >again. However, the SCR was approved in the required form, with >only the GEO node dissenting. The "when" question was addressed by >our accompanying recommendation that only missions archiving to v3.8 >and higher of the standards should be required to comply with the >SCR, if approved. >