IPAC 2MASS Working Group Meeting #98 Minutes

IPAC 2MASS Working Group Meeting #98 Minutes 7/16/96

Attendees: R. Beck, T. Chester, R. Cutri, T. Evans, J. Fowler, L. Fullmer, T. Jarrett, G. Kopan, B. Light, H. McCallon, J. White

AGENDA

  1. Use of Stderr
  2. Subsystem Interfaces with EXEC/PCP
  3. QA & DM Review
  4. Camera Status
  5. 2MAPPS Vsn 0.1
  6. DLT Tape Budget

DISCUSSION

  1. Use of Stderr -- The design of the EXEC and PCP subsystems has progressed to the point where some clarification of the use of stdout and stderr has occurred. These standard unix devices ("standard output" and "standard error", respectively), are accessed in FORTRAN by writing to units 6 and 0, respectively. Subsystem implementers were previously requested to use stderr for all warning and error messages, anticipating that the MONITOR program (part of EXEC) would intercept stderr for display and run control purposes. Furthermore, the same messages were to be sent to stdout for print summary tracking. It has now become clear that the MONITOR will need to read both stdout and stderr, so that such messages would now occur twice in the MONITOR input if written as previously requested.

    In most cases, a simple deletion of all references to stderr in existing subsystem code would suffice for this change in MONITOR design. The method deemed best by the cognizant engineer for each individual subsystem will be acceptable. The goals are:

    Since the MONITOR will read all stdout from every subsystem, excess verbiage should be avoided in production mode. Debug output in the system environment is a desired capability, but it must be able to be turned off. Two typical ways of doing this are: These can also be combined.

  2. Subsystem Interfaces with EXEC/PCP -- A memo is being distributed by J. White which details subsystem coding conventions that will ease integration with PCP and EXEC. It also specifies severity levels for exit codes; these should be implemented consistently in all subsystems.

    The EXEC/PCP script is being implemented in a form that involves calling subsystems via a csh script wrapper. In most cases, this subsystem script need not do very much beyond invoking the subsystem executable program with appropriate command line parameters, but these parameters need to be set up so that the names used are consistent with environment variables provided by the main scripts (i.e., EXEC/PCP). There is also a need to control path information and perhaps other standard operating mode declarations.

    Since most subsystem cognizant engineers have not yet seen the EXEC/PCP definitions of supplied environment variables, J. White has stated that he expects to support each Cog E in the writing of the wrapper script. It is also expected that some additional environment variables will be needed, and he will add them as he is informed of such needs.

    Since the subsystem scripts comprise an interface between EXEC or PCP with individual subsystems, it is highly desirable for an interface definition to be prepared in each case that supplies all information needed to assure consistency; this would include signoff lines for cognizant individuals involved in the interface. A new SIS format is probably required, and might take the form of a simple listing of the script along with parameter definitions. The EXEC/PCP SDS will document all environment variables, so to a large extent, simply referencing the version of the EXEC/PCP SDS may suffice in the SIS's.

    The memo also details file naming schemes, use of a single NAMELIST file to provide multiple NAMELIST input, and other such considerations. Team members are requested to become familiar with the contents so that they can be discussed more fully at next week's meeting.

  3. QA & DM Review -- R. Cutri reported that the 2MAPPS CDR has been rescheduled, and its focus has been modified. It will now be the Quality Assurance and Data Management Review and is scheduled for November 13, 1996. It is expected to be held at IPAC. "Data Management" refers to end-to-end pipeline data flow, including database processing. In support of the "Quality Assurance" part, Roc requests that the Cog E's identify the five most useful quality judgement parameters for the corresponding subsystem by August 21 and supply a SIS for passing these to the QUALITY subsystem.

  4. Camera Status -- R. Cutri reported that the science-grade arrays had been installed in the camera and tested. It was found that one quadrant of the H array was bad because of electrical contact problems. It was returned to the manufacturer for repair and is due back at UMASS by the end of this week. 60 Hz noise with an amplitude of about 300 electrons was also observed; this is considered typical of the problems encountered in first-time power-ups of this kind of equipment and is not a cause for alarm. All such startup transients are expected to be smoothed out within a few weeks.

  5. 2MAPPS Vsn 0.1 -- For some time we have been aiming at the target date of 12 September for exercising the first integrated version of the 2MAPPS pipeline, dubbed herein as version 0.1 because many of the modules may be stubs. The goal remains to have a pipeline that can pass data end-to-end by this date.

  6. DLT Tape Budget -- T. Chester reported that the cost of DLT tapes in quantities supporting R. Cutri's recent data volume estimates is much higher than anticipated and actually threatens the hardware budget. If purchased now (which will of course not be done), the cost would be about $500,000, i.e., half the hardware budget. Some discrepancies were encountered in various team members' perceptions of the rate at which tape storage costs are declining, but it seems some relief is surely possible because of that. Nevertheless, it appears that some extra steps will be necessary to avoid wasted tape space, such as stacking backup raw data and processed data on single tapes, recycling tapes from nights found to be unacceptable, and certainly the widespread use of compression is indicated. R. Cutri stated that his estimates were designed to illustrate the result of expected data acquisition rates and are probably not overestimated, but that if tapes are not written at the observatory on non-photometric nights, the tape volume might drop by as much as 30%. In any case, we have been expecting to monitor the development of more cost effective data storage technologies anyway, and we will continue to revisit this issue. In the short term, we will not buy any more tapes than are clearly needed for purposes in the immediate future.