- FDD Conformance --
In studying the 2MAPPS documentation (FDD, certain SDS's and SIS's), S.
Wheelock discovered two inconsistencies between the FDD and other documents.
Section 2.6 of the FDD states "...file formats representing position para-
meters must be accurate to three decimal places when represented in seconds
of time, two decimal places when represented in seconds of arc or camera
pixels, and six decimal places when represented in floating-point degrees."
Since it is clear that the intent is not accuracy but precision, this
section will be reworded when the next version of the FDD is prepared, with
"must provide precision of" replacing "must be accurate to". The main point
is that some SIS's indicate position degree formats of F10.5, which provides
a precision of only 0.036 arcseconds where 0.02 is required. Subsystem
cognizant engineers are requested to check their SIS's and correct them in
accordance with the six-decimal-place requirement where necessary, being
sure that all users of the interface are aware of the changes.
The other inconsistency involves the interpretation of subsystem exit
codes. When a subsystem terminates, an exit code is passed to the calling
script (EXEC, PCP, or a subsystem wrapper-script). The FORTRAN default is
zero, and the value can be changed by using CALL EXIT(N) instead of STOP
(other programming languages have similar control over the exit code), where
N is set to the value desired for the exit code. When section 2.5 of the FDD
was written, the intent was to allow for expansion of non-critical exit code
meanings, and the convention was established that lower-numbered nonzero
codes were the most severe. The PCP SDS and most of the subsystem coding has
adopted higher-numbered codes as more critical. The less expensive resolu-
tion of this conflict is to change the FDD, since the intent is to ensure
uniformity rather than any particular convention. The cognizant engineers
should employ the conventions stated in the PCP SDS, which are as follows
(quoted from the SDS).
|
7) SUBSYSTEM RETURN CODES
Each subsystem will return to the PCP a completion code by executing
an exit (code). This code will be written to the "log" file and can
effect continued processing of the pipe. The return codes have the
following meaning:
Value Effect
0 AOK : SS Processing complete with no problems
1
2 Alert : proceed nominally; note possible problem
3
4 Stop scan : Abort PCP but proceed with the next scan
5
6 Stop EXEC : Abort THIS scan and start no NEW ones
7 Abort EXEC: Abort ALL Scans ASAP
Setting any status code other than 0 will force the "no Delete" PCP
option at EOS.
Possible non-zero return code examples:
code 2 : Posman computes extracted src positions with large uncertainties
code 4 : Rdframe found no PAR file for this scan.
code 6 : the output disk is full.
|
- NAMELIST File Delivery and Utilization --
R. Beck, who has taken over software delivery duties from the departing
G. Laughlin, requested a clarification of NAMELIST file delivery require-
ments with the following issues to be addressed:
- Must NAMELIST files be delivered with each delivery of executable
software?
- Can NAMELIST file deliveries be made without delivering executable
software at the same time?
- How will hemisphere dependence be handled?
- How will time dependence be handled?
The last two issues are also involved in the question of how EXEC/PCP is
supposed to get the right versions of NAMELIST files to the subsystems in
the context of specific observation dates (ObsDates), scans, and hemi-
spheres. Specifically regarding delivery by a subsystem cognizant engineer
to the system, the following standards were adopted.
- There will be two dedicated hemisphere-specific subdirectories for
all data, including NAMELIST files, above the subsystem level. For
delivery purposes, cognizant engineers will place a NAMELIST file in
each subdirectory, one for each hemisphere (identical if there is no
hemisphere dependence; but see item D below) named /2massc/del/datan
and /2massc/del/datas for the northern and southern hemispheres, respec-
tively. To avoid naming conflicts, the NAMELIST files will be named
nl., where is the name of the subsystem or module
that will read that file; for example, BANDMERGE will read nl.bandmerge,
and the POSMAN module PFPREP will read nl.pfprep. The names will be the
same for both hemispheres, but the files will reside in the separate
directories as stated above.
- NAMELIST files will be delivered as part of any software delivery;
when only the NAMELIST files have changed, they may be delivered to the
directories above without redelivering the rest of the software in its
dedicated delivery directory (e.g., /2massc/del/src/pixcal).
- If there is a possibility of time dependence (e.g., the northern
telescope had to be recollimated on 980123, and POSFRM's distortion
model had to be recalculated), then the NAMELIST should contain para-
meters indicating the valid date range for the other data, and the
software should compare the ObsDate to this range before accepting the
NAMELIST (or other data file) input. When multiple date ranges must be
available, the different sets of data should be concatenated in a single
file for EXEC/PCP to handle, and the input file should be reread as
necessary to obtain a valid date range. Note that EXEC/PCP provides a
special environment variable that can be used to override the standard
setup of data directories for a subsystem; this may be useful for system-
level testing or other purposes, so any cognizant engineer who feels a
need for more flexible data setup should consult with R. Beck and J.
White to learn how to use this option.
- The EXEC/PCP cognizant engineers request that every file existing
in separate hemisphere-dependent locations contain either the text
hemis='n' or hemis='s' (lower case, no embedded blanks) to tag its
corresponding hemisphere, even if there is no other real hemisphere
dependence. This is to allow certain scripts to grep on this text and
perform standardized operations on files containing it. Note that such
text may be placed in a table-file header; it may also be placed in
NAMELIST files as part of the NAMELIST dictionary or as a comment
between NAMELIST groups (e.g., after the &end and before the next input
group, if any, begins with another &).
- If a subsystem requires script operations above the scan level, the
cognizant engineer may arrange with EXEC/PCP to include those opera-
tions in the system level initialization or termination scripts, but
not via a general-purpose calling of subsystem scripts; the system-
level scripts must contain the code and be maintained at the system
level. For example, the system-level initialization script will handle
copying the NAMELIST (and other) files for the appropriate hemisphere
to the appropriate scan directories for the subsystems to use; similar
sorts of operations for subsystem-peculiar requirements can be included
via understandings worked out with the EXEC/PCP cognizant engineers.
These conventions will be put into effect during the 2MAPPS 0.1 delivery
exercise, not before, since two subsystems have already been delivered prior
to the adoption of these conventions. Many changes will be found necessary
during the tests about to start, and conforming to these standards can be
included among them as work progresses. It should also be noted that the
descriptions given above are hereby being presented to the entire team for
the first time in such specific terms and reflect the author's interpretation
of the working group discussion; if errors are found, a full corrected state-
ment of all of the above aspects will be included in future meeting minutes.
Team members are requested to make known any problems they may see in these
conventions as described above.
- QUALITY Q&A --
L. Fullmer answered questions regarding how subsystems interact with the
QUALITY subsystem, such as what directories files should be written in and
what information is needed for QUALITY to make use of the information. The
answer to the first question is that the directory structure will include a
scan subdirectory named qual, i.e., just as the northern-hemisphere scan
number 123 for the observation date 980204 will involve a directory named
980204n/s123/flat for flattened-frame data, it will involve a directory
named 980204n/s123/qual to which data intended as input to the QUALITY
subsystem should be written. In addition, there will be a higher-level
directory for the night, 980204n/datequal, where the QUALITY subsystem itself
will operate.
It was pointed out that the best way for a subsystem to get its QUALITY
output to the right directory is to write it to the current working directory
and then have the subsystem wrapper script move it to the proper directory.
This will be possible via simple relative-address directory specification
(e.g., "../qual") or via environment variables provided by EXEC/PCP.
As far as how to interpret its input, QUALITY will rely on the expertise
of the subsystem engineers to provide thresholds for defining out-of-limits
conditions, warning levels, guidance on what sort of graphical representa-
tions would be useful for judging quality, etc. It is recognized that such
expertise is just now being accumulated, and that considerable work with
survey shakedown data will be needed before the vast range of possibilities
can be honed down to the truly useful diagnostics. Thus this exercise of
developing quality assessment tools will be an ongoing activity for which
the primary individuals in charge will be R. Cutri, L. Fullmer, and J.
Mazzarella.
- Cumulative File Handling --
Several (if not most) subsystems need to have scan-independent cumula-
tive data files, and the question of how to handle this need arose. For
example, the CALMON subsystem needs a cumulative data base for each hemi-
sphere in which calibration data can be accumulated over time, thereby
reducing parameter uncertainties as more information is gathered. A similar
need exists for the POSMAN and BANDMERGE subsystems.
Because of the asynchronous parallel processing of many scans, one
cannot assume that a subsystem can simply open a cumulative file and write
to it, because that file could be in use by another executing copy of the
same subsystem. File-locking techniques exist, but all simple methods tried
so far have been found to have certain quirks that would be best avoided.
For this reason a different approach is to be taken: updates to cumulative
files should be written to the current working directory, and then the EXEC
subsystem will run a termination script that cleans up the entire run,
including moving/appending these updates to their cumulative directories-
/files. This appears to be workable for all the cases where a need is
currently recognized, because only ASCII-file updates appear to be needed.
Cognizant engineers who need this service should schedule a discussion with
J. White and R. Beck to arrange for the proper script coding to be included.
- 2MAPPS 0.1 Status --
Software deliveries for the version 0.1 2MAPPS integration are on sched-
ule for September 18. A plan for bridging certain gaps has been formulated.
The key gaps are: DARKS, POSFRM, and POSPTS. A program that can strip out
FITS files from the scan data has been written and will be used to set up
twilight flat and dark frames for hand analysis by R. Cutri to produce respon-
sivity images, masks, and darks; this replaces the DARKS subsystem.
The POSMAN program PFPREP will be enhanced in several ways, the relevant
one here being to expand the content of its erstwhile POS09 file to match
the format and content of the POS01 file but with preliminary estimates for
all quantities (it already contained such estimates for the frame offsets
and several other parameters in common with POS01). The key element is the
connection to J2000 coordinates, which at this early stage is made exclusively
via the a priori position information passed from the observatory input. Since
no associations with astrometric catalogs will be made until the real POSFRM
comes online in a few months, tying extractions to external astrometric
information will not be possible in version 0.1, nor will attenuation of the
random walk in the frame-to-frame offset solutions, which will themselves be
only the preliminary J-band-only estimates (since the first processing will be
with replicated, K-band data, the limitation to single-band information will
not be a real degradation at this stage).
A POSFRM stand-in program denoted as version -0.9 will use the PFPREP
POS01-like output and the FREXAS extractions to create a file in the format of
POS02 that should be suitable for testing MAPCOR, and the POS01 substitute
should suffice for BANDMERGE and POSPTS version -0.8, where the latter is
another stand-in program that will provide the POSPTS function of converting
the BANDMERGE output point-source list to J2000; the associations field of
each record, however, will indicate that no association has been made, since
no catalog or capability to use it has been set up yet.
The output of 2MAPPS 0.1 should resemble the real output in format and
general content, but with position and photometric accuracies not meeting the
project requirements, although not so erroneous as to be unrecognizable. As
stated in the minutes of previous meetings, many functions will not be per-
formed, and associated files will not be generated. For more information on
these limitations, see the minutes to meeting number 101, 8/20/96.