Date: Wed, 26 Nov 1997 17:01:24 -0800 (PST)
From: jwf@ipac.caltech.edu
To: 2mass@ipac.caltech.edu
Cc: chas@ipac.caltech.edu, sstrom@donald.phast.umass.edu,
    stiening@ipac.caltech.edu
Subject: 2MASS WG Mtg #137 Minutes

           IPAC 2MASS Working Group Meeting #137 Minutes
                            11/25/97

Attendees: R. Beck, T. Chester, R. Cutri, D. Engler, T. Evans,
           J. Fowler, L. Fullmer, G. Kopan, J. Mazzarella,
           H. McCallon, B. Nelson, B. Wheaton, S. Wheelock,
           J. White


AGENDA

1.) 970731n Analysis
2.) PSF Model Status
3.) FREXCLEAN
4.) Survey Visualization
5.) K-Band "Split" Sources



DISCUSSION


1.) 970731n Analysis

    R. Cutri reported that the 970731n analysis is tying up disk
space needed for pipeline processing, which cannot resume until
more space is made available. People analyzing the 970731n
before-and-after results are requested to wrap up their work as
soon as possible. The decision was made to spool the 970731n data
to tape for the holiday weekend so that the pipeline processing
can proceed, then the pipeline outputs will be spooled next
Monday, and the 970731n data will be restored.
    The analysis involves the impact of changing the way PICMAN
handles masked pixels. Since the coadding interpolation kernel
covers a significant number of pixels in each frame (the exact
number depends on the kernel), if the presence of a dead pixel
anywhere in the kernel invalidates the entire contribution of
that frame to the image, the incidence of thin coverage and
"holes" in the images can become serious enough to result in an
unacceptable amount of degradation.
    Several members reported progress in their analyses, but no
one has quite finished. Each reported intermediate results that
ranged from neutral to positive.


2.) PSF Model Status

    B. Wheaton reported that last Friday had been a good day for
the PSF generator. Despite continued IEEE errors, mostly at
reduced frequencies, the PSF grid is fairly well populated in
each band for shape parameters between 0.94 and 1.08. More work
is needed to extend the the low and high shape parameter regions,
and there is some difficulty in finding scans with good enough
seeing to supply low-end scans. There are plenty of scans for the
high end, but scans with large shape parameters also tend to have
large dispersions in shape parameter.
    B. Wheaton also reported that K. Marsh is close to having a
version of the PSF generator that does not use the "filter"
routine to detect point sources; it will use a list of existing
detections. This will speed up the program so that execution time
should not be an issue, and perhaps some use of the code for
calibration scans inside the production pipeline might be
feasible.
    J. Fowler requested that thought be given to analysing the
behavior of the PSF models as functions of shape parameter. There
is a possibility that the results from separate scans will
provide shape dependence that is not smooth, even possibly non-monotonic, 
and some smoothing might be prudent. Furthermore, this
would lead naturally to an interpolation/extrapolation capability
that might prove useful in the near term before enough acceptable
scans can be found and processed to populate the extreme shape
parameter regions. R. Cutri pointed out the difficulty in
treating the PSF variance image in this same manner, and G. Kopan
pointed out that other difficulties must also be weighted, such
as focus variations not being modelled. J. Fowler and G. Kopan
agreed to investigate PSF dependence on shape parameter once the
PSF grids are put into usable shape.


3.) FREXCLEAN

    T. Evans reported that a new routine, FREXCLEAN, has been
delivered. This routine removes FREXAS Read2-Read1 detections
that are within a thresholded distance of bright Read1
detections. The reason for doing this is that such detections
have been found to cause problems in the merging performed by
PFPREP, and they are usually either saturated or artifacts that
carry no useful position information. Rather than helping PFPRPEP
compute frame offsets, they create confusion and position
dispersion that hinders it.
    FREXCLEAN will be used in the pipeline between FREXAS and
PFPREP despite the fact that the problem that motivated its
creation was solved independently by using the latest estimates
of certain POSMAN parameters in PFPREP. PFPREP has to solve for
the frame offsets iteratively, since it has to merge detections
in order to compute frame offsets, but frame offsets must be
known in order to merge detections. It starts with an estimate of
the frame offsets based on previous average values. Apparently
the better initial estimates of these average frame offsets were
sufficient in the case examined to eliminate the merge failures
that caused "split" and confused sources. Nevertheless, there is
clearly a sensitivity to the initial estimates that prudence
demands be alleviated as much as possible, so FREXCLEAN will
provide upstream support, and as discussed in item 5 below,
downstream support will also be added.


4.) Survey Visualization

    T. Evans and J. Mazzarella displayed a new "survey
visualization" tool: an image of about a hundred square degrees
of sky taken from the IRAS Sky Survey Atlas with the 2MASS tiles
superposed with black-line boundaries and actual scans superposed
with white-line boundaries. The presence of the scans indicates
the progress of the survey in the vicinity, and the image will be
available on computer displays for accessing images from scanned
areas by clicking on the desired area.


5.) K-Band "Split" Sources

    Several failure modes have now been identified that yield
"split" sources, and progress has been made in fixing such
problems, which have been found through the analyses performed by
a number of people, most often by S. Wheelock, and recently by M.
Skrutskie and M. Weinberg. Three types of split-source problem
have been discovered and will be described separately. A "split"
source is a point source downstream from band merging that is
missing one or two of its detections while those detections are
in the immediate vicinity but not merged in the combination that
seems obvious.
    The simplest problem of this sort results from extractions
for which PROPHOT is unable to compute a profile-fit magnitude;
currently 3-arcsec positions are assigned to such extractions.
Frequently these are associated with bright objects, and multiple
extractions from deblending or unmerged Read1 detections may all
be present. When these are processed by BANDMERGE, the large
uncertainties yield inappropriately small chi-square values,
allowing these less optimal representations to attach themselves
to good detections in other bands. The result is that what should
be JHK sources have a 99.999 magnitude in one band while a
detection with a good magnitude in that band sits nearby,
unmerged because its position uncertainty was not artifically
large. This problem appears to have been fixed by three changes:
(a.) improved tuning in MAPCOR, which reduces the amount of
unmerged Read1 detections; (b.) BANDMERGE now clips position
uncertainties on the high side (default value: 1.0 arcsec),
greatly reducing the range over which the sub-optimal detections
can reach (a trick learned from the asteroid processor); (c.)
BANDMERGE now includes a status-flag vector pseudo-dot-product
term in its confusion-resolving code that penalizes matches
between detections that have one good magnitude and one 99.999
magnitude.
    The second kind of "split" source results from PFPREP failing
to merge all Read1 detections of a given source. If a merge seed
happens to be on the outside of the position scatter of its
various single-frame detections, it may not pick all of the more
distant ones up, and the leftovers eventually become a different
source. When Read1 detections are involved, a single source
becomes two separate Read1 sources for MAPCOR, which must pass
both on to BANDMERGE, and a "split" source results. This problem
can be solved only by better merging in PFPREP, and in the case
studied most thoroughly, the problem evaporated when better
initial estimates for the frame offsets were used, as in item 3
above. Again, a sensitivity to initial estimates needs to be
alleviated, and a plan to do this was suggested by H. McCallon:
when PFPREP terminates, it has information about how far off its
initial estimates were with respect to its final solutions; these
differences could be used in a test to trigger a second run of
PFPREP with a processing flag set that would cause it to use the
first run's final solutions as its "initial" estimates. This
should eliminate the "split" source problems described so far
herein, which may or may not be related to the hysteresis-
afflicted band-to-band position shifts observed before. The
latter are sufficient reason to install the double-pass
capability, but previously the priority for this was not as high
as it perhaps should now become. It does not appear that a large
run-time cost would be involved, since most scans would probably
not trigger the second pass, and PFPREP is not one of the more
CPU-intensive modules in the pipeline.
    The third kind of "split" source is not yet fully understood.
The case studied is scan 016 of 971109n. This is a calibration
scan with a lot of JH sources that obviously should have K
components, and those K components lie within about 1 arcsec in
the form of single-band objects. What makes this case unusual is
that there are plenty of JHK sources that do not have this
problem. H. McCallon compared the FREXAS extractions for this
scan to the corresponding PROPHOT extractions and found that J
and H were consistent, whereas two populations of K detections
were identified: (a.) population 1 was well behaved, having
typically small position discrepancies between FREXAS and PROPHOT
versions, extending the length of the scan (these yielded the
correct JHK sources); (b.) population 2 also extended the length
of the scan, but had large mean discrepancies in Y (about 1.0
arcsec) with small dispersion (about 0.11 arcsec), and large mean
discrepancies in X (about 0.9 arcsec) with a large dispersion
(about 0.33 arcsec) and a possible dependence on Y position.
    It was found that the position discrepancies had shown up in
the BANDMERGE statistics file, and the spectral-combination
anomalies had shown up in the BANDMERGE QUALITY file; both were
quite different from the corresponding files for scan 017, which
covered the same calibration region but had none of the
anomalies. The backgrounds were found to be higher on scan 016
than on scan 017, surprisingly in K more than in J, but also in
H, where for some reason PROPHOT produced less than half as many
detections for s016 than for s017, while the FREXAS output was
only slightly less.
    Another clue was found: 90% of the population 2 detections
were from the right-hand side of the focal plane. A check of the
"3-by-3" PSF grids for s016 and s017 (a diagnostic computed for
each scan for use by GALWORKS, not used by PROPHOT) showed
noticeably bad K PSF estimates on the right-hand side relative to
the left for s016, but the sampling was not dense enough to draw
firm conclusions, and a similar tendency was seen in s017,
although not as bad. The PSFs for the other bands were also not
entirely missing this phenomenon. Another possible clue is that
the mean magnitudes differed significantly, with the first
population having a mean of 12.7 and the second 14.1.
    It appears that this scan would not be acceptable because of
atmospheric conditions, but concern was expressed over the
failure to understand why the anomalies took the form of two K
populations, as well as whether other scans had suffered similar
anomalies. J. Fowler will examine a large sample of BANDMERGE
position statistics files and QUALITY files to see if the pattern
can be found elsewhere.
    [Editor's note: after the meeting, it was found that the
latest version of the pipeline software had processed this scan
without the anomalies. This convinced H. McCallon that this too
is tied in with correct merging in PFPREP, adding priority to the
double-pass capability for that program discussed above. But the
concern still remains regarding the lack of understanding of the
exact mechanism that produced two K populations. Analysis of this
issue will continue.]