Protocamera Data Processing Status
To: 2MASS Science Team 6 June 1995
From: Roc Cutri
RE: Protocamera Data Processing Status
1. Observing Run Executive Summary
As most of you are probably aware, we were blessed with 9 nights of
photometric weather during the Protocamera run at the KPNO 50" telescope.
During those nights, over 480 sq.deg. of sky were scanned, divided
roughly equally between J, H and Ks measurements to produce
~40 Gbytes of data. Approximately 340 sq.deg. were covered in long,
6 degree, survey-type scans, and 146 sq.deg. were observed during
calibration measurements. A summary of the targets observed is
provided in the 2MASS Project Information Web page at UMASS
(http://pegasus.astro.umass.edu/2MASS/). As Mike Skrutskie has
already reported, virtually all of the highest priority
observations were accomplished successfully, as well as most of the second
and third priority measurements. Observations were also carried out on
an additional 9 non-photometric nights, and another ~30 Gb of data were
collected in various regions, including the galactic center, a number
of star formation regions, bright galaxies and clusters and other
miscellaneous fields.
2. Preliminary Processing and Pipeline Preparation
Preliminary processing of selected data has been carried out over the
past several weeks to tune and prepare the operational (ops) pipeline.
Data from the nights of 4-22 and 4-25 were used for the preparation.
As a result of the analysis of earlier Protocamera data and
recommendations by the Science Team, several refinements have been
incorporated into the operational processor to improve photometric
accuracy and processing efficiency. These include:
a. Evaluation of a seeing estimation parameter (PFRAC) that measures
the fraction of encircled energy within fixed diameters.
b. Use of a seeing-parameterized grid of psf's, rather than a single psf,
as was the case for the 6/94 data processing, or scan-by-scan psf
generation. Currently, the grid contains 3 psf's per band.
c. Hybrid aperture/KAMPHOT photometry is derived for intermediate
brightness point sources.
d. Inclusion of photometry using a series of fixed circular apertures
for extended sources.
During preliminary pipeline testing, a number of unusual features were
identified in the data and in the pipeline. These are summarized below,
and also given are the suspected origin of these abnormalities and the
preliminary solutions that have been incorporated into the operational
pipeline.
Known Abnormalities in Data and Processing
------------------------------------------
a. Systematic offsets between in-scan positions of R1 and R2-R1 sources
i. Suspected cause - Telescope/mechanical. Secondary has not yet
settled before R1.
ii. Ops solution - Measure offsets for large numbers of objects and
apply correction to source positions in frame coordinates
b. POSREC start-up errors in very high source density and/or high extinction
regions. This affects less than 10% of the current scans.
i. Suspected causes:
- In several cases initial telescope position reported incorrectly
- POSREC jumpstart uses optical catalog (GSC) - bears little
resemblance to near IR star fields in these regions.
ii. Ops solution: These scans will not be position reconstructed at this
time, although photometry is extracted. If IR-bright sources
with reasonable astrometry can be identified in these scans,
POSREC will be re-initialized and rerun. Longer term solution will
be to incorporate more rigorous position search and matching
techniques developed by H. McCallon for position reconstruction of
IRTS data. Test application of these methods have successfully
reconstructed data in one high density field.
c. Systematic offset between aperture photometry and KAMPHOT psf-fit
photometry in periods of poor seeing.
i. Suspected cause: Seeing disk overfilling aperture causing
underestimation of point source brightness in aperture photometry.
(We had a wealth of nights with moderate to poor seeing!).
ii.Ops solution: Revert to KAMPHOT photometric measurement when
seeing parameter drops below a fixed value (PFRAC < 0.4).
d. J-band ghost still present.
i. Suspected cause: Optics.
ii. Ops solution: Incorporate ghost parameters in persistence cleaning.
3. Pipeline Processing
After a few minor start-up delays, operational pipeline processing of the
April/May 2MASS Protocamera data was started on June 3rd.
The status of processing for sets of scans from each night can be followed
by referring to the 2MASS Science Team Information WWW page
(http://www.ipac.caltech.edu/2mass/2mass.html). Select the Protocamera
Processing Status line; this is a restricted access page so please refer
to Mike Skrutskie's memo of May 24, 1995 for password information.
Each night has a separate entry on the status page. Clicking on a night
will bring up a table containing all scans taken on that night,
and the status of each processing phase for that night:
TAPE INPUT: Raw data successfully extracted from tape
POSREC: Position reconstruction successful
MERGE: R1, and R2-R1 sources consolidated, aperture and KAMPHOT photometry
consolidated and persistence, ghosts and other artifacts removed.
GAL: Extended source processing complete
DATA POSTED: Processed source lists and coadded images available
on line
PATH: Location of processed data
All Science Team members are encouraged to review the processed data files
and provide feedback. Data will be held on "lugosi", the primary
production computer, for approximately 1 week, and then will be
spooled to the archive server. This will be reflected in the PATH
entry in the status table.
For each night, the data are arranged in separate sub-directories for
each scan (i.e. /l5/prod/95-04-22/s016 for scan 16 from the night
of April 22), and within each scan directory are sub-directories
containing image, point and extended sources, along with ancillary
information that is retained for analysis and development purposes.
Of particular interest will be the following files (examples are
given for scan 16 on 95-04-22):
.../snnn/image/*.Z - Compressed coadded FITS image files (512x1024) rebinned
to 1"/pixel scale.
(i.e. /l5/prod/95-04-22/s016/image/ki0300009.fits.Z)
.../snnn/src/*.p.ph - Position reconstructed point source aperture photometry.
(i.e. /l5/prod/95-04-22/s016/src/k016.p.ph)
.../snnn/src/*.p.ps - Position reconstructed point source KAMPHOT photometry.
(i.e. /l5/prod/95-04-22/s016/src/k016.p.ps)
.../snnn/src/*.p.ps_s - Position reconstructed point source merged and cleaned
photometry.
(i.e. /l5/prod/95-04-22/s016/src/k016.p.ps_s)
.../snnn/extended/*.gal - Position reconstructed extended source photometry.
Includes selection of fixed apertures.
(i.e. /l5/prod/95-04-22/s016/extended/k016.gals)
Source data files have the general form bnnn... where "b" is the band (jhk),
and "nnn" is the scan. If a scan failed any phase of the processing, such
as POSREC, some of these files may be missing. A summary of "bad" scans is
provided at the end of each night's status file. Users are strongly
recommended to review that file since sometimes only a few of the
scans within a set may have had problems. Such sets are noted with
an asterisk.
4. Calibration
Currently, the photometric data in the ../src subdirectories have only
been corrected for approximate photometric zero points. These
zero-points are likely accurate to about 10%. More accurate
zero-point and extinction corrections will be evaluated for each night
and posted at a later time.
5. Processing Priority
As outlined in the Protocamera Run Processing Plan, data from the
Photometric nights will be processed first. Data from non-photometric
nights will be processed following the completion of the photometric
data. On each night, calibration scans are processed first, followed by
survey and test scans.