Options for Use of Position/Uncertainty Corrections
---------------------------------------------------
(H.L. McCallon 981015)
I. Introduction
This memo discusses options on how to bring improved positions and
position uncertainties into the database. A description of how the
improvements can be calculated using information from the database
has been separated out into another memo. For the purpose of this
discussion it should be sufficient to note that the position and
position uncertainty corrections for each scan segment (12 segments
by Dec per scan) will be saved to file. A routine will be supplied
which interpolates out of that file to return corrections for any
desired point. Great care will need be taken to assure that
corrections are applied to the scan and processing run for which they
were computed.
There are actually two issues to be addressed here. The first stems
from the fact that the position uncertainties currently in the database
do not properly represent the accuracy of the positions. Most of the
time they are conservative, but on occasion they can be much too low.
This results from the fact that there is no time-variant uncertainty
model in PosFrm. The second issue arises because it is possible to
significantly improve the pipeline positions by global minimization
of scan overlap discrepancies weighted against ACT reference star
match information.
II. Position Consistency
The following table calls out the various position parameters which
would need to be updated to maintain consistency. I've intentionally
left out parameters where sub-asec corrections would not be significant.
I've spoken to a number of team members (some more than once - thanks for
your time) while compiling this table, but I could easily have missed
something. If so, let me know. Definitions of the affected parameters
can be found in the corresponding database SIS's which are called out.
Positions to Update for Consistency
-------------------------------------------------------------------------
| Description | Subsystem | Database | Position | Uncertainty |
| | SIS | SIS | Parameters | Parameters |
|-----------------------|-----------|----------|------------|-------------|
| Point Sources | CAL01 | DBM01 | ra | err_maj |
| | | | dec | err_min |
| | | | glon | err_ang |
| | | | glat | |
| | | | x,y,z | |
| | | | x-scan | |
| | | | y-scan | |
|-----------------------|-----------|----------|------------|-------------|
| Coadd Headers | PXP05 | DBM02 | ra | |
| | | | dec | |
| | | | x,y,z | |
|-----------------------|-----------|----------|------------|-------------|
| Extended Sources | GAL01 | DBM02 | ra | |
| | | | dec | |
| | | | glon | |
| | | | glat | |
| | | | x,y,z | |
| | | | sup_ra | |
| | | | sup_dec | |
|-----------------------|-----------|----------|------------|-------------|
| Galaxy-Stamp Headers | TBS | TBS | ra | |
| | | | dec | |
|-----------------------|-----------|----------|------------|-------------|
| Asteroid Associations | MPC03 | DBM03 | dX* | mch_cnf |
| | | | dY* | |
|-----------------------|-----------|----------|------------|-------------|
| Optical Associations | CAL01 | DBM01 | dist_opt | |
| | | | phi_opt | |
-------------------------------------------------------------------------
* Note: Not currently in database
III. Task Descriptions
Seven tasks are identified below, which taken together address the
two issues discussed above and all the items from the table. The tasks
are ordered according to my estimate of decreasing benefit/cost ratio.
The task descriptions are set up such that (with the exception of
the original incorrect point-source uncertainty parameters) the
corrected values are added as new parameters. Alternatively the
original values could be stored under new parameter names and corrected
values under the original names. There are arguments for doing it each
way. This is something which needs to be decided. In any case, as long
as the original values are retained the updates can be redone or removed.
Task 1: Correct uncertainties associated with original point source positions
------
A) Move original incorrect error ellipse parameters to new columns
in the point source table:
1) err_maj => ERR_MAJo
2) err_min => ERR_MINo
3) err_ang => ERR_ANGo
Step A is done solely to allow corrections to be backed out later
B) Write corrected error ellipse parameters for original positions back
into the columns previously occupied by the incorrect values in the
point source table:
1) err_maj = major axis of corrected error ellipse for original positions
2) err_min = minor axis of corrected error ellipse for original positions
3) err_ang = angle of corrected error ellipse major axis (E of N) for
original positions
Task 2: Add improved point source positions with associated uncertainties
------
Add the following columns to the point source table:
1) RAc = corrected RA
2) DECc = corrected Dec
3) ERR_MAJc = major axis of error ellipse for corrected RA/Dec
4) ERR_MINc = minor axis of error ellipse for corrected RA/Dec
5) ERR_ANGc = angle of error ellipse major axis (E of N) for cor. RA/Dec
Task 3: Add corrected positions to coadd headers
------
Add corrected positions to coadd headers while retaining original
positions. Given a routine which provides corrections, Gene Kopan
has volunteered to make the fits header updates. Existing database
software should be able to pull off these fits headers and update
the database.
Task 4: Add improved extended source positions
------
Add the following columns to the extended source table:
1) RApC = corrected peak-pixel RA
2) DECpC = corrected peak-pixel Dec
3) RAcC = corrected super-coadd centroid RA
4) DECcC = corrected super-coadd centroid Dec
Another approach here would be to add only two columns with
corrections (dRA & dDEC) which could be applied to both the
peak-pixel and centroid positions when desired. As pointed
out by Tom Jarrett, this would not only save two columns but
the remaining two would need fewer significant figures. The
down side is that corrections would have to be made on the fly
at access time. Tracy Evans feels that might turn out to be
a bigger penalty than the extra file size.
Task 5: Add corrected positions to galaxy-stamp headers
------
Add corrected positions to galaxy-stamp coadd headers while
retaining original positions. Given a routine which provides
corrections, Tom Jarrett has volunteered to make the fits header
updates. Existing database software should be able to pull off
these fits headers and update the database.
Task 6: Update difference data from asteroid associations
------
Update asteroid position differences and match confidence. To
really do it right one would need to redo the asteroid associations.
Task 7: Update difference data from optical associations
------
Update position difference distances and angles from optical
associations. To do this right would require redoing the associations
themselves. This would be a formidable task, but would also
allow all USONA associations to be w.r.t. the same catalog.
In general these tasks involve pulling information from the database,
operating on it, then writing new information back. One approach would be
to pull the needed inputs out of the database in convenient scan length
blocks written to a table file. A stand-alone program would read that
file, perform the necessary computations and write out out a new table
file which would then be incorporated back into the database. A second
approach would be to have the database software access the needed inputs
a point at a time, call a subroutine which would return the new
parameters, then insert them back into the database as it goes.
The first approach would require very little additional coding on the
database side but the second approach might be faster. This needs to
be investigated.
IV. Options
Since the original positions/uncertainties are maintained in tasks 1
through 5 it should be possible to sidestep the consistency issue and
combine tasks into a set of options independent of that consideration.
Although many options are possible the following make the most sense
to me:
Option 1: (Task 1 only)
This is necessary for any release since without it the uncertainties
can be very misleading to the user. Not doing this task would reflect
badly on the project.
Option 2: (Tasks 1 through 3)
Most of the overhead for computing the corrected uncertainties for the
original positions is in extracting the needed information from the
database. Computing the position corrections at the same time has a
relatively small impact. Writing back to the database is also relatively
unaffected since all parameters from tasks 1 and 2 have the same keys.
Correcting the positions would collapse the long tails in the original
error distribution and provide a significant improvement to the user.
Gene Kopan has existing code which can be readily modified to generate
new fits headers for the coadds. Mih Seh Kong has existing code which
which can be used "as is" to load those headers into the database. It
also helps that the header counts are relatively small.
Option 3: (Tasks 1 through 5)
Adding the corrected extended source positions to the database cannot
be done as a tag-along to task 1 since the keys will be different
from the point sources. The additional overhead per source will be
greater but that is mitigated by the fact that the number of extended
sources is much less.
Tom Jarrett has existing code which can be readily modified to generate
new fits headers for the galaxy stamps. The remaining comments made under
option 2 regarding the coadd headers also apply to the galaxy-stamp
headers.
Option 4: (Tasks 1 through 7)
It will be necessary to step through every point source regardless of the
option selected. The asteroid and optical associations could be redone
as each point source is addressed. The cost for doing so would be somewhat
higher, both in development and cpu time, than for the first five tasks.
We might want to save this for the final catalog release.
A less expensive approach would be to accept the original associations
and simply adjust the position differences and match confidence values.
V. Recommendations
For the intermediate releases (starting with the March release), I
believe we should go through task 3 (option 2) as a minimum and
through task 5 (option 3) if time allows.
For the final catalog release one could go to option 4, but a more
attractive alternative exists. The alternate approach would be to do
a global refinement of the positions before starting the 2MAPPS v3.0
reprocessing of the data. By that time the version of Tycho with
3 million sources may even be available for use in the refinement.
The frame positions for the reprocessing would be tied to the refined
global solution and the associated position uncertainties could be
placed in the fpos files as well. This would provide the desired
position improvement and realistic time-variant position uncertainties
while assuring total consistency of the various position related
parameters throughout the database. It would also require much less
cpu time than option 4.