• Using imap.pro
• Description

 
• Setting up data in IDL
• Cleaning data
• Masking out bright sources
• Vignetting correction
• Drift correction
• Skyflattening
• Finishing up
• Comparison
• imap.pro
• Questions

---

Using the imap.pro

1. PIA --> SPD level

The P32 data needs to be saved by the PIA as an internal SPD file at the SPD level .   This file ends with ".ISPD".

Start an IDL session from the directory which contains the imap.pro file and the saved SPD file.  Then type the following at the IDL prompt.  You should have imap.pro saved to a file in advance.

IDL> .r imap.pro
IDL> imap,'filename'

Then follow the comments and questions at the IDL prompt.

The imap.pro displays several images at appropriate times to aid the user visually.  In addition, five of these displays, one from each of the major sections of program, are saved to imap_result.ps file, so that the user can produce hardcopies easily.

In the end, the user has the option of saving the imap.pro results to a new internal SPD file or exit without saving any changes.  The new file has ".ISPD_IMAP" ending to distinguish it from the original PIA result.

Now you can load this new file into the PIA and construct the map as usual.  Please note that to read this file into the PIA properly, you need to delete the _IMAP ending.  Refer to the PIA User Manual for questions regarding the map construction.  Also, you should disable the Flat Fielding with the PIA mapping because IMAP already applied skyflattening.

Description

SETTING UP DATA IN IDL

• Restores the PIA SPD file and extracts headers and data arrays.
 
• Prints out the following information on the screen :
• name of the object as defined in the header
• name of the PI
• TDT number
• detector used
• acquisition date
• wavelength of the filter used in the measurement
• size of the map
• step sizes in both y and z directions

 
• Assigns relative (Y,Z) coordinates to each pixel at each chopper plateau.

 
• The position of each pixel at each chopper plateau is defined by the following arrays.
• phtispd.det contains the detector information (C100 or C200)
• phtispd.cpos contains the chopper positions (ranging from -90 to +90 degrees)
• phtispd.rpid contains the raster point IDs

(Please refer to the Appendix B: Internal PIA Buffer Structure Definitions of the PIA User Manual.)
• Determines Y and Z coordinates of the detector center at each chopper plateau (i.e., the center of the central pixel if it is the C100 detector, and in the case of the C200 detector, the mid-point where the gaps between the pixels cross).  These are defined as (y0,z0) in the program.  Note that y0 and z0 are arrays with their lengths equal to the number of chopper plateaus, n.
• From the (y0, z0) arrays, the Y and Z coordinates of each pixel center, (yout, zout), are calculated from the known detector dimensions.   The pixel arrangement for the C100 detector is shown below.  The center of the detector coincides exactly with the center of the pixel #5.

The pixel arrangement for the C200 detector is as follows.

For both diagrams above, the +Y is to the right and +Z is upwards.

DISPLAY   Displays the individual pixel centers with yellow dots, and the detector centers at each chopper plateau with green plus signs.  Above the plot, the name of the object, TDT number, date of observation, and wavelength of the filter used are given.   The gray square boxes represent the individual detector pixels for the top left most chopper plateau.  These boxes are correctly centered at the pixel centers.  However, these are NOT drawn to scale, and the aspect ratio of the display is not correct; this is simply to aid the user with the visualization.

• Divides the map into a user-defined number of cells for masking out deviated points.
 
• A   First, prints out the size of the region cover by the map (specifically by the pixel centers, the yellow dots in the display) in arcsecs, then asks the user to input values for a cell size in arcsecs and for a threshold in units of r.m.s. noise for sigma clipping.  Examples are given to guide the user.


DISPLAY   After the number of cells and their locations are calculated, the result is plotted.  As before, the yellow dots represent the pixel centers, and the green plus signs show the chopper plateaus.  Now, the red asterisks represent the centers of the cells.  The cells should not be too closely spaced.
 

• The data points are grouped into different cells.
• For each given cell, calculates the mean and sigma of the phtispd.mnpw (mean in-band power per chopper plateau) values of the data points grouped into that cell and whose corresponding phtispd.flag values are zero.  In phtispd.flag array, a value of 0 means the signal is valid, 1 means the uncertainty associated with the signal is not valid, and 2 mean the signal is not valid.  Then, all the signals that deviate from their corresponding cell mean by more than the sigma times the user-defined threshold are flagged (by setting their FLAG values equal to 3).
• For each cell, the following parameters are printed out on the screen.  The user may need to make the window wider to avoid the output spilling over to the next line.  The (Y-cell, Z-cell) are index numbers used to designate each cell whose coordinates are given by (Y-center, Z-center).  For each cell, the number of valid data points, their mean and sigma, and the number of points flagged are printed.
 

Y-cell	Z-cell	Y-center	Z-center	number of valid data points	mean	sigma	number of points flagged
number	number	arcsec	arcsec	number	watts	watts	number

 
• Now asks the user if he/she would like to ignore this result and try using different cell size and/or different threshold for the sigma-clipping.  This is a yes/no question.  The user should make the decision based on the displayed cell positions and the above table - the number of valid points per cell should not be too small.  If the user answers yes, then all the FLAG = 3 will be reset back to 0 and the program goes back to the top of this section marked with A.  If the answer is no, then the program moves to the next section keeping all the results.

• Masks out bright point sources using a sigma-based threshold and a growing radius.  This is to pick out the background region to be used for further corrections.
 
• Using only the data points with FLAG = 0, the MEAN power is recalculated for each cell.  Then, the mean and sigma of these MEAN powers are calculated.  These are called MEANMEAN and MEANSIGMA respectively in the program.
• B   At the command line, the user is asked to input a threshold such that any cell whose mean signal deviates by more than this number of sigmas will be masked out (e.g. 2.5).  Next, it asks the user to input a value for the growth radius using the units of the cell size.  Any points found within this radius around the masked out cells (if any) will also be masked out.  If this instruction is not clear, the user is encouraged to try different values and study the changes in the display result.


DISPLAY   An IDL display window designed to fit four plots appears.  The two plots on the left are surface plots of the mean values of all the cells, and each of the two plots on the right shows the view of the corresponding surface plot to its left as seen from the point directly above it with overplots of contours and dots which represent the centers of each pixel at each chopper plateau.  The X in the lower right plot will show all the flagged points.  If there is no X present in the lower right plot, then the two surface plots on the left are exactly the same except for the scale along the z-direction.  This display is saved into a postscript file named imap_result.ps.
 

• All cells whose mean is above the mean of all cell means by more than the threshold times the sigma will be masked out, and all points found within the radius of these cell centers with FLAG = 0 will also be masked out by having their FLAG changed to 4.  The total number of cells and the number of masked out cells are printed on the screen, and the result completes the current IDL window display.
• Now the user is asked if he/she would like to ignore this result and try using different threshold and/or growing radius.  Again, this is a yes/no question, where no means the program moves to the next section saving all the work done so far.  If the answer is yes, then all the FLAG = 4 gets reset to 0 and the program moves back to the point B of the program to start this section over again.

• Statistically determines the chopper vignetting.
 
• First, defines a new two-dimensional array CPVIGN whose first index has size equal to the total number of pixels (9 for C100 and 4 for C200) and the second has size equal to the total number of different chopper positions.  Hence, in the case of C200, the 1st index had 4 elements and the 2nd index has 7 elements.  In the case of C100, the 1st index has 9 elements and the 2nd has 13 elements.
• Using only those with FLAG = 0, the program groups the data points according to their pixel and chopper position.  The phtispd.cstp array which contains the chopper step information (contains repeating 1-13 for C100 and 1-7 for C200) is used in a loop over pixels.  An array CPV is used to store all data points according to different pixels and chopper positions.  The CPVIGN is simply the median of CPV.  That is, CPVIGN is the median of all data points for a given pixel and a given chopper position.
• For each pixel, the normalized vignetting factors are found by dividing all of the CPVIGN elements by its middle element which corresponds to the chopper throw or position of 0 degree.
• Now these vignetting factors for each pixel are displayed along with their polynomial fits.  After each pixel display, the user is asked to select which fit to use for correction.


DISPLAY   For each pixel, its CPVIGN is fitted with polynomials of order 2, 3 and 4 which are plotted as yellow solid line, green dotted line, and red dashed line respectively.  Please refer to the following setup for the pixel arrangement on the IDL display window.
 

C100 pixel arrangement

pixel #1	pixel #2	pixel #3
pixel #4	pixel #5	pixel #6
pixel #7	pixel #8	pixel #9

 

C200 pixel arrangement

pixel #1	pixel #2
pixel #3	pixel #4

 
• Asks the user if he/she would like to apply these selected fittings to the data.  If yes, then the the correction is applied to all data points irrespective of their FLAG values by dividing each signal by the corresponding CPVIGN element (i.e., the same pixel and the chopper position).  If no, the vignetting correction is skipped.

• Corrects for the long-term drift using a moving median method.
 
• C   Uses a moving median method to calculate the average drift of each pixel.  First, it prints out the length of the time series N and the suggested width for the moving median DELTA which is equal to 2/3 the length of the time series.  DELTA is the length over which the median is taken.  Next, it asks the user to enter his/her choice of the width for median.  The value should not be too small.
• Using only the data points with FLAG = 0, it calculates the moving median for each pixel P and calls the result MEDIAN(P,N).  To understand how a moving median is computed in the program, follow the example given below.
 
An Example:  Suppose the data has 1000 points (i.e., N = 1000), and the chosen width for the moving median is 600 (i.e., DELTA = 600).  Then, the quarter of DELTA is 150, and this number of points at the start all have the same value which is equal to their median.  The value assigned to the 151th point is the median of all points between the first and the 300th point (This is the half width of DELTA).  Again, for the 152th point, it is the median of all points between the second and the 301th point.  This continues similarly till the 300th point.  After that, median is taken from 600 data points (the full width of DELTA) until the 300th point from the last point is reached.  Then, only the half of DELTA is used again, and the last 150 points all have the same value which is their median.   Hence, the pattern of grouping the data points for the median is done symmetrically around the mid-point.


DISPLAY   For each pixel, the data points are plotted along with the moving median just calculated.  The x-axis is the time in seconds, and the y-axis is the signal in watts.  The green dots represent the data points with FLAG = 0 and 1,  yellow plus signs show the masked out bright signals, red dots denote the data points with FLAG = 2, red Xs depict the flagged points using the cells, and finally the white line is the calculated moving median.  This result is also saved to the postscript file, imap_result.ps.  In the postscript file, different symbols are used to distinguish between different data points instead of colors as in the actual window display.
 

• Immediately, the user is asked if he/she wants to zoom in on the moving median curves.  The y-axis scale gets blown up considerably when they are zoomed in, so the user should not be alarmed by the greatly exaggerated structures in the moving median curves.
• The user is now asked if he/she wants to heavily smooth these curves.  The user is encouraged to smooth the curves to rid of any isolated features in the curves.  Each time the answer is yes, the moving median curves are smoothed five times using a box car average of width, DELTA/5.


DISPLAY   If the answer to any of the above two questions is yes, then the zoomed in plots show up.  All the conventions are the same as before, but now the smoothed curves will be overplotted.
 

• Unusually low signals are detected where there is a bright source.  To correct for this, the program asks the user to input a number of sigmas such that any data points found below the smoothed curves by more than this amount should be flagged.  The value of 1.0 is suggested, but a large value can be used to skip this sigma clipping.  These bad points get their FLAG set to 5.


DISPLAY   The result of this clipping under the bright source is displayed.  The green dots show all the good data points, the red Xs represent the flagged points from the cleaning with cells, and the red dots show the flagged points whose signals are below the moving median by more than the user-defined number of sigmas.  This result is saved to the imap_result.ps file.  In this postscript result, only the data points with FLAG = 0 and the moving median curves are displayed to avoid clutter.
 

• Asks the user  to enter one of the following options:
• enter 1 to accept these fits (i.e., the moving median curves) as the long-term drifts
• enter 2 to redo the fits using a different width or
• enter 0 to skip this long-term drift correction all together
If the answer is 1, then the MEDIAN is first normalized by its final part.  This means each element in the MEDIAN is divided by the median of its last DELTA number of points.  Next, each data point is divided by this resulting MEDIAN irrespective of their FLAG values for the long-term drift correction.  If the answer is 2, then the program moves back to the point Cof the program.  If the answer is 0 or anything other than 1 and 2, then the program moves to the next part without applying this correction to all data points; however, the result of any points flagged from the clipping below the bright source is saved.

• Applies skyflattening.
 
• Using only the data points found within the central region of the map, 92 arcsecs smaller in all four sides (referring to the plot of pixel centers - i.e., yellow dots in the first two displays at the beginning of the program), and have FLAG = 0, the program takes the median of all the signals for each pixel (SKYFLAT is the resulting array in the program).  Taking the mean of SKYFLAT, the skyflat factors are calculated by dividing this mean value by individual SKYFLAT elements.
• The result is printed out on the screen with the following table format.  The first one is for C100 and the next for C200.
 
Pixel Layout    Multiplying Factor from Skyflat
9    6    3     skyflat(9)    skyflat(6)    skyflat(3)
8    5    2     skyflat(8)    skyflat(5)    skyflat(2)
7    4    1     skyflat(7)    skyflat(4)    skyflat(1)
 
Pixel Layout    Multiplying Factor from Skyflat
   4    3         skyflat(4)    skyflat(3)
   1    2         skyflat(1)    skyflat(2)
 
• Asks the user if this skyflattening correction be applied to the data.  If the answer is yes, then all the data points are multiplied by the skyflat factor according to their pixel label irrespective of their FLAG values.


DISPLAY   The final result is plotted on two different display windows.  The first one shows the individual pixel data with time in seconds as the x-axis and the signal in watts as the y-axis.  The yellow dots represent the valid data points, and the red dots show all the flagged points.
DISPLAY   On the second display window, 3D surface plots are displayed (These are very much like the ones shown before when masking bright sources).  On the top, it shows the data as  before any of it changed from this program, and the final resulting data is plotted on the bottom.  These two displays are also saved to the imap_result.ps file.

• The results need to be saved in a PIA/SPD file format if it is to be fed back into the PIA for mapping.
 
• Asks the user if he/she want to save the results to a new PIA/SPD file.  If the answer is yes, then set all FLAG = 3 and 5 to FLAG = 2 and FLAG = 4 to FLAG =0, and a new PIA/SPD file is created with ".ISPD_IMAP" ending containing the modified data.  If the answer is no, then the results are not saved but lost.   Appropriate message is issued on the screen before "DONE" shows up to signal the end of the program.

Questions

Nanyao Y. Lu
lu@ipac.caltech.edu
(626) 397-9505

Min Y. Hur
hur@ipac.caltech.edu
(626) 397-7045

Mail Stop 100-22
770 S. Wilson Avenue
Pasadena, CA 91125
Fax : (626) 397-7018