at J, H, and Ks, respectively) to
define the faint end of the lowest bin in each waveband.
ii. Reliability
The Level 1 Specifications call for 99% reliability at galactic
latitude > 20° from the Galactic plane, and 80% reliability between 10°
and 20°. The specification indicates that completeness and reliability be
calculated "per magnitude in the faintest magnitude bin." 2MASS has used the
Level 1 sensitivity specification (15.0, 14.3, and 13.5 mag for
10- at J, H, and Ks, respectively) to
define the faint end of the lowest bin in each waveband.
The XSC consists of two subsets, Extended Sources and Galaxies. Each candidate source was given a numeric flag, rating how extended it was and how galaxy-like it was, and sources passing a given threshold on either parameter were placed in the Catalog.
Because the goal of the XSC was to produce a reliable galaxy catalog, the Level 1 Specifications were interpreted as the galaxy reliability of the sources that exceeded the threshold for the galaxy flag. This is much more stringent than simply requiring that a source be extended. Double stars below the resolution of 2MASS are correctly picked up as extended sources, but they count as false sources for the galaxy reliability, since they are not galaxies.
Reliability is usually the hardest specification to measure, primarily due to the large variations in conditions encountered in an all-sky catalog. That is, one can analyze a subset of the data to determine its reliability, but this gives no assurance that different reliability problems do not occur elsewhere.
However, nearly one-third of the entire 2MASS XSC has been reviewed source by source by eye (see IV.5c1), with each source verified against its 2MASS image and an optical DSS image, when available. As far as we know, this produces the best reliability determination for any such large catalog at the time of the Release.
For bright sources, this reliability determination is definitive, and the entire set of sources in the XSC is virtually 100% reliable, with only a few false sources, due to missed artifacts. However, for fainter sources, a few percent of the sources cannot be definitively classified as galaxies, double stars, or misclassified single stars. Fortunately, a statistical analysis (below) of these sources has found that at least 85% of them are most likely true galaxies. We use that minimum estimate of 85% to calculate our best estimate of the reliability below.
The reliability from visual inspection (see below) for all sources with g_score < 1.4 (i.e., the ones that passed the "galaxy" threshold) with |b| > 25°, is as follows:
| Band | N_gal | N_not_gal | N_unk | Rel_min | Rel_max | Rel_est |
|---|---|---|---|---|---|---|
| J | 59,999 | 19 | 1919 | 96.9% | 100.0% | 99.5% |
| H | 94,284 | 212 | 2696 | 97.0% | 99.8% | 99.4% |
| K | 93,030 | 259 | 3589 | 96.0% | 99.7% | 99.2% |
As can be seen, it is quite likely that the XSC exceeds the stringent Level 1 specifications for galaxy reliability for |b| > 20°.
Figure 1 shows the reliability vs. magnitude, ignoring the unknowns.
|
| Figure 1 |
In addition, we compared all sources with g_score = 1 against the SDSS EDR, finding a reliability ~99% at the Level 1 specifications. This comparison is shown in Figure 2.
|
| Figure 2 |
(The dip in the 2MASS/SDSS match rate at 8.5 to 11 mag is apparently a feature in the SDSS. The unmatched XSC sources in this range are without question galaxies, with NED counterparts, and often with spiral structure visible in the 2MASS Atlas Images. In particular, T. Jarrett has looked at every single XSC source brighter than Ks = 12 mag, and nearly every one of them is clearly a galaxy. The reliability of this set from the visual analysis is 100.0%, to one decimal place, as shown in Figure 2.)
For galactic latitudes between 10 and 20°, we significantly exceed the Level 1 specification. From the visual inspection, the reliability is well above 90% in the appropriate magnitude bins, as shown in the first plot above.
A number of other tests have been performed, including:
All of these analyses verify the high reliability of the XSC.
An analysis of the calibration field repeats was done, but the number of galaxies in the field was too small to demonstrate anything, other than consistency with the above analyses. For additional analysis, see below.
In order to verify the results of the 2MASS galaxy processing, over 150,000
sources were examined by hand and classified as galaxies, apparently single
stars, double stars, triple stars, and artifacts. The parameters calculated
by GALWORKS, the 2MASS Atlas Images, and the DPOSS images were all
employed.
However, at high latitudes and in the half magnitude bin above the XSC
completeness specification, 4--10% of all sources defied categorization in that
they could not clearly be placed into either the galaxy or single-star classes.
These sources were left as a class of "unknown" sources.
A few unknowns have been followed up with higher resolution data, and most of
them have turned out to be true galaxies. However, the number of such
followups is extremely small, and because all of them have occurred in known
galaxy clusters, that subset of the unknown sources might not be representative
of the class as a whole. In the absence of definitive information, the
unknowns are considered as nearly always true galaxies, or that they were
nearly always falsely-detected single stars.
Finally, enough statistics have been accumulated to state with some confidence
that most of the unknowns that have J magnitudes between 14.5 and 15.0,
found above |b| = 30°, are likely true galaxies. It is possible
that a significant minority (~15%) could still be false sources.
In the following, the term single stars or just singles will
always mean "detected extended sources" classified as single stars.
Since the unknowns are either true galaxies, falsely-detected single stars, or
a mixture of the two classes, we can compare the characteristics of the
unknowns to the two possible source classes. Thus, consider the following two
hypotheses, and their implications:
This hypothesis predicts that the characteristics of the unknowns are very
similar to those of true galaxies, except that the unknowns should be fainter
on average and have somewhat lower measured source extents, accounting for the
less visible fuzz.
Unfortunately, we cannot predict from first principles the characteristics that
this implies, since we have not yet analyzed the single star classifications
to find the main failure modes that allowed these stars to be detected as
extended. The most likely suspect is a brief instance of untracked seeing
that was either too brief to be reliably detected by the automatic
see_track "seeing-tracking" processing, or that was associated with a
marginally-poor see_track score that fell just under the threshold for
rejecting the data. Untracked seeing is a well-known cause that creates
extended sources out of true point sources.
In spite of not knowing the precise cause that creates the category of single
stars, this hypothesis predicts that the characteristics of the unknowns should
closely match the characteristics of the single stars picked up as extended
sources. There must be some modification to account for the placement of each
source into the unknown category, rather than the single star category, but we
cannot predict what those modifications would be, without further understanding
of the failure mode that produces the single stars. For example,
if the single stars are due to untracked seeing, then one scenario
consistent with this hypothesis is that the unknowns result from instances of
worse seeing than the occasions when a single star category was clear. The
worse seeing would mimic a fuzzier source, preventing a confident assignment
of the source to the single star category.
With these two hypotheses in mind, it is now a simple matter to analyze all
the main characteristics of these source populations and compare them to the
predictions of each hypothesis. Each characteristic will be considered in turn.
In all analyses below, we consider the set of classified sources with
14.5 < J(mag) < 15.0 that are unconfused as indicated by their
cc_flg, found at |b| > 30°. There are 3467 galaxies, 41
singles, 25 doubles, 0 triples and 178 unknowns.
(See below.) Due to the large difference in the number
of galaxies, singles and unknowns, in some of the figures below the number of
galaxies and singles have been scaled to the number of unknowns, to make the
comparison of the distributions clearer.
Galaxies have an approximately uniform distribution on the sky, whereas stars
are highly concentrated to the Galactic plane. If the unknowns follow one
distribution, rather than the other, this test by itself is probably the most
powerful test by far, giving the answer directly.
The distribution of stars falsely identified as extended by untracked seeing
will not be precisely the same as the stellar distribution, since untracked
seeing is somewhat more likely to occur in low source density areas, where the
seeing is tracked with a lower frequency. However, seeing "flare-ups" do occur
on time scales too short to be tracked even with the high stellar densities of
the Galactic plane. In any case, the theoretical distribution is not needed,
since the observed distribution of the single stars is available.
Because the area covered by this subset of the "truth table" is not readily
available, we instead simply compare the histogram of number of sources vs.
density, in Figure 3. (Density is the
log number of sources with Ks < 14.0 mag.)
The Nature of Galaxy Truth Table "Unknowns"
 |
| Figure 3 |
The histogram of unknowns vs. density is almost an exact match of the histogram of galaxies vs. density below density = 2.9, and definitely disagrees with the histogram of singles vs. density below density = 2.9. This is the most powerful evidence that most of the unknowns are, in fact, true galaxies.
Above density = 2.9 there is an excess of unknown sources, implying that a small percentage of unknowns may, in fact, be false sources in those high density areas. There are 27 more unknowns above density = 2.9 than predicted by the hypothesis that 4.4% of all galaxies fall into the unknown category. This implies that ~27/178 = 15% of the J unknowns are singles, and that ~85% of the J unknowns are true galaxies.
In the figures below, the unknowns will be separated into sources found in areas with density < 2.9 and with density > 2.9.
Galaxies have a range of measured extent parameters, ranging from small compact galaxies to large more diffuse galaxies. The hypothesis that unknowns are galaxies whose non-nuclear component has fallen below some visibility threshold implies that the unknowns should continue to show a variation in their extent parameters, but that the group of unknowns as a whole must be fainter (see below) and therefore have smaller extent parameters.
If these predictions are not obvious, consider two specific cases: First, a small compact galaxy whose fuzz is bright enough to be seen on the 2MASS Images, and therefore is classified as a galaxy. If that galaxy is simply farther away, it becomes smaller and fainter, with a smaller measured extent. Correspondingly, it gets harder to classify it as a galaxy. Second, consider a large diffuse galaxy, also classified as a galaxy. If that galaxy is farther away, it also becomes smaller and fainter, with a smaller measured extent. However, this galaxy still has a large measured extent, compared to the first galaxy, even though it too has become too faint to classify as a galaxy. Hence, the group of galaxies as a whole has become fainter and smaller, but still retains some variation in the measured extent parameters.
The J shape score (j_sh_sc) is the measured source extent divided
by the one scatter in the shapes of point sources.
Comparing that score vs. J magnitude directly shows the range in measured
galaxy source size at a given total J magnitude, as well as the general
decrease in measured source extent, as the total J magnitude becomes fainter.
The relation for unknowns (Figure 4) is exactly that predicted for galaxies (Figure 5), and very different from that for singles (Figure 6). Only one single has a j_sh_sc > 20, whereas a considerable number of the unknowns have scores above 20. Further, the unknowns show a clear trend of declining score with fainter magnitude, not present in the singles.
| 
| 
|
| Figure 4 | Figure 5 | Figure 6 |
We conclude that we have strong confirming evidence that the bulk of the unknowns are true galaxies.
The distribution of the J and Ks measured shapes are significantly different for galaxies (Figure 7) and singles (Figure 8). Galaxies show a high correlation between their shapes measured at J and Ks, whereas the singles considered here, detected at J, tend to not have very high Ks shapes. This latter fact may be caused by the J and Ks fitted seeing curves differing significantly in areas with untracked seeing.
| 
| 
|
| Figure 7 | Figure 8 | Figure 9 |
Regardless of the source of the difference between galaxies and singles, there is no doubt that the unknowns (Figure 9) strongly resemble galaxies, and not singles.
For completeness, we present other possible comparisons that use the 2MASS data, but which are not expected on theoretical grounds to show clear differences between galaxies and stars.
The color-color diagrams for galaxies (Figure 10), singles (Figure 11) and unknowns (Figure 12) closely resemble each other, due to the color selection used to assign the g_score. Nonetheless, it is noticeable that the singles include relatively more blue sources (J-H < 0.4) than are seen in the galaxy and unknown plots.
| 
| 
|
| Figure 10 | Figure 11 | Figure 12 |
A histogram of the number as a function of magnitude (Figure 13) for the three source classes is difficult to interpret, since there are many effects at work. For example, it is expected that the unknowns would show an increased number at fainter magnitudes if they are galaxies, due to the selection effect of being classified as unknowns. It is also expected that stars falsely classified as extended would increase rapidly at fainter magnitudes, due to the larger errors in derived parameters at fainter magnitudes. Thus, we can draw no firm conclusions from this figure.
|
| Figure 13 |
Three powerful comparisons strongly support the hypothesis that the unknowns are largely (but not necessarily entirely) galaxies and strongly reject the hypothesis that the unknowns are single stars. The small differences in the distribution of parameters for unknowns vs. galaxies are exactly those predicted by the hypothesis that the unknowns are simply galaxies that have become faint enough, so that their "fuzz" is not clearly distinguishable in human analysis of images.
These comparisons leave little doubt that the unknowns are largely galaxies. Additional confirming evidence may remove any doubt once we understand the origin of the singles. For example, if the singles are due to untracked seeing in one way or another, that gives another distribution to which the unknowns and galaxies can be compared.
Of course, the ultimate truth is further followup of a number of the unknowns. The very limited work to date has found that the unknowns are indeed galaxies, but more followup is needed. Again, it is possible that ~15% of the unknowns are false detections of single stars. Once the origin of the singles is known, it may be possible to accurately determine the percentage of the unknowns that are galaxies and the percentage that are single stars.
T. Jarrett has put postage-stamp pictures of these unknowns online for those who wish to peruse them.
The XSC is first and foremost a catalog of resolved
objects, the bulk of which are galaxies in the local Universe (redshift < 0.3).
A very small percentage are Milky Way stars (point sources), masquerading as
resolved, extended objects. The contamination level rises with the confusion
noise. The degree of stellar confusion is measured by the density metric
(see IV.5c).
The point source contamination or reliability of the XSC may be gauged using
repeated observations of the Abell 3558 cluster of galaxies (located behind
a region of the Milky Way rich in stars). The results are
documented in
Galaxy Cluster Repeatibility Analysis. Results are also shown in
Figure 14.
Further Analysis of Reliability
|
| Figure 14 |
Approximately one-third of the XSC has been visually inspected (see IV.5c1), with nearly 100% completeness at the bright end: Ks < 12.5 mag. The results of this inspection then represent a measure of the reliability of the Catalog, particularly at the bright end, since it was uniformly inspected. The results are divided into four samples, each representing a "density" or confusion noise regime:
The density metric and its relation to the Galactic Plane are explained in IV.5c1. The "raw," or aggregate, reliability is summarized in Figure 15.
|
| Figure 15 |
The bottom line is that the XSC reliablity is better than 98% for unconfused regions of the sky, and better than 90% (Ks < 13.0 mag) for regions that include the Milky Way. The tabular results for the low stellar density case are summarized here:
| mag | Ngal| Nbog| Rj | Nunk| Ngal| Nbog| Rh | Nunk| Ngal| Nbog| Rk | Nunk|
| (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8)|(9) | (10) | (11)|(12) | (13)|
9.000 985 0 100.0 4 2109 0 100.0 10 2994 0 100.0 17
10.500 2016 0 100.0 5 5207 1 100.0 29 8090 2 100.0 29
11.250 2512 0 100.0 19 6927 0 100.0 17 10761 0 100.0 55
11.750 5178 1 100.0 14 13247 1 100.0 72 21540 2 100.0 244
12.250 9867 1 100.0 40 26045 1 100.0 395 44374 0 100.0 1214
12.750 19147 0 100.0 197 50682 0 100.0 1164 61565 64 99.9 1215
13.250 37481 1 100.0 1159 51421 86 99.8 1231 32297 328 99.0 2575
13.750 58739 31 99.9 854 28617 332 98.9 2461 23769 563 97.7 6425
14.250 37439 295 99.2 2008 21318 556 97.5 6142 14581 677 95.6 6211
14.750 23068 473 98.0 4226 11526 411 96.6 4978 801 46 94.6 437
15.250 18368 688 96.4 7076 504 17 96.7 397 27 2 93.1 28
15.750 5217 125 97.7 2548 15 1 93.8 18 6 0 - 1
Notes:
Statistics are for low density regions of the sky: |glat| > 25 deg
column (1) is the elliptical isophotal mag
columns (2)-(5) are the J-band results, with (2) number of extended sources, (3) number of false
sources, (4) %Reliability, and (5) the number of 'unknown' sources
columns (6-9) are the H-band results; see above
columns (10)-13) are the K-band results; see above
The Level-1 Science Requirements set specific completeness and reliability requirements. Here the requirements allow the application of the g_score, to improve the galaxy reliability (representing most of the XSC, in any case). The bottom line is that the XSC galaxy reliablity is better than 99% for unconfused regions of the sky, and better than 90% (Ks < 13.0 mag) for regions that include the Milky Way. The tabular results for the low stellar density case are summarized here:
| mag | Ngal| Nbog| Rj | Nunk| Ngal| Nbog| Rh | Nunk| Ngal| Nbog| Rk | Nunk| | (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8)|(9) | (10) | (11)|(12) | (13)| 9.000 595 0 100.0 4 1628 0 100.0 9 2457 0 100.0 15 10.500 1882 0 100.0 5 5030 1 100.0 27 7865 1 100.0 26 11.250 2434 0 100.0 18 6806 0 100.0 15 10616 0 100.0 44 11.750 5063 1 100.0 14 13092 1 100.0 70 21315 1 100.0 235 12.250 9742 1 100.0 34 25770 0 100.0 382 43958 0 100.0 1168 12.750 18944 0 100.0 190 50281 0 100.0 1117 61095 42 99.9 1137 13.250 37151 0 100.0 1134 51089 61 99.9 1180 31935 217 99.3 2452 13.750 58368 19 100.0 785 28304 238 99.2 2364 22950 348 98.5 5924 14.250 37213 220 99.4 1940 20698 344 98.4 5801 13372 402 97.1 5464 14.750 22786 338 98.5 4116 10608 248 97.7 4368 729 31 95.9 383 15.250 17650 417 97.7 6579 439 10 97.8 336 26 1 96.3 24 15.750 4485 46 99.0 2044 14 1 93.3 14 5 0 100.0 1
[Last Updated: 2003 Jan 31; by T. Jarrett and T. Chester]