|
|
|
The Refractivity of Air
IntroductionThe refractivity of air is difficult to measure accurately, so that there have been many re-measurements, and several different formulae have been used to represent the dispersion curve of air by different authors. Worse yet, the older formulae, which have long been known to be incorrect, have become fossilized in handbooks, and copied by more recent handbooks from the older ones, so that obsolete and inaccurate formulae are often cited and used. Note that the refractivity is usually given for 15° C, and refers to dry air, not air with some water vapor in it. The argument is usually vacuum wavenumber, not just the reciprocal of the wavelength in air. Sometimes the formula given refers to air free of CO2 as well as water vapor. Because the refractivity, (n−1), is generally less than 3×10−4, it's quite common to have the formula give either 106 or 108 times the actual refractivity. Bear all these facts in mind when using any refractivity formula. Read all the fine print before using any formula! Finally, there is more than one “refractive index” of interest, and the right one to use depends on the kind of measurement being made. For angular refraction, the appropriate index is the “phase index,” which is the one usually meant if no such distinction is made. However, for radar and laser geodimetry, where an electromagnetic signal is modulated, it is usually the group velocity rather than the phase velocity that is of interest; then the “group index” is required. If observations are made in or near absorption lines, things become still more complex. Fortunately, these niceties are mainly of concern for the geodesists, and will be ignored here.
Chronology
Early measurementsAlthough the first crude laboratory measurements of the refractivity of air were made around 1700, and the refractivity was estimated by Newton about this time from astronomical refraction observations, fairly accurate measurements first became available a century later, through the work ofArago and Biot. From the middle of the 19th Century onward, the refractive index of air has been measured repeatedly, as improvements in technique allowed better results to be obtained.
The Barrell & Sears formula (1939)In its day, the discussion by
was quite influential, though it is now quite obsolete. Its refractivity error approaches a part in 1000 near the atmospheric transmission cutoff just below 300 nm, and is about 1 part in 5000 or 6000 through most of the visible spectrum. (However, the discussion of the physical effects that must be taken into account is still worth reading, even though the numerical values have been superseded.) Unfortunately, one occasionally still finds papers that use it. (The tip-off that this obsolete formula is being used is that it has the form of a 2-term Cauchy dispersion formula: a constant, a term in λ−2, and a term in λ−4.) A recent example is the paper by J. Gubler and D. Tytler in PASP 110, 738 (1998). They got the B&S formula from A. T. Sinclair's NAO Technical Note (1982), which doesn't sound too obsolete. But Sinclair got it from the IAG's Bulletin Géodésique (1963), p. 390 — which of course got it from Barrell & Sears (1939). By the time Gubler & Tytler picked it up, the formula was out of date by almost 60 years! Unfortunately, this incorrect and obsolete formula was also used by Hohenkerk & Sinclair (1985), from which it was copied in the new (1992) edition of the Explanatory Supplement (see the middle of p. 142 for the obsolete formula there); so we can expect to continue to see it crop up again in the future. The old Edlén formula (1953)The oldest formula still commonly encountered is the “old Edlén” formula from 1953, which involves 3 terms (a constant and two fractions):B. Edlén This is the formula with a 41 in the denominator of the last term. (A modified form of this formula was also included in the IAG's Bulletin Géodésique (1963), p. 390. Recently, the IAG finally got clued in and dumped these obsolete formulae.) Though it's still often cited, it's known to be wrong. Don't use it! It's inexcusable that it was still given as “the” refractive index of air on p. 262 of the 4th edition of Allen's Astrophysical Quantities in the year 2000 — 36 years after it was superseded. (Worse yet, this handbook even misprints it with an egregious typo that reduces the formula to rubbish, on p. 257. They're charging $99 for this?) The new Edlén formula (1966)In the 1960s, it became plain that something was wrong with this formula. Edlén re-did the work, and published a new fit. So we got the “new Edlén” formula, of the same form, but with 38.9 instead of 41 in the last denominator:
Owens (1967)While the “new Edlén” formula still appears in some handbooks, it was already shown to be slightly incorrect the very next year:
Owens, like Barrell & Sears, separated out the contribution from carbon dioxide. He claimed an accuracy of 1 part in 109 for the refractive index, or 3 parts in 106 for the refractivity. Like Barrell & Sears, Owens considered compressibility effects (i.e., deviations from the ideal-gas law); he found them to be about 10−4 at 245 K, increasing rapidly at lower T. (From his figure, it looks like 2 × 10−4 at 200K). Owens showed that Edlén (1966) is off by 7×10−8 at −30°C, and by 4×10−7 at 45°C, 100% RH. Unfortunately, the 1966 Edlén formula is still widely quoted and used.
Peck & Reeder (1972)A few years later,
made new measurements in the infrared. They showed that Edlén was certainly wrong in the IR, and that one could obtain accuracy as good as the data (except at wavelengths below 0.23 µm) with a 2-term Sellmeier dispersion formula, which involves only 4 parameters instead of 5. This accuracy is a little better than 2 parts in 109 for the refractive index, from 0.23µm in the UV to 1.69µm in the near IR. Personally, I like to use their simple formula for calculating atmospheric refraction. It's “good enough” for most purposes, as long as you don't need to worry about water vapor, or the middle infrared.
Jones (1981)
reconsidered the real-gas effects, and performed a careful error analysis:
Unfortunately, he seems to have overlooked the Peck – Reeder work. He produced a formula good for the visible region with an error in the refractivity of a few parts in 108.
Water-vapor ProblemsMatsumoto (1982)
measured the effects of water vapor, and found considerable effects due to its near-IR absorptions.
Birch & Downs (1988)Then additional errors due to water vapor, even in the visible, were found by
K. P. Birch, M. J. Downs and confirmed by
J. Beers, T. Doiron
Ciddor (1996)Currently, the best refractivity discussion I know of is:
We can regard Ciddor's results as definitive, at least for the present. They have been adopted as the basis of a new standard by the International Association of Geodesy (IAG). Geodetic applications often require the group rather than the phase index; see
Bönsch and PotulskiSoon after Ciddor's first paper, a new set of highly accurate measurements at 4 wavelengths in the visible spectrum was published by G. Bönsch and E. PotulskiTheir work focuses primarily on determining the effects of water vapor and CO2 content, and takes account of a slight adjustment between older practical temperature scales and the ITS-90 scale, which is almost a hundredth of a degree. They start with the “new” Edlén formula of 1966. The aim is to provide the highest possible accuracy for laboratory conditions, so they consider only a small temperature range about 20°C and a CO2 mixing ratio near 400 ppm, where linear formulae are adequate; their results are not sufficiently general for field use in the actual atmosphere. They give a nice practical summary of their results in an Appendix. As they essentially confirm the results of previous recent studies — in particular, the water-vapor corrections of Birch & Downs and more recent workers — I would regard their paper as indirectly supporting Ciddor's results. The uncertainty in the refractive index of air is about 1 unit in the 8th decimal place. For further discussion of these matters, see the Web pages of NIST and the IAG. Infrared dispersion effects of water vaporThe above discussion refers to the effects of water vapor in the visible spectrum. But water vapor also has strong absorptions in the near infrared, which (by classical dispersion theory) produce considerable effects on the dispersion curve there. The following reference gives an example of the size of these effects out to 25 microns (in spite of the narrower scope suggested by its title):
Required AccuracyIt's worth bearing in mind that the refractive index of air at NTP changes by just about 10−6 per degree; that's a change of about 1 part in 300 in the refractivity. As the refraction at the horizon is on the order of 2000 arc sec, we can expect that temperature changes of 0.01° may be significant in accurate positional measurements. Such tiny temperature changes in the temperature of air are exceedingly difficult to measure, even in the laboratory, and are impossible for the observing astronomer to measure.Similarly, a change of 0.1 mm of mercury changes the density and hence the refractivity of air by about a part in 104, so that still smaller pressure changes might be significant. This is also the level at which deviations from the ideal gas law become appreciable. Stone says that, for accurate positional astrometry, errors should be kept below 0.05 arc seconds at zenith distances up to 70°, where the sea-level refraction is about 147 arc seconds. This refraction is nearly 3000 times the allowed error. For most purposes, we can neglect effects smaller than 1 or 2 seconds of arc at the horizon, corresponding to relative errors in density of 1 part in 1000, or perhaps 1 part in 2000. It appears that an error of 1 part in a few thousand is small enough for astronomers. This means an accuracy of 1 part in 3000 for the refractivity, or 1 part in 107 for the refractive index, is adequate for most astronomical purposes. That would make the Edlén formulae acceptable. But, as the more recent formulae — in particular, the Peck – Reeder formula — are more reliable, as well as less expensive to calculate, I think one of them should be used. Note that 1 part in 3000 requires measuring temperatures to 0.1°C or better accuracy (not just precision); pressures must be accurate to about 1/3 of a millibar; and the water-vapor mixing ratio should be known to a relative accuracy of about 10%. For astronomical refraction calculations in the visible spectrum, the Peck – Reeder formula seems a good compromise between accuracy and economy of calculation. I recommend it. For other purposes, such as geodesy, more accuracy may well be required. Then one should adopt Ciddor's results.
Used with permission. Copyright © 2003 – 2007 Andrew T. Young
|
