Daily Simulation of Ozone and Fine Particulates over New York State: Findings and Challenges (2024)

a. The Eta/CMAQ modeling system

The forecasting system developed by NWS/NOAA/EPA (Davidson et al. 2004; McQueen et al. 2004; Otte et al. 2005), and also utilized in this study, consists of operational weather forecasts from the NWS Eta Model (Black 1994) at a horizontal grid spacing of 12 km, the “PREMAQ” emissions and meteorology preprocessor (Otte et al. 2004, 2005), and the Community Multiscale Air Quality (CMAQ) model (Byun and Schere 2006; Binkowski and Roselle 2003). As discussed by Mathur et al. (2004), the emission inventories used by the CMAQ system are updated annually to best represent the forecast period. Because estimating mobile source emissions with day-specific temperatures using the “MOBILE6” model (U.S. EPA 2003b) is computationally expensive and inefficient for real-time applications, mobile source emissions are estimated using approximations to the MOBILE6 model as discussed by Pouliot et al. (2003). The Biogenic Emission Inventory System, version 3.12 (BEIS3.12; Pierce et al. 2002) was used to estimate the biogenic emissions. Further details on the model setup can be found in Otte et al. (2005), Pouliot et al. (2003), Pouliot (2005), Pleim and Mathur (2005), Otte et al. (2004), and Mathur et al. (2004). In contrast to the operational air quality forecasts performed by NWS/NOAA/EPA that provide forecast guidance maps for ozone only, the simulations presented in this study include both ozone and aerosol species.

Each CMAQ simulation was performed for 48 h utilizing 48-h Eta forecasts starting at 1200 UTC and was initialized with concentration fields from the previous day’s simulation. For the analysis presented in this study, only CMAQ simulations for forecast hours 17–41 (i.e., 0500–0500 UTC) are utilized because these hours correspond to a full day in the eastern time zone [0000–0000 eastern standard time (EST)] in which New York State is located. In other words, the first 17 and last 7 h of the CMAQ 48-h forecast initialized at 1200 UTC on the previous day are discarded when comparing the daily maximum 8-h ozone and 24-h-averaged PM_2.5 CMAQ simulations with observations.

CMAQ simulations were performed for July–September 2004, January–March 2005, and June–October 2005. Although the 12-km Eta domain remained identical for all simulation periods, these Eta fields were interpolated to different horizontal CMAQ domains for the first two time periods versus the third time period because of operational changes. A map showing the two CMAQ domains is provided in the left panel of Fig. 1a, and it can be seen that both CMAQ domains cover New York State, which is the focus of the analysis presented in this study. Both CMAQ domains have a horizontal grid spacing of 12 km and differ only in their map projection. The right panel of Fig. 1a shows individual CMAQ 12-km grid boxes from both simulations (solid and dashed as in the left panel) superimposed over the New York City area, that is, Manhattan, the Bronx, Queens, Brooklyn, and Staten Island. Dots indicate the location of the four PM_2.5 speciation monitors in this urban area and illustrate that the choice of modeling domain may affect point-by-point comparisons between observations and model predictions that are influenced by the relative distributions of land and water, emission sources, and so on, within each grid cell. To reduce such ambiguities, a smaller horizontal grid spacing would be necessary.

b. Observational database

Table 1 lists the time periods for which measurements and model predictions were available for analysis in this study. Observations of hourly ozone were obtained from the EPA Air Quality System (AQS) for all monitors in New York State, and daily maximum 8-h ozone mixing ratios were then determined from the hourly ozone data. For PM_2.5, there are a variety of distinct observational networks and measurement techniques. Therefore, evaluation of CMAQ PM_2.5 simulations encompasses the synthesis of multifaceted pieces of information obtained from the comparison of model predictions against specific networks and measurement types. In this study, CMAQ predictions of PM_2.5 were compared with four distinct types of observations.

First, CMAQ predictions of total PM_2.5 were compared with filter-based 24-h-averaged PM_2.5 concentrations based on the Federal Reference Method (FRM), which were obtained from the EPA AQS for all monitors located in New York State. At 30 out of 36 monitors, measurements are taken once every 3 days, while the measurement frequency is once per day at four monitors and once every 6 days at the remaining two monitors.

Second, to analyze CMAQ predictions of the various species contributing to the total PM_2.5 mass, filter-based 24-h-averaged concentrations from Speciation Trends Network (STN) monitors located in New York State were also obtained. At seven out of these eight monitors, measurements are taken once every third day; at the remaining site, measurements are taken once every sixth day.

Third, hourly observations of total PM_2.5 mass, sulfate, nitrate, elemental carbon, and organic carbon for a monitoring site in Bronx County, New York, were retrieved from AQS to elucidate diurnal patterns in these species. The instruments used to measure hourly sulfate, nitrate, and elemental carbon (EC)/organic carbon (OC) were TECO 5020, R&P 8400N, and a semicontinuous EC/OC field instrument from Sunset Laboratories, respectively.

Hourly values of total PM_2.5 concentrations for monitors in New York State were downloaded from the EPA “AIRNOW” system, which is used to inform the public about the ambient AQI (U.S. EPA 1999). For monitors located in New York State, the PM_2.5 concentrations stored in this database are based on hourly measurements by tapered element oscillating microbalance (TEOM) instruments operating at a temperature of 50°C. However, as required by the AIRNOW system, prior to upload to the database, the raw TEOM measurements are adjusted for known instrument biases such as the loss of some volatile species to be more compatible with PM_2.5 as measured on filters by the FRM (U.S. EPA 2002). In other words, 24-h-averaged PM_2.5 concentrations derived from these hourly “FRM like” AIRNOW values rather than filter-based FRM measurements were utilized to compare the observed and predicted AQI in section 4 on a daily basis because the actual filter-based FRM measurements are available only once every third day at most monitors. An in-depth comparison between TEOM and FRM PM_2.5 measurements at two sites in New York State is given by Schwab et al. (2004a). It should also be noted that the focus of the AIRNOW database is on providing access to monitoring information in near–real time. Therefore, while basic quality assurance is performed via automated checks on minimum/maximum values, rates of change, etc., these data are considered preliminary and are subject to more complete quality assurance prior to integration into the AQS database.

c. NYSDEC routine AQI forecasts

Routine daily AQI forecasts are issued by the NYSDEC for the eight regions in New York State (see Fig. 1b). These NYSDEC routine AQI forecasts are based on measured concentrations, climatology, weather forecasts, and expert judgment. In this approach, a single combined O₃–PM_2.5 AQI is predicted for each region, and this combined AQI for a given region is constructed as follows. First, both ozone and PM_2.5 concentrations are predicted separately for each of the eight regions. Next, these predicted concentrations are converted to the AQI as described in U.S. EPA (1999), and the AQI predicted for a given region is the greater of the ozone AQI and PM_2.5 AQI. These routine AQI forecasts were archived electronically starting in February 2005.

a. Daily maximum 8-h ozone and 24-h-averaged total PM_2.5

While the FB is negative at rural monitors and close to zero for urban and suburban monitors in upstate New York, there is a strong tendency for CMAQ to overpredict PM_2.5 concentrations at monitors in the New York City metropolitan area where the FB exceeds 70% in some cases. Before investigating the possible reasons for this overestimation of PM_2.5 concentrations in the New York City metropolitan region, it is also instructive to assess the ability of the CMAQ system to capture the temporal fluctuations of observed daily maximum 8-h ozone and daily averaged PM_2.5 concentrations. To this end, correlation coefficients between observations and model predictions were computed at each monitoring stations and are displayed in Figs. 3a–b. For ozone, Fig. 3a illustrates that correlations between observed and predicted concentrations for the time periods simulated in this study are greater than 0.6 at all but one station. These correlations are comparable to those reported for hindcast simulations such as the ones analyzed in Hogrefe et al. (2001). For PM_2.5, correlations are greater than 0.65 for all but two locations in upstate New York and range from 0.45 to 0.75 in the New York City metropolitan area. Even though the correlation estimates for PM_2.5 tend to be lower than those for ozone, they are significant and indicate that the modeling system does have skill in predicting the temporal evolution of PM_2.5 pollution episodes.

b. Evaluation of speciated daily averaged PM_2.5 concentrations over New York State

To investigate PM_2.5 model performance further, predictions of daily averaged concentrations of the various components of PM_2.5 mass were compared with measurements taken by the eight STN monitors located in New York State. Two of these monitors (Pinnacle State Park and Whiteface Mountain Lodge) are located in rural areas in upstate New York, two other monitors (Buffalo and Rochester) are located in urban areas in upstate New York, and the remaining four monitors (Bronx: 200th Street/Botanical Gardens, Bronx: IS52 and Queens College, and Manhattan: Canal Street) are located in the New York City metropolitan area. Figures 4a–c display the observed and predicted species concentrations at each of the eight monitors grouped by region averaged for summer 2004, winter 2005, and summer 2005, respectively. The organic mass (OM) shown for the observations was determined by performing a sampler-specific blank correction of the measured OC (Rao et al. 2003), followed by multiplication with 1.4 to account for the presence of oxygen and nitrogen compounds (Turpin and Huntzicker 1995). It should be noted that the OM/OC ratio likely varies both spatially and temporally, depending on source contributions and photochemistry. For example, Turpin and Lim (2001) have argued that a factor of 1.6 might be more appropriate for urban sites and factors as high as 2.1 might be more representative of the chemical composition of OM in nonurban areas. This uncertainty needs to be considered in the interpretation of Figs. 4a–c.

These figures show several noteworthy features. First, there is a strong seasonal component to the composition of PM_2.5 at all sites in both observations and model predictions. While sulfates and carbonaceous aerosols (elemental carbon plus organic mass) dominate in summer, nitrate concentrations increase in wintertime and become comparable to or larger than sulfate concentrations at most monitors with the exception of Whiteface Mountain, a high-elevation site dominated by long-range transport. CMAQ captured this seasonal fluctuation in the observed species that is typical of the eastern United States and has been reported previously (Schwab et al. 2004b; Malm et al. 2004; Morris et al. 2005). The second noticeable feature in the observations is the contrast between the rural and urban monitors in terms of total PM_2.5 mass and species composition. The fraction of PM_2.5 attributable to primary sources (elemental carbon and crustal material) is significantly lower at the rural than at the urban monitors in all seasons. While CMAQ captures this phenomenon, the fraction of these components tends to be strongly overestimated at grid cells corresponding to monitors in the New York City metropolitan area. For example, predicted EC and crustal concentrations are about 8 μg m⁻³ each at the Canal Street monitor for summer 2004 while the observations show that the sum of these components is less than 2 μg m⁻³. Another feature evident in these figures is the general underestimation of organic mass at both rural and urban monitors in upstate New York, but the tendency for overestimation in the New York City metropolitan area. This may point to an underestimation of secondary organic aerosol production from biogenic VOC emissions that are a likely major contributor to the observed OM concentrations in upstate New York. Last, it is noteworthy that there are distinct differences between the model performance at individual sites in the New York City metropolitan area between summer 2004 and winter 2005 on the one hand and summer 2005 on the other hand, with the largest overprediction shifting from the Canal Street monitor to the IS52 monitor. This may be partially attributable to the change in the horizontal modeling grids for the summer 2005 simulations shown in the right panel of Fig. 1a because the new grid cell containing the Canal Street monitor covered less land area and, therefore, had lower primary PM_2.5 emissions relative to those of the old grid projection.

c. Evaluation of hourly predictions of speciated PM_2.5 in the Bronx, New York City

Figures 4a–c discussed above illustrate that the general overprediction of PM_2.5 in the New York City metropolitan area (see Fig. 2) is mainly due to an overprediction of the organic (EC + OM) and crustal components, while an overprediction is less evident for the inorganic components (sulfate, nitrate, and ammonium) in most seasons at these four monitors. To investigate possible reasons for the discrepancy between observed and predicted PM_2.5 concentrations in this region, hourly predictions of total PM_2.5, sulfate, nitrate, EC, and OM were compared with hourly measurements taken at the IS52 monitor in the Bronx, New York. The monitor is located in an urban setting in the south Bronx about 0.25 miles away from a major interstate highway. Because hourly EC and OM measurements are only available since July 2005, this comparison was performed for July–September 2005 only. Figure 5 shows the average observed and predicted diurnal cycles of total PM_2.5 and the four measured components. Note that the y-axis scale for observations and CMAQ predictions is different by a factor of 3. These diurnal cycles confirm that 24-h-averaged concentrations of all PM_2.5 components were overestimated at this site (cf. Fig. 4c), and they also illustrate that a substantial portion of this overprediction is due to severe overpredictions during the early morning and evening hours. CMAQ predictions have a pronounced double-peak structure that is either not present at all (sulfate, OM) or much weaker (nitrate, EC) in the observations. In contrast to the other species investigated here, EC does not have a secondary formation mechanism, that is, ambient concentrations are determined by the combined effects of emissions, horizontal and vertical transport, and deposition. To investigate which of these processes may be responsible for the model-predicted double-peak structure, Fig. 6 presents a time–height cross section of the predicted diurnal cycle of EC as a function of height for the time period from July to September 2005, along with average diurnal cycles of the mixed-layer height and total emissions for the same time period for the grid cell containing the IS52 monitor. Note that the shape of the emission curve is determined by the application of many different diurnal profiles to a large variety of emission sources during the temporal and spatial allocation of annual total county-level emissions by PREMAQ. Figure 6 illustrates that the model-predicted peak concentrations from 0400 to 0800 and from 2000 to 2400 EST coincide with the time periods in which the mixed layer is low and emissions are still relatively high. Conversely, the large predicted drop in afternoon EC concentrations at the surface (cf. Fig. 5) coincides with the timing of the highest mixed-layer height, and the time–height cross section of EC concentrations indeed shows relative small vertical concentration gradients from the surface to a height of about 1.5 km. Therefore, this suggests that any or all of the following factors could cause the erroneous shape of the predicted diurnal profile: 1) an inaccurate temporal allocation of primary EC emissions from some or all of the relevant source categories, 2) an underestimation of nighttime vertical mixing in the New York City urban area characterized by urban heat island effects and land–sea contrasts in nearby grid cells, and/or 3) an underestimation of deposition processes. Further study is needed to isolate the causes for this behavior and to improve model performance in this complex urban area.

In summary, this section showed that daily maximum 8-h ozone and daily averaged total PM_2.5 predictions from the CMAQ modeling system have similar temporal correlations with observations in New York State as those reported for hindcast studies. Correlations tend to be higher for ozone than for PM_2.5 and for upstate than downstate monitors. Absolute 24-h-averaged total PM_2.5 concentrations are overestimated by CMAQ, especially in the New York City metropolitan area. Further analysis of speciated observations and model predictions showed that most of this overprediction stems from the organic and crustal components, and analysis of hourly speciated data measured at one site suggests that a combination of vertical mixing, temporal allocation of emissions, and deposition processes all contribute to this overprediction in the complex urban area.

a. Evaluation of categorical AQI simulations

The evaluation results presented in section 3 characterized the performance of CMAQ in predicting observed concentrations of ozone and total and speciated PM_2.5 over New York State. In this section, we focus on evaluating estimations of the AQI, a dimensionless parameter utilized to convey air quality information to the public. EPA has established piecewise linear relationships between concentrations and the AQI for different pollutants (U.S. EPA 1999). For example, daily maximum 8-h ozone concentrations of 65 ppb and daily averaged PM_2.5 concentrations of 15.5 μg m⁻³ each are equivalent to an AQI of 50, while daily maximum 8-h ozone concentrations of 85 ppb and daily averaged PM_2.5 concentrations of 45.5 μg m⁻³ each are equivalent to an AQI of 100. While the dimensionless AQI for any given pollutant can have discrete values between 0 and 500, local, state, and federal agencies in the United States utilize five broad AQI categories for conveying air quality forecasts to the public. These five AQI categories are associated with specific color codes (green, yellow, orange, red, and maroon) and specific levels of health concern (good, moderate, unhealthy for sensitive groups, unhealthy, and very unhealthy) and are separated by AQI “break points” of 50, 100, 150, and 200, respectively (U.S. EPA 1999).

Categorical simulations can be evaluated using contingency tables and metrics constructed from contingency tables as shown in Tables 5 and 6 (U.S. EPA 2003a; Kang et al. 2005). For the following analysis, CMAQ ozone and total PM_2.5 predictions for grid cells containing ozone/PM_2.5 monitors in New York State were converted to AQI categories. These predictions were then compared with the ozone and total PM_2.5 AQI derived from observations. Daily maximum 8-h ozone observations were derived from hourly ozone observations from the AQS database, while daily averaged PM_2.5 concentrations were derived from hourly PM_2.5 data from the AIRNOW database. As described in section 2, these hourly PM_2.5 data were constructed by applying calibration factors to the original hourly measurements to ensure compatibility between these continuous data and filter-based 24-h-averaged PM_2.5 concentrations using the FRM. Two separate thresholds each for ozone and PM_2.5 were applied to define exceedances/nonexceedances, corresponding to AQI values of 50 (transition from good to moderate) and 100 (transition from moderate to unhealthy for sensitive groups).

The results for the categorical analysis are shown in Tables 7 and 8 for ozone and PM_2.5, respectively, and are grouped by the eight air quality forecast regions used by New York State identified in Fig. 1b. Results are averaged over all monitors within a given air quality forecast region. The results indicate that CMAQ correctly predicted the observed ozone AQI category in the eight air quality forecast regions in New York State between 81.1% and 94.9% of the time. Depending on the threshold, the false-alarm ratio for the ozone AQI ranges from 32.9% to 82.5%, accuracy (which measures both correctly predicted exceedances and nonexceedances) ranges from 84.0% to 99.8%, the probability of detection ranges from 0% to 84.8%, and the critical success index ranges from 0% to 53.2%. Model performance is generally better for the AQI threshold of 50 than the AQI threshold of 100. For comparison, in a prior evaluation study for regression-type ozone forecasting in California, Dye et al. (2000) reported values of ∼85%–90% for accuracy, ∼70% for the probability of detection, and ∼40% for the false-alarm ratio. McHenry et al. (2004) reported a probability of detection of 49%, a false-alarm ratio of 13%, an accuracy of 80%, and a critical success index of 34% for Multiscale Air Quality Simulation Platform-Real Time (MAQSIP-RT) ozone forecasts at 67 monitors in New England during a pollution episode from 1 to 10 August 2001. Kang et al. (2005) reported probabilities of detection of 7%–37%, accuracies of 76%–90%, false-alarm ratios of 64%–76%, and a critical success index of 6%–18% for daily maximum 8-h ozone concentrations predicted by three modeling systems [MAQSIP-RT, MM5-Chem, and Hybrid Single-Particle Lagrangian Integrated Trajectory Chemistry Model (HYSPLIT-Chem)] over the northeastern United States during the summer of 2002.

For PM_2.5, Table 8 shows that CMAQ correctly predicted the observed AQI category in the eight air quality forecast regions in New York State between 52.0% and 89.7% of the time. For an AQI threshold of 50, the false-alarm ratio ranges from 25% to 55%, accuracy ranges from 60.8% to 89.7%, probability of detection ranges from 24.3% to 90.9%, and critical success index ranges from 22.5% to 53.7%. For an AQI threshold of 100, the number of observed exceedances decreases with the lowest number of observed exceedances in regions 4–7. CMAQ accuracy ranges from 91.4% to 99.7%, and the false-alarm ratio and probability of detection are 100% (or undefined because of no predicted exceedances) and 0%, respectively, with the exception of region 2, that is, the New York City metropolitan area. This indicates that CMAQ did not capture any of the limited number of days on which the PM_2.5 AQI exceeded 100 anywhere but in New York City. Here, because of the significant overprediction of total PM_2.5 discussed in section 3, the probability of detection is very high at 44.7%, but the false-alarm ratio is even higher at 96.2%, indicating that CMAQ predicts many exceedances of the PM_2.5 AQI threshold of 100, many of which never occurred in the New York City area.

b. Comparison of CMAQ simulations with other forecast methods for New York State

In this section, we compare the CMAQ AQI simulations with the routine daily AQI forecasts that are issued by the NYSDEC for the eight forecast regions in New York State. These routine NYSDEC AQI forecasts are based on measured concentrations, climatology, weather forecasts, and expert judgment. In this approach, the overall AQI predicted for a given air quality region is based on predictions of both ozone and PM_2.5 for that region. First, both ozone and PM_2.5 concentrations are predicted separately for each of the eight regions. Second, these predicted concentrations are converted to the AQI. Last, the AQI predicted for a given region is the greater of the ozone AQI and PM_2.5 AQI. Correspondingly, the CMAQ ozone–PM_2.5 AQI for a given region was determined from the maximum ozone–PM_2.5 concentration simulated at all grid cells containing ozone–PM_2.5 monitors within that region. This analysis was performed for February–March 2005 and June–October 2005, the only time periods for which both archived routine AQI forecasts and CMAQ simulations as well as observations from AIRNOW were available electronically. Only days for which both CMAQ and routine NYSDEC forecasts were available were included in the analysis.

Frequency distributions of observed, CMAQ-simulated, and NYSDEC-predicted AQI categories for all eight forecast regions were constructed and are displayed in Fig. 7. This figure illustrates that the observed combined ozone–PM₂₅ AQI is generally lower in central upstate New York relative to Long Island, the New York City metropolitan area, and western New York. In most regions, both the NYSDEC forecasts and CMAQ closely match the observed distribution of AQI categories, with the NYSDEC forecasts showing slightly better agreement than CMAQ. A notable exception to this general good agreement is forecast region 2, that is, the New York City metropolitan area, where CMAQ simulations consistently show poorer air quality than that observed.

The results presented so far indicate that CMAQ simulations suffer from a systematic overprediction of PM_2.5 concentration at a number of monitors. To assess whether CMAQ successfully captures day-to-day changes in the AQI despite this systematic overprediction, a comparison was performed between observed and predicted AQI tendencies. The AQI tendency is defined as the sign of the difference between today’s and tomorrow’s AQI. Therefore, the AQI tendency can either be positive, negative, or zero in the case of constant conditions. Note that for this analysis discrete AQI values rather than the five AQI categories were utilized. For each day, CMAQ and NYSDEC tendencies were compared with observed tendencies. If the predicted and observed tendencies matched (i.e., correctly predicted increasing, decreasing, or constant AQI), the forecast was categorized as “successful,” while it was considered “unsuccessful” when the tendencies differed. Figure 8 shows the frequency distributions of CMAQ and NYSDEC forecasts categorized this way, with the green area indicating the percentage of correctly predicted tendencies and the red area indicating the percentage of the incorrectly predicted tendencies. For comparison, Fig. 8 also shows results for the 2-day persistent tendency forecast. This no-skill forecast assumes that if today value is higher than yesterday, tomorrow will be higher than today, etc. Results indicate that CMAQ and NYSDEC forecasts have very comparable skill in predicting AQI tendencies and consistently higher skill than the 2-day persistent tendency forecast, which only can capture observed tendencies when the same tendencies persist for at least two consecutive days. In particular, it is of interest to note that the CMAQ simulations perform almost identically to the NYSDEC forecasts in the greater New York City metropolitan area despite the overprediction of absolute AQI values in this region shown before.

Table 1.

Time periods of CMAQ simulations and observations utilized in this study.

Table 2.

Evaluation statistics for CMAQ predictions of daily maximum 8-h O₃ concentrations at AQS monitors in New York State.

Table 3.

Evaluation statistics for CMAQ predictions of daily averaged PM_2.5 concentrations at AQS FRM monitors in New York State.

Table 4.

Evaluation statistics for CMAQ predictions of daily average PM_2.5 concentrations at AQS FRM monitors in New York State grouped by land use.

Table 5.

Contingency tables used for defining the quantities A–D shown in Table 6, below.

Table 6.

Categorical forecast evaluation metrics as defined in U.S. EPA (2003a) and Kang et al. (2005).

Table 7.

Categorical CMAQ forecast evaluation results for daily maximum 8-h ozone for two AQI thresholds (50 and 100). All results shown are averaged over all monitors in each of the eight forecast region in New York State.

Table 8.

Categorical CMAQ forecast evaluation results for 24-h-averaged PM_2.5 for two AQI thresholds (50 and 100). All results shown are averaged over all monitors in each of the eight forecast region in New York State.