Contents Blowdown Y2K Ready? FX-Net Development 3-D Visualization FX-Connect Turbulence Etcetera Publications Contact Us Design: Wilfred von Dauster

Clear-Air Turbulence: Assessing Algorithm Performance By Adrian Marroquin Under the sponsorship of the Federal Aviation Administration (FAA), several diagnostic turbulence forecasting algorithms (e.g., DTF3, DTF4, and DTF5) have been designed and tested at FSL. These algorithms were designed using the best and most recent turbulence theories (formulations based on simplification of the E–[epsilon] turbulence model; here E represents turbulent kinetic energy and [epsilon] is the dissipation rate of turbulence). The aim is to use the best physics to link the physical processes conducive to atmospheric turbulence and the response of the diverse types of aircraft to this turbulence. This approach is also geared toward keeping pace with the new effort to provide automated in-situ eddy dissipation rates (EDRs) from aircraft. In response to the need for these and other more traditional algorithms (e.g., the Richardson number and Ellrod index) to be systematically compared and verified, FSL designed a data archive for algorithm development. This archive provides a tool to easily calibrate, compare, and verify several turbulence forecasting algorithms for the same time period, the same model output, and the same turbulence observations. The archive serves as a research tool capable of offering the user the flexibility to manipulate algorithm parameters and external parameters, and to add new algorithms. It is the first stage of evaluation in algorithm development. After passing this developer's verification, the Real-Time Verification System developed in FSL's Aviation Division, tracks algorithm performance in an independent assessment. Forecaster feedback to the algorithm developers is critical for further algorithm calibration and testing. Here we present the developer's verification results from the calibration, comparison, and verification of the DTF3 and DTF5 algorithms. (Results from other algorithms will be featured in a future Forum issue.) The algorithms discussed here were modified to diagnose only clear-air turbulence above 20,000 ft (in jet/frontal systems). The algorithms are verified with pilot reports (PIREPs) using forecast output from the Mesoscale Analysis and Prediction System (MAPS) model. The PIREPs relating to convection or gravity wave breaking were not excluded, so the results are difficult to interpret because of this contamination. Algorithm Verification The optimization (calibration process, not shown) of each algorithm was done by running short verifications, using archived data for the first week of December 1997. In these short verifications the internal parameters of each algorithm (formulation-dependent) were changed to obtain the maximum probability of detection (POD) for yes- and no-turbulence using the PIREP data for a range of thresholds. After the calibration, the verification process was initiated by computing the POD of yes- (PODy) and no- (PODn) turbulence for each algorithm with several thresholds and external parameters (turbulence intensity, flight level, and aircraft type). The verification period covered 1 November 1997 to 30 June 1998, with the 40-km MAPS model output available hourly (from 0001–2300 UTC). The turbulence variable at the PIREP location is the volume average of the variable computed (using the model's basic variables in native coordinates) at each grid point of a box, 4x4x3, with the PIREP located in the innermost grid box. The box (1000-ft thickness) is vertically centered at the elevation of the aircraft. The verification included PIREPs within a time interval of 30 minutes around each forecast time. The external parameters for the verification are aircraft flight level, weight, and PIREP turbulence intensity. The aircraft categories are heavy aircraft (HA) and special aviation (SA). The HA category includes aircraft with weights equal to or greater than 120,000 lbs and flight levels equal to or higher than 20,000 ft; the SA category includes all aircraft types flying above 20,000 ft. The two main turbulence intensity categories are defined as I1 (to designate light or greater) and I4 (to designate moderate to severe or greater). Diagrams of PODy and PODn as a function of threshold are computed monthly (see the August 1996 FSL Forum ). The curves intersect at a point where PODy equals PODn. The threshold and POD at this intersection are used to compare the algorithms. The intersection point simultaneously maximizes PODy and PODn. Tables 1 and 2 show the PODs and thresholds for DTF3 and DTF5, respectively. Results Figure 1 shows the behavior of the monthly PODs (PODy = PODn) and thresholds for DTF5 using the values from Table 2. Similar results were obtained for DTF3 (not shown). The PODs for the I1 turbulence category (Figure 1a) remain about the same throughout the verification period with a POD of about 60%, while those for the I4 turbulence category show a dramatic variation with two local maxima for the months of December and March. Figure 1b shows the thresholds that have to be used to maintain the optimum POD performance shown in Figure 1a. The two local maxima for December 1997 and March 1998 seem to coincide with corresponding local minima of the convective activity over the continental United States for these two months. This was true for the case for December 1997, using the total lightning counts for each month (not shown). Further, the POD decrease toward the summer is the result of an increase in convective activity, and in the organization of upper jets and fronts toward the warm season. Figure 1. Monthly PODs (a) and thresholds (b) from the verification of DTF5 with PIREPs using 40-km MAPS output from 1 November 1997 to 30 June 1998. The vertical axis (b) indicates the range of thresholds that was used (see Table 2). The solid and short-long-dashed curves show the PODs and thresholds for the I4 (moderate-to-severe or greater intensity) turbulence category and for HA and SA aircraft types, respectively. The short-dashed and long-dashed curves show the PODs and thresholds for the I1 (light or greater intensity) turbulence category and for HA and SA aircraft types, respectively. Conclusions The results of the comparison of DTF3 and DTF5 show that the algorithms perform about the same despite the fact that they are based on better turbulence formulations than the Ri and Ellrod index, which are traditionally used to forecast turbulence. This verification shows that larger POD values are obtained for the turbulence intensity category I4 (moderate to severe) than the I1 (light or greater). This result can be attributed to the ambiguity of reporting the light to moderate turbulence intensities, or it can also mean that the models perform better on more intense events. The small differences in PODs between the heavy (HA) and special aviation (SA) aircraft types occur because very few light aircraft are found above 20,000 ft. The presence of the two local maxima (Figure 1A), corresponding to minima in convective activity, suggests that DTF3 and DTF5 work well to diagnose turbulence in upper tropospheric fronts above 20,000 ft. Furthermore, it is important to remove PIREPs reporting turbulence from convection and gravity wave breaking in order to clearly see the regime for which the algorithms perform the best. Some of the possible reasons for the same performance between algorithms include: Coarse horizontal and vertical resolutions of the model Damping of turbulence due to model's diffusion Ambiguities in the PIREPs Inadequate verification techniques Simplifications in the physics of the algorithms. These results show that it is difficult to make progress in turbulence forecasting and algorithm development while serious uncertainties are present in the turbulence observations (PIREPs) and known deficiencies exist in the models. One of the many uncertainties is that different turbulent kinetic energy is associated with different aircraft reporting the same intensity. Further work in turbulence forecasting must involve use of high-resolution models and better turbulence observations, such as the vertical accelerometer data and the in-situ EDR from commercial aircraft. A turbulence forecast product is available on the FSL Web page. This page provides more information on the eddy dissipation rate of turbulence generated with the algorithm DTF5.0, using the 40-km RUC-2 model output. A complete list of published articles relating to turbulence research in FSL is also available on the Web. (Dr. Adrian Marroquin is a researcher in the Mesoscale Applications Branch, headed by Dr. Cecilia M.I.R. Girz. He can be reached here.)