NOAA / Space Weather Prediction Center

Relativistic Electron Forecast Model (REFM)

Presented by the USAF and NOAA/ Space Weather Prediction Center


It is well established that high-energy (relativistic) electrons can cause deep dielectric charging of satellites, which can in turn lead to spacecraft upsets and/or complete satellite failures [Reagan et al, 1983; Baker et al, 1987]. These relativistic electrons are monitored at geo-synchronous orbit by GOES satellite instruments; plots of the data are available on SWPC’s web page. To warn customers of possible damaging effects, SWPC issues alerts when the >2 MeV electron flux at GOES exceeds 103 particle flux units (pfu) for more than three consecutive five-minute periods. An even more significant threshold is when the cumulative fluence exceeds 109 pfu over a 72-hour period.

As the data plots show, the flux of relativistic electrons at geo-synchronous orbit undergoes significant diurnal variation. The cumulative daily fluence can also show a day to day variation of many orders of magnitude. Yet even with this wide variation, scientists have found a statistical link between the daily electron fluence and average solar wind speed. Baker et al [1990] showed that the daily >2 MeV electron fluence measured at geo-synchronous orbit could be predicted one day in advance using a linear filter technique with solar wind speed as the input. SWPC has made further refinements to this technique, improving the forecast skill and extending the lead-time, resulting in the Relativistic Electron Forecast Model (REFM).

REFM forecasts are intended to supplement current SWPC alerts by providing guidance as to the future likelihood of damaging relativistic electron events. Since the REFM output is a daily fluence forecast, not flux, it applies most directly to the high fluence threshold. In fact the graphical model output displays a warning message during periods when the 72-hour fluence exceeds or is forecast to exceed 109 pfu. High 24-hour fluence values can also be used as a proxy for high flux levels. A study of data measured between 1995 and 2000 showed that when the 24-hour fluence exceeds 3 × 107 pfu the SWPC flux alert criteria is nearly always met (false alarm ratio = 0.07, miss frequency = 0.06). With this threshold, REFM fluence forecasts can be used to predict SWPC flux alerts.

Output from the REFM, which runs once at the start of each UT day, is made available as both graphical plots and text files on the main web page. The REFM web page presents output from two different operational modes. The first mode, ACE, uses real-time solar wind speeds measured by NASA's Advanced Composition Explorer satellite to create forecasts with 1 to 3 days lead-time. The second mode, WS, takes advantage of predicted solar wind values calculated using the Wang-Sheeley technique to extend the predicted electron fluence up to eight days in advance.

Return to Top

Using the REFM Web Page

Opening the main REFM web page loads the most recent model run. A plot of the output dominates the page. In addition to the plot, the page contains links to text files of model output and recent performance statistics for both the ACE and Wang-Sheeley (WS) modes. There is also a link to a verification plot.

Standard Plot

The standard REFM plot displays roughly 30 days of observed and forecast data. Output from both the ACE and WS modes are shown on the same plot; previous forecast values are kept on screen for comparison with observed data. An example from a period of high electron fluence in 1996 is given below, along with a description of the plot's features.

Example of REFM Output Plot

Plot Features:
  1. Plot symbols correspond to the 24-hour >2 MeV electron fluence at geo-synchronous orbit, either observed or forecast. For example, in the plot shown above, the most recent observed value corresponds to data from 24/00 to 25/00 UTC; the +1 Day Forecast is valid 25/00 to 26/00 UTC.
  2. A legend in the lower left corner indicates the symbol and color-coding used for the Observed, Forecast (ACE), and Forecast (WS) values.
  3. Two vertical dashed lines indicate the most recently observed 24-hour period; in the example above this corresponds to 24/00 to 25/00 UTC.
  4. The lower right corner displays the latest observed and 1-4 day forecast values in a tabular format. The values are also color-coded in the same manner as the plot symbols. The date shown is valid at the beginning of the 24-hour period.
  5. Missing values, observed or forecast, are indicated on the bottom of the plot with a color-coded dashed line; the word "Missing ---" also appears in the legend.
  6. When the 72-hour fluence exceeds 109 pfu, a warning message is displayed. Red lines (solid for observed and dashed for forecast) appear at the top of the plot, corresponding to the applicable days. A warning message also appears in the legend.

Verification Plot

The main difference between the standard and verification plots is that the Verification Plot displays all of the previous forecast values in order to compare them to the observed data. While useful, this makes the plot more difficult to read than the standard plot.

Return to Top

Model Specifics

The Relativistic Electron Forecast Model (REFM) predicts the >2 MeV 24-hour electron fluence at geo-synchronous orbit. It is based on a linear prediction filter (Baker, 1990) that uses average solar wind speed as its input. An offset is employed to help account for additional physical processes that can dramatically affect the electron fluence (the fluence can drop 2-3 decades in 24 hours without a corresponding change in solar wind speed).

The linear prediction filter is a relatively simple statistical technique in which correlation coefficients are created from historical data by solving a set of linear equations (Levinson, 1947). These coefficients are then applied to real-time data. The REFM uses 30 days of solar wind speed data (currently from the ACE spacecraft) to predict the 24-hour fluence with +1 to +8 days of lead-time. In the ACE mode the +1, +2, and +3 day forecasts each use a different set of coefficients. In the Wang-Sheeley mode, forecast solar wind speed values are substituted for observed data to extend the electron fluence forecast out to +8 days.

As mentioned earlier, the electron fluence can exhibit dramatic day-to-day variability, typically associated with strong shocks and/or geomagnetic storming. This is believed to be due to both magnetic field stretching, in which the electron population is temporarily shifted into or out of geo-synchronous orbit, and sudden electron loss to either the magnetopause or upper atmosphere. The basic linear filter is unable to account for these processes; given high solar wind speeds it will forecast high electron fluences, even if the electrons actually experience a sudden decrease or increase.

To help account for this short-term variability as well as longer-term drifts, REFM adjusts the output of the basic linear prediction filter with a flux offset. REFM compares previous forecasts to the observed values, computes an offset, and applies it as a correction to the current forecast. The number of days used to compute the offset varies; for example, the +1 Day ACE mode uses only the most recent forecast/observed pair, while the +2 and +3 Day ACE mode and WS mode all use the past 20 days of forecast/observed pairs. These numbers were chosen to maximize the performance statistics of each mode.

Although the offset improves the statistics, at best the flux offset's response lags a day behind the environment's variability. This causes an associated rise in model errors during solar maximum when coronal mass ejections (CMEs) and subsequent strong geomagnetic storms are prevalent. For this reason, the REFM performs best during solar minimum. In addition to the problems associated with electron variability, the WS mode is limited by the capability of the Wang-Sheeley technique. The WS forecasts of solar wind speed assume a "quiet sun", i.e. no CMEs, further increasing the problems for the WS mode during solar max. Fortunately the largest electron events occur during solar minimum when both the ACE and WS modes perform at their best.

Both the electron sensor on GOES and the SWEPAM sensor on ACE are subject to contamination during proton events. To avoid this contamination, the REFM will not use these data when the >30 MeV protons exceed 50 pfu (a fairly conservative threshold). Without solar wind speed data for the current day, the REFM (ACE mode) will not produce a forecast. An outage in ACE coverage or a satellite maneuver can also cause a data gap resulting in no forecast. Because the Wang-Sheeley mode uses forecast wind speeds, it is not as reliant on ACE data, but the WS model requires observations from ground based observatories, and can therefore have gaps due to clouds or maintenance.

Return to Top

Model Verification

Verification was done using standard measures of skill. The Skill Score (SS) is a relative measure in which performance of the target forecast is compared to a reference forecast. The SS is defined as

Skill Score Equation

The most commonly used reference forecast is simply the average observation (i.e. sample mean); when this is used, the SS is referred to as the Prediction Efficiency (PE). Other common reference forecasts include Persistence (using the most recent observation as a forecast) and Recurrence (using the observation from 27 days prior). Perfect predictions result in an SS = 1.0, positive and negative values respectively indicate performance better or worse than the reference forecast.

The Skill Score given above is appropriate for gauging model performance as a whole. To measure the model's ability to forecast rare events, such as 72-hour fluence exceeding 109 pfu, we used the True Skill Statistic (TSS) which is an absolute measure based on contingency tables (Doswell, 1990). The TSS ranges from –1.0 (no correct forecasts) to 1.0 (perfectly correct), with positive values indicating a forecast skill exceeding that expected from pure chance.

As explained in Model Specifics, the REFM performs best during solar minimum conditions, when large geomagnetic storms are less frequent. Therefore, in order to test the model under optimal conditions, scores were tabulated using data from the fall of Cycle 22 into the rise of Cycle 23, 1996 through 1999. Model coefficients for this test were generated using data from 1995, guaranteeing independent results. Due to the large day-to-day variations in electron fluence, the Skill Scores were calculated using the logarithm (base 10) of the observed and forecast values.

The following table lists the Skill Scores for each forecast period, +1 to +8 days, and each forecast mode, ACE or WS. Three standard reference forecasts were used: sample mean (Prediction Efficiency), persistence, and recurrence.

REFM Skill Scores
1/1/1996 through 12/31/1999
Skill Score versus
Forecast Mean (PE) Persistence Recurrence
+1 Day (ACE) 0.71 0.13 0.79
+2 Day (ACE) 0.39 0.15 0.56
+3 Day (ACE) 0.24 0.26 0.45
+2 Day (WS) 0.32 0.00 0.48
+3 Day (WS) 0.19 0.18 0.39
+4 Day (WS) 0.09 0.26 0.31
+5 Day (WS) 0.06 0.33 0.29
+6 Day (WS) 0.03 0.35 0.26
+7 Day (WS) -0.01 0.36 0.23
+8 Day (WS) -0.02 0.37 0.22

The results show that the ACE mode has good forecast skill, especially on day one, but that in the short-term the results are only slightly better than using persistence. The WS mode shows lower skill (PE) that continues to drop as the lead-time increases. This is not too surprising since the WS mode relies on other model output to create its forecast. In fact, for the +2 and +3 day forecasts, the ACE mode out-performs the WS mode in all three categories. In both modes the Prediction Efficiencies drop with increasing lead-time, while the skill relative to Persistence increases.

The next table shows the True Skill Statistics for the model's ability to predict 72-hour fluences that exceed 109 pfu. For comparison, scores are also shown when persistence is used to forecast these events.

True Skill Statistic for Predicting 72-hour Fluence Exceeding 109 pfu
1/1/1996 through 12/31/1999
Forecast True Skill Statistic
Lead ACE mode WS mode Persistence
+1 Day 0.64 - 0.91
+2 Day 0.36 0.24 0.65
+3 Day 0.22 0.20 0.23
+4 Day - 0.20 0.11
+5 Day - 0.20 0.02
+6 Day - 0.12 0.02
+7 Day - 0.11 0.02
+8 Day - 0.11 0.00

These results show that in the short-term (1-2 days) Persistence has higher skill scores than either REFM mode. This is due to the fact that when the 72-hour fluence exceeds 109 pfu, it tends to stay elevated for a few days. Of course as the lead-time increases, the model's performance quickly overtakes persistence.

Return to Top


Baker, D.N., et al, "Deep Dielectric Charging Effects Due to High Energy Electrons in the Earth’s Outer Magnetosphere", J. Electrost., 20, 3, 1987.

Baker, D.N., et al, "Linear Prediction Filter Analysis of Relativistic Electron Properties at 6.6 Re", J. Geophysical Res., 95, A9, 1990.

Doswell, et al, "On Summary Measures of Skill in Rare Event Forecasting Based on Contingency Tables", Weather and Forecasting, 5, December 1990.

Reagan, J.B., et al, "Space Charging Currents and their Effects on Spacecraft Systems", IEEE Trans. Electr. Insul., EI-18, 345, 1983.

Return to To