Lesson
The theory โ The Moving Average as a Filter
An M-point moving average replaces each value of a series by the mean of M consecutive values. It is a convolution with a box of M weights 1/M, so by the convolution theorem its effect on a cycle of f cycles per sample is a multiplication by the box's Fourier transform, the gain H(f) = sin(ฯfM)/(M sin ฯf). H is zero at f = 1/M, 2/M, โฆ, negative between the first and second zero, and it is real when each average is placed at the middle of its window; placed at the window's end, the same values carry a delay of (M โ 1)/2 samples.
What each symbol means
M- the number of days in each average
f- the frequency of a cycle, in cycles per day, from 0 to ยฝ
H(f)- the gain: what the average multiplies a cycle of frequency f by; negative means inverted
- Assumes
- One value per day with no gaps, so that the window always spans M calendar days. The cancellation needs the reported values themselves to repeat every 7 days. A reporting delay that is always the same keeps them weekly and changes nothing; a missing day, a holiday, or a delay that varies from week to week breaks it.
Problem solved in full
-
Average the average 4 steps
A 7-day average turns a 5-day wiggle upside down. A colleague suggests taking the 7-day average of the 7-day average. What happens to the 5-day wave now, to the weekly cycle, and what does the second pass cost? Press A 5-day wave first: the page gives the gain of one pass.
-
One pass, from the page:
-
A second pass multiplies the wave by the same gain again, and a negative times a negative is positive:
-
The two passes together are one average over 13 days with triangular weights, the box convolved with itself:
-
Its half-power point moves out, from Hยฒ = ยฝ for one pass to Hโด = ยฝ for two:
Answer
The 5-day wave comes out upright, at 0.0534 of its size, and the weekly cycle is still deleted, since 0 ร 0 = 0. The price is a smoother that also softens slower cycles: the period whose power is halved moves from 15.66 days to 21.74, and a centred version needs 6 days on each side instead of 3, so it stops 6 days before the data does. Hyndman and Athanasopoulos call this kind of thing a moving average of a moving average, a 3ร3-MA in their example; here it is a 7ร7-MA.
-
Learning path
The Fourier transform in practice
References (3)
- The source of the page's real series: the daily difference of Germany's cumulative confirmed cases in the JHU CSSE COVID-19 Data Repository, licensed CC BY 4.0, with the dashboard paper its terms of use ask users to cite: COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University, time_series_covid19_confirmed_global.csv (archived 10 March 2023). E. Dong, H. Du and L. Gardner, An interactive web-based dashboard to track COVID-19 in real time. The Lancet Infectious Diseases 20(5), 533โ534 (2020). doi:10.1016/S1473-3099(20)30120-1
- The moving average as a convolution with a rectangular pulse, the centred and one-sided forms, and its frequency response H[f] = sin(ฯfM)/(M sin ฯf) (equation 15-2), with the verdict the second insight quotes: "an exceptionally good smoothing filter", "an exceptionally bad low-pass filter": S. W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing. California Technical Publishing (online edition, ยฉ 1997โ2011). Chapter 15, Moving Average Filters.
- The m-MA of a time series, why symmetric moving averages have odd order, the 2รm-MA for an even seasonal period, the statement that "a 7-MA can be used to estimate the trend-cycle of daily data with a weekly seasonality" while other orders leave the estimate contaminated by the seasonality, and moving averages of moving averages (the 3ร3-MA), which the worked problem extends to 7ร7: R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and Practice, 3rd edition. OTexts, Melbourne, 2021. Section 3.3, Moving averages.