Tidal records and ENSO

I think this is a possible direction to go, based on some very promising results that I worked on over the past day.

http://contextearth.com/2014/09/16/using-tidal-gauges-to-estimate-enso

For all I know, this may be a well known correlation, but I discovered that the tidal gauge data in Sydney Harbor aligns with the SOI behavior remarkably well, especially if a delay differencing of about 2 years is applied.

ddmodel

The tidal data itself has a significant annual and bi-annual periodicity modulated by an erratic envelope, potentially modeled by a DiffEq as shown below.

sydney

Another interesting teleconnection to consider !

Paul Pukite

«1

Comments

  • 1.

    Paul I got the same, redid you calcs, this is terrific

    Dara

    Comment Source:Paul I got the same, redid you calcs, this is terrific Dara
  • 2.

    Dara, Thanks for the sanity check.

    I am trying to understand the significance of the two-year differencing.

    A two year delay differencing can act as a very granular derivative, so it may be a measure of the speed at which the sea-level is rising in the harbor.

    Or it could be an indicator of a previous value that has reflected back.

    Timely that Blake Pollard wrote the blog post with the animated GIF of SST, which is a result of the water volume instability.

    animated

    Contrast that with a dynamic simulation of sloshing -- be patient with this one as it builds up over time. This is a solution to a forced Mathieu equation [1]

    sloshing

    [1] S. S. Kolukula and P. Chellapandi, “Finite Element Simulation of Dynamic Stability of plane free-surface of a liquid under vertical excitation.”

    Comment Source:Dara, Thanks for the sanity check. I am trying to understand the significance of the two-year differencing. A two year delay differencing can act as a very granular derivative, so it may be a measure of the speed at which the sea-level is rising in the harbor. Or it could be an indicator of a previous value that has reflected back. Timely that Blake Pollard wrote [the blog post](http://johncarlosbaez.wordpress.com/2014/09/16/exploring-climate-data-part-2/) with the animated GIF of SST, which is a result of the water volume instability. ![animated](http://math.ucr.edu/home/baez/ecological/el_nino/1976-1978_equatorial_surface_temperature_animation_pollard.gif) Contrast that with a dynamic simulation of sloshing -- be patient with this one as it builds up over time. This is a solution to a forced Mathieu equation [1] ![sloshing](http://imagizer.imageshack.us/a/img823/6842/6fd5.gif) [1] S. S. Kolukula and P. Chellapandi, “Finite Element Simulation of Dynamic Stability of plane free-surface of a liquid under vertical excitation.”
  • 3.

    A two year delay differencing can act as a very granular derivative, so it may be a measure of the speed at which the sea-level is rising in the harbor.

    Why don't you cook up a time series so I could wavelet transform it and see this 2 year periodicity.

    I have a question: Why do you think these periodicals need to have a cognitive or rational explanation e.g. sloshing. Maybe the dynamical system has 'beats' like waves canceling some parts of other waves.

    Comment Source:>A two year delay differencing can act as a very granular derivative, so it may be a measure of the speed at which the sea-level is rising in the harbor. Why don't you cook up a time series so I could wavelet transform it and see this 2 year periodicity. I have a question: Why do you think these periodicals need to have a cognitive or rational explanation e.g. sloshing. Maybe the dynamical system has 'beats' like waves canceling some parts of other waves.
  • 4.

    Dara, I always need to work against a physical explanation. I have learned from experience that it is common to be accused of numerology or "trendology" if one doesn't have a physical basis.

    "Maybe the dynamical system has ’beats’ like waves canceling some parts of other waves."
    

    Yet that is essentially what constitutes Bloch waves -- a set of waveforms that are allowed based on the constraints of the system. Within the math abstraction of sloshing, standing waves develop which interfere with each other.

    Adding delay differencing adds another layer of complexity. I am not even certain that the 2-year delay manifests as a periodicity.

    In digital signal processing a delay difference is a $1 - 1/Z$ term, which is a building block for a filter. And unless I am mistaken, this should eliminate 2-year periodicities because it acts as a notch filter. Think about it, if one samples a sin wave of 2 year period and takes the difference at any two points separated by two years, one gets a zero result !

    Comment Source:Dara, I always need to work against a physical explanation. I have learned from experience that it is common to be accused of numerology or "trendology" if one doesn't have a physical basis. "Maybe the dynamical system has ’beats’ like waves canceling some parts of other waves." Yet that is essentially what constitutes Bloch waves -- a set of waveforms that are allowed based on the constraints of the system. Within the math abstraction of sloshing, standing waves develop which interfere with each other. Adding delay differencing adds another layer of complexity. I am not even certain that the 2-year delay manifests as a periodicity. In digital signal processing a delay difference is a $1 - 1/Z$ term, which is a building block for a filter. And unless I am mistaken, this should eliminate 2-year periodicities because it acts as a notch filter. Think about it, if one samples a sin wave of 2 year period and takes the difference at any two points separated by two years, one gets a zero result !
  • 5.
    WHT,
    SOI is atmospheric pressure Tahiti-Darwin, so -SOI correlates with atmospheric pressure at Darwin, not all that far from Sydney. I wonder how atmospheric pressure at Sydney would fit in?
    Comment Source:WHT, SOI is atmospheric pressure Tahiti-Darwin, so -SOI correlates with atmospheric pressure at Darwin, not all that far from Sydney. I wonder how atmospheric pressure at Sydney would fit in?
  • 6.

    Thanks Nick, I will look that up. Pressure does effect sea-level, via the inverted barometer effect decribed here.

    "In addition, oceanographic effects such as Southern Ocean Oscillation (El Niño) can produce large scale variations in mean sea level of up to 0.5m with corresponding changes in rate and direction of tidal streams."
    

    I will post an interesting fit to the tidal variation shortly, which involves an exact two-year sinusoidal forcing function. This is starting to make some sense.

    Comment Source:Thanks Nick, I will look that up. Pressure does effect sea-level, via the *inverted barometer effect* decribed [here](http://www.bom.gov.au/oceanography/tides/met_effects.shtml). "In addition, oceanographic effects such as Southern Ocean Oscillation (El Niño) can produce large scale variations in mean sea level of up to 0.5m with corresponding changes in rate and direction of tidal streams." I will post an interesting fit to the tidal variation shortly, which involves an exact two-year sinusoidal forcing function. This is starting to make some sense.
  • 7.

    Dara, I always need to work against a physical explanation.

    Paul could I challenge you on this statement?

    Dara

    Comment Source:>Dara, I always need to work against a physical explanation. Paul could I challenge you on this statement? Dara
  • 8.

    Paul I think our ideas on PHYSICAL EXPLANATION are antiquated and primitive, we could have much more sophisticated explanations but machine-generated. I think we have reached the boudaries of human explanation for physical phenomenon, we need to think of a new machine-aided paradigm.

    Best example is what we are seeing with atmospherics and oceanics, there is no reasonable human explanation at all.

    Dara

    Comment Source:Paul I think our ideas on PHYSICAL EXPLANATION are antiquated and primitive, we could have much more sophisticated explanations but machine-generated. I think we have reached the boudaries of human explanation for physical phenomenon, we need to think of a new machine-aided paradigm. Best example is what we are seeing with atmospherics and oceanics, there is no reasonable human explanation at all. Dara
  • 9.

    In the tidal data there are strong frequency components at 1 year and 1/2 year. These are understandable from seasonal variations caused by the earth's rotation.

    So I placed a simple 1-year box-filter on the Sydney tidal data and I also filtered out the low-freq components and overall trend.

    Next I added a forcing function of exactly a 2 year period on the RHS of a Mathieu-type DiffEq and optimized over an interval that I used for the validation fit earlier:

    tdal2year

    This is a promising path because the formulation is very simple yet seems to be able to capture the erratic nature of the waveform.

    The need for the 2-year delay differencing may now make more sense. It is possible that ENSO is not sensitive to the 2-year tidal cycle so that to extract the ENSO factor from the tidal data, it is important to filter out that periodicity. As I said before the delay difference of 2 years acts as a notch filter to selectively remove that period. It may be that what remains after this filtering is an erratic gradient that tracks the ENSO behavior.

    This is like untying a knot ! As one is never sure if one is making things more knotty or less knotty.

    BTW, I have no good idea as to what physical mechanism drives the 2-year period, unless some nonlinear behavior is causing a period-doubling of the seasonal behavior -- perhaps a bifurcation?

    Comment Source:In the tidal data there are strong frequency components at 1 year and 1/2 year. These are understandable from seasonal variations caused by the earth's rotation. So I placed a simple 1-year box-filter on the Sydney tidal data and I also filtered out the low-freq components and overall trend. Next I added a forcing function of exactly a 2 year period on the RHS of a Mathieu-type DiffEq and optimized over an interval that I used for the validation fit earlier: ![tdal2year](http://imageshack.com/a/img904/6197/f7qRxC.gif) This is a promising path because the formulation is very simple yet seems to be able to capture the erratic nature of the waveform. The need for the 2-year delay differencing may now make more sense. It is possible that ENSO is *not* sensitive to the 2-year tidal cycle so that to extract the ENSO factor from the tidal data, it is important to filter out that periodicity. As I said before the delay difference of 2 years acts as a notch filter to selectively remove that period. It may be that what remains after this filtering is an erratic gradient that tracks the ENSO behavior. This is like untying a knot ! As one is never sure if one is making things more knotty or less knotty. BTW, I have no good idea as to what physical mechanism drives the 2-year period, unless some nonlinear behavior is causing a period-doubling of the seasonal behavior -- perhaps a bifurcation?
  • 10.

    Dara said:

    "Best example is what we are seeing with atmospherics and oceanics, there is no reasonable human explanation at all."
    

    I am going on the assumption that the climate science establishment has overlooked something simple. I realize that machine learning may be able to deduce new connections, but the combination of machine learning with guided insight is what can prune unproductive paths -- and it is fun and challenging to work this way.

    BTW, I have heard it said there is also no reasonable human explanation for quantum mechanics, you just take it as a given. Yet one can scientifically reason based on applying the physical explanation of wave interference, e.g. in doing electron diffraction.

    Comment Source:Dara said: "Best example is what we are seeing with atmospherics and oceanics, there is no reasonable human explanation at all." I am going on the assumption that the climate science establishment has overlooked something simple. I realize that machine learning may be able to deduce new connections, but the combination of machine learning with guided insight is what can prune unproductive paths -- and it is fun and challenging to work this way. BTW, I have heard it said there is also no reasonable human explanation for quantum mechanics, you just take it as a given. Yet one can scientifically reason based on applying the physical explanation of wave interference, e.g. in doing electron diffraction.
  • 11.

    BTW, I have heard it said there is also no reasonable human explanation for quantum mechanics,

    This is true if you were quoting Feynman, may he rest in peace. The proposed algebras that model quantum behaviour are human cognizable but the results obtained from them are not e.g. quantum tunneling or time reversal.

    So the written code that does certain algorithm is human cognizable but the results it produces are not! This is almost true for any application of Machine Learning and Machine Vision as well, that humans could not explain the output of these code/algorithms. Yet the results are useful and model some dynamical systems well.

    If we remove the code and the algorithm and try to explain how to navigate a UAV in a storm or how to balance a 1-legged robot or how to fly an airplane with a broken damaged wing, we cannot.

    Almost the same is true for solutions to a system of differential equations.

    So my comment to you was: DO USE SLOSHING like arguments and computations, but we do not have to be stuck with known formulas or patterns of behaviour to model a dynamical system.

    Dara

    Comment Source:>BTW, I have heard it said there is also no reasonable human explanation for quantum mechanics, This is true if you were quoting Feynman, may he rest in peace. The proposed algebras that model quantum behaviour are human cognizable but the results obtained from them are not e.g. quantum tunneling or time reversal. So the written code that does certain algorithm is human cognizable but the results it produces are not! This is almost true for any application of Machine Learning and Machine Vision as well, that humans could not explain the output of these code/algorithms. Yet the results are useful and model some dynamical systems well. If we remove the code and the algorithm and try to explain how to navigate a UAV in a storm or how to balance a 1-legged robot or how to fly an airplane with a broken damaged wing, we cannot. Almost the same is true for solutions to a system of differential equations. So my comment to you was: DO USE SLOSHING like arguments and computations, but we do not have to be stuck with known formulas or patterns of behaviour to model a dynamical system. Dara
  • 12.

    One thing to remember about 2nd-order DiffEq formulations is that the results are attenuated based on the frequency of the RHS forcing. So that if a natural resonant frequency of $\omega_0$ =1.5 rads/year is excited with equal magnitudes of frequencies $\omega$ = 1, $\pi$, $2\pi$, and $4\pi$ rads\year, the amount of attenuation will increase in the order, according to this fraction :

    $ \frac{1}{1+(\omega/\omega_0)^2} $

    And a Mathieu-type non-linearity in the DiffEq can further selectively amplify or attenuate frequencies.

    This helps explain how lower frequencies of relative small magnitudes can emerge, even though they aren't the strongest perturbation. And why the higher frequencies don't become erratic as well, since they are not as close to the threshold of resonance as the lower frequencies are.

    The 2-year delay differential is a convenient way of notching each of the $\pi$, $2\pi$, and $4\pi$ forcing factors from contributing to the lower frequency ENSO dynamics. They may still be there but whenever anyone plots SOI, typically a yearly smoothing filter is applied beforehand so they don't appear in the chart. Otherwise , the SOI will get obscured by this seasonal noise.

    I can stand to be corrected on this interpretation. The main objective I have is to understand the rationale for being able to isolate the ENSO features in the tidal gauge data. I am concerned that the $\pi$, $2\pi$, and $4\pi$ forcing factors may all have an impact and that by throwing this data away, and in particular, when we potentially have a non-linear dynamic occurring, may not be wise.

    Comment Source:One thing to remember about 2nd-order DiffEq formulations is that the results are attenuated based on the frequency of the RHS forcing. So that if a natural resonant frequency of $\omega_0$ =1.5 rads/year is excited with equal magnitudes of frequencies $\omega$ = 1, $\pi$, $2\pi$, and $4\pi$ rads\year, the amount of attenuation will increase in the order, according to this fraction : $ \frac{1}{1+(\omega/\omega_0)^2} $ And a Mathieu-type non-linearity in the DiffEq can further selectively amplify or attenuate frequencies. This helps explain how lower frequencies of relative small magnitudes can emerge, even though they aren't the strongest perturbation. And why the higher frequencies don't become erratic as well, since they are not as close to the threshold of resonance as the lower frequencies are. The 2-year delay differential is a convenient way of notching each of the $\pi$, $2\pi$, and $4\pi$ forcing factors from contributing to the lower frequency ENSO dynamics. They may still be there but whenever anyone plots SOI, typically a yearly smoothing filter is applied beforehand so they don't appear in the chart. Otherwise , the SOI will get obscured by this seasonal noise. I can stand to be corrected on this interpretation. The main objective I have is to understand the rationale for being able to isolate the ENSO features in the tidal gauge data. I am concerned that the $\pi$, $2\pi$, and $4\pi$ forcing factors may all have an impact and that by throwing this data away, and in particular, when we potentially have a non-linear dynamic occurring, may not be wise.
  • 13.

    I am freeing up, so I will fire off more solutions and track your work

    Comment Source:I am freeing up, so I will fire off more solutions and track your work
  • 14.

    I created another variant of the chart in #10. Besides the biennial forcing function on the RHS of the model, I added annual and biannual forcing factors to give the model fine structure. This doesn't improve the correlation coefficient significantly but it indicates where the details in the signal may be coming from.

    highlighted

    The fit is qualitatively good and it appears on the verge of matching exactly, except for areas highlighted in yellow. These appear to flip as mirror images around the zero level, perhaps indicating a phase reversal on a zero crossing. If the forcing indeed is a biennial function, the phase on an even and odd year is metastable, as neither is physically preferred.

    Post year 2000, the phase reverses completely, and the forcing waveform needs to be shifted to get the model in synch with the tidal data:

    post2000

    This is uncharted territory AFAIAC and so it will be interesting what it leads to.

    Comment Source:I created another variant of the chart in #10. Besides the biennial forcing function on the RHS of the model, I added annual and biannual forcing factors to give the model fine structure. This doesn't improve the correlation coefficient significantly but it indicates where the details in the signal may be coming from. ![highlighted](http://imageshack.com/a/img745/8371/9mG964.gif) The fit is qualitatively good and it appears on the verge of matching exactly, except for areas highlighted in yellow. These appear to flip as mirror images around the zero level, perhaps indicating a phase reversal on a zero crossing. If the forcing indeed is a biennial function, the phase on an even and odd year is metastable, as neither is physically preferred. Post year 2000, the phase reverses completely, and the forcing waveform needs to be shifted to get the model in synch with the tidal data: ![post2000](http://imageshack.com/a/img904/5303/AVqe3N.gif) This is uncharted territory AFAIAC and so it will be interesting what it leads to.
  • 15.

    So far the only feedback that I have received on the tidalGauge/ENSO correlation concerns applying a 3 month offset-delay to the tidal data data before doing the correlation. Two others elsewhere mentioned this, and Nick's suggestion was to use Sydney atmospheric pressure data instead, which I inferred meant that a time shift $\delta T$ would be unnecessary. I am still looking for the Sydney pressure data, but in the interim I decided to show how subtle this 3-month effect is.

    This was what was fed into the optimizer:

    $$ SOI(t) = k (Tide(t - \delta T) - Tide(t - \delta T - \Delta T)) $$ In the figure below, the top chart has the delay and the bottom has no offset delay, $\delta T =0$, with $\Delta T$ unchanged at 23 months. The difference in correlation coefficient is 0.65 vs 0.62. The correlation coefficient was used as the measure to minimize against via the optimizer and the 3-month offset and the 23-month delay difference is what the machine learning determined as the best correlation.

    correlation

    I don't think this offset delay is the most interesting feature in the correlation, but since three people made essentially the same observation I figured I would dive a little deeper into the optimization approach. Bottom-line is that I am not putting a finger on the scale, and this is what the machine learning discovered.

    Comment Source:So far the only feedback that I have received on the tidalGauge/ENSO correlation concerns applying a 3 month offset-delay to the tidal data data before doing the correlation. Two others elsewhere mentioned this, and Nick's suggestion was to use Sydney atmospheric pressure data instead, which I inferred meant that a time shift $\delta T$ would be unnecessary. I am still looking for the Sydney pressure data, but in the interim I decided to show how subtle this 3-month effect is. This was what was fed into the optimizer: $$ SOI(t) = k (Tide(t - \delta T) - Tide(t - \delta T - \Delta T)) $$ In the figure below, the top chart has the delay and the bottom has no offset delay, $\delta T =0$, with $\Delta T$ unchanged at 23 months. The difference in correlation coefficient is 0.65 vs 0.62. The correlation coefficient was used as the measure to minimize against via the optimizer and the 3-month offset and the 23-month delay difference is what the machine learning determined as the best correlation. ![correlation](http://imageshack.com/a/img631/488/J8RcwQ.gif) I don't think this offset delay is the most interesting feature in the correlation, but since three people made essentially the same observation I figured I would dive a little deeper into the optimization approach. Bottom-line is that I am not putting a finger on the scale, and this is what the machine learning discovered.
  • 16.

    I was searching for any references to a strict biennial == 2-year periodicity in the climate science literature and found [1] by Kim in 2003. This doesn't have many cites other than by the author to his own work, but it may hold the key.

    From the abstract:

    "The first CSEOF is stochastic in nature and represents a standing mode of SST variability associated with a basinwide change in the surface wind. The second CSEOF exhibits a strong deterministic component describing a biennial oscillation between a warm phase and a cold phase. The surface wind directional change in the far-western Pacific appears to be instrumental for the oscillation between the two phases. Because of the distinct nature of evolution, dynamical and thermodynamical responses of the two modes are different. Further, the predictability of the two modes is different. Specifically, the biennial mode is more predictable because of the strong deterministic component associated with its evolution. The distinction of the two modes, therefore, may be important for predicting ENSO. The irregular interplay of the two modes seems to explain some inter-ENSO variability, namely, variable duration of ENSO events, approximate phase-locking property, and irregular onset and termination times."
    

    [1] Kim, Kwang-Yul, James J. O'Brien, and Albert I. Barcilon. "The principal physical modes of variability over the tropical Pacific." Earth Interactions 7.3 (2003): 1-32. PDF

    Here is a selected passage where they describe the even and odd year effects

    "As they propagate eastward, the upwelling (negative) Kelvin waves erode the thermocline structure favorable for the maintenance of the positive sea surface temperature anomaly condition in the central and eastern Pacific. This terminates the warm phase of the biennial oscillation. It takes about 3 months for the Kelvin waves to reach the eastern boundary; then, the colder SST condition is observed in the eastern Pacific. In the winters of even years, the situation reverses. An easterly surface wind anomaly is observed in the far-western Pacific, and henceforth the thermocline deepens. The warmer subsurface water at the thermocline depth propagates eastward as downwelling Kelvin waves as the direction of the surface wind anomaly changes in the western Pacific by the end of the even year. Then, a warmer surface condition is observed in the following spring"
    

    This seems like a compelling story, but why isn't there more on this topic of even versus odd year ENSO effects? It is possible that since this was published in 2003, that something happened to cause the even/odd years to switch places and so scientists reading the paper later would think it was not applicable. It is difficult for me to see how an odd year is special over an even year, and so one should be open to the idea that it could reverse.

    Notice also the reference to the "about 3 months for the Kelvin waves to reach the eastern boundary", which happens to be the delay from Sydney harbor tidal readings to the corresponding SOI readings that I discovered in the correlation.


    There are many reference to biennial variations in Pacific Ocean salmon catch -- google Salmon "Odd Year" "Even Year".

    In recent times , the salmon catch has been higher in odd years, but this has flipped in the past.

    [2] Irvine, J. R., et al. "Increasing Dominance of Odd-Year Returning Pink Salmon." Transactions of the American Fisheries Society 143.4 (2014): 939-956. PDF here

    Comment Source:I was searching for any references to a strict biennial == 2-year periodicity in the climate science literature and found [1] by Kim in 2003. This doesn't have many cites other than by the author to his own work, but it may hold the key. From the abstract: "The first CSEOF is stochastic in nature and represents a standing mode of SST variability associated with a basinwide change in the surface wind. The second CSEOF exhibits a strong deterministic component describing a biennial oscillation between a warm phase and a cold phase. The surface wind directional change in the far-western Pacific appears to be instrumental for the oscillation between the two phases. Because of the distinct nature of evolution, dynamical and thermodynamical responses of the two modes are different. Further, the predictability of the two modes is different. Specifically, the biennial mode is more predictable because of the strong deterministic component associated with its evolution. The distinction of the two modes, therefore, may be important for predicting ENSO. The irregular interplay of the two modes seems to explain some inter-ENSO variability, namely, variable duration of ENSO events, approximate phase-locking property, and irregular onset and termination times." [1] Kim, Kwang-Yul, James J. O'Brien, and Albert I. Barcilon. "The principal physical modes of variability over the tropical Pacific." Earth Interactions 7.3 (2003): 1-32. [PDF](http://journals.ametsoc.org/doi/pdf/10.1175/1087-3562%282003%29007%3C0001%3ATPPMOV%3E2.0.CO%3B2) Here is a selected passage where they describe the even and odd year effects "As they propagate eastward, the upwelling (negative) Kelvin waves erode the thermocline structure favorable for the maintenance of the positive sea surface temperature anomaly condition in the central and eastern Pacific. This terminates the warm phase of the biennial oscillation. It takes about 3 months for the Kelvin waves to reach the eastern boundary; then, the colder SST condition is observed in the eastern Pacific. In the winters of even years, the situation reverses. An easterly surface wind anomaly is observed in the far-western Pacific, and henceforth the thermocline deepens. The warmer subsurface water at the thermocline depth propagates eastward as downwelling Kelvin waves as the direction of the surface wind anomaly changes in the western Pacific by the end of the even year. Then, a warmer surface condition is observed in the following spring" This seems like a compelling story, but why isn't there more on this topic of even versus odd year ENSO effects? It is possible that since this was published in 2003, that something happened to cause the even/odd years to switch places and so scientists reading the paper later would think it was not applicable. It is difficult for me to see how an odd year is special over an even year, and so one should be open to the idea that it could reverse. Notice also the reference to the *"about 3 months for the Kelvin waves to reach the eastern boundary"*, which happens to be the delay from Sydney harbor tidal readings to the corresponding SOI readings that I discovered in the correlation. --- There are many reference to biennial variations in Pacific Ocean salmon catch -- google Salmon "Odd Year" "Even Year". In recent times , the salmon catch has been higher in odd years, but this has flipped in the past. [2] Irvine, J. R., et al. "Increasing Dominance of Odd-Year Returning Pink Salmon." Transactions of the American Fisheries Society 143.4 (2014): 939-956. [PDF here](http://www.tandfonline.com/doi/full/10.1080/00028487.2014.889747#.VBsx07RtR8w)
  • 17.

    Hello Paul, not referring to your work, these papers are too verbose for me. I have no idea what this fellow is saying in such ambiguous verbiage e.g. KELVIN WAVES ERODE THE THERMOCLINE STRUCTURE... I have no idea what is ERODE nor STRUCTURE.

    This is why I read these papers and while I understand every line and most of the math, I have no idea what in the world they are saying as an engineer.

    These English sentences reek with ambiguity and double meanings. Say I was coding some model for these, I will imagine several versions of the code completely different.

    I think these paper are written so that if someone critic them, they could always say: O that is not what I meant I meant this... and there is much room for interpretation.

    I prefer what you do, there is data, math and code and a bit of English to explain the model and let the CPU chug along

    Dara

    Comment Source:Hello Paul, not referring to your work, these papers are too verbose for me. I have no idea what this fellow is saying in such ambiguous verbiage e.g. KELVIN WAVES ERODE THE THERMOCLINE STRUCTURE... I have no idea what is ERODE nor STRUCTURE. This is why I read these papers and while I understand every line and most of the math, I have no idea what in the world they are saying as an engineer. These English sentences reek with ambiguity and double meanings. Say I was coding some model for these, I will imagine several versions of the code completely different. I think these paper are written so that if someone critic them, they could always say: O that is not what I meant I meant this... and there is much room for interpretation. I prefer what you do, there is data, math and code and a bit of English to explain the model and let the CPU chug along Dara
  • 18.

    Same here, Dara. These wordy narratives are often referred to as "just so" stories. They are built on a sequence of premises "just so" they start to make plausible sense. Whether they are actually correct is another matter. I include these articles so that we can use them to cite previous work should we make definite progress.

    For now, this is my toe-hold to try to explain why the tidal data has the magic 2-year number, both in the biennial oscillations and in the delay differencing factor.

    As my own variation of a "just so" story, I could hypothesize that the surface of the ocean is sloshing with a certain behavior, while the thermocline layer below is sloshing with a similar behavior, but delayed or lagged by two years. The difference between the two could be the maximally exposed sub-surface temperature leading to SST and atmospheric pressure changes. Whether this working hypothesis is valid, who knows? but it gives us some other ideas for data to look at.

    Comment Source:Same here, Dara. These wordy narratives are often referred to as "just so" stories. They are built on a sequence of premises "just so" they start to make plausible sense. Whether they are actually correct is another matter. I include these articles so that we can use them to cite previous work should we make definite progress. For now, this is my toe-hold to try to explain why the tidal data has the magic 2-year number, both in the biennial oscillations and in the delay differencing factor. As my own variation of a "just so" story, I could hypothesize that the surface of the ocean is sloshing with a certain behavior, while the thermocline layer below is sloshing with a similar behavior, but delayed or lagged by two years. The difference between the two could be the maximally exposed sub-surface temperature leading to SST and atmospheric pressure changes. Whether this working hypothesis is valid, who knows? but it gives us some other ideas for data to look at.
  • 19.

    Dear Paul let me know if I am getting on your nerves.

    I buy the magic 2-year number form your code or some wavelet code or something similar which shows the periodicity as such, the rest of JUST SO English I do not buy.

    Also on the sloshing: The sloshing might occur between warmer less dense waters sloshing against much colder denser water! I wonder why we should only think that sloshing happened against the shorelines?

    Comment Source:Dear Paul let me know if I am getting on your nerves. I buy the magic 2-year number form your code or some wavelet code or something similar which shows the periodicity as such, the rest of JUST SO English I do not buy. Also on the sloshing: The sloshing might occur between warmer less dense waters sloshing against much colder denser water! I wonder why we should only think that sloshing happened against the shorelines?
  • 20.

    That's it exactly. We will see how far the computational math can take us and have fun speculating on what is happening as we go along.

    Comment Source:That's it exactly. We will see how far the computational math can take us and have fun speculating on what is happening as we go along.
  • 21.

    We will see how far the computational math can take us and have fun speculating on what is happening as we go along.

    Ok count me in

    Comment Source:> We will see how far the computational math can take us and have fun speculating on what is happening as we go along. Ok count me in
  • 22.

    Latest information on the Tide Gauge / ENSO connection is here:

    http://contextearth.com/2014/09/21/an-enso-predictor-based-on-a-tide-gauge-data-model/

    The model is a Mathieu DiffEq with a RHS forcing that includes annual, biannual, and biennial sinusoidal factors. The biennial is most critical, while the annual and biannual factors add detail.

    I timed posting this to coincide with the People's Climate March and am getting some feedback.

    Bottomline I think the reason the model is working so well is that the Tide Gauge data is more directly sensitive to sloshing of the ocean, which makes sense since sloshing is a height differential phenomena. Whereas ENSO indices such as SOI are sensitive but only though an affine transformation to the sloshing motion.

    This is the latest model fit:

    fit

    The correlation coefficient is "only" 0.53 (highlighted in blue) but that is one of those measures that is dependent on the structure of the data. It also depends on not encountering bad stretches such as that highlighted in yellow. These stretches essentially negate and then detract because they show almost perfect anti-correlation. The task now is to find out if there is any way that this quality of fit is merely fortuitous and coincidental. I am looking for a cookbook approach to verifying that is not the case.

    Comment Source:Latest information on the Tide Gauge / ENSO connection is here: <http://contextearth.com/2014/09/21/an-enso-predictor-based-on-a-tide-gauge-data-model/> The model is a Mathieu DiffEq with a RHS forcing that includes annual, biannual, and biennial sinusoidal factors. The biennial is most critical, while the annual and biannual factors add detail. I timed posting this to coincide with the People's Climate March and am getting some feedback. Bottomline I think the reason the model is working so well is that the Tide Gauge data is more directly sensitive to sloshing of the ocean, which makes sense since sloshing is a height differential phenomena. Whereas ENSO indices such as SOI are sensitive but only though an affine transformation to the sloshing motion. This is the latest model fit: ![fit](http://imagizer.imageshack.us/a/img903/7049/yxkLF2.gif) The correlation coefficient is "only" 0.53 (highlighted in blue) but that is one of those measures that is dependent on the structure of the data. It also depends on not encountering bad stretches such as that highlighted in yellow. These stretches essentially negate and then detract because they show almost perfect anti-correlation. The task now is to find out if there is any way that this quality of fit is merely fortuitous and coincidental. I am looking for a cookbook approach to verifying that is not the case.
  • 23.

    Early on there was a discussion of how Ludescher et al were comparing peaks and classifying success in terms of how many of the peaks matched.

    I am thinking that something similar can be done with the tidal sloshing model. In the figure below, I highlighted peaks that matched in yellow and circled in red those that appeared out of phase.

    comp

    What is interesting about the red regions is that they appear in anti-phase pairs, perhaps indicating that a metastable phase reversal lasts for a full cycle before it flips back. .

    Comment Source:Early on there was a discussion of how Ludescher et al were comparing peaks and classifying success in terms of how many of the peaks matched. I am thinking that something similar can be done with the tidal sloshing model. In the figure below, I highlighted peaks that matched in yellow and circled in red those that appeared out of phase. ![comp](http://imageshack.com/a/img913/8029/6fJqZC.gif) What is interesting about the red regions is that they appear in anti-phase pairs, perhaps indicating that a metastable phase reversal lasts for a full cycle before it flips back. .
  • 24.

    One thing to notice on the phase reversals in #24 is that they occur near the origin. Recall the Mathieu differential equation for sloshing:

    $ f''(t)+[a-2q\cos(2\omega t)]f(t)=F(t) $

    where $F(t)$ is $ A sin(\pi t ) + B sin(2\pi t) + C sin(4\pi t ) $ ignoring all the phase constants for illustrative purposes

    As $f(t)$ crosses zero, $f''(t)$ also nears an inflection point of zero if the waveform is near sinusoidal and has a zero mean value. This means that the forcing RHS, $F(t)$, has a greater influence on the direction that the waveform will take as it crosses the axis.

    The premise is that the biennial forcing factor, $ A sin(\pi t ) $, has to be metastable to begin with since the earth favors neither odd years nor even years. And so the question becomes -- is this zero-crossing the point where the biennial oscillation could change from odd/even pairing in years to even/odd (or vice versa) ? And if there is a bias in the metastability, that it could change back on the next cycle, because one pairing is slightly favored ?

    Comment Source:One thing to notice on the phase reversals in #24 is that they occur near the origin. Recall the Mathieu differential equation for sloshing: $ f''(t)+[a-2q\cos(2\omega t)]f(t)=F(t) $ where $F(t)$ is $ A sin(\pi t ) + B sin(2\pi t) + C sin(4\pi t ) $ ignoring all the phase constants for illustrative purposes As $f(t)$ crosses zero, $f''(t)$ also nears an inflection point of zero if the waveform is near sinusoidal and has a zero mean value. This means that the forcing RHS, $F(t)$, has a greater influence on the direction that the waveform will take as it crosses the axis. The premise is that the biennial forcing factor, $ A sin(\pi t ) $, has to be metastable to begin with since the earth favors neither odd years nor even years. And so the question becomes -- is this zero-crossing the point where the biennial oscillation could change from odd/even pairing in years to even/odd (or vice versa) ? And if there is a bias in the metastability, that it could change back on the next cycle, because one pairing is slightly favored ?
  • 25.

    Paul we need John's input on these equations, I know you are dang smart but John's opinions are vastly important. In particular I am interested, per John's earlier write up, to have non-smooth F(t).

    We are having a CDF problem with version 10, so I am not releasing the code above for you.

    D

    Comment Source:Paul we need John's input on these equations, I know you are dang smart but John's opinions are vastly important. In particular I am interested, per John's earlier write up, to have non-smooth F(t). We are having a CDF problem with version 10, so I am not releasing the code above for you. D
  • 26.

    Here is a post on finding a second mode in the ENSO time series, in addition to the biennial forcing mode

    http://contextearth.com/2014/09/23/two-modes-to-enso-variability/

    The paper that the analysis aligns well with is

    Kim, Kwang-Yul, James J O’Brien, and Albert I Barcilon. “The Principal Physical Modes of Variability over the Tropical Pacific.” Earth Interactions 7, no. 3 (2003): 1–32. -- PDF

    After having read this paper, I was amazed that it only has 8 citations and those were mainly self-directed. Kim et al are one of the few papers where one finds references to the biennial periodicity of ENSO.

    Comment Source:Here is a post on finding a second mode in the ENSO time series, in addition to the biennial forcing mode <http://contextearth.com/2014/09/23/two-modes-to-enso-variability/> The paper that the analysis aligns well with is Kim, Kwang-Yul, James J O’Brien, and Albert I Barcilon. “The Principal Physical Modes of Variability over the Tropical Pacific.” Earth Interactions 7, no. 3 (2003): 1–32. -- [PDF](http://journals.ametsoc.org/doi/pdf/10.1175/1087-3562%282003%29007%3C0001%3ATPPMOV%3E2.0.CO%3B2) After having read this paper, I was amazed that it only has 8 citations and those were mainly self-directed. Kim et al are one of the few papers where one finds references to the biennial periodicity of ENSO.
  • 27.
    edited September 2014

    For anybody who want to get nearer the road than any of us are in this algorithm chucking game (which I think possibly everybody is forced to play, given the complexity), there is this CSEOF Fortran code (apparently derived from that used by Kim et al.).

    I don't have the mileage to download the datasets so I'm not trying it :).

    Anyway I just ground my way through the just so, convoluted text of this paper trying to extract any strong statements I could. I wanted to compare its description of El Nino and techniques with Ian Ross's description which is among the clearest I've come across. I always start trying to substitute the greek squiggles with v. bad pseudocode due to bad maths fu.

    My notes should be here

    Comment Source:For anybody who want to get nearer the road than any of us are in this algorithm chucking game (which I think possibly everybody is forced to play, given the complexity), there is this [CSEOF Fortran](http://www.cgd.ucar.edu/cas/software/SUBR_CYCLOSTATIONARY_WD.f90.html) code (apparently derived from that used by Kim et al.). I don't have the mileage to download the datasets so I'm not trying it :). Anyway I just ground my way through the just so, convoluted text of this paper trying to extract any strong statements I could. I wanted to compare its description of El Nino and techniques with Ian Ross's description which is among the clearest I've come across. I always start trying to substitute the greek squiggles with v. bad pseudocode due to bad maths fu. My notes should be [here](https://dl.dropboxusercontent.com/u/61621163/Docs/QBOKim.md)
  • 28.
    edited September 2014

    Jim, Thanks for that note summary. I agree that the Kim paper is a tough slog, with a perhaps too facile "just so" narrative, but I have learned that this is quite typical for climate science papers.

    Yet the authors did somehow pull this biennial odd/even periodicity out of the muck, which is what intrigued me since I am seeing this effect as well.

    Part of what sets my approach apart is that I am not using any of the conventional climate science methods -- I am not comfortable at all with EOFs for instance. I base my entire premise on the fact that we have this humongous body of water in the Pacific and trying to understand what happens when it starts to slosh back and forth. There is very little viscosity in this motion from what I understand, so that it should oscillate once it gets set in motion. And the simplest first-order inviscid perturbation to the oscillation has the form of the Mathieu differential equation.

    There could be two major modes of the oscillation, separable I am guessing in orthogonal directions aligned with the geographic axis.

    In the past I had noticed that one of the authors, O'Brien a highly regarded emeritus climatologist, had written several papers (again low-cited) on solving the hydrodynamics problem with a co-author D.Muller.

    D. Müller and J. J. O’Brien, Shallow water waves on the rotating sphere, Phys. Rev. E 51, 4418 , 1995

    As I recall, they had some references to a Mathieu equation formulation, but did not pursue it. I will dig this paper out in my PDF stash. EDIT: PDF here

    Amazing how these guys can keep any of this straight in their minds.

    ---------------- EDIT ADDED BELOW ----------------------------

    This is an extract from the Muller & O'Brien paper referenced above

    muller

    Because of the spheroid nature of the earth, they punted on the problem. They may be formally correct that one can not apply a full 2D treatment to a spheroidal problem, but that does not mean we should avoid a first-order perturbation approach to at least capture the quasi-periodidicity.

    There is a gaping hole in the research on this topic. Nothing below the horribly complex.

    Comment Source:Jim, Thanks for that note summary. I agree that the Kim paper is a tough slog, with a perhaps too facile "just so" narrative, but I have learned that this is quite typical for climate science papers. Yet the authors did somehow pull this biennial odd/even periodicity out of the muck, which is what intrigued me since I am seeing this effect as well. Part of what sets my approach apart is that I am not using any of the conventional climate science methods -- I am not comfortable at all with EOFs for instance. I base my entire premise on the fact that we have this humongous body of water in the Pacific and trying to understand what happens when it starts to slosh back and forth. There is very little viscosity in this motion from what I understand, so that it should oscillate once it gets set in motion. And the simplest first-order inviscid perturbation to the oscillation has the form of the Mathieu differential equation. There could be two major modes of the oscillation, separable I am guessing in orthogonal directions aligned with the geographic axis. In the past I had noticed that one of the authors, O'Brien a highly regarded emeritus climatologist, had written several papers (again low-cited) on solving the hydrodynamics problem with a co-author D.Muller. D. Müller and J. J. O’Brien, Shallow water waves on the rotating sphere, Phys. Rev. E 51, 4418 , 1995 As I recall, they had some references to a Mathieu equation formulation, but did not pursue it. I will dig this paper out in my PDF stash. EDIT: [PDF here](http://coaps.fsu.edu/bios/95-4.pdf) Amazing how these guys can keep any of this straight in their minds. ---------------- EDIT ADDED BELOW ---------------------------- This is an extract from the Muller & O'Brien paper referenced above ![muller](http://imageshack.com/a/img909/2811/4pbHAh.gif) Because of the spheroid nature of the earth, they punted on the problem. They may be formally correct that one can not apply a full 2D treatment to a spheroidal problem, but that does not mean we should avoid a first-order perturbation approach to at least capture the quasi-periodidicity. There is a gaping hole in the research on this topic. Nothing below the horribly complex.
  • 29.

    Hello Paul

    I think the problem is the Hairy Ball Theorem:

    Hairy Ball Theorem

    And this comes from the standard formulation of any GLOBAL waves on 2-Sphere, however if you used wavelets with compact support or vanishing support (as t->infinity) then you have local waves (not global) and you might be able to find localized solutions.

    That was why I was asking you to include e^-r^2 in your wave-forms.

    Comment Source:Hello Paul I think the problem is the Hairy Ball Theorem: [Hairy Ball Theorem](http://en.wikipedia.org/wiki/Hairy_ball_theorem) And this comes from the standard formulation of any GLOBAL waves on 2-Sphere, however if you used wavelets with compact support or vanishing support (as t->infinity) then you have local waves (not global) and you might be able to find localized solutions. That was why I was asking you to include e^-r^2 in your wave-forms.
  • 30.

    If you use the 3D volumetric data for the planet or 3-sphere, then Hairy Ball Theorem does not hold, you should be able to have global waves in the volumetric data, I really really really need John's comments on these since I am at undergrad level with manifold theory.

    Dara

    Comment Source:If you use the 3D volumetric data for the planet or 3-sphere, then Hairy Ball Theorem does not hold, you should be able to have global waves in the volumetric data, I really really really need John's comments on these since I am at undergrad level with manifold theory. Dara
Sign In or Register to comment.