International global precipitation measurement mission data goes public

Dear John

A dream come true! This is perhaps one of the most important events of sciences' history, we will be able to see dynamical systems of earth from space, in volumetric data:

International Global Precipitation Measurement Mission Data Goes Public

Dara

Comments

  • 1.
    Comment Source:[LEVEL 1](http://pmm.nasa.gov/data-access/downloads/gpm)
  • 2.

    I am not sure that people are aware of this, but NASA has a competition going on to make use of their huge stores of cloud-served data in terms of climate science topics.

    Prizes are like $25,000 for winning the challenge

    More info here: https://www.innocentive.com/ar/challenge/9933585

    and here http://phys.org/news/2014-08-nasa-earth-ideas-climate-apps.html

    closing date is October I believe

    Comment Source:I am not sure that people are aware of this, but NASA has a competition going on to make use of their huge stores of cloud-served data in terms of climate science topics. Prizes are like $25,000 for winning the challenge More info here: <https://www.innocentive.com/ar/challenge/9933585> and here <http://phys.org/news/2014-08-nasa-earth-ideas-climate-apps.html> closing date is October I believe
  • 3.
    nad
    edited September 2014

    I had already written about that here. But of course it usually makes sense to repeat that kind of announcements. By the way do you know which types of data is the openNex data collection encompassing? I have problems to give a good meaning to that data poll cryptic acronyms like what exactly is Prisn monthly (4 km and 800m)? I couldn't even find information on that on the Wikipedia page: .

    Comment Source:I had already written about that <a href="http://azimuth.mathforge.org/discussion/1402/earth-science-data-challenge/?Focus=11674#Comment_11674">here.</a> But of course it usually makes sense to repeat that kind of announcements. By the way do you know which types of data is the openNex data collection encompassing? I have problems to give a good meaning to that data poll <a href="https://nex.nasa.gov/nex/static/htdocs/site/extra/opennex/form_input.html">cryptic acronyms</a> like what exactly is Prisn monthly (4 km and 800m)? I couldn't even find information on that on the Wikipedia page: <a href="http://en.wikipedia.org/wiki/Prism_(disambiguation)">.
  • 4.

    I missed that nad. I came across the announcement on a twitter feed.

    Your argument about it not being a good use of time given the relatively meager prize money is a good one.

    I might just submit something that I worked on from before.

    Comment Source:I missed that nad. I came across the announcement on a twitter feed. Your argument about it not being a good use of time given the relatively meager prize money is a good one. I might just submit something that I worked on from before.
  • 5.

    Here is the first image of the entire planet's precipitation we put together for about a day ago:

    GPM sample

    Several satellites' orbitals were glued to make the image.

    The resolution is huge! so we need to reduce the resolution by large factors to have reasonable size data.

    D

    Comment Source:Here is the first image of the entire planet's precipitation we put together for about a day ago: [GPM sample](http://files.lossofgenerality.com/20140905-S120000-E240000.jpg) Several satellites' orbitals were glued to make the image. The resolution is huge! so we need to reduce the resolution by large factors to have reasonable size data. D
  • 6.

    I might just submit something that I worked on from before.

    Does it conform to the 4 winner ideas?:

    This Challenge focuses on building an application using themes based on the best topics from the first Ideation Challenge, combined with any new ideas you may have and a clever implementation.

    The submission to the Challenge should include the following:

    A detailed description of the proposed Solution, addressing the specific Solution Requirements presented in the Detailed Description of the Challenge.

    and phys.org wrote:

    Abdal Elhassani of Indiana University, Bloomington, proposed an app to predict how plant hardiness zones will change in the future with a changing climate. Edward Aboufadel of Grand Valley State University, Allendale, Michigan, suggested using the data to compare a local community's future predicted climate with the historical record of another community.

    A team led by Raymond Milowski of San Francisco proposed converting the storehouse of OpenNEX climate model data to formats compatible with the Open Web Platform to facilitate wider use by web developers. Reuben Cummings from Peoria, Illinois, suggested a web application to map potential and actual climate-related environmental hazards such as wildfires, flood, and drought across the United States.

    Comment Source:>I might just submit something that I worked on from before. Does it conform to the 4 winner ideas?: >This Challenge focuses on building an application using themes based on the best topics from the first Ideation Challenge, combined with any new ideas you may have and a clever implementation. >The submission to the Challenge should include the following: > A detailed description of the proposed Solution, addressing the specific Solution Requirements presented in the Detailed Description of the Challenge. and phys.org wrote: >Abdal Elhassani of Indiana University, Bloomington, proposed an app to predict how plant hardiness zones will change in the future with a changing climate. Edward Aboufadel of Grand Valley State University, Allendale, Michigan, suggested using the data to compare a local community's future predicted climate with the historical record of another community. >A team led by Raymond Milowski of San Francisco proposed converting the storehouse of OpenNEX climate model data to formats compatible with the Open Web Platform to facilitate wider use by web developers. Reuben Cummings from Peoria, Illinois, suggested a web application to map potential and actual climate-related environmental hazards such as wildfires, flood, and drought across the United States.
  • 7.

    Dara, What I did was write a white paper on Modeling via the Semantic Web

    http://entroplet.com/ref/foundation/D-knowledge_based_enviromental_modeling.pdf

    One of the problems is the huge amount of data out there, and being able to unambiguously identify what applies to a topic that you are interested in. I worked with one of the NASA JPL guys in applying their so-called SWEET ontology to create a web server that could organize models and data. This probably doesn't directly address what they are looking for, but it is interesting and worth repackaging in some way.

    Comment Source:Dara, What I did was write a white paper on Modeling via the Semantic Web <http://entroplet.com/ref/foundation/D-knowledge_based_enviromental_modeling.pdf> One of the problems is the huge amount of data out there, and being able to unambiguously identify what applies to a topic that you are interested in. I worked with one of the NASA JPL guys in applying their so-called [SWEET ontology](http://sweet.jpl.nasa.gov/) to create a web server that could organize models and data. This probably doesn't directly address what they are looking for, but it is interesting and worth repackaging in some way.
  • 8.

    Thanx Paul, I will read your writeup and reply tonight.

    Comment Source:Thanx Paul, I will read your writeup and reply tonight.
  • 9.

    WOW! Paul great!

    I am making several replies to this paper.

    Currently I am interested in fresh water estimations for land and raid, with focus on forecast and clustering.

    I like to apply differential operators and other transforms to the data, not interested in the RAW data itself. For example Laplacian of the water on land and then forecast the Laplacian to figure out the water level changes.

    If we get the volumetric data from GPM, then do the latter for a volume grid of the atmosphere i.e. 3D cubes of atmosphere with one face being the surface land, and again forecast e.g. Laplacian.

    Dara

    Comment Source:WOW! Paul great! I am making several replies to this paper. Currently I am interested in fresh water estimations for land and raid, with focus on forecast and clustering. I like to apply differential operators and other transforms to the data, not interested in the RAW data itself. For example Laplacian of the water on land and then forecast the Laplacian to figure out the water level changes. If we get the volumetric data from GPM, then do the latter for a volume grid of the atmosphere i.e. 3D cubes of atmosphere with one face being the surface land, and again forecast e.g. Laplacian. Dara
  • 10.

    Hello Paul

    Th Wind Statistics, if they are satellite based I would be interested.

    So I envisage the satellite data synched to the ground maps, for water and wind

    Comment Source:Hello Paul Th Wind Statistics, if they are satellite based I would be interested. So I envisage the satellite data synched to the ground maps, for water and wind
  • 11.

    Dara, The wind statistics that I used were associated with localized regions such as Oregon, Germany, and Ontario. These were of course measurements closer to the ground and associated with energy cooperatives, such as Bonneville Power Authority (BPA) in Oregon. These places have very extensive data for monitoring wind turbine energy efficiency.

    What I was trying to do was evaluate wind variability in terms of maximum entropy probability density functions. The well known PDF is the Rayleigh distribution in velocity |v| or Boltzmann in Energy ~ v^2, but adding a layer of MaxEnt uncertainty to the mean, one gets a modified BesselK PDF, which seems to fit the BPA data closely.

    BPA wind

    MaxEnt distributions are simple and can usually be characterized by a single moment -- in this case the mean, which is an average wind speed of 12 MPH. The higher order moments fall out of the distribution.

    Certainly global wind data is nice to have. Understanding wind variability has huge implications for wind energy applications, which is why I wanted to look into it.

    Comment Source:Dara, The wind statistics that I used were associated with localized regions such as Oregon, Germany, and Ontario. These were of course measurements closer to the ground and associated with energy cooperatives, such as Bonneville Power Authority (BPA) in Oregon. These places have very extensive data for monitoring wind turbine energy efficiency. What I was trying to do was evaluate wind variability in terms of maximum entropy probability density functions. The well known PDF is the Rayleigh distribution in velocity |v| or Boltzmann in Energy ~ v^2, but adding a layer of MaxEnt uncertainty to the mean, one gets a modified BesselK PDF, which seems to fit the BPA data closely. ![BPA wind](http://1.bp.blogspot.com/-yPtFex5vI1w/TzWsB_ZLQUI/AAAAAAAAA84/lOAPPxYKB_E/s1600/bpa_wind_all_semilog.gif) MaxEnt distributions are simple and can usually be characterized by a single moment -- in this case the mean, which is an average wind speed of 12 MPH. The higher order moments fall out of the distribution. Certainly global wind data is nice to have. Understanding wind variability has huge implications for wind energy applications, which is why I wanted to look into it.
  • 12.

    Paul where did you get your wind data?

    Comment Source:Paul where did you get your wind data?
  • 13.

    Hello John and hello to everyone

    surface precipitation Eurasia Spet 05 2014

    Please note the resolution and size of the satellite scan for surface precipitation. The map is that of Eurasia.

    Note the top of the map has patterned artifact due to mapping the circular orbital to rectangular region, we need to study them in sept to see how to handle best for machine learning and machine vision algorithms.

    What you see is combined data of quite a few satellites.

    We will look at the GPM/JAXA for both high resolution and volumetric data to study and test algorithms for.

    This is the dawn of space-climatology :)

    Dara

    Comment Source:Hello John and hello to everyone [surface precipitation Eurasia Spet 05 2014](http://files.lossofgenerality.com/20140905_S0000000_E0000000_N000.00-N090.00_E020.00-W180.00_SurfacePrecipitation.jpg) Please note the resolution and size of the satellite scan for surface precipitation. The map is that of Eurasia. Note the top of the map has patterned artifact due to mapping the circular orbital to rectangular region, we need to study them in sept to see how to handle best for machine learning and machine vision algorithms. What you see is combined data of quite a few satellites. We will look at the GPM/JAXA for both high resolution and volumetric data to study and test algorithms for. This is the dawn of space-climatology :) Dara
  • 14.

    "Paul where did you get your wind data?"

    Dara, That data for wind was taken from all the surface stations under the BPA's jurisdiction -- which has over 20 meteorological stations set up around northern Oregon. The download consisted of over 2.5 million data points collected at 5 minute intervals, archived over the span of a little less than 2 years.

    The data is still there and more than two years have elapsed since I took the data last:

    http://transmission.bpa.gov/Business/Operations/Wind/MetData.aspx

    It would be interesting to see if the PDF has remained invariant over this time ... but priorities beckon.

    Comment Source:"Paul where did you get your wind data?" Dara, That data for wind was taken from all the surface stations under the BPA's jurisdiction -- which has over 20 meteorological stations set up around northern Oregon. The download consisted of over 2.5 million data points collected at 5 minute intervals, archived over the span of a little less than 2 years. The data is still there and more than two years have elapsed since I took the data last: <http://transmission.bpa.gov/Business/Operations/Wind/MetData.aspx> It would be interesting to see if the PDF has remained invariant over this time ... but priorities beckon.
Sign In or Register to comment.