- Snowfall, Temp(Avg,Min,Max) & Δ in min/max distance YOY
An empirical analysis into temperature change and precipitation over time for the Rocky Mountain region.
Hypothesis: As Time Tn increases the temperature differences between min max events will increase.
Secondary hypothesis: Record setting snowfall events at Tn; is a function of Temperature swings and precipitation as a derivative of temperature,humidity,elevation(mBar)
Front page of noaa.gov (2-3-2020)
The front page of NOAA.gov above claims January as the hottest single January on record. In hypothesis: The average monthly temperature for January 2020 > the average of each January for year n where n < 2020; we will test this as well.
I. Introduction
In an effort to make more available the information allowing people to learn to "fish" for their own conclusions in the climate change misunderstanding. I will supply the sources as well as some automated tools for extracting ongoing climate data yourself. We will test the 3 hypothesis for each case and submit conclusions on each. References will be linked directly if possible as well as in the appendix. This is science previously done by esteemed researchers simply replicated and tested as the logical method of finding truth. This seeks to be a simple representation of the scientific method in application as a tool for risk management and understanding.
II. Data Acquisition
To create a test we need to collect knowledge(data) and create a function that can manufacture understanding(palpable intelligence). Initially I tried to buy some segmented and "Cleaned" data from open weather map. This was easy to attain and import. While it had pattern consistent temp,min,max data the snowfall recording clearly had something fishy. Vail, CO clearly had a jump in average overall snowfall around the year 2000 while winter park seemed to remain in pattern with stated research from NOAA. I'll need to get a larger sample to verify no report bias.
I wanted to buy some cleaned data because i wanted to save some time. None the less this was a proof of concept for the hypothesis. The temperature values in this set were enough to see distance from 0 increases in min & max temperatures over rime. In the figure above we can see the the average temp increase as closer to 5 degrees for these to locations rather than the stated 2 degrees global on noaa.gov
This data alone showed a 20+ degree increase in variation as maximums increase and minimums decrease over time. I had to construct a proof and for that we needed another source. The NOAA database is a beautiful testament to people logging the climate to understand patterns as far as i could tell back to 1900 archives for some daily summary data. The data is in small dispersed form, meaning if we wanted a full image of every weather station recording since it first came on line functionally 1900- present we were going to have to submit some file requests. I'm aware they have an API and it can be used for small requests but for full image requests its not very nice to load out an underfunded server like that. So how many records are in the image? Indexed over 120 years, the world has globally cataloged just north of 140 million data points to the NOAA repository. This is beautiful but i'm not going to click 500,000 file downloads as well as check back for current data daily; insert python automation. The first run took some time but once the catalog was built i had quick daily updates of data.
Then came the import process, you could use python and do it in line while grabbing the image with the above code but its fairly slow to open and read each file ~1file/5sec. Here is some import code for a full SQL instance:
import pyodbc
import csv
import os
import pandas as pd
import time
colnames = []
sqlid = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=*SERVER*;DATABASE=*DB*;Trusted_Connection=yes;')
cursor = sqlid.cursor()
fileloc = '*Archive Loc*'
for file in os.listdir(fileloc):
if file.endswith(".csv"):
file = str(os.path.join(fileloc, file))
data = pd.read_csv(file, error_bad_lines=False)
fd = pd.DataFrame(data)
cursor = sqlid.cursor()
for ind in fd.index:
query = "insert into *Table* values('"+ fd['StnID']
[ind] +"',"+ str(fd['Lat'][ind]) +","+
str(fd['Lon'][ind]) +","+ str(fd['Elev'][ind])
+",'"+ str(fd['Year-Month-Day'][ind]) +"',"+
str(fd['DlySum'][ind])+",
+-'Grab only needed fields after clean')"
cursor.execute(query)
cursor.commit()I used an SSIS package for this to downsize the timeline ~1file/.6sec. This changed the import timeline for 500,000 files(@ ~ 300 records/file), from 20+ days to 3.4 days. I'm sure there are better ways to do this but it was something i could set aside for a couple days to get a job done.
III. Literature Review
The literature is mostly segmented into sections of science into biome's of our climate. from resource economics, biology, astronomy, geology. The regression models range from linear to polynomial as well as some neural networks and other LSTM models. I found my favorite in (Solow 1987) where they use a segmented or two phase regression model to explain trends. Breakpoints allow us to sub aggregate individual trends within series over time. Remember that time series in never kind to the data scientist. we use a broken time system as it reefers to mathematics. Decimal numbers in our day values (365.25 days in a year). Segmenting by day when it is not wholly divisible by the year segment. Either way this yields attainable complexity for computation (remember we have millions of data points) while not over fitting in the long run. After all we want to extrapolate constant trends from the overall complex global climate system while controlling for dynamic extremes in a noise variable of sorts. We will call this variable as the difference measure by year. This will maintain the information of Max - Min as difference while also allowing us to directly weight its impact to the overall fit with normalization.
Another source (7) on the marine environment expanded my understanding of the system in that the sea, land, air temp and content are all part of an intricate cycle that is kept in balance by land and marine ecosystems. This source: Climate change impacts on marine ecosystems, shows the change in acidity as well as chemical content in to solution that is our oceans. The figure below shows this effect over time.
NOAA(3) has a good representation of this system interacting with the atmosphere:
This system between the ocean solution and the atmosphere exchanges gasses and those gases are converted into carbonates as well as others resulting in acidity increase. The ecosystems within the marine environment are directly impacted by this change in habitat. most importantly algae and corals.
Another source (6): Global climate change impacts in the united states. States similar results in the atmosphere as is to be expected if these two biome's are understood to be interacting with each other.
We can see the same chemicals in the atmosphere transited to solution in the marine environment. CO2 = carbonic acid, CH4 = formic acid, N2O = nitric acid in the marine environment.
The aside graphic from the same source shows a more informative interpretation of a 1 unit change in attribute x at time n holding all other variables constant yields a value for influence to each variable.
The Attributes contain prior and post industrial revolution dates allowing for a control projection from that set and a control projection from the set post industrial revolution (Human impact).
The positive impact to temperature increase is show to have a 1.6 to 1 positive correlation of Carbon Dioxide. Other greenhouse gases combined are roughly 1 to 1 positive correlation.
Some, factors are inversely correlated:
Reflective properties of the earth surface and climate can cool the climate from (.2 -.6) to 1 with some outlier events in cloud reflective effect. Net effect of naturals yield small positive correlation.
The sum of these parts of literature support our understanding of needs in the variable space. With this we also see more complexity of system to iterate any regression reasonably. I'm reminded of:
"The whole is greater than the sum of the parts"
- Kurt Koffka
The polymath and more famously Psychologist said this of noise understanding in complex system variate such as Gestalt Psychology. While we are not transferring an equation we are going to use some insight into grand schema understanding that is captured here. The pool of the whole (data) is segmented by (Air/land/sea). Each interacts with the other while effect is over time and not instant. Therefore the segment of clarity is by Data/Segment/Time = N/3/365 = N/1095 (see Stratified Sampling). We want to address changes locally while allowing limited cross pollination. In a scaled example this would need to be clustered by Geo location and or Statistical clustering archetype:
I see a three model function Where: Air/land/sea each have contributing variables independent of each other, yet the overlaying principal equation will be a function of these parts including a T+n lag assumed to be gradual over time. We will test time further.
IV. Data & Cleaning
While the mass data collection was in progress I set my sights on validating weather stations in the area that are registered with NOAA. It looks like Open weather map is potentially consolidating sources with multiple recording standards or methods which would skew any distribution blow.
While the variation in weather density is highly compartmentalized in the mountains. The more thorough data set from NOAA showed more pattern consistent data. With a mass of data in hand, the task of laying out the data environment can begin.
The First thing i had to do was segment the set and narrow down our sample to the local rocky mountain region. I tried to encompass as many resorts as possible without extending the bounds too far.
<
I came up with the case:
Where LAT between 37 and 44.5 and LON between -114 and -103
^
>This provides the following station array: calling roughly 1.5 million records.
V. Correlations & Milticolinearity
The initial Set has plenty of data gaps and correlations as well as some variables that we don't need or have no explanatory input to our models:
we will remove the correlated columns as well as non explanatory variables such that our set is:
VI. Model Specification
The model's will consist of 3 of 5 parts:
Part 1: Temp is increasing over time?
=:
TEMPyn = β0 + βTEMPn + βYEARn - βELEVnPart 2: Hottest January on Record?
=:
TEMP(2020-01,n) > ALLyn where Y(n-m(01))= (Sum(TEMPn)/Count(TEMPn))Part 3: As Time Tn increases the temperature differences between min max events increases?
=:
DIFFn = β0 + βMAXn - βMINnPart 4: Record setting snowfall events at Tn; is a function of Temperature swings and precipitation as a derivative of temperature,humidity,elevation?
=:
SNOWn = β0 + βPRCPn - βMINn -βMAXnwhere PRCP(x) = (
Temp,
DEWP
Where DEWP(X) = (dew point, Tdp, given just the actual ("dry bulb")
air temperature, T (in degrees Celsius) and
relative humidity(in percent), RH, is the Magnus formula:
Part 5: Climate Change is occurring in a positive Temp/Diff direction?
=:
CLIMATE𐤃n = β0 + βTEMPyn + βDIFFyn VII. Tests
Model 1: There is a positive correlation to TEMP And YEAR | Accept Null
Model 2: January 2020 is the Maximum average TEMP in the Data set. | Accept Null
Model 3: There is a positive correlation between TIME and DIFF | Accept Null
Model 4: There is a Positive correlation between MAX snow and TIME | Accept Null
Model 5: There is a positive correlation between TEMP, DIFF and TIME | Accept Null
VIII. Results
As we can see by the tests we accept the null hypothesis for each case due to positive correlation or in the case of model 2 we return the boolean response TRUE for the condition. While we knew that science has told us all these things are true thus far it is the duty of science to test the same hypothesis and conclude the same results or disprove and further the knowledge in an area. That is why it is imperative when trying to understanding something to search for scholarly articles as much as common info. When this process is upheld it does a service for human understanding. When it is not it pollutes the confidence in understanding, otherwise recently know as "Fake News". Climate change is real and testable to a high degree of significance across multiple metrics as we have shown. Among our cases as well as other cases published by esteemed researchers I can can find no reference to a null being rejected in reference to climate change in temperature or min max difference. Additionally no model is showing an overall negative correlation between TIME and TEMP. This meaning no research shows temp to decrease where time increases. It is worth mentioning that some regions in particular do have an decreased overall average temp with time. However this is a outlier in an otherwise increasing average.
The Min Max difference Increases with time. It is not clear the exact inputs to this but if the climate is like any other complex system I've modeled, an increase in energy applied to the system often results in more distant oscillation from the mean. The mean(avg Temp) is increasing in our case. If we say that temperature is a function of residual radiation in the atmosphere and the retention of radiation within the atmosphere is increasing. It can then be said that as average temp increases we can assume the distance between the Min and Max events will increase. That is why in media you hear that an average increase in global temp of 2 degrees can cause catastrophic changes. While a 2 degree difference will not impact you personally Max events > 110 degrees and Min events < -30 degrees can kill humans within minutes. As seen in the graphic above we trended past those bookend values around 1990. In 30 years since then we have added roughly 10 degrees to the overall difference measure. This is the trend that must be tended to with the upmost diligence. An increase in avg temp of 2 degrees resulted in a greater than 10 degree difference in Min/Max over 30 years. If we linearly extend that trend. By 2050 the average Max will be 120 degrees and the average Min will be -45 degrees. keep in mind this is along the 37 to 44th latitude which contains most of the US an Europe countries. These are not temps for the poles or equator where we see grander extremes to either max or min. No this is the global temperate space beyond the tropics where a mass of the words population lives.
Precipitation has changed over time as we can see in the aside graphic. Trends have increased after a drought and heat wave of 1980. It would be interesting to see the data for 1900 to 1950 so that the effect of drought and monsoon year cyclicality can be controlled for. As temperatures in high elevations decrease in Min events it pushes more moisture out of weather cycles moving thorough the region. Thus we see an increase in snowfall YOY when we also see a increase in avg temp. This is counter intuitive but true when single snow events in min extremes are adding more snow to the total year sum then the averages. While temp averages may shorten a ski/snowboard season at its bounds ie: season starts late, ends early as temps melt more snow when larger Max evens come in warmer months. The Snow totals/yr, if trend continues, will increase. However, if you are familiar with changing conditions; this means powder will condense and cause a melt freeze layer problems more consistently throughout the year. This can mean larger avalanche cycles in January and February months for this Continental snow pack.
- Pros of climate change for snow sports:
1. Larger Snow volumes during storm
- Cons of climate change for snow sports:
1. Shorter season
2. Shorter powder cycles
3. Snow will not stay soft long after storms
Resorts seek to apply large sums to snow making systems and in general can expect to see less operational days offered per year as time goes on. However the rider can expect to see larger powder days while they arrive in less regularity. As any company, why not tend to the wound rather than apply a band-aid solution(snow making)? Because other large corps are not? This logic is asinine and intolerable and something that organizations like POW are trying to combat at the source. The situation is so grandiose that we loose sight of it easily being part of the massive system. Simply, no human, corporation, or any other biological life form on this planet will remain unaffected. Many, as extinction rates exponentially increase YOY, will cease to exist permanently. So will we stand lazily by as lesser capable forms die around us? What then will we eat? When have we confirmed enough the suspicions of the holocaust to come?
"Ashes and diamonds, foe and friend, we were all equal in the end"
- Pink Floyd (Two Suns In The Sunset)
I'm interested in applying AI and Data Science to world problems. If you have a research topic suggestion or would like to talk about projects please reach out! Here
VIIII. References:
5. Solow, A. R. (1987). Testing for climate change: An application of the two-phase regression model. Journal of Climate and Applied Meteorology, 26(10), 1401-1405.
6. Karl, T. R., Melillo, J. M., Peterson, T. C., & Hassol, S. J. (Eds.). (2009). Global climate change impacts in the United States. Cambridge University Press.
7. Doney, S. C., Ruckelshaus, M., Duffy, J. E., Barry, J. P., Chan, F., English, C. A., ... & Polovina, J. (2011). Climate change impacts on marine ecosystems.