In this blog, we are gonna perform the analysis on the Meteorological data, and prove the hypothesis based on visualization. The Null Hypothesis H0 is "Has the Apparent temperature and humidity compared monthly across 10 years of the data indicate an increase due to Global warming". The H0 means we need to find whether the average Apparent temperature for the month of a month says April starting from 2006 to 2016 and the average humidity for the same period has increased or not. This monthly analysis has to be done for all 12 months over the 10 year period. So let's start: ## Importing libraries numpy as np   ## linear algebra pandas as pd   ## data manipulation visualization matplotlib.pyplot as plt
%matplotlib inline seaborn as sns

weather_df = pd.read_csv( )

Formatted Date	Summary	Precip Type	Temperature (C)	Apparent Temperature (C)	Humidity	Wind Speed (km/h)	Wind Bearing (degrees)	Visibility (km)	Pressure (millibars)	Daily Summary : : + Partly Cloudy	rain Partly cloudy throughout the day. : : + Partly Cloudy	rain Partly cloudy throughout the day. : : + Mostly Cloudy	rain Partly cloudy throughout the day. : : + Partly Cloudy	rain Partly cloudy throughout the day. : : + Mostly Cloudy	rain Partly cloudy throughout the day. import for import for and import import 'weatherHistory.csv' 0 2006 -04 -01 00 00 00.000 0200 9.472222 7.388889 0.89 14.1197 251 15.8263 1015.13 1 2006 -04 -01 01 00 00.000 0200 9.355556 7.227778 0.86 14.2646 259 15.8263 1015.63 2 2006 -04 -01 02 00 00.000 0200 9.377778 9.377778 0.89 3.9284 204 14.9569 1015.94 3 2006 -04 -01 03 00 00.000 0200 8.288889 5.944444 0.83 14.1036 269 15.8263 1016.41 4 2006 -04 -01 04 00 00.000 0200 8.755556 6.977778 0.83 11.0446 259 15.8263 1016.51 weather_df. ()

< >
Range : entries, Data (total ):
 # Non- Count  Dtype Formatted non- non- Precip non- Temperature (C) non- float64 Apparent Temperature (C) non- float64 Humidity non- float64 Wind Speed (km/h) non- float64 Wind Bearing (degrees) non- int64 Visibility (km) non- float64 Pressure (millibars) non- float64 Daily non- dtypes: float64( ), int64( ), ( )
memory : + MB

weather_df. ().sum()

Formatted Precip Temperature (C) Apparent Temperature (C) Humidity Wind Speed (km/h) Wind Bearing (degrees) Visibility (km) Pressure (millibars) Daily dtype: int64 info class 'pandas.core.frame.DataFrame' Index 96453 0 to 96452 columns 11 columns Column Null ---  ------                    --------------  ----- 0 Date 96453 null object 1 Summary 96453 null object 2 Type 95936 null object 3 96453 null 4 96453 null 5 96453 null 6 96453 null 7 96453 null 8 96453 null 9 96453 null 10 Summary 96453 null object 6 1 object 4 usage 8.1 isnull Date 0 Summary 0 Type 517 0 0 0 0 0 0 0 Summary 0 As it is clear from above that our desired features, Apparent Temperature, Humidity and Formatted Date has no null values. So no need to perform interpolation here. Here our Formmated Date column is of an object type, hence first we will convert it into DateTime format. We can do that as follows: datetime datetime
weather_df[ ] = pd.to_datetime(weather_df[ ], utc=True)

weather_df[ ] = weather_df[ ].dt.year

weather_df[ ] = weather_df[ ].dt.month

weather_df[ ] = weather_df[ ].dt.day

weather_df.head()

	Formatted Date	Summary	Precip Type	Temperature (C)	Apparent Temperature (C)	Humidity	Wind Speed (km/h)	Wind Bearing (degrees)	Visibility (km)	Pressure (millibars)	Daily Summary	Year	Month	Day : : + : Partly Cloudy	rain Partly cloudy throughout the day. : : + : Partly Cloudy	rain Partly cloudy throughout the day. : : + : Mostly Cloudy	rain Partly cloudy throughout the day. : : + : Partly Cloudy	rain Partly cloudy throughout the day. : : + : Mostly Cloudy	rain Partly cloudy throughout the day. from import 'Formatted Date' 'Formatted Date' 'Year' 'Formatted Date' 'Month' 'Formatted Date' 'Day' 'Formatted Date' 0 2006 -03 -31 22 00 00 00 00 9.472222 7.388889 0.89 14.1197 251 15.8263 1015.13 2006 3 31 1 2006 -03 -31 23 00 00 00 00 9.355556 7.227778 0.86 14.2646 259 15.8263 1015.63 2006 3 31 2 2006 -04 -01 00 00 00 00 00 9.377778 9.377778 0.89 3.9284 204 14.9569 1015.94 2006 4 1 3 2006 -04 -01 01 00 00 00 00 8.288889 5.944444 0.83 14.1036 269 15.8263 1016.41 2006 4 1 4 2006 -04 -01 02 00 00 00 00 8.755556 6.977778 0.83 11.0446 259 15.8263 1016.51 2006 4 1 We have extracted year, month, and days from the Date attribute. Now we have cleaned our data, it is time to move on to prove the null Hypothesis. That is to check if Apparent Temperature and Humidity has increased during the last 10 years due to Global Warming. To prove the check the hypothesis, we will visualize variation the attributes yearly for each month. I am gonna use and plotly cufflinks libraries first to plot bar graphs. plotly.offline iplot plotly py plotly.tools tls cufflinks cf .offline.init_notebook_mode(connected= ) .go_offline() from import import as import as import as py True cf Now we have imported and connected our notebook with Plotly, so let's visualize the attributes to get some insights. jan = weather_df.loc[weather_df[ ]== ]

jan.iplot(x= , y=[ , ], kind= ) 'Month' 1 "Year" "Humidity" "Apparent Temperature (C)" "bar" We can analyze from this plot that Humidity for January month for each year is constant, does not vary with the year. But Apparent Temperature shows variations year by year. But this plot does not give us an exact idea about the variation measure in the Apparent Temperature. So now we will first resample the desired attributes by its mean(average) and then will visualize our hypothesis. weather_df.set_index( , inplace = True)

weather_df = weather_df[[ , ]].resample( ).mean()

weather_df.head()

Formatted Date : : + : : : + : : : + : : : + : : : + : 'Formatted Date' "Humidity" "Apparent Temperature (C)" 'MS' 2005 -12 -01 00 00 00 00 00 0.890000 -4.050000 2006 -01 -01 00 00 00 00 00 0.834610 -4.173708 2006 -02 -01 00 00 00 00 00 0.843467 -2.990716 2006 -03 -01 00 00 00 00 00 0.778737 1.969780 2006 -04 -01 00 00 00 00 00 0.728625 12.098827 plt.figure(figsize= ( , ))
plt.plot(weather_df[ ], = , = , linestyle = )
plt.plot(weather_df[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 january =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(january[ ], = , = , linestyle = )
plt.plot(january[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 1 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 As we can analyze there is not any change in humidity in the past 10 years(2006–2016) for the month of January. whereas Apparent temperature increases sharply in 2006 and drops in 2007 and again increases in 2010 but drops in 2014 for the rest of the years there isn’t any sharp change in the temperature. This plot has given us a clear idea about the variations. So let's do the same plotting for each month as well. february =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(february[ ], = , = , linestyle = )
plt.plot(february[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 2 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 march =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(march[ ], = , = , linestyle = )
plt.plot(march[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 3 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 april =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(april[ ], = , = , linestyle = )
plt.plot(april[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 4 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 may =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(may[ ], = , = , linestyle = )
plt.plot(may[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 5 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 june =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(june[ ], = , = , linestyle = )
plt.plot(june[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 6 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 july =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(july[ ], = , = , linestyle = )
plt.plot(july[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 7 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 august =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(august[ ], = , = , linestyle = )
plt.plot(august[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 8 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 september =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(september[ ], = , = , linestyle = )
plt.plot(september[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 9 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 october =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(october[ ], = , = , linestyle = )
plt.plot(october[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 10 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 November november =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(november[ ], = , = , linestyle = )
plt.plot(november[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 11 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 december =weather_df[weather_df == ]
plt.figure(figsize= ( , ))
plt.plot(december[ ], = , = , linestyle = )
plt.plot(december[ ], = , = )
plt.title( )
plt.legend(loc= , fontsize = ) .xticks(fontsize = ) .yticks(fontsize = ) .index .month 12 15 3 'Humidity' label 'HUmidity' color 'orange' 'dashed' 'Apparent Temperature (C)' label 'Apparent temoerature' color 'green' 'Appenrent Temp and Humidity variation yearly' 0 8 #plt 8 #plt 8 Now we have plotted the variation plots for each month with respect to 10 years period of time. : From the month of april to the month of august there is slightly change in Apparent Temperature but nearly no change in humidity for the 10 years(2006-2016). Whereas, for the month from september to march there is a vast variation in the temperature but again humidity remains unchanged.So our null hypothesis is not so true. Anlaysis We have visualized the variation, now we will check this by performing T-test. scipy.stats stats

_,p_value=stats.ttest_rel(a=weather_df[ ],b=weather_df[ ]) (p_value) p_value < :    # alpha value or % ( ) : ( )

we are rejecting hypothesis import as 'Humidity' 'Apparent Temperature (C)' print 6.686806829267691e-24 if 0.05 is 0.05 5 print " we are rejecting null hypothesis" else print "we are accepting null hypothesis" null NIce! We have verified our analysis by T-test as well. So the conclusion here is that we are i.e. Apparent Temperature and Humidity have not been increasing since the last 10 years(2006-2016) due to the Global Warming. rejecting Null Hypothesis That is it for this blog, if you find anything incorrect or have any suggestions/feedback, feel free to reach . me I am thankful to mentors at https://internship.suvenconsultants.com for providing awesome problem statements and giving many of us a Coding Internship Experience. Thank you www.suvenconsultants.com ".

Visualization of Hypothesis on Meteorological data

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Building Handwritten Digits Recognizer using Support Vector Machine

10 Great Articles On Data Science And Programming!

10 FinTech Trends in 2021 [Part II]

10 Financial Predictions for 2021 to Protect Your Wealth

13 API Metrics That Every Platform Team Should be Tracking

If You’re Trying to Talk to Everybody, You’re Not Reaching Anybody

Building Handwritten Digits Recognizer using Support Vector Machine

10 Great Articles On Data Science And Programming!

10 FinTech Trends in 2021 [Part II]

10 Financial Predictions for 2021 to Protect Your Wealth

13 API Metrics That Every Platform Team Should be Tracking

If You’re Trying to Talk to Everybody, You’re Not Reaching Anybody

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps