Okay, let’s go back to looking at charting the weather condition data for a given day. Something that, when I started on it a while back, generated a lengthy period of data processing. Manually and programatically.
Weather Condition Counts
To save you a trip back to the earlier page, here’s the ouput of my query at the time.
(dbd-3.13) PS R:\learn\dashboard\utils> python weather_db.py
current month/day: 09.24
condition cnt
0 mainly clear 1
1 mostly cloudy 2
2 partly cloudy 1
And, here is the current state of affairs.
(dbd-3.13) PS R:\learn\dashboard\utils> python weather_db.py
current month/day: 09.24
condition cnt
0 cloudy 2
1 heavy rain 1
2 mostly cloudy 5
3 partly cloudy 2
Clearly, considerably more data. For this date, we in fact have a weather condition for every year currently in the database. Not something that is likely going to be the case for very many days.
Get Counts Method for Weather_db Class
Okay, I was planning on having a chart showing the count of weather conditions for the given date for all available years. So, let’s look at developing the code for that.
Right now the dataframe is being sorted alphabetically by condition. I am not sure that’s what I want. I would probably like to have it ordered in some fashion related to the basic weather condition. I.E. clear, cloudy, rain, snow. With the appropriate variations for each of those from least to most severity. E.G. clear, mainly clear, mostly clear, partly cloudy, mostly cloudy, cloudy, etc. Not sure how to go about that.
But expect I will need to write a custom function to specify sort order for the dataframe sort method. Let’s give that a look.
Well turns out pandas has this situation covered: pandas.Categorical. The idea is we will make the weather condition column a categorical series, assigning it a custom sort order. Once that is done we sort on that column and bingo. I am using the last dataframe shown above.
print(f"current month/day: {c_md}")
# not sure it is the best order but it is the one I am, for now, going to use
custom_order = ["clear", "mainly clear", "mostly clear",
"partly cloudy", "mostly cloudy", "cloudy",
"fog",
"light rain", "rain", "heavy rain",
"light snow", "snow", "heavy snow",
]
w_cnt["condition"] = pd.Categorical(w_cnt["condition"], categories=custom_order, ordered=True)
wc_sort = w_cnt.sort_values("condition")
print("\n", w_cnt, "\n\n", wc_sort)
And the terminal output was as follows.
(dbd-3.13) PS R:\learn\dashboard\utils> python weather_db.py
current month/day: 09.24
condition cnt
0 cloudy 2
1 heavy rain 1
2 mostly cloudy 5
3 partly cloudy 2
condition cnt
3 partly cloudy 2
2 mostly cloudy 5
0 cloudy 2
1 heavy rain 1
And, that seems to have worked. So a new method for the Weather_db class.
def get_wc_count(self, mn_dy):
"""Get count of weather conditions over the available years
for a specific month and day. The counts will be sorted
in a custom order.
params: mn_dy, month and day to query on, string as mm.dd
returns: dataframe of data, sorted according in custom ordering
"""
t_nm = self.tnms["tp_tnm"]
# okay want the weather condition counts in a specific order
custom_order = ["clear", "mainly clear", "mostly clear",
"partly cloudy", "mostly cloudy", "cloudy",
"fog",
"light rain", "rain", "heavy rain",
"light snow", "snow", "heavy snow",
]
# do not include weather condition for current year
q_wcnt = f"""SELECT condition, COUNT(*) as cnt
FROM {t_nm}
WHERE SUBSTR(datetime, 6, 5)='{mn_dy}' AND condition!='' AND condition IS NOT Null
AND SUBSTR(datetime,1,4)!='{c_yr}'
GROUP BY condition;"""
w_cnt = self.qry_pd(q_wcnt)
w_cnt["condition"] = pd.Categorical(w_cnt["condition"], categories=custom_order, ordered=True)
wc_sort = w_cnt.sort_values("condition")
return wc_sort
And a quick test produced the same result as my test code for 09.24. A few other test dates produced reasonable results. Well, once I got rid of leading blanks in some of the condition values in the table. Not many of them, but…
Charting Counts
Let’s look at producing a chart. I was thinking bar chart. But, you know, I think I will start by looking at a pie chart. Something different, eh!
For development, I will just plot the chart, allowing it to open in a browser tab.
So, in my db_charts module, a new if block for testing things out.
Note, the text size in the call to go.Pie was selected to make some later refactoring display nicely. May yet need to be changed when I look at adding the chart to the dashboard.
... ...
if do_wc_chart:
md_dy = "09.24"
wc_2dy = "heavy rain"
wc_cnt = rfall.get_wc_count(md_dy)
print(wc_cnt)
wcs = wc_cnt['condition'].to_list()
cnts = wc_cnt['cnt'].to_list()
print(f"\n{wcs}\n{cnts}")
fig = go.Figure(data=[go.Pie(labels=wcs, values=cnts, textfont_size=18)])
fig.update_layout(autosize=False, width=600, height=600)
fig.show()
In the terminal I got the following.
dbd-3.13) PS R:\learn\dashboard\utils> python db_charts.py
condition cnt
3 partly cloudy 2
2 mostly cloudy 5
0 cloudy 2
1 heavy rain 1
['partly cloudy', 'mostly cloudy', 'cloudy', 'heavy rain']
[2, 5, 2, 1]
And the pie chart looked like the following. Note, for now only showing a screen capture. So no interactivity. Will hopefully have that with the final version of the chart.
Next, I wanted to get rid of the legend and put the weather condition and count in the chart. As well as add a title. The pertinent bits of code follow.
... ...
fig = go.Figure(data=[go.Pie(labels=wcs, values=cnts, textinfo='label+value',
textfont_size=18)])
fig.update_traces(showlegend=False, selector=dict(type='pie'))
fig.update_layout(autosize=False, width=800, height=600,
title_text=f"Historical Morning Weather for {md_dy}", title_x=0.5,
title_font_size=22)
... ...
And, the chart now looks like this.
Finally, if the current days weather condition is in the historical data, I want to highlight that segment of the chart. I will pull it out a small amount. I also add some parameters to reduce the space being taken up by the chart title. Mainly by controlling the y position of the title and adjusting the size of the top margin.
A bit of trouble with the title if a segment is being pulled or not. So needed to set up 2 sets of top margin and title y-position.
... ...
wcs = wc_cnt['condition'].to_list()
a_pull = [0] * len(wcs)
t_y, t_m = 0.99, 30
if wc_2dy in wcs:
wc_idx = wcs.index(wc_2dy)
a_pull[wc_idx] = 0.2
t_y, t_m = 0.96, 10
cnts = wc_cnt['cnt'].to_list()
fig = go.Figure(data=[go.Pie(labels=wcs, values=cnts, pull=a_pull,
textinfo='label+value', textfont_size=18)])
fig.update_traces(showlegend=False, selector=dict(type='pie'))
fig.update_layout(autosize=False, width=600, height=600,
margin=dict(l=0, r=0, b=0, t=t_m, pad=0),
title_text=f"Historical Morning Weather for {md_dy}", title_x=0.5, title_y=t_y,
title_font_size=22)
... ...
And, here’s the final result in it’s interactive form.
I really think that’s a good or novel way to show what the morning weather condition is for the current day. But will need to come up with something else if today’s condition is not in the historical data.
Definitely going to need to work on the colours. Hard to read the text in some cases.
And, I definitely prefer a pie chart over a bar chart in this case.
Done
I think this post is done. Fair bit of fooling around to get things coded and working more or less as I wanted. So, will take a break before looking at where to go next. Some other weather related chart. Or move onto temperature related charts.
Until next time, enjoy your days coding.
Resources
- API reference | pandas arrays, scalars, and data types | pandas.Categorical
- Plotly Pie Charts in Python