Past 10 days activity heatmap on Hive and Leofinance. When are users active ? with python codes

in LeoFinance5 months ago (edited)

Good morning to everyone , hope you all are doing well . We are seeing a raise in price , a raise in activity and so much more on Hive Blockchain . I am sure many think about what's the best time to post on LeoFinance or Hive ? I was wondering about this too and thought of seeing when are people active ( in terms of posting and commenting ) on Hive .

Hive - posting + commenting data .

For some strange reason , HiveSQL wasn't working for me yesterday , I had to use Beem API to get the comment and posts data as well as voting data.

Important code snippets -

comment_list=[]
vote_list=[]
for op in b.stream(start=start1, stop=stop1, max_batch_size=50):
    try:
        if 'type' in op:
            if(op['type']=='comment'):
                comment_list.append(op)
            elif op['type']=='vote':
                vote_list.append(op)
    except:
        print(op)     

Everything ran smoothly , the code didn't enter the except clause .

What I am doing here is I am checking all the operations from 18th Feb to 27th Feb and storing the details in a list .

  1. Comment_list stores comment details
  2. Vote_list stores vote details
store_activity=[]
c=0
for i in range(0,len(comment_list)):
    try:
        json_app=json.loads(comment_list[i]['json_metadata'])
        if 'app' in json_app:
            store_activity.append([comment_list[i]['timestamp'],comment_list[i]['author'],json_app['app']])
    except:
        store_activity.append([comment_list[i]['timestamp'],comment_list[i]['author'],''])
        c=c+1

vote_activity=[]
for i in range(0,len(vote_list)):
    vote_activity.append([vote_list[i]['timestamp'],vote_list[i]['author'],vote_list[i]['weight']])

Then I go through each comment and store only three info

  1. Date
  2. Author
  3. App

For voting , I save the following info

  1. Date
  2. Author
  3. Weight

Now I will only post code for one dataframe -

df_activity=pd.DataFrame(store_activity,columns=['date','author','app'])
hours=pd.to_datetime(df_activity['date'])
df_activity['hours']=hours.dt.hour
df_activity['date']=df_activity['date'].dt.date
df_activity=df_activity.groupby(['hours','date']).count()['author'].to_frame()
df_activity=df_activity.unstack(level=0)
df_activity=df_activity.rename(columns={'author':'hour'})

To put it simple I am doing the following

  1. Create a new column hours
  2. Group the whole posts + comments based on date and hours
  3. Pivot the table

Then I have used seaborn to get the heatmap .

Heatmap - posts + comments - Hive

image.png

So this is how it looks ,

  1. The color intensity is directly proportional to the number of comments+ posts .
  2. The Lighter the color , the less activity .
  3. I have mentioned the hour in the top - 0,1,2,3 = all these are in UTC timings .

What do you observe from this ?

  1. The activity is less from 0-12 UTC when compared to 13-24 UTC . The evening ( in UTC timings ) is the busiest .
  2. The highest comments per hour we had ( past 10 days ) was on 22nd which is Monday .
    • Monday 22nd Feb 19-21 UTC saw almost 3000 comments+ posts on Hive .
  3. The minimum score was - 379 on Thursday 25th Feb at 4-5 UTC .

Can you observe anything else?

Heatmap - votes - hive

image.png

Well I initially thought the heatmap for posts+comments and the votes will be the same or similar , why?
Because of autovoting obviously but that isn't it , votes ( number of votes ) are more distributed throughout the day when compared to posting .

There is no huge difference when you see this .

What about just LeoFinance ?

Note- the below data is only for those posts and comments posted from LeoFinance front-end

image.png

  1. Well the 0-12 less activity and 13-24 more activity holds good here too.

  2. Monday which is 22nd has highest activity here too . Still the activity seems more distributed from 12-24 UTC when compared to Hive .

Update :

As @minus-pi pointed out - the above includes both first time posts/comments + edits ( since blockchain takes it as a new one ) . If I retreived from HiveSQL it would have retrieved only unique but since I used Beem API , both are retrieved ( new post + edits )

Although we may take the update to a post as an activity , it shouldn't be considered as +1 comment/ post . So here is the activity for posts and comments by not adding +1 when edited .

I checked the data and number of edits were 13k out of 172k . So I removed it and here is the update -

Hive posts/comments (without considering edits)

image.png

Leofinance posts/comments (without considering edits)

image.png

Well not much changes when we consider activity into count . Edits or no edits , its 99% similar.

Number of posts and number of votes per day

Number of posts per day ( and comments )

image.png

Number of votes per day

image.png

Number of posts and comments per day posted from Leofinance.io

image.png

Not bad at all huh?


That's it from me for now , what do you feel about this? Did you observe something which I missed? Let me know in the comments .
Regards,
MR.

Posted Using LeoFinance Beta

Sort:  

Hey .... great job!

Thank you very much @dalz :) Means a lot coming from you.

Posted Using LeoFinance Beta

Wow I am shocked to be honest, my post is like exact replica of that .

I am glad I am walking in the right path then lol .

Posted Using LeoFinance Beta

Interesting to see more activity in Europe afternoon/evening time frame. I would have expected this to be more towards US afternoon/evening times.
Didn't check your code carefully, but post/comment numbers include edits to the same posts, right? It probably makes sense for activity stats, only the total number of posts could contain some duplicates depending on how to count

Thanks for pointing it out . Although the heatmap doesn't change much , I had actually considered edits too into the heatmap . I updated with unique posts/comments without edits.

Posted Using LeoFinance Beta

Doue it means it is EST/US morning ?
We have to remove task and onealfa data to understand it better :) :)

Posted Using LeoFinance Beta

All the data is in UTC timings :)

We have to remove task and onealfa data to understand it better

Lol true , I can do that .

Posted Using LeoFinance Beta

Awesome job, man!!

These heatmaps look awesome and a lot of valuable information in them! I will choose wisely when I publish my posts :) Thanks for pointing out these things!

Posted Using LeoFinance Beta

Thank you @ph1102 .

I had often wondered when to post too so I pulled these data . I will be glad if it helps you.

Posted Using LeoFinance Beta

This statistics shows that the real activity starts in the evening be it comments or votes. i think this is a good guidance while posting

Yes you are right , if you like to engage with others , the right time to hop on is evening 19-21 preferably .

Posted Using LeoFinance Beta

Unless you are in the US, in which case it's best to do it during the day, 10:00-17:00.

Posted Using LeoFinance Beta

Oh that's correct but whenever I mention time it is in UTC lol.

Posted Using LeoFinance Beta

Congratulations @amr008! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :

You made more than 5000 comments.
Your next target is to reach 5500 comments.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Check out the last post from @hivebuzz:

Hive Tour Update - Financial stages
Hive Power Up Day - March 1st 2021 - Hive Power Delegation

Super good info. It sure beats doing my own trial and error "study" as to which times seem to produce better results. :-) I wonder what it would be like to have a rudimentary knowledge of coding? lol

Posted Using LeoFinance Beta

wonder what it would be like to have a rudimentary knowledge of coding? lol

Lol what do you mean by rudimentary knowledge?

Posted Using LeoFinance Beta

Like I know nothing. Zip. Zero. I'm assuming doing what you did is not rocket science for someone who understands coding (or whatever you'd call it). I may be wrong on that so forgive my ignorance if it's difficult. But it seems that although it may take some time, if you have a basic understanding of what you're doing, it shouldn't be too hard to figure out how to do it.