Hello all
I am working on building an app that sends the user a message when their voting power is 100%
@trostparadox gave me an idea that it would be great if we could notify a user on when we have 100% voting power again. I came up with so far 2 different methods. Web-scraping and hive data.
Method 1: Web-scraping
If you are new to hive you might now be familiar with a website called hivestats. If you type in
browser
https://hivestats.io/@your-user-name
This website will display many different statistics about your account. One of these stats is your hive power.
There are many different python and javascript libraries for web-scraping. A pretty straight forward python library called beautifulsoup4 can work great. For some reason I am having an error though. But here are the basics.
In the image below you can see there are different dividers in this ginormous html code. Using beautifulsoup4 we can get the url and parse the html to have all the different html components to sort through.
The code is something like this:
python
import requests
from bs4 import BeautifulSoup
r = requests.get("https://hivestats.io/@armoredbanana")
soup = BeautifulSoup(r.content, 'html.parser')
print(soup.prettify())
output:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<link href="/favicon.ico" rel="icon"/>
<meta content="width=device-width,initial-scale=1" name="viewport"/>
<meta content="#000000" name="theme-color"/>
<meta content="Track, Analyze and Optimize Your Hive Account" name="description"/>
<link href="/manifest.json" rel="manifest"/>
<title>
HiveStats
</title>
<script defer="defer" src="/static/js/main.1ebe7a24.js">
</script>
<link href="/static/css/main.9cd37ad2.css" rel="stylesheet"/>
</head>
<script async="" data-skip-dnt="true" defer="defer" src="https://api.hivestats.io/latest.js">
</script>
<noscript>
<img alt="" src="https://api.hivestats.io/noscript.gif?ignore-dnt=true"/>
</noscript>
<body>
<noscript>
You need to enable JavaScript to run this app.
</noscript>
<div id="root">
</div>
<div id="spawn">
</div>
</body>
</html>
As you can see this outputs the parsed html, but it ends at the div with id spawn.
On other web pages, if you scrape them, the amount of parsed html displayed is quite more. And as you can see from the image above, our information is within this root divider, but somehow we cannot access it as of right now.
For example if you run this code
python
import requests
from bs4 import BeautifulSoup
r = requests.get("https://hivestats.io/@armoredbanana")
soup = BeautifulSoup(r.content, 'html.parser')
#print(soup.prettify())
s = soup.find('div', id = 'root')
x = s.find_all('div')
print(x)
our code outputs:
python
[]
I am looking into other methods of webscraping from javascript enabled websites
Method 2: hive blockchain data
The second method I am looking into is can I get data through the hive blockchain. Hivestats.io gets their data somehow, we just need to figure out how!
Thank you for reading and Part 2 will come soon!
