Why I Love HiveSQL!

in LeoFinancelast year


HiveSQL is a centralized database that stores all of the Hive blockchain data and is available for free to all Hive people. It is created and maintained by @arcange. It is probably one of the best if not the best tool on Hive. Let me explain why.

Interacting with the Hive blockchain or any blockchain when you need to get massive amounts of data to analyze is not a very efficient. You can't simply use commonly used programming language, SQL to get data from Hive blockchain directly. HiveSQL takes all the data in the Hive blockchain and stores on a separate centralized database. As the naming suggest we call can get this data with simple SQL queries. This works really fast. I have tried to get data from the Hive blockchain using python in the past and still do sometimes. When you need a lot of data and need to sort and filter things, it gets really messy and slow.

HiveSQL is a great solution. You write simple queries and define your filters and get what you wanted super fast. This project is funded by the community through Decentralized Hive Fund. That's why it is free for all Hive people. In essence, the services you get are already paid for by the community and Hive blockchain. Why don't you use it? I know it is still funded because my queries still work. When they stop working, it will alert me that the service is not funded anymore. We need to understand that this service stores massive amount of data and maintaining it must be very expensive to any one person to provide for free.

I learned SQL because of HiveSQL. I had zero knowledge about SQL and had no clue how it worked. I started learning using other people's scripts here on Hive years ago. The main reason was to make curating process more efficient. My first year on Hive (previously known as something else) was heavily dedicated to curating and I made my first 10k Hive Power mainly by curating activities for Curie. Back then I wasn't much of a blogger and only posted once in a while. Curating paid much more than authoring a post, especially for a new bee.

I would spend endless hours going through hundreds of posts on a daily basis. There had to be a more efficient way, I thought. And there was HiveSQL, previously known as something else. But it was free to use and I thought why not try. I am really glad I did, because it motivated me to learn programming. To use HiveSQL you only need some understanding of SQL. But I wanted more, and started learning python as well, because I wanted to implement a full automation.

SQL with python is the best. Without python, only using SQL may require to use third party UI software. It is ok. You can do that if you prefer this path. What I wanted to do is to have an ability to process the received data more and achieve this with as little clicks as possible. Moreover learning python was important for me, because not only I wanted to use HiveSQL, I wanted to understand how blockchain itself works and wanted to try getting data directly from the blockchain too.

This enabled me to pursue learning both languages. I don't regret it a bit. Trying getting data both ways: directly from the blockchain, and using HiveSQL demonstrates how efficient and power HiveSQL truly is. There was time when maintaining this service became too expensive for Arcange and it became a paid service. I think I stopped using it for a while, then had to restart using it anyway.
I am sure it is more expensive to maintain it now with the growth of the blockchain and the data stored in it. Yet, it is completely free for any Hive person. Just need to get the login credentials, which is easy to do.

So, if you want to learn a little bit of programming, HiveSQL is a great playground to do so. That's how I learned. You can too. I am not a very skillful programmer at all. I only know the basics. But these basics save me a lot of time. If you have time, I would strongly recommend trying. Learning SQL will not take too long. Because all you need is the query basics. You can learn that in a couple of weeks. There are tons of information available for free online. Then start experimenting with HiveSQL. Who knows where it may lead you.

Combining SQL skills with python will boost your skills and and make this process even more efficient. Learning python may take a little bit longer. Learning the basics should not take more than a month. Python is so popular there are tons of learning tools online for free. In essence, one can get basic understanding and skills how to use SQL and python in no longer than two months, without any supervision or need for attending any classes.

What if you could make this learning experience into a money generating experience too? Hive is a great place to learn. Learn something, share your experience on Hive and get rewarded. You get to learn a new skill and make some Hive along the way. But most importantly such experience will make you understand how Hive works a little bit better, and these skills can be used outside of Hive as well. So, if you are interested to learn a little bit of programming HiveSQL is the way to go. If you already have programming skills and don't use HiveSQL, then shame on you. :)

For me it started with curating and a great interest in understanding how the blockchain works. I was lucky to get to know fellow Curie curators who had some programming skills and they were so kind to share their knowledge and scripts. That helped a lot.

I am still involved in curating for Curie. I have been curating on a daily basis from my other account, and you probably wouldn't even notice. For this purpose I have created a Hive Search tool, basically a copy of previously active tools like HiveLookUp. If you haven't seen it yet, it is part of the Librarian app, but has a separate website link - https://hive-search.herokuapp.com. It helps me a lot with curating. If you are a curator or would like to have an ability to find the posts based on certain parameters, you may want to try it. I use it myself everyday. And it is all possible thanks to HiveSQL. It converts the parameters in the app into SQL query and returns the data received from HiveSQL.

One of my first posts about this service was how to setup HiveSQL (previously known as something else) within a python script on a Mac. It wasn't easy task to do. I thought the problems I have experienced could help others too. I sure did. To this day that post has been one of my best posts. Not because of the payout, but because of how many people found it useful in their journey to HiveSQL, new friends made, and collaborating on other topics. It a was a pleasant surprise when I saw even some very talented and experienced programmers finding the guide useful. Since then I have posted the same guide few more time with updates. If you are planning to use HiveSQl with python there are several modules that can help with that. Feel free to read this post to choose the best. I highly recommend using HiveSQL with python.

Another great thing happened during my early learning times. Anarcotech contacted me to hire for a freelance job. Back then Anarcotech and Starkerz were working on a project called "promo". It seems like Anarcotech is not active on Hive anymore, but Starkerz is one of the founders of 3Speak. Anyway, it was great to see someone asking me to create a custom script to get certain kind of data so they can use it in their project. It was kinda funny, because I knew nothing and was just learning, and all of the sudden being asked to do something important. It was a great learning experience. I did it for free of course, but was a great experience. I remember afterwards, Starkerz coming and upvoting all my pending posts as a thank you, lol. The best part is that I could call myself a freelance programmer after that. :)

Since then I found myself involved in running weekly scripts for various projects like Openmic winners, Curie comment contest, etc. I haven't used any of these activities on my resume. I doubt they would do anything, but it was just great to be involved in some of these projects.

For all of these reasons and more, I love HiveSQL. If you use HiveSQL great, if you don't yet, what are you waiting for? If you have time, do it now. It is free. It is fun. It is rewarding. If you have any questions or need any help in your learning journey feel free to ask or contact me on discord. There are many more Hive people who use HiveSQL and all of them are very helpful. That's another great aspect of HiveSQL.

Vote for HiveSQL proposal, if you haven't done so yet. It is indeed a great asset for Hive blockchain and the community.

Posted Using LeoFinance Beta


Thank you for the shout out about HiveSQL and the the work I do to keep it running with top performances. Tho, your post is missing a reference to the HiveSQL website: https://hivesql.io ;)

if you want to learn a little bit of programming, HiveSQL is a great playground to do so

Please don't! DO NOT use HiveSQL as a playground!

HiveSQL is a shared infrastructure, sometimes with billions of rows in its tables. Someone who does not use it in the right way could harm the performance of the whole and negatively impact the activity of other users.

Should you want to learn SQL, deploy your own database, inject data into it and play with your own toy! When you better master SQL syntax, then you are welcome to issue your queries to HiveSQL

Yes, I forgot to mention the website. Thank you for sharing it here.

You are right, it is best to use the service after getting a proper knowledge of how SQL works. Actually that's what I had in mind with playground. Applying learned skills to something more practical and useful.

Thank you @arcange!

I love HiveSQL too. I use it to find people for the #Britlist and have found other uses too. I combine it with Python and the beem library. I should really explore what else I can do with it. I have done plenty of SQL for work stuff, so that side is fairly easy for me. We are lucky to have such great tools on Hive. Maybe I will work on some bigger project eventually.

Keep up the good work.


I agree combining HiveSQL with Beem works great, especially for those who use python.

Is there a good version of Beem right now?
I know awhile back Holger disappeared and some users had to fork it to fix somethings.

To be honest I have not kept up on it.

Any insight?

It is still same. I know it is not being updated anymore. I think someone made a patch to make certain things function properly. I would recommend asking at the Beem discord.

Thank you for writing this post @geekgirl. Love the enthusiasm you show, along with the practical input for our consideration.

I worked with systems containing SQL capabilities for years, in the latter part of my career, as SQL (I'm sure you know this), is the de facto standard for "talking" to a database.

Now, to try HiveSQL, I just have to find the time ... Nice to read that @arcange has made this freely available. That was news to me. Last time I checked into it a bit, it was a paid service.

I have bookmarked this and your other post in Peakd, for future reference. I'm sure the day will come when I can look into this more closely. Lots and lots of applications come to mind very quickly ...

All the best to you, for a better tomorrow. 👍🙂 Onward and upward! 🦁🚀

Posted Using LeoFinance Beta

Thank you @roleerob! Great to hear from you as usual. Yes, the service has been free for a while now. You can see how to get login credentials on hivesql.io.

"... the service has been free for a while now."

Saw your comment below and just found the DHF proposal, so have now voted the stake I control in support of keeping it this way!

And the new HAF system is going to allow anyone to build a specific HAF service which collects the SQL for a specific project.

Links to more information about HAF? I hear people mention it, but still all I know is the acronym. LOL

Partially joking here, but there should be a bot that downvotes posts that mention HAF without linking to something useful.

"하이브 애플리케이션 프레임워크". 약어입니다. 이것은 좋은 출발점입니다. 감사 해요. 나는 여기에서 파다.


That will be great!

I am interested to learn more about HAF. Last time I checked I couldn't figure out how it works. But I saw Blocktrades post there is more in the documentation now. I will try to learn more about it soon. I still need to learn more about podping and v4v. Just need to dedicate some time.

can't wait for this to be more fully flushed out.

I am a Software Engineer and it's great to hear about HiveSQL! Thanks for this!

Great post @geekgirl - I recently found out about HiveSQL and I'm so excited to start sharing my reports with you all. I really believe HiveSQL is currently a hidden gem. With so many transactions happening on HIVE, being able to query any area of the chain quickly and efficiently is so important to understanding how people are using Hive. Of course we have some really good frontends like Peakd and Hiveblockexplorer, that show some of the story - but the details stay hidden in HiveSQL, until we query them!

Exactly! And it is fun to use.

Yes and a little challenge :D

You did it again! I still didn't start (seriously) learning to program, but (your) each new post makes me closer to it.
Besides that, I'm almost 100% sure you will know when I start, as I'm probably going to hang around here to bore you with my questions. 😃

That's great.

I voted for the proposal a while back because it did affect the engagement project by amr008 when it got defunded. Anyways I think it can be quite useful to gather data and I do know both SQL and Python to an extent. So I guess I could always just write some code in the future if I need to use it in the future.

Posted Using LeoFinance Beta

Yes. It will always be there when you need it.

Is HiveSQL free or you need to pay anything in order to use it?

Posted Using LeoFinance Beta

Yes it is free, because it is funded by DHF. I think as long as it is funded by DHF it will remain free. See hivesql.io website to find out how to get login credentials.

This tuesday i have my first class of SQL, thanks a lot for this articule i will read complete later. i Love programing :D

Awesome. Have fun with SQL.

Your early journey on hive must have been so lit . Hive SQL is something I would really love to learn. I should really look it up . More lit and educative posts like these from you always .. Best Regards!

Posted Using LeoFinance Beta

You may want to learn a little bit about SQL first. Thanks.

I know right . I would not mind that at all ..

Up Vote .. You are Great 100% @geekgirl
Full Vote Report Here

wow, this is great tool. Thank you.

I'm interested to know more.

Keep up the good work @geekgirl I really appreciate every part of it

Great information in this post. I am gonna have to find some time to give this a try!

Posted Using LeoFinance Beta

Thank you for this great information.

Great read. Is there a place to find code snips?
!PIZZA !hivebits

Thanks for sharing this piece, I hope to see more innovations with interesting concepts like the decentralized wallet application from https://atomicwallet.io/, with features to store different coins across the blockchain networks

:P Some more term I hear sometimes, but have no real clue about.. SQL and PYTHON. Cool, now I at least know more about them. I should really try and learn more..

Honestly, you are really doing it great and keeping your good work. Really appreciate it more and more

The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.

@geekgirl! The Hive.Pizza team manually curated this post.

PIZZA Holders sent $PIZZA tips in this post's comments:
wanderingmoon tipped geekgirl (x1)
@steevc(3/5) tipped @geekgirl (x1)

Join us in Discord!

Congratulations @geekgirl! Your post has been a top performer on the Hive blockchain and you have been rewarded with the following badge:

Post with the most upvotes of the day.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Check out the last post from @hivebuzz:

Hive Power Up Month - Feedback from February day 5
Hive Power Up Month challenge - Feedback from February day 3
Our Hive Power delegations to the last Power Up Month challenge Winners

This is really cool weldone @geekgirl 🤝.

I've bookmarked HIVESQL long ago to do such a learning experience but didn't find the time to start out yet (laziness sucks sometimes)
Your posts are as always a great motivation :))

Thanks for this. I tried it, it's no longer free and require 1 HBD to get the credentials. I'm still trying to figure how stuff from games are stored in TxCustom, wish it was FTS indexed like Comments table