HeyStack launch!

in LeoFinance3 years ago
HeyStack!

Alpha version launch!

For those who haven't seen one of these posts before, HeyStack is a Hive-orientated crypto portfolio tracker project.
You can use it to look at your own Hive journey, or to track your whole crypto portfolio using Hive data and exchange data.

Where is it?

You can see a live version here:
https://miniature-tiger.github.io/HeyStack/

You can find the github repository here:
https://github.com/miniature-tiger/HeyStack

Instructions for use

This week I'll put out some instructions for use, showcasing what HeyStack can do and how to use it.

Today, an introduction to loading your own Hive journey...

(1) Intro page:

Intro.png

  • Read it please!

(2) Add a wallet:

addWallet.png

  • Click on WALLETS in the header at the top to access the wallets tab.
  • Click on HIVE button (top left).
  • Type your account address into the input box (top right).
  • Click ADD WALLET.

There are no keys or passwords involved. HeyStack just reads the publicly available data from the blockchain for the address chosen. The same way you can look at someone's wallet page. Or the account history on hiveblocks.

(3) Load data:

verified.png

  • Your wallet should appear if you typed it right. Now you are ready to load some data!
  • Note that the volume of data downloaded can be significant. Please use an appropriate wifi / broadband connection to manage your data allowances and avoid charges.
  • Native ops to load shows the number of transactions to be loaded (Hive loading filters only the financial transactions so the data volume is actually lower than that shown).
  • Once you are happy, click on LOAD DATA. And wait!
  • The 'Native ops to load' should reduce down to zero.
  • If your account existed pre Hive airdrop, then a number of transactions will be passed across to 'Prior ops to load'. The countdown will continue for these transactions.
  • A Steem wallet also automatically appears at this point.
  • The load is complete when both ops to load figures are zero. There should be a message.
  • My own account is 120k transactions and this takes a few minutes. Larger accounts can take a while!
  • If you just want to test a smaller account to start with then @laxman97 is a useful small example and @hiveio a little larger. Click on HIVE again and go through the steps in (2) above but with the different account name.

At the end of the Hive load the Hive airdrops are calculated (by roll up of all pre-Hive transactions) and entries representing these amounts for each Hive currency are added to the Hive wallet.

(4) Load Steem data:

clickTable.png

  • At this point your Hive account is fully loaded!
  • If you are also interested in your Steem data post the Hive airdrop you can load this through the Steem API.
  • Click on the Steem wallet in the table.
  • Click LOAD DATA.
  • Wait some more.

Overlapping transactions (those pre-Hive airdrop) are not loaded twice. They are loaded through the Hive API and placed into the Steem wallet storage in step (3). The Steem 'Native ops to load' will drop suddenly down to zero during the Steem load once an overlapping transaction is reached (the loads all go backwards).

(5) A little more loading!

  • At this point all the ops to load figures should be zero. However if your account is particularly active there may have been some transactions while the load took place.
  • Just click on the relevant wallet and click LOAD DATA again. Should be super-fast.

(6) Check the balance roll-up!

check.png

  • Probably the best thing to do next is to do is check that the roll-up of your loaded transactions equals your actual wallet account balances (as obtained through the API).
  • Click on JOURNEY in the header to access the journey charts tab.
  • Click on ADD TO CHART to draw a chart of your balances over time in the upper window.
  • There may be a short pause here - a few seconds or so if much transactions.
  • Hover over the right arrow on CHART (top right, second button) to switch to CHECK.
  • Hover over the right arrow on UPPER CHART to switch to LOWER CHART (bottom right).
  • Click on ADD TO CHART to add a check on your roll up.

The Hive and HBD amounts should be exact. The Hive Vests amount should be very close, typically with less than 1% error overall. There are some elements (power up) that have to be estimated.

If you have larger errors than this, please screen grab the check and drop it in the comment to this post. Also if it works let me know too!

Enough for today. Go play. More instructions tomorrow assuming I'm not having to fix it all!

Posted Using LeoFinance Beta

Sort:  

Issue with inconsistent account history numbers. Please try again later.

Any idea why I get this message when I try to load account history?

The get_account_history API call is based on a number for each transaction. So for my account with 130k transactions I can, for example, make a call for a group of transactions starting from number 130000 and going back to 129000 (it goes backwards).

However, these numbers can jump around. If this happens then when I call the next group of transactions (129000 to 128000) transaction 129000 may not be the same as in the previous call.

This could result in a history overlap (and duplication issues - less of a problem) or a gap (and missed transactions - more of a problem).

This is rare on the accounts that I've tested on Hive. On my account it doesn't happen at all. Similarly for accounts up to 500k transactions it doesn't seem to happen. It looks like it may be more of a problem on larger accounts, particularly very large accounts.

To prevent the loading issues described above I chain each batch of transactions together using the block number (plus more detailed transaction id info). So if transaction 129000 (the anchor transaction) is a different transaction in each of the two API calls the check fails.

At this point the load is stopped. My assumption has been that the jumping is temporary and would revert after a short delay, allowing the load to be restarted. However as the problem comes and goes it's difficult to test explicitly.

The issue is further complicated by 'empty' call returns which are caused by long periods with no financial transactions in the account history (lots of json for example). In this situation there is no anchor transaction and the 129000 number becomes more important. Jumping around at this point becomes more problematic.

It's something on my list to improve. I should be able to solve it eventually! @blocktrades team is currently working on the account_history side of things so this is going to lead to some changes that may make things easier.

Posted Using LeoFinance Beta

However, these numbers can jump around. If this happens then when I call the next group of transactions (129000 to 128000) transaction 129000 may not be the same as in the previous call.

No thats inaccurate. Transaction numbers should be the same on every call. If you start at trx 12000, it should start at the same place every single time, since new transactions are added at higher indexes and you only go towards lower indexes.

Posted Using LeoFinance Beta

The issue is that in one call a transaction is being given the number 2484376:

snrcoco1.png

But in another call that transaction is being given the number 2484378:

snrcoco2.png

So I agree it's not a case of new transactions interspersing themselves inbetween the existing transactions since, as you say, new transactions are added as the end and the loading is backwards.

It's some kind of re-indexing issue. Perhaps indirectly caused by new transactions being added forcing a re-indexing? It needs to be something that mainly affects larger accounts (which typically have more frequent transactions).

Or the call being directed to a different source which has different numbers. A different node somehow? My knowledge of how the API returns are actually generated is sketchy.

Posted Using LeoFinance Beta

Oh thats actually really weird. Do you know if 2 different api nodes were used in there or if it was the one? There's a chance different nodes might have different actions at different indexes. Account history is being redone so it should hopefully be better in the future. On my call using @deathwing's node I got it at index id in the picture.

Screen Shot 2021-03-18 at 3.33.28 PM.png

I'm just using 'https://api.hive.blog' for all calls.

It happens on Steem as well so I think it must be a pretty old issue.

Posted Using LeoFinance Beta

Try using https://api.deathwing.me for now. This should hopefully be fixed on launch of new account history.

OK, that's great advice thanks!

I used one of Roeland's nodes in some of my testing which may explain why I didn't see this error occurring as much previously.

Posted Using LeoFinance Beta

@miniature-tiger
I get the same error. It says Loading complete yet immediately turns to:

Too much data?
:

The same explanation as above for taskmaster4450.

I was using your account to investigate and I had the same issue to start with. However I deleted the wallet, added it again and after a couple of repeats of this it loaded straight through.

The issue does seem to come and go.

ericlemark.png

However the load does seem to have dropped a transaction somewhere. Looks like a difference of exactly 200 Hive on the check above.

Posted Using LeoFinance Beta

Thanks for your answer. After several tries, the data is loading now.

Changing the node used for the API calls seems to resolve this issue. I tested for your account and it seems to be running better (although I haven't run it all the way through). Worth another try now.

Posted Using LeoFinance Beta

damn, this looks amazing and for someone like myself, this could make my job of tracking everything much easier. Plus my token holders are able to keep me honest and check the total account history much will build my projects reputation higher.

Thanks! Glad to hear it!

Looks like your account loads without any problems too.

spinvest.png

Posted Using LeoFinance Beta

Great idea with a crypto portfolio tracking application, especially relevant to me (a person with many different HE tokens, the cost of which is so difficult to track). I'll try to use it tomorrow. Today the head already refuses to work normally.

Posted Using LeoFinance Beta

Thanks!

It's just layer 1 at the moment. Hive Engine would be another set of API and another set of prices (not coingecko). Something to think about for the future but not in the short term.

Posted Using LeoFinance Beta

I look forward to further articles explaining what your service is capable of. While everyone is crouching. Thank.

image.png

Posted Using LeoFinance Beta

Fantastic tool! Thank you for creating this. I was able to extract my entire Hive history and export it as well. Took a little while to load the data, but it completed without error. I've got a lot of playing with this to do to figure out all the data I can extract. One question, in future updates would it be possible to add the price of the coin to the exported data?

Great to hear! I'll put up some more details/instructions tomorrow but from the sounds of it people are already working things out.

Prices/values (in fiat) of transactions for export should be possible. It's definitely something I'll be working on shortly on the trade/portfolio side of things (for income/gains reasons). I should be able to port something similar across to the transactions side.

Posted Using LeoFinance Beta

This tool can be really useful! I just tried it and it worked for me. I'll test more options later on.

Posted Using LeoFinance Beta

Good to hear of some accounts where it's working!

Posted Using LeoFinance Beta

Congratulations @miniature-tiger! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :

You received more than 15000 upvotes.
Your next target is to reach 20000 upvotes.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Load looks good.

hivebuzz.png

Posted Using LeoFinance Beta

All I get is the same as Taskmaster after several attempts I'm afraid!

Having seen it now in a few accounts I've got a pretty good idea of how to fix it. I'll try and get a patch out.

Posted Using LeoFinance Beta

Hey cheers!

Changing the node is a quicker fix. It should run through for you now.

Posted Using LeoFinance Beta

Hi, it's still not working, nice interface though,

Is it the same error message?

I managed to get yours to load all the way through:

reviseSoc.png

Maybe try the DELETE ALL (double click) on the wallets page and start again?

Posted Using LeoFinance Beta

Ah cheers, delete all I will try

Ah, it won't let me delete, I click and nothing happens

Hmmm. Could you screen copy the error message from the console log?

(1) Right click
(2) Inspect
(3) Select Console (it shows Elements by default)
(4) Screen grab the error message in red.

Thanks!

Posted Using LeoFinance Beta

Ah hey no worries it's working!

So much work you've put in! Looks great

Any chance we can get the HP option for total rather than non-delegated?

Cool!

It doesn't take delegation into account. So you should be seeing something close to 66k on Hive Power. Maybe you're seeing USD rather than COIN AMOUNT for the DISPLAY VALUES IN button?

That said, when I export the chart I get 64.7k Hive Power rather than 66k.

It's because I convert the Vests earned to Hive Power at each date then aggregate. So it's right for transactions like author rewards at each date. But this will lose the Hive Power interest over time for the overall balance roll-up when I aggregate.

A problem for another day!

Posted Using LeoFinance Beta

cool bro ... you still a hive believer?

I'm still around, although posting less often than back in the day!

yeah, good stuff bruh, crypto still surging, i'm 75% btc, 20% eth and back up to 5% alts again ... still hold my silver and gold as well