Hive Punks - Enhanced Market History

avatar

IDnN_DMbHAMUF7VAdycEiA_r.jpg

A while ago I put the Hive Punks Market History update on hold due to Hive Engine not having complete information to faciliate it.

Hive Engine has built-in market history that I use to feed the current https://punks.usehive.com/markethistory page but it is truncated every 24 hours. This is why the market history is limited to the last 24 hours.

I've always planned to replace this with my own market history but when I looked into doing this I realized Hive Engine does not provide the necessary information to do this in the NFT Market settlement operation.

For example, this is a recent Hive Punk sale as reported by Hive Engine.

{
  "transactions": [
    {
      "block_num": 14259607,
      "ref_steem_block_num": 61246825,
      "trx_id": "8c8335f66ca381188c113980e2c11e3333a5ff21",
      "sender": "handtalk5",
      "contract": "nftmarket",
      "action": "buy",
      "payload": {
        "symbol": "PUNK",
        "nfts": [
          "765"
        ],
        "market_account": "blockheadgames",
        "exp_price": "17.50000000",
        "exp_price_symbol": "SWAP.HIVE",
        "is_signed_with_active_key": true
      },
      "logs": {
        "events": [
          {
            "contract": "tokens",
            "event": "transfer",
            "data": {
              "from": "handtalk5",
              "to": "blockheadgames",
              "symbol": "SWAP.HIVE",
              "quantity": "0.70000000"
            }
          },
          {
            "contract": "tokens",
            "event": "transfer",
            "data": {
              "from": "handtalk5",
              "to": "blockheadgames",
              "symbol": "SWAP.HIVE",
              "quantity": "0.17500000"
            }
          },
          {
            "contract": "tokens",
            "event": "transfer",
            "data": {
              "from": "handtalk5",
              "to": "supportbase",
              "symbol": "SWAP.HIVE",
              "quantity": "16.62500000"
            }
          },
          {
            "contract": "nft",
            "event": "transfer",
            "data": {
              "from": "nftmarket",
              "from_type": "c",
              "to": "handtalk5",
              "to_type": "u",
              "symbol": "PUNK",
              "id": "765"
            }
          },
          {
            "contract": "nftmarket",
            "event": "hitSellOrder",
            "data": {
              "symbol": "PUNK",
              "price_symbol": "SWAP.HIVE",
              "account": "handtalk5",
              "owned_by": "u",
              "sellers": [
                {
                  "account": "supportbase",
                  "owned_by": "u",
                  "nft_ids": [
                    "765"
                  ],
                  "payment_total": "16.62500000"
                }
              ],
              "payment_total": "16.62500000",
              "market_account": "blockheadgames",
              "fee_total": "0.70000000",
              "agent_account": "blockheadgames",
              "agent_fee_total": "0.17500000"
            }
          }
        ]
      },
      "executed_code_hash": "a3060c1e7365ee3f2d2356de4898c6c4a6e31c7416ea5c5697ba8d88f3c1ab0d815ad74975ef2099fb0d332f42eb5503e27c52c78002f3d408fb0ca67bc37edb815ad74975ef2099fb0d332f42eb5503e27c52c78002f3d408fb0ca67bc37edb815ad74975ef2099fb0d332f42eb5503e27c52c78002f3d408fb0ca67bc37edb389babf88e5a3f8966d947525fc60bee5048ec4fa6abf3724198bffa30f7e5a9",
      "hash": "880649ae06f9adf242073e3f0270fb16210d422306f83cb3933016c19c1ab6e7",
      "database_hash": "ded0e813cdac6c98ff9a815f022b9a5d2409d07dce1319f6d668af88de3964b7",
      "timestamp": "2022-01-25T21:05:24.000Z"
    }
  ],
  "query": {
    "start": "2022-01-25T23:41:16.032+00:00",
    "elapsed": 0.000123584,
    "params": {
      "controller": "transactions",
      "action": "show",
      "trx_id": "8c8335f66ca381188c113980e2c11e3333a5ff21",
      "format": "json"
    }
  }
}

Looking at this is pretty overwhelming, but we are mostly only interested in the hitSellOrder operation in the nftmarket contract.

You can see this section here:

          {
            "contract": "nftmarket",
            "event": "hitSellOrder",
            "data": {
              "symbol": "PUNK",
              "price_symbol": "SWAP.HIVE",
              "account": "handtalk5",
              "owned_by": "u",
              "sellers": [
                {
                  "account": "supportbase",
                  "owned_by": "u",
                  "nft_ids": [
                    "765"
                  ],
                  "payment_total": "16.62500000"
                }
              ],
              "payment_total": "16.62500000",
              "market_account": "blockheadgames",
              "fee_total": "0.70000000",
              "agent_account": "blockheadgames",
              "agent_fee_total": "0.17500000"
            }
          }

From here I can collect the seller, buyer, NFT ID, and price. Here is where the problem comes in though, the price is specified as payment_total, this is the amount that goes to the seller without factoring fees.

If you are a developer, you will also notice "sellers" is an array, meaning multiple Punks can be in the same transaction. Yet the fee (both market and agent) are in the parent section of the transaction bundle, meaning it is shared with all the NFT sold. In English, this means it is possible someone can buy two Hive Punks in the same transaction with a single set of fees for both NFTs.

Almost all Hive Punks sold are sold one at a time, so this shouldn't be a problem, but it is possible more than one is sold at once, as such it has to be accounted for.

You would think you could just add the payment_total for all the NFT's sold, figure out what percentage of total each NFT is, and then split the fees that way. This in theory should be fine, but it is possible two Punks were put on the market through two different interfaces and have different market fees. There is a minimum market fee allowed by the contract but other interfaces can change this. Also Punks launched with a 10% market fee but are now at 5%. If one user put two Punks into their basket and they have different market fees, there is no way to calculate the final sell price for either. This situation isn't likely, but it is possible, so it has to be handled.

Because of this complexity, I haven't been able to do an improved market history as I was originally hoping for. I also didn't expect all Punks to sell out in 36 hours so I figured I'd have a lot more time to add in features.

I recently started to revisit dealing with the market history problem, and come up with a solution. I managed to get a dump of all Hive Engine blocks since the launch of Hive Punks (~2 months) and modified my market history fetcher to process these blocks and strip out the Hive Punk market sales. To date, there have been 1,218 Punks sold on the market. I am going to process these market orders and enter them into a database to drive the new Market History.

Did you know 2 months of Hive Engine blocks came out to 59 Gigabytes?

As I build my own local database for the market history, a lot of new possibilies open up. I can do things like show market volume for more than just 24 hours, average sell price, and potentially even more complex analytics like average sell price per rarity.

I was planning on analyzing the current market sales when I put them into a database and see if there have been any trades that would prevent using the payment_total + market fee + agent fee formula to calculate the final price accurately. If not, I will be able to continue to use the final sale price on the new Market History.

I also got word there is a pull request for Hive Engine that will this problem going forward by including the final sale price in the settlement operation.

image.png

This change means you don't have to calculate the final sale price and hope all NFTs sold in the transaction have the same market fee percentage. The change isn't live yet and only affects transactions going forward, and will not affect any in the past. I will still need to use my current code to get the new Market History database up to date.

This change will also help other dapps that use NFTs to have this data as well.

If you haven't gathered already, this is a new Market History coming out that will go back further than 24 hours, I will let everyone know when it is ready.


YouCut.gif
Thanks @doze!

Posted Using LeoFinance Beta

hivepunks hive-engine proofofbrain oneup vyb ctp nft technology leofinance
0
0
0.000
34 comments
avatar

This change will also help other dapps that use NFTs to have this data as well

It is pretty to know more about this information happening in the blockchain

0
0
0.000
Reply
avatar

Thanks Marky that will be a nice addition to the current Market History.

I look at it every day, waiting to jump in and grab some more ladies in Pink Hats!

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

How many you have out of the 30 now?

0
0
0.000
Reply
avatar

8! I still look every day at the current pricing and availability. :)

0
0
0.000
Reply
avatar

If you want to look at my collection and see if I have any of those, let me know and we can work something out.

0
0
0.000
Reply
avatar

I've been trying to collect certain attributes and you have a few punks that I would need to finish my collection, would you be interested in working out a deal with me too?

0
0
0.000
Reply
avatar

Sure - let me know what you're looking for.

0
0
0.000
Reply
avatar

There's a lot you have done with the hibepunks and made the blockchain more interesting.
For me, I learnt something I new, I didn't realise that hive engine truncates and it's been a while I went to the marketplace after I tried getting my first Punk.

Nice one Markymark and love your picture, just perfect.

Posted using LeoFinance Mobile

0
0
0.000
Reply
avatar

Nothing can be easy it seems with that.

Oh well. I am highly confident you will figure it out. Seems like you have your mind set on getting it the way you need to be.

Over the long haul, I think Hive Punks is really going to add to the ecosystem. It was a great addition.

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

Very great to know more about the Blockchain info so that you will exactly what to in the crypto market

0
0
0.000
Reply
avatar

I'd like to get some punks in the days to come. Looks like NFT is the new deal.

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

I was wondering why this was only 24 hours, but now I get it, you're dependent on the data that HE provides. Good to know you're exploring the options to expand it though, will be good to see some more data. Can't wait, and this reminds me that I really really should start writing another Punk story :)

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

Thanks Marky, can't wait to see the new updates!

0
0
0.000
Reply
avatar

Wooo, I made it in your blog. :-D

0
0
0.000
Reply
avatar

If you can make it here, you can make it anywhere... :)

0
0
0.000
Reply
avatar

Thanks for all the hard work that you have put into Punks.

I do this weird thing where whenever I see code, I begin inspecting it. I can't even begin to tell you how many times a day I click "view page source" source, with an expectation to modify the code and hack my way to greatness, only to realize moments later, that I'm an idiot: I don't understand any of it.

0
0
0.000
Reply
avatar

Very insightful and helps us to improve our understanding about NFTs also.

0
0
0.000
Reply
avatar

That will be awesome to see if any of mypunks sold because I honestly have no clue

0
0
0.000
Reply
avatar

Looks like you sold 27 Punks for a total of 745 Hive.

0
0
0.000
Reply
avatar

Great addition for any kind of NFT related projects. Having past prices / date / etc… is a must when investing in this type of assets.
Thanks for doing the behind the scene work @themarkymark

Posted using LeoFinance Mobile

0
0
0.000
Reply
avatar

Did you know 2 months of Hive Engine blocks came out to 59 Gigabytes?

Holy shit. Incredible work, making NFTs better for Hive as a whole!

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

cool stuff there. I can't imagine that just 2 months of data history would take 59 GB already. whoa! Good Luck to u in getting it all set. ps. that coffee pic is awesome, how'd u make it? or it just some really really good barista skillz??

0
0
0.000
Reply
avatar

ps. that coffee pic is awesome, how'd u make it?

Just image manipulation. There are tools that do all the hard work.

banksy.gif

0
0
0.000
Reply
avatar

Great work. I wonder if we can have market listed NFT, I mean that I have listed some NFT for sale few weeks ago but I cannot see it not until I sort it out page by page. Hope there is an easy way to see this

0
0
0.000
Reply
avatar

On the market page, you can click My Punks and it should show your punks you have listed. If you don't see any, they likely sold or are back in your gallery.

image.png

0
0
0.000
Reply
avatar

You convinced me for a coffee with the first photo :D very very useful post, thank you!

0
0
0.000
Reply
avatar

within 36h ? well when some prime invited investors could buy hundreds of them in bundles for 1 hive ..for sure it went fast in the selling .

Posted Using LeoFinance Beta

0
0
0.000
Reply
avatar

I can't believe how fast they sold out, but that explains why so many of us missed minting our punks. Still, there is the punk market, and I suspect the value wil just continue to rise.

0
0
0.000
Reply