Context
In these posts I try looking at Hive as a black box.
I keep poking at it, to get some better insight on how it works.
Hive Ops
Heard on the street: Hive has ops.
Virtual ops on the block.
I investigate. 😎
block_api
I've demonstrated how it should be possible to build a whole game engine around block_ap.get_block_range, by just looking at all operations in a live stream.
- api.py:
import requests
def get_block_range(start, count, url):
data = '{"jsonrpc":"2.0", "method":"block_api.get_block_range","params":{"starting_block_num":'+str(start)+',"count": '+str(count)+'},"id":1}'
return requests.post(url, data)
Same 'api' as before. Returns the response raw.
- get_ops:
def get_ops_with_block_range(start, count, url):
response = api.get_block_range(start, count, url)
blocks = response.json()['result']['blocks']
ops = []
for block in blocks:
for transaction in block['transactions']:
for operation in transaction['operations']:
ops.append(operation)
return ops
Operations are in transactions, transactions are in blocks.
I need to iterate through that nest to collect all ops.
condenser
condenser_api.get_ops_in_block can also return ops.
- api:
def condenser_get_ops_in_block(block, url, only_virtual):
data = '{"jsonrpc":"2.0", "method":"condenser_api.get_ops_in_block", "params":['+str(block)+','+only_virtual+'], "id":1}'
return requests.post(url, data)
- get_ops:
def get_ops_with_condenser(num, url, only_virtual= 'false'):
response = api.condenser_get_ops_in_block(num, url, only_virtual)
ops = response.json()['result']
return ops
That should return the same ops as with block_api get_ops...
Test
import time
url = 'https://api.hive.blog'
print(len(get_ops_with_block_range(89040499, 1, url)))
time.sleep(2)
print(len(get_ops_with_condenser(89040499, url)))
returns:
91
143
Virtual Operations
There is a difference of 52.
You can read more on virtual operations in the documentation. btw: the examples there don't really work 🙄, but:
Virtual operations (curation rewards, etc) are derived from blockchain activity, but aren’t actually stored as operations themselves. They happen based on consensus from the blockchain based on other user initiated operations. These virtual operations are NOT available on the head block, so a 100% live feed of this information would not be possible. In order then to follow these operations you would have to stream the last_irreversible_block. To get a feed of virtual operations, each of the block transactions needs to be investigated for the type of the operations.
I would explain it more like this:
'Normal' operations are user operations. They are explicit.
They need to be signed and broadcast by a user: Alice makes a post.
Virtual operations are implicit: When Alice makes a post, determines when the post payout will be.
Post and curation rewards must happen exactly 7 day later and are accounted for as virtual operations.
But this is just a coding adventure, not a guide...
Test
print(len(get_ops_with_condenser(89040499, url, only_virtual = 'true')))
>> 52
account_history
In more practical terms, the best endpoint I could find to get to virtual operations:
account_history_api.enum_virtual_ops
I think account_history plugin (not api) must be activated on all nodes by default...
Information behind all of this is scarce...
- api.py:
def account_history_enum_virtual_ops(start, stop, url):
data = '{"jsonrpc":"2.0", "method":"account_history_api.enum_virtual_ops", "params":{"block_range_begin":'+str(start)+',"block_range_end":'+str(stop)+'}, "id":1}'
return requests.post(url, data)
- get_ops:
def get_ops_with_enum(start, stop, url):
response = api.account_history_enum_virtual_ops(start, stop, url)
return response.json()['result']['ops']
Once again, the documentation is wrong.
The ['ops'] is important, as the response actually looks like this:
next_block_range_begin
next_operation_begin
ops
ops_by_block
'next' is used with 'limit' for pagination.
Anyways, after navigating to 'ops':
print(len(get_ops_with_enum(89040499, 89040500, url)))
>> 52
Reversibility
If you actually visited the links from above, you probably noticed reversibility being mentioned.
In the documentation there are parts about head block and such.
Blocks used to be reversible for 1 minute.
That was an issue. The HAF has a huge module 'fork manager' to deal with reversibility.
But the Hive protocol got updated and blocks are kind of insta-irreversible now.
...for this post I am just hacking away at operations. I don't understand it well enough to try explaining things, but I'll investigate further...
Conclusion
There's more to it, than just looking at blocks.
Virtual operations are not accessible by block_api.
In terms of automating something: If all you ever need is user operations, streaming block_api could be reliable enough. Virtual ops are mostly dealing with delayed things like posting rewards - depending on what you want to build, it could matter...
I was asking myself what those virtual ops were since I first encountered them... thanks for explaining it!
I may have to update the scripts I was working on, because at the time I wasn't aware of this difference between the two APIs - and I can't remember what API I used!
btw: where are you running the scripts, if I may ask?
At home or do you rent server space? Should I write something about server rental on privex?
Right now I'm only running them locally on my pc when I test them 😅
If I will ever become decent at coding I may consider to buy something small and cheap to create my own personal server, but I'd be happy to know something more about server rental :)
I recommend a used laptop. Runs efficiently, comes with UPS, keyboard, display, touchpad...
An old, cheap one is enough.
@privex accepts HBD. I'll write a post about how to set up a server.
I'm going to remember it :) I may also have access to one where I work!
That would really cool! Building something on Hive and having the server's expenses paid in HDB seems perfect.