Repository
https://github.com/holgern/ua-python
user authority - a way to measure trust
UA works on the assumption that a user follow when he want's to see new posts from the account he starts to follow in his feed. When an account has a high UA value, it means that the account is followed by many other accounts as it has produced good content.
The whole thing doesn't work, though, since there are accounts that are followed by countless accounts without ever having produced any content at all.
One solution to this problem is the establishment of trusted accounts. Only accounts that were followed direct or indirect by trusted accounts have a UA value higher than zero. Who should be set as a trusted account? This is not an easy task. As witnesses in the steem blockchain have been elected, witnesses can be seen as somehow trusted accounts.
It can be seen on those figures from the paper "Combating Web Spam with TrustRank" from Z. Gyöngyi et al. how this works. Each trusted node gets an initial seed value, which propagates through the follow connection to followed accounts. The amount which is propagated depends on the number of following accounts.
There is damping. This means that an account which is not directly followed by a trusted account can accumulate less UA than an account which is directly followed by a trusted account.
Mathematics
We are starting with a static score distribution d which depends on the received witness votes (non-witnesses are set to zero)
d = []
for d in accounts:
if d is witness:
d.append(witness_votes)
else:
d.append(0)
The sum of d is normalized to 1
d = d/sum(d)
Now the UA_raw value can be computed by:
UA_raw = d
while True:
UA_raw = alpha * T * UA_raw + (1-alpha) * d
where alpha is the decay factor and normally set to 0.85 and T is the transition matrix.
T*UA_raw is a matrix multiplication and is implemented as:
for a in accounts:
new_value_a = 0
for follow_a in a["followers"]:
new_value_a += UA_raw_old[follow_a] / len(accounts[follow_a]["following])
UA_raw_new[a] = alpha * new_value_a + (1-alpha) * d[a]
The algorithm is now extended by a virtual omega account. Whenever an account is nobody following (also called dangling account), a virtual follow to the virtual omega account which is following itself is added:
new_value_omega = 0
for follow_a in zero_followings:
new_value_omega += UA_raw_old[follow_a]
UA_raw_omega = alpha * new_value_omega
Implementing the omega account is necessary, as without, the algorithm does not converge.
The sum of UA_raw_new + UA_raw_omega is now 1.
Example
In this simple example, B is followed by A. B is a dangling account. When no omega account is added, the sum is not 1 for every iteration. The algorithm is stable after two steps.
| step | A | B | sum |
|---|---|---|---|
| 0 | 1 | 0 | 1 |
| 1 | 1 * 0.15 | 1 * 0.85 | 1 |
| 2 | 1 * 0.15 | 0.15 * 0.85 | 0.277 |
Now, the omega account is added and assure that the sum is always 1. Now B is followed by A and omega is followed by itself and B.
| step | A | B | omega | sum |
|---|---|---|---|---|
| 0 | 1 | 0 | 0 | 1 |
| 1 | 1 * 0.15 | 1 * 0.85 | 0 | 1 |
| 2 | 1 * 0.15 | 0.15 * 0.85 | 0.85 * 0.85 | 1 |
| 3 | 1 * 0.15 | 0.15 * 0.85 | (0.85 * 0.85 + 0.15 * 0.85) * 0.85 | 1 |
Mapping UA_raw
UA_raw_new is mapped to a logarithm scale by:
def UA(x, N):
score = math.log10(x * N + 1/N)*2 + 1
if score < 0:
score = 0
if score > 10:
score = 10
return round(score, 3)
Technology Stack
The calculation of UA is divided into the following scripts:
streamCustomJsonOps.py
In order to calculate UA, all follow connection of all account must be freeze for the calculation. Thus, fetching just the follower for each account using the API does not work as this would take several hours and the first fetched accounts have a different follow state than the last ones.
The python script fetches all account creation operation and custom_json with follow id from the first block. By doing it this way, it is guaranteed that all follow connections for all accounts are known for the exact same time. The script parses account creation operation which are:
- account_create
- account_create_with_delegation
- pow
- pow2
- account_create_with_delegation
and custom_json operation.
When a new account operation was found, it is stored into the database:
account_data = {"block_num": block_num, "type": type_id, "account_index": next_account_id, "account_name": account_name}
Whenever a custom_json with id=follow is found it is stored in the database. Stored are follower, following, the what field and the current block number.
build_follow_db.py
This script goes through the stored account creation and follow/unfollow operation and stores the current follow network state for a specific time point. The generated database contains all available account names and ids, and all follow connections.
In order to keep the database is small as possible, all follow connections are stored with the account number. Each follow is one data entry with:
{"follower": follower, "following": following, "state": state, "block_num": data["block_num"]}
where follower and following are the account ids.
At the moment this database has 112,732,000 entries. For the calculation of UA, all follow connection are needed. As they cannot be stored into RAM, they need to be read out when need from the database.
This process could be speeded up by storing the follow connection into a mongo database. The possibility to use a mongo database was added later and is optional.
When setting use_mongodb=True the accounts and its follower/followings are stored into a mongo database. This is the default setting.
init_accounts_for_ua.py
In this script, all parameter for UA calculation are reset and initialized.
set_trusted_accounts.py
Fetches all active witnesses and calculates the static score distribution for all accounts.
build_ua3.py
Implements the iterative loop to calculate UA_raw for all accounts. The calculation is stopped when 20 iterations are reached or when the difference between two iterations is smaller than 1e-8.
ua_top100.py
This script is used to rank all accounts and to calculate the top 100 accounts.
Roadmap
We will work on open sourcing more UA-related components and try to improve the documentation readability of the code.
How to contribute
Everyone can fork the repository and do a pull request. You can also contact us in our discord.
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thank you for your review, @helo! Keep up the good work!
Very happy to finally see this and have little bit of an understanding on how things are working behind the scenes. @steem-ua is my second largest delegation. These systems can contribute a lot to the ecosystem on the long run due to being a system that is hard to game. It's also really nice to see that my UA Rank is basically 1500 ahead of my follower rankings. Thanks for everything you've done and Best of Luck For The Future!
nice to see this go OS!
This is not true, i follow lots of people who dont put out good content, imo.
Ive seen nothing out of this effort that doesnt lead me to conclude that it is anything but a funnel from the newbs to the top account holders.
Not voting people that dont delegate, accepting delegations from newbs to vote established accounts, and wrapping yourself in rhetoric of altruism while doing these things leads me to conclude that this effort will go the route of the bid bots.
Scam money out of the newbs until they are finally rejected by a diminished community.
But i am not a newb, and got a good idea of how steem's math works.
Care to rebut my assertions, or will you let them stand?
Either way they will be in the blickchain for anybody that cares to look, in 3, 2,....
Yup.
https://steempeak.com/dregs/@dregs/that-revolving-door
Can you say lol yet?
I just smdh,...
I mean, it's indeed altruistic. The intention here is not the exploitation of newbs
Posted using Partiko Android
No, its not.
They take delegations from newbs that dont know better and vote the established people.
Its a redistribution scam.
From those that dont math to those that do.
According to the numbers, it's often worth even for newbs to invest in it.
They are rewarded proportionally, which involves huge potentials for a good content creator.
Maybe not the biggest ROI on the platform but is a good proportion for value.
If you can't see it then I can't help you. Listen to reason, not emotions
Posted using Partiko Android
Got any graphs of data, or am I supposed to take you word on it?
As you can see, I'm writing from Android - not really in a position which allows me to create colorful charts and stuff. You can check the outgoing votes of @steem-ua on steemworld.org, as well as the data of the posts voted on.
You can compare those with the delegation of the post author, the numbers (in terms of ROI) will speak for themselves.
As for quality, that needs a bigger sample. For that, I'm following @trufflepig, a machine learning bot trained on Steem posts - check out how many of posts highlighted by it are also supported by SteemUA, that should be considered an objective measurement maybe.
Do your own research, I'm just here to spread the word and goodwill (which in turn means that you may take my word on it as well, I consider myself credible in the matter anyway)
Posted using Partiko Android
Using ninjamined stake denies a level playing field to those that arent so favored.
When does ua stop taking a subsidy from my vote?
I will find somebody to do the math, give me a little time.
Sup bud.
I totally get where you are coming from. IMHO @steem-ua is a far cry from an ideal reputation metric but it's certainly better than what we got with that fake ass number next to people's names.
As robust as the UA system is, it presupposes certain things that both of us reject namely that being an established witness confers trust.
We both know that is a crock of shit because we both have seen at least one top 20 witness doing underhanded proxy bidding of their posts. That sketchy behavior and I wouldn't trust that witness as far as I can throw them.
If UA is to be a real determinant of the trust or integrity of a user, there should be rules to penalize that sort of behavior.
That includes circle jerking. UA doesn't care if witnesses are involved in collusive voting. We know these behaviors speak volumes of the trust of a witness.
For UA to be a more ideal rep metric, it must not put witnesses on a pedestal and hold them to a rigorous standard. As witnesses, they should be held to a higher standard but it seems rather that the lay users are instead.
Unless I see changed on this front, I am doubtful for the future of UA as being as revolutionary as I had hoped it would be...
Full disclaimer: I delegate to @steem-ua at the moment.
Posted using Partiko Android
I think ganging up to increase rewards shows a lack of class.
Picking winners disadvantages all not chosen.
Which is the game we are playing until we start scheming to keep some down for personal reasons.
If they voted 'good' content i would care less, but they vote based on criteria other than a 'good' post.
And yes, we are well aware that reputation, and adulation, dont follow 'good' content, either.
Have you crunched any numbers?
Does it pay for newbs to play their game?
Yeah, damn it. You and your red pills! Lol
Yeah I just saw the post yesterday and would be nice to run a deep analysis. I'm not really sure about the numbers at this time.
It was nice to see that I have a better UA than a few of those vote buying Trenders. I'm just sick of bid bot people having the leg up on me and seeing their smug little faces. Kind of seen this as an equalizer of sorts.
I'm going to need to give this some careful thought. :/
Posted using Partiko Android
I mean, if we have to put up with spoiled rich kids, and their sycophants, in real life, who would join steem just to get more of the same?
As long as the inflation goes to the haves, as it does under linear rewards, why would havenots stick around to watch it?
Let alone invest to watch their money line somebody else's pocket.
Stinc intentionally hobbled us.
At some point a fork us inevitable, if we want steem to thrive.
Nice. Open source doesn’t mean losing but gaining in the long run. Good job 👍
Awesome @steem-ua. Thank you for making this open source. I hope people will be able to improve this amazing algo to combat spam.
I was never keen on this inititiave, but it is more my belief everything becomes one big gratification ring here. I do keep an eye in my ranking out of curiosity. For most part, it seems reasonable. I know you’re trying to do good on the blockchain, so for that I am appreciative.
a great step in the right direction. Not sure about the 100% self vote??????
Gotta get that curation rewards you know.
hey, finally :)
You said it
Wow, very nice, great to see this.
Thank you very much for taking that step. I've been waiting very long for this. ;-)
Brilliant. Thanks for spelling this out so succinctly, and for providing a viable alternative to reputation.
Wow that all looks foreign to me hahah.
From what reputation is an account considered 'trusted'?
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPTo support your work, I also upvoted your post!
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPGreat step, well done for this, I've wanted this since day 1 😋
Hey, @steem-ua!
Thanks for contributing on Utopian.
Congratulations! Your contribution was Staff Picked to receive a maximum vote for the development category on Utopian for being of significant value to the project and the open source community.
We’re already looking forward to your next contribution!
Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
How can I delegate to steem ua?
Posted using Partiko Android
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPYour level lowered and you are now a Minnow!
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPchecking it out
Posted using Partiko iOS
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @steem-ua! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOP