https://github.com/utopian-io/utopian-bot
If you've been submitting Utopian contributions lately you might have noticed we have been having trouble keeping up with the amount of contributions. The amount of voting power (or mana, I never know which) used is more than can be recharged in a day, which of course eventually became a problem. Because of this @elear asked me to completely rework the behaviour of the bot, and after some misunderstandings, I am glad to say it's finally done! In this post I will try to explain how exactly it works. In case this is a bit too in depth for your liking I think there will be a more general overview posted by @utopian-io soon as well.
New behaviour
Instead of upvoting one contribution and review comment at a time, we will be moving to batch voting and trying to use exactly (or as close as possible to) 20% voting power per day. The bot will start voting once it hit 100%, and will first vote comments, then contributions and then finally all trail posts (e.g. @steemstem).
Upvoting comments
To upvote all comments the first thing the bot will do is find out which comment, that hasn't been upvoted, is the oldest. Once it has found this comment, it will then get all comments made in the 24 hours following that one, as long as they are older than 2 days. It will then calculate the weights that will be used to upvote each comment, depending on the category it's in, as seen in image below.
Example of the voting weight that will be used for each comment per category
If the amount of voting power used to upvote all of these comments exceeds a certain amount (currently 3.2%), then each weight will be multiplied by a constant so that the used voting power is exactly this amount.
A category's share
After voting all our moderator's comments it will start voting the highest quality (determined by its score) contributions currently pending. There are currently 14 categories that we review. Some of these categories have a lot more activity than others, which in turn means that the majority of @utopian-io's voting power is spent on them. As you can see in the table below (data is from a couple of days ago), nearly half of the contributions submitted to Utopian are translations!
Category activity
| Category | Reviewed | # active moderators | Reviews per moderator |
|---|---|---|---|
| all | 197 | 46 | 4.3 |
| translations | 89 | 24 | 3.7 |
| development | 30 | 5 | 6.0 |
| blog | 20 | 2 | 10.0 |
| graphics | 17 | 2 | 8.5 |
| tutorials | 13 | 3 | 4.3 |
| iamutopian | 7 | 2 | 3.5 |
| analysis | 5 | 2 | 2.5 |
| anti-abuse | 4 | 3 | 1.3 |
| video-tutorials | 4 | 1 | 4.0 |
| ideas | 3 | 3 | 1.0 |
| social | 2 | 1 | 2.0 |
| task-development | 2 | 1 | 2.0 |
| bug-hunting | 1 | 1 | 1.0 |
To give an equal chance to each category it has been decided that each category will get an even share of the allocated voting power for contributions (currently a minimum of 14.8%). If upvoting all contributions of a certain category doesn't use up its entire share, then the remaining part of its share will be distributed to the categories that need it. An example of the initial share per category is given in the table below.
Voting power share per category
As you can see in this example a total of 15.73% voting power is allocated to upvote all the highest quality contributions in the current voting round. Taking all of the currently pending contributions and calculating the amount of voting power that would be used to upvote all of them results in the table show in the image below.
Voting power usage per category
Some categories use much more than their share, while others don't even use it entirely. The total amount of voting power used is also much more than has been allocated to the contributions. As I mentioned earlier the bot will distribute the voting power over the categories that need it and then use that to upvote everything. This results in the following image, which shows each category's new share.
Each category's new share of the allocated voting power
The bot will then start upvoting the contributions after sorting them by score. For each contribution the voting power usage is calculated and subtracted from the category's share. If the voting power usage exceeds the this share, then no more contributions of that category will be upvoted in this voting round. Obviously this means that it might happen that a contribution won't be upvoted if its score is not high enough compared to other contributions in the same category. This also means it creates competition within the categories themselves without affecting the contributions in other categories.
I am Utopian
This part is only really relevant for moderators and managers of Utopian, so you can skip this if you aren't. Before this change the #iamutopian posts were upvoted manually by @elear. This won't be the case any longer, and these posts will be considered as contributions like in any other category. Because of this, please keep in mind that you should use the #iamutopian tag before any other category's tag, for example: iamutopian, analysis, utopian-io will result in it being submitted to the iamutopian category. Using the tags utopian-io, analysis, iamutopian will result in it being submitted to the analysis category. I hope you get my point.
My misunderstanding
An example of the unused voting weight multiplier
In my eyes the biggest problem I had to solve was to make sure everything that gets a positive review should get upvoted. I thought of a way to make sure that every contribution in a category was upvoted without it affecting the voting weight of contributions in other categories. To do this I calculated the voting power usage per category, and if it was more than their share, then each contribution in that category's voting weight was multiplied by a constant (calculating a way to do this was surprisingly difficult). This made sure that each contribution was upvoted, while using exactly the amount of voting power that was allocated to the category. After last week's voice meeting it became clear that this is not what @elear had intended, and so it was changed. He explained that it wouldn't be fair for the high quality contributions in a particular category to get their voting weight scaled down simply because of the amount of contributions (and thus voting power usage) in that category.
The trail
After finishing the bot I was asked to also add the trail functionality to the bot. This was initially implemented in JavaScript in another repository, but of course it would be better to keep all voting functionality in one place. So I set out to port it to Python so it would work with the new bot! I didn't really use much of the code from the JavaScript bot, so I will describe pretty much everything about it below.
Configuration
Example configuration
For each trail it's important to define some behaviour for the bot:
- If it should follow self votes
- If a post's context should be checked with IBM Watson (more on that later)
- What the minimum voting weight for following the trail is
- What the max voting weight and weight multiplier is
- What the weekly upvote limit is
- If the trail should be prioritised above others
Putting together the trail's batch of posts
Once the behaviour has been defined the trail's batch can be formed! It first of all creates a list of posts that it should potentially upvote by checking the vote history of the trailed accounts (48 hours). If the votes are above the minimum weight then the post is taken into further consideration.
What I mean by this is that for some trails the context of the post is checked using IBM Watson, which returns certain labels under which the post is categories, like "/technology and computing/operating systems/linux". If IBM Watson is sufficiently confident that this is the case (score > 0.67), then the post is added to the batch. While doing this the post's voting weight is calculated by using the weight multiplier and the max weight (if applicable)
Some trails are also prioritised above others, and this is indicated at this time as well. Of course if a post has already been upvoted by @utopian-io, then it won't be included in the trail's batch.
Upvoting the trail
After upvoting all comments and contributions submitted to Utopian, there will be a certain amount of voting power left over (minimum of 2%). This is used to upvote all the posts in the trail's batch, sorted by priority and voting weight. If the voting power of @utopian-io will drop below 80% after voting one of the trail's posts, then it stops voting. This also means that not every post in the trail is guaranteed an upvote.
Some words
It took quite a long time to implement everything, especially the reward multiplier which didn't end up getting used anyway. A lot of people provided some form of help, either by sharing their thoughts and ideas, testing stuff, solving problems and pointing out mistakes in the code and much more. I would especially like to thank @crokkon, @espoem, @holger80, @miniature-tiger, @nothingismagick and @scipio - I really appreciate your help!
I did my best to explain how everything will work from now on, but if you are still confused or have some questions, then please don't hesitate to ask for clarification. I'll do my best to answer any questions to the best of my ability - however it's my girlfriend's birthday today so I probably won't respond immediately.
Brilliant.
Thanks for all the work you do in the last week for the Utopian bot.
This will help us scale better before the Utopian V2 release.
I see you have added constants.py into the .gitignore, but there is still a constants.py in the repository?
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thanks for the review, @emrebeyler! @elear asked me to add constants.py to the .gitignore but I forgot to remove it, thanks for reminding me haha.
Thank you for your review, @emrebeyler! Keep up the good work!
Beautiful work @amos I hope it functions perfectly.
@utopian-io is lucky to have a great a hard working guy like you.
Thanks for the kind words! I hope the same, haha.
This is an excellent post and the work involved behind the scene is huge. Congratulations on taming this bad boy.
I can't wait to work with you on the V2 integration ;)
Awesome work, now we know where were you busy the whole week :P
Thank you for the update @amosbastian
Interesting, you are leveraging IBM watson? Is the code on github? I would like to read more.
It's only used for some of the posts in the trail. I think it's currently only used for the @steemmakers trail, but I could be wrong.
I just noticed the github reference at the top, my bad.
Nice work @amosbastian :D
Although I've never really thought the bot wasn't doing a good job, these changes are likely to make things go even smoother. Hopefully the new process can catch approved posts sooner so they can get the publicity in Hot/Trending that they deserve.
Cheers!
Thanks! I hope everything goes smoothly as well, will save me a lot of stress lol. It should definitely mean that high quality contributions are upvoted as soon as possible instead of when they are (almost) 6 days old as was happening before.
I guess one nice feature would be to vote early enough to let post access to the trending list. We could decide that post are review between day 0 and 3 and the votes arrived on day 4 or 5.
If a contribution's score is high enough to enter the voting round, then they will be upvoted no matter their age. This should mean that once a high quality contribution (compared to others in the same category) has been reviewed, it will be upvoted within 24 hours or less.
Thanks for bringing up so much clarity on utopian bot services. Its been long that i didnt made any contribution..but looking at the new things it pulling me again.....steem on
Posted using Partiko Android
Incredible work!
I have a question:
If the allocated VP is exhausted for a category and some posts go unvoted, will they remain in the queue to possibly get an upvote another day or will they be removed?
In other words, is there a chance that before payout a not-that-great post, will get voted simply because there was some leftover VP?
Keep it up, that's some awesome work you've done there 😁
Yes, basically every round it will upvote the highest scored contributions until the required voting power to upvote the highest scored remaining contribution will deplete the category's share (< 0%). For example (using made up numbers):
Category X has share of 1%
So this means that category X will have 0.16% left in its share which won't be used, and in the next voting round it will start with contribution D (unless contributions with a higher score are reviewed in the meantime). I hope this contrived example clears things up for you, haha.
Haha it actually made a lot of sense (you wrote it in a developer's way so it helped a lot xD )
Thank you :)
Sounds like you did a great job.
Well done and thanks for the hard work.
Thanks! Hopefully this means I now have time to help you out with PyRules, haha.
Hi, @amosbastian!
You just got a 0.31% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.
Hi @amosbastian!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server
Hey, @amosbastian!
Thanks for contributing on Utopian.
Congratulations! Your contribution was Staff Picked to receive a maximum vote for the development category on Utopian for being of significant value to the project and the open source community.
We’re already looking forward to your next contribution!
Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!
Thanks for all your work @amosbastian.
Is there any word on when we will see the return of community ratings?
I have no idea to be honest. If anyone would know it is definitely @elear, because if it returns, it will be part of the new website.
Yeah, true. @elear I hope community raitings will return. The moderators do good work, but as this tweak to upvote bot behavior seems to once again concentrate curation power in the hands of a few. It's useful to project owners and developers to be exposed to multiple perspectives.