Hivemind API update to support decentralized blacklists

in HiveDevs5 years ago (edited)

Target audience for this post

Please note that this post is not of general interest: the target audience is Hive software developers (especially those creating Hive API documentation), so it will probably be pretty much gibberish to anyone else.

This post is describing new and not-yet-released API features, to allow developers to prepare for new capabilities that will be available for testing on the testnet in the coming week.

"follow" API call modified to support blacklisting and following of blacklists

To add support for decentralized blacklists, we only needed to extend the capability of the existing "follow" API call (a custom-json call used to communicate with Hivemind servers) and add some new fields to the hive_follows database table in Hivemind.

Original behavior of follow call

Here’s an example of a follow call:

[“follow”,{"follower":"jes2850","following":"netuoso","what":["blog"]}]

This call tells Hivemind that Hive account jes2850 wants to follow the blog of account netuoso. The what parameter tells Hivemind the nature of the follow call. Previously, the what parameter could only be set to two values:

  • blog (tells Hivemind to toggle the following state for the follower)
  • ignore (tells Hivemind to toggle the mute state for the follower)

These states are mutually exclusive, so Hivemind internally used a single value to represent whether a user was not following/not muting (default state, stored in Hivemind database as integer value of 0), following (integer value of 1), or muted (integer value of 2). Since the states are exclusive, if a user has muted someone, then follows them, the muting is overwritten. Similarly, if a user was following someone, then mutes them, the following is overwritten.

As mentioned above, these commands are toggle commands. So assuming a user is initially not following another user, if the follow call is made twice, with what set to blog, the first call causes the user to begin following the following account, and the second call causes the user to stop following the account.

Enhanced version of follow call

All the above behavior is unchanged in the new version of the follow call, so existing API consumers do not need to make changes unless they want to support the new capabilities for creating and following decentralized blacklists.

The enhanced API allows the what parameter to be set to the following new values:

  • blacklist: adds the “following” account to the follower’s personal blacklist
  • unblacklist: removes the “following” account from the follower’s personal blacklist
  • follow_blacklist: follows the blacklist of the account “following”
  • unfollow_blacklist: unfollows the blacklist of the account “following”

It’s important to note that these new states are not mutually exclusive with the following/mute state, so two new boolean fields had to be added to the hive_follows table to store the blacklist and follow_blacklist states.

This means it is possible, for instance, to simultaneously mute someone, blacklist them, and still follow their blacklist. Probably more usefully, a user can follow someone and follow their blacklist (or they can just follow either their blog or follow their blacklist).

Frontend changes needed to use the new API features

At a minimum, a frontend will need to allow users to mark the accounts they want to blacklist and the accounts they want to follow for blacklisting.

But current UIs for doing this are probably insufficient for creating large blacklists (this is feedback from one of the maintainers of the current global blacklists). So it will be useful to have a new UI feature that supports rapid entry of many account names (e.g. cut-and-paste of a large number of comma or space-separated account names).

Fortunately, the new blacklist command is not a toggle command, so this new UI feature won’t need to distinguish between accounts that have already been blacklisted: it can just make blacklist calls on all the accounts specified.

It’s also likely that most users won’t need such a feature, so it may be confined to being a special purpose tool, instead of building the capability into the frontends intended for non-power-users.

Sort:  

Would it be possible to create more than one blacklist as an individual (different reasons)? Or to blacklist people for X reason, where followers can opt-in by checking X reason?

It may be extra, but perhaps pin-pointed focus on specific subjects would improve the workload and infrastructure in the future for Hivemind.

Would it be possible to create more than one blacklist as an individual (different reasons)?

For this functionality, it's only one list per account. So if an individual wanted multiple lists, they would need to create multiple accounts (potentially linking them in some advertised way if that was desired).

Or to blacklist people for X reason, where followers can opt-in by checking X reason?

I'm assuming that anyone who wants to advertise their list for following purposes will probably provide the criteria by which they add people to their list. The current global lists usually publish such criteria. My guess is that we will see more "specialized" lists as a result of this change. Some of the types of lists I expect to see are:

  • account names similar to service accounts such as exchanges
  • accounts that have been trying to scam people or publishing links to phishing sites
  • accounts that are plagiarizing content
  • accounts that are spamming repetitive or informationless posts

For this functionality, it's only one list per account. So if an individual wanted multiple lists, they would need to create multiple accounts (potentially linking them in some advertised way if that was desired).

That's a clever solution!
Are you planning to run your own BL?

I hadn't really planned on it, since I figured other people would do it.

But we do maintain an internal blacklist at BlockTrades to try to prevent users from accidentally sending to phishing accounts similar to exchange accounts and hive services (and also some accounts that have tricked people into giving up their account keys), which we could publish.

I'd guess that some of the bigger blacklists will already cover those accounts. But maybe it would be useful to publish it for someone who just wanted to blacklist those accounts and no others. I guess we'll wait to see what decentralized lists emerge, then decided it if would be useful to do it or not.

We've just had a case that makes it clear how damaging blacklisting is when it has nothing to do with plagiarism. Have you heard of it? I don't want to mention it.

I don't see what these tools will bring to the community if they're going to be used for public scorn.
I think witnesses should invest in the positive things like SMT that we've been waiting for years

I admire your work and I'm very grateful, but in this case, no

As to why we're looking at doing this change now, it's being driven by performance problems associated with the current way that the blacklists are used. We're working to speed up hivemind performance, and we found that blacklisting as coded now was a performance problem. So since we needed to speed up performance anyways, it seemed like a good time to make some quick changes to make them opt-in and give each user the choice of how he interacts with those lists.

One of the points of this change is to make blacklisting "opt-in". Currently, they are not: when you use a given interface like hive.blog or peakd.com, the blacklists are chosen for you. This change will allow each user to make choices about which blacklists they want to subscribe to (or if they don't want to subscribe to any at all). From what you're saying, I would think you would be in favor of this change.

Any type of blacklist should be opt-in and I appreciate your push to make it so. Having blacklists chosen for me is troublesome because I have no idea why someone is blacklisted(there literally is no reason given) and it also sets a precedent that POWER USERS who have elevated technocratic power know better than me on who I want to mute/not see posts from. There are some people who are blacklisted in the #informationwar tag for example and they stopped posting awhile ago because they couldn't get off the blacklist.

Starting off on a fresh account on any other social media I have nobody blocked/muted to begin with and that is how it should be. There should also be an option to import a given person's blacklist AND make changes to your own copy.

For example, if I wanted to use the hivewatcher blacklist because they are correct about spammers/plagiarists 99% of the time, well that 1% of the time they are wrong I would not want those innocent users on the blacklist I follow. The ability to use hivewatchers blacklist and EDIT MY OWN copy would be great. It would have properties of following everything hivewatchers does, when they update the blacklist it updates mine as well, BUT I would like to have the option to manually remove someone from that blacklist that ONLY is applied for my account. If they muted @informationwar then I would remove myself from that list so that on my @truthforce account I can see it, I would still be on the hivewatchers global blacklist though just not on the local copy @truthforce follows if that makes sense.

Also an added ability to submit counter blacklists could be a function in the future, or a proposal to "remove this person from blacklist due to error" that would notify hivewatchers.

The ability to "use a whitelist to filter a user's blacklist" has been suggested, and it seems like a useful idea, but I didn't consider it essential to do it right away, as blacklists don't actually mute anyone, they just add some information beside the account name (and warn on transfers).

But it could be interesting in conjunction with the ability to also follow someone else's whitelist. Some good ideas in your comment, thx!

That sounds better. Thanks for explaining. I also hope you take into account what happened yesterday for all that change, greetings

As a short term fix, this is the best thing I could think of to improve the process, and reduce misuse of blacklists. Of course, there was also the option to just entirely drop blacklists, but they do serve important functions, especially to prevent phishing accounts with similar names to Hive services causing people to lose their funds due to typos.

Greetings, thanks for your working for a better platform

A huge hug 🤗 and a little bit of !BEER 🍻 from @amico!


Un caro abbraccio 🤗 e un po' di BEER 🍻 da @amico!

No not gibberish just another nail in the coffin of Hive

Out of curiosity, did you actually read the previous post (linked at the top of this post) that describes what this new functionality does?

Rrrrrrigght. /Shrug

thanks for the hardwork but I hope some people won't use it just for trolling.

I'm having difficulty imagining how that would work. Adding someone to your personal blacklist will have no effect on other people (even the person you are trying to troll), unless people follow your blacklist. And if you are in the habit of trying to troll people with your blacklist, it's pretty certain no one reasonable will follow your blacklist.

nah, I don't like trolling anyway and thanks for replying. You're enlightened my ignorance.. 🤗🤗

does it mean that frontends will not force a blacklist upon users? but they can still show their blacklist to whoever chooses to follow it.
if correct i think is fair

Yes, that's correct. It will be up to users to select any blacklists they want to use. In theory, a frontend could still force a blacklist, but previously it was pretty much a binary choice that the frontend had to make: use blacklists for it's users or not. Now they can let their users indidvidually choose which if any blacklist they want to follow. So I think that all frontends will quickly adopt this feature.

Thanks for the information. Hopefully I have never been blacklisted.

How does it matter?

Hi hello I have developed an hive application in mobile app by directly loading the web app in it.

Good work. I think it might help promote freedom of speech in the future.

a blacklist promoting freedom of speech... ummmmm... how?

Because there are currently just a few who control blacklists and the people who are listed on it.

yeah ... i guess it does have an aspect of speech that we can all keep our individual blacklists on chain ... and I agree it's a good thing to decentralize the publishing of blacklists on steem ... but it's still blacklists and they are inherently more useful to surpress speech than to liberate it.... jmho

Wouldn't it be great to collab with the entire community to fight plagiarism, phishing and scam together? Right now, there are a few who are in control of blacklists, only they can add and remove people who are blacklisted. This is troublesome in several ways. People who run/control a blacklist will be held responsible and this can either result in pin-pointed targeting or their honesty will be questioned.

While we currently are forced to see a blacklist, once the new system is implemented, we can choose to use whatever blacklist we want, including none. It will be up to you, and anyone else, to use or not to use a blacklist. The goal is to protect our community from those who intently want to harm it and its users. Yes, this will suppress speech, but it will be opinionated from (hopefully) an objective perspective.

I see it more as an extension of the mute option. While we can already mute (suppress), we can now add extra data to a similar option, as we do not mute/sensor but mark, we can still monitor those who exploit, abuse and harm the community on an individual level. Keep in mind that we all hold stake, big and small, and we should be responsible for not our own, but for all stake that is on the platform.

thank you for this improvement of the chain! It was a much needed feature to decentralize the power given to people currently in charge of managing black lists. 👍

Good jhoob

thank you for updating for audience interest

Congratulations @blocktrades! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :

Your post got the highest payout of the day

You can view your badges on your board And compare to others on the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Do not miss the last post from @hivebuzz:

Project Activity Update
Support the HiveBuzz project. Vote for our proposal!

Just a q, when BCH to HIVE?

Just added, per your request, but it make take a little bit to show up on the site.

Thank you I use bitcoin cash so this is going to be very useful for me

I really believe this is the right path to handling BL. This will provide added information about each account and will in some way block malicious threats(it will require some time but thats fine).

Anyway, it would be great to have an option on each profile where a user could follow the blacklist options of another account. A bit like trail voting but with a more direct approach. Basically what i mean is that:

I (or any other user) do trust @blocktrades , that means i trust his choices. I would just need to open blocktrades profile and click on a button to prompt me if i wanted to follow his BL.

Like providing our witness vote but for BL.

The feature you're describing, the ability to follow someone else's blacklist, will be available.

That is great, but will it be easy (userfriendly) for any user to do it?? A lot of features are there but sometimes we needed additional work (another frontend) in order to access that feature. It would be great to have them accessible without having to use any third party frontend but just directly through each of our hive profiles.

That's up to the interface using the feature, not to the feature.

Blacklist, Just call them lists.

To some extent I agree, but the primary initial function of the new feature is to replace global blacklists, and if they were just discussed as "lists", we probably wouldn't get as much feedback (including ideas for the future).

Great job! 👍

Is it possible to use different blacklists for me at the same time?
For example, I want to follow both your and gtg's blacklists:

["follow",{"follower":"oflyhigh","following":"blocktrades","what":["follow_blacklist"]}]
["follow",{"follower":"oflyhigh","following":"gtg","what":["follow_blacklist"]}]

Will we consider introducing blacklists in the community? Only specified post can be mute now, such as:

["mutePost",{"community":"hive-105017","account":"xxxx","permlink":"xxxxxx","notes":"test"}]

It would be much more convenient if we can mute the user directly or we can follow_blacklist for the community.

Yes, you can follow as many blacklists as you want.

The ability to mute a user in a community sounds pretty useful. I suspect that's a feature that is worthy of inclusion, when we start reviewing potential new API calls (a big task that will need a lot of input).

Thank you for your reply.

@blocktrades Very interesting ;)

Why does the Hive 2nd-in-command go down Boating?

I don't understand your question.

Why do you down boating my posts?

Let's live in peace.

I did the upboat of your post.

I want to stay in Hive, too.

Ok, I didn't understand what you were asking.

You are asking why my account has downvoted some of your posts: It actually has nothing to do with you or your posts.

My account is currently setup to counter downvotes from a pair of large accounts called @haejin and @ranchorelaxo. These accounts have been doing randomized voting of posts, often near the end of the 6.5 day payout period.

These two accounts are considered the largest abusive voters on the blockchain, and since I have a large account that can counter those votes, a bunch of people have asked me to help do so. So I have a bot setup now that automatically partially counters their vote. It doesn't nullify their vote entirely, but it reduces it to a more reasonable level.

So, you won't normally be downvoted by the bot. It will only happen when you get huge votes from those two accounts.

I hope this makes the issue clear and you understand it's nothing personal.

If you have any more questions, let me know.