You are viewing a single comment's thread from:

RE: AI is stupid: Humans Judgment Matters

in #steemit • 7 years ago

Great post, I'm very much in agreement with you. @rycharde pointed me in your direction and I'm glad he did. 😊 Happy you're getting use out of FOSSbot too. Have you forked it? I presume considering your background that you've made some changes. If you have please consider making a pull request, I have not had time to make any improvements in the last months but it really needs some.

I'd argue instead that our tools should get better filtering so we can avoid content that we don't like, but recognise that it could be considered quality content by another community.

As you know this is pretty much what FOSSbot does and I think we need something like that in the main Steemit.com interface. Either that or we need to make a new tool, but I'd love if it was integrated. I argued for this here not too long ago. I wonder what your take is, how would you suggest better filtering be implemented in the UI?

We can use AI to make humans more productive, but humans must make the final decisions.

I couldn't agree more. I've been thinking of a radical idea for Steem, that posts should be reviewed by people as part of the system. I am working out the details as a thought experiment, but the gist is that reward payout would be affected by whether or not a post passes a randomly assigned review.

There are a lot of issues with the idea, and I have solutions for many (but not yet all) of them, but considering the content of this post I'd like to hear your thoughts on that kind of idea in general. Or if you prefer, and answer to this question: should we have humans in the loops, as a balanced community duty, to disincentivize spam and / or objectively low quality posts?

Sort:  

Hey! Thanks for making such a great tool!
I have not forked FOSSBot. After some tweaking, its parameters are sufficient for my needs for now. I do have a list of improvements I might make - but these are quite far down the priority list. If I do fork then I'll consider a pull request if I think my changes are of interest to anybody.

My position is that filtering is a UI level task, not a blockchain task. How that gets implemented is really up to the UI creators and their own philosophy on how they'd like their UI to operate. If I were to make an attempt at such a thing, I'd try to learn individual user preferences for content by observing their behaviour.

I think the review panel idea is another thing that shouldn't be at the level of the blockchain but instead the tools and ecosystem that surround the blockchain. To some extend what you're proposing (if I have it correct) is an up-powered version of curation.
If what you're proposing is at the level of the blockchain then bots can outscale the human reviewers and just accept the random loses. The economic inflection point for that is when the profit before a bot is discovered and black-listed is less than the cost of creating an account. I'm also not convinced the slowness of human review really work in a social network and there's the corruptibility of the reviewers to consider.
I think putting more tools and SP behind our human curators is probably going to acheive the same aims with less of the downsides.

Hey, great, any contribution towards improvements are welcome. There's also some tickets there ... juz sayin' 😜

I would be ideal if the filtering were done as a UI task, though obviously not client side. With the reorganization of the Steem services with Hive and so on, I think it will be offloaded from the basic blockchain operations, which are far too tightly coupled with it's internal node database and the RPC services on that. I think we need to wait for that but there's a chance to contribute directly to that code and perhaps really make what we're talking about here happen. I'd wait for a beta release though before I start poking around so as not to waste my time!

I had the opportunity to put an idea related to the review panel to Ned recently and he believes SMTs will take care of this. After considering it I'm inclined to agree, either that or it's an idea for something else, not Steemit after all.

On the idea itself though, there are many holes and I'm constantly reworking it. The latest best version is the creation of a curators SMT which uses a network of willing reviewers to kind of super charge a post. So instead of changing how Steem fundamentally works, it's an additional layer of reward, avouchment and community building. Speaking of which, it will be interesting to see how communities affect things.

On the AI accepting random losses, you can work around it I think. For example a multiple choice out of 4 with one right answer you have 25% change of randomly selecting the right one. But if you negatively score accounts that get a loss you can reduce the 1/4 takings to almost nothing if they choose uniformly randomly. Of course a human cabal of answers could collude against the system so I'm trying to come up with a way to build a chain of responsibility between reviewers as a way to mitigate this and the bot attack.

I'm coming up with a beta version (or a general system for testing) that might be able to operate in a live simulation mode without SMTs as they're not available yet.

I think your idea sounds promising. But, yeah so much wait-and-see: SMTs Hive.
A reputation based system for reviewers - sort what curie has, but not so monolithic might work.

It's a pity but yes, we must wait and see. The balance there is that any implementation would take at least as long as it will take to see some of these changes, and would perhaps become obsolete or in need of so much update that it'd be better to wait.

However I do think we could prototype some of these ideas before that time, perhaps using a BitShares asset or just a simple DB or on chain version of that by adding comments to a particular post, etc. It wouldn't have real assets but the ideas could be tested.

There's also the SMT test net, though it looks complicated and unfinished.

I thought about this a bit more. FOSSBot is quite a good tool really. A Curation team could use FOSSBot to mark posts that fit broad criteria with a dust-sized upvote. Then, they could track the upvotes to decide what to curate. That'd make for a much easier curation workflow build on best of breed existing tools.

Interesting idea! Dust vote tagging is one way if you want to keep it totally on chain, better than an annoying comment for sure. I think this idea could feed into the curation SMT idea in my more lengthy comment above 😊 What do you think?

The dust vote tagging was more a way to separate concerns between a coarse parameter driven bot and something that took the output from that to a human. It doesn't have to be on chain but, 1) FOSSBot already does the coarse parameter search well and can leave dust votes on the chain. 2) A human curator is going to have to interact with the chain anyways so why introduce a second layer of comms - unless we're worried about chainbloat.

I wouldn't be worried about chainbloat for a project like that no. Makes sense, good idea.

Would you like to co-propose this on Utopian.io ?

I only like to formally propose when I'm willing to put the work into implementation. Right now, my priorities are elsewhere. If you're thinking about making better curation tools - then please do go ahead!

For me, I'm not going to do another project for Steemit or related where I am the sole major contributor so I'll wait for the right time instead. I'm going to keep talking about these ideas so we'll see what comes up between all the people and all the ideas 🙂

You might also enjoy the comment by @drmake. I'll @ mention you there.