RE: Opening the Kimono of Onboarding Data | Time for Full Transparency

[ura-soul]

Great to see you are taking steps to provide the necessary insights for people to independently verify claims being made. This kind of responsibility is one of the few tools that creators have in a decentralised environment to be able to establish themselves as trustworthy.

As I mentioned in our DMs, the key factor for me here is not so much the data and numbers themselves, but the ability to verify that the new users are real people who bring some kind of value to the chain. One of the primary goals of digital marketing is to not only bring in traffic, but to attempt to bring in targeted and QUALIFIED traffic, that is likely to bring value and to convert into some kind of behaviour that achieves the goals of the organisation.

Tracking the wallet sizes and engagement levels of the new accounts is certainly relevant to this process, but to fully respond to the claims of malfeasance, it is necessary to go a few steps further. Providing the full data dump should hopefully be sufficient to allow the community to do whatever due diligence they prefer with regard to clarifying the situation for themselves. However, as I mentioned previously, the best possible test here (and for all hive projects that onboard new users) is to find ways for the new users to meet and greet with existing users.

The age of AI unfortunately means that we can no longer trust the majority of internet activity as having originated in a human being. While it is an interesting conundrum that AIs could theoretically be given accounts on Hive and completely fool almost everyone into thinking they are human - earning tokens along the way - we are at least currently able to verify people through video calls and other methods. In the face of accusations of fraud and where this avenue is the only one that is open to really prove the situation one way or another, I look forward to some steps being taken to address this.

I want to reiterate that this is a problem that faces ALL projects on Hive that seek funding for marketing or that make claims about onboarding - in that no-one can truly know the accuracy of the claims without such human-to-human interactions being involved. In many ways this is similar to social support funding operated by governments, which are hugely targeted with false claims - just because false claims can be made does not imply that those running the schemes are scammers, but it does mean that some kind of quality testing is needed.

Cheers!

Comments

[khaleelkazi]

Hey Ura, thanks for taking the time to drop by and leave your feedback. Thank you for all the feedback as well as it helped guide the development of this dashboard.

As I mentioned in our DMs, the key factor for me here is not so much the data and numbers themselves, but the ability to verify that the new users are real people who bring some kind of value to the chain. One of the primary goals of digital marketing is to not only bring in traffic, but to attempt to bring in targeted and QUALIFIED traffic, that is likely to bring value and to convert into some kind of behaviour that achieves the goals of the organisation.

I fully agree with this. The way we parse the data we're retrieving for the dashboard has some filtering when we parse for "Real Creators"

For example, here on the MACs trend, you can see ~2120 accounts were removed from the data as we found them to be AI generated/operated (or at least appeared to be).

The way we did that is by parsing the data with specific filters. We've created a handful of filters like these - and as AI gets smarter, we will need to continually tweak our filters. Hive is no stranger to people trying to game the system, even for pennies worth of rewards. Our goal is to filter out as much of that as we are capable through both automated and manual tactics. A few of the automated filters:

1. Time between comments - bots/AI tend to have very specific and rigid standards. If an account is posting every 7 minutes on the dot, then we are able to flag it as a potential bot and then remove it from our data
2. Time from sign up to first post - some of these bots have abnormal abilities to make their first post/thread within seconds of signing up. We've found a way to filter those out as well
3. Profile image - many of these AI accounts either have no profile image or have an AI generated one. We can easily filter out no profile image but the AI generated ones are a bit harder to flag. We've found some ways to flag them as "potential AI" then see if another part of our filtering flags them for something else
4. ...ongoing. As all of this evolves (both the tools from AI and the tools on our end), we have to keep evolving our filters. I didn't list them all here as part of the game theory is that the real people who are making these AI accounts may be looking for all the exact filters in order to game the system. We'll continue to evolve our methodologies. The Dashboard released as part of this post will continue to flag and remove AI accounts from the data both manually and automated based on our filtering

I invite anyone to download the raw data and confirm our filtering. If anyone has suggestions on how to filter better after doing so, I am all ears and will implement more filtering mechanics.

Providing the full data dump should hopefully be sufficient to allow the community to do whatever due diligence they prefer with regard to clarifying the situation for themselves

I fully agree here. After speaking with you and others, I realized the best way to respond to this is to build & release this dashboard and also the raw data that is pulled onchain to display the data.

We use the #newlions tag and a few other tags to filter for new accounts created. We also have a referral tracking system that helps users see who was onboarded from their links (i.e. if you share a blog post link, it attaches your referral ID and if anyone signs up from it within 30 days, they are attached to your account and you can view them in your Referral Dashboard on INLEO). We modeled this after Amazon's Affiliate program (30 day cookie dropped in the link clicker's browser. If they sign up within 30 days, the original referrer gets credit).

Quality testing is absolutely essential. I 100% agree. We will continue to work hard to filter people out, parse data better and have both automated and manual ways of finding the humans. Thank you for all your feedback 🦁

[ura-soul]

No problem, I am glad to help.
It's great that you are filtering for bots, though it may make sense to not publish the filters being used in future! ;)
I'll check out the full data dump once it's available and update asap.