Failure to understand how the trending page works on Steemit and get my posts to the top, I have decided to look at some data to see what insights I can find out.
On 2nd Aug I scraped www.steemit.com/trending and I extracted 249 posts which I dumped into an Excel Spreadsheet.
The info that I scrapped gave me the post title, author, number of votes, the tag used and the payout value.
From this data alone, what can we establish?
In total 171 Authors have posts on the trending page scraped.
20% (51) of the posts on trending are made up of 7% (12 authors) of all authors on Trending. Below is a table showing the 20% of author sorted by the number of posts they created that is on the trending page.
Excluding negative votes, the minimum vote count for a post on Trend is only 27, however the max post count found on the posts on trending was 3678. Below is the top 12 authors based on the average vote count per post on trending
Jerrybanfield, dollarvigilante and kingscrown are all in both the Authors with the most number of posts in trending and the highest average votes on a post.
And below is the bottom 10 based on average vote count
On the payments side the average post payment is $161, the highest being $840 and lowest is $55. Who got the highest average post value? The table below shows the top 12 authors with posts on trending sorted by average pay out value.
Kingscrown is the only author that is now on the top of each report shown so far, 4 posts on the trending page, the average votes per post is in the top 12 and average payout is also in the top 12.
Other accounts that appear on two lists are Jerrybanfield, dollarvigilante, joshsigurdson, dan and bookingteam.com
And below is the bottom 10 authors with posts on the trending page sorted by lowest average pay out.
What is interesting about the lower 10’s is that many of these authors only have 1 post on trending
Next I had a look at the tags
First I sorted by the tag with the most number of trending posts
Then by average number of votes
And by average post value
The only real trend here is that there are 4 tags that seem to do well in terms of hitting the trending page. Cn, Bitcoin, life and art. These are popular tags anyway so it would be expected to see a few posts from these tags on the trending page. But none of these are on the top average number of votes or average post value list. This is a good thing, well it seems the niches do well as the tags do seem to vary
I have still not got any close to finding out the magic secret on getting my post to trending. I know posts need to have between 27 and 3679 votes with a value of $55 and $840. To be honest this tells me nada.
But I have identified a few authors that might hold the secret so I think it’s time to mash this data up with data from the SteemitSQL Database and get some real insights.
Are you ready to see what I have uncovered?
First I connected to the TxVotes table, so I can see who is voting for these posts and for these top authors. The problem I faced here was that the data I scraped from the web has no common field with the TXVotes data.
To overcome this, I needed to also connect to the Comments table, as this table contains the post title, and my web scraped data also has this field. So I merged the TXVotes table with the comments table based on the permalink field to pull in the post title to the TXVotes table.
Now I have data I can work with…(I love this stuff!!!)
Who are voting for the Authors with posts on the trending page? Are there any patterns or people in common?
Above is a network chart. The colour dots represent authors on the trending page. The grey dots with the black text represent the voters. It’s obvious from this visualisation that there are a number of accounts standing out as a common voter on many posts on the trending page.
Zooming in a little so you can make out some names. We are looking in particular the name in the black as these accounts seem to be voting on a number of trending post authors
I looked at some of these accounts in black….noting interesting really. They might even be voting bots. There are new accounts and have not been delegated any Steem power by anyone (I feel like an investigation journalist looking for something sinister.)
Trending posts and votes from Whales
I have also taken a look at the voting relationship between the Authors on the trending page and the top 250 Whales. To do this I needed to add a new table to my data set. So I have scraped data for the top 250 whales from https://steemwhales.com and added this to my data set.
It is very interesting to observe that the authors with the most posts on the trending page seems to get a number of votes from other whales…………..
In total there were 66875 votes on the 249 trending posts. Of this 1907 was a vote from a Whale in the top 250 whales. That’s 3% of total votes on trending posts. And averages out at 7.6 whale votes per post on trending page. So who is getting above that average?
On the bottom, it is interesting to see that some posts only have 1 whale vote and the lowest voting weight is 20%. This is interesting. I have a chance of getting a post to the trending page with 1 whale vote at 20%
I also took a look at the whales that were voting and at what % there are voting on trending posts. The average voting weight is just 50% although there are a considerable number of whales voting for trending posts at 100%
Conclusion
The charts and data I have shown are only part of the analysis I carried out to see how to get a post on the trending page. What I have established is that the value of a post is of utmost importance.
In order to get value quickly you need to rely on votes from whales. We have seen that a number of authors have a few posts on the trending page, and tend to trend on a regular basis. Well know I know why. It is these same authors that are getting high volumes of whale votes.
Can us minnows compete with that and get a place on the trending page? Highly unlikely. Unless of course Steemit changes the way the Trending page works.
I also found some other stuff during this analysis that I have not shared. The reason, well if I am write, or wrong, playing the name or shame game will only get me down voted ( I don’t like down vote bullies). I found a Whale voting accounts that seem odd to me. Seem funded and seem to be voting for the same 4 authors on trending time and time again.
A big thanks to @furion for steemdata and @heimindanger for steemwhales.com
If you enjoyed this post please do follow me, I post on Excel, PowerBI, Business Intelligence, Steemit, Data, Stats and other things of interest
here is a full list of authors with a post on trending as I scraped on the 2nd Aug
dan kingscrown keiserreport officialfuzzy rea penguinpablo oflyhigh bookingteam.com jerrybanfield frostedace joseph kevinwong deanliu modprobe gavvet joshsigurdson dollarvigilante b0y2k zararina someone crypticwyrm steemcleaners jphenderson anarchospace sirlunchthehost witness-category buildteam kyriacos rivalhw steemdevelopers slowwalker jayjayjeffery myfirst kencode oldstone schmid donkeypong sebi99p oldtimer sndbox hilarski minnowbooster lukestokes awgbibb patelincho sportspodium canadian-coconut helene madiba ericvancewalton dragonslayer krystle dianargent stacee rheteric marty-arts jubi cherryblossom-sa siddartha nicolemoker mylifeasgoda birds clayton senzenfrenz crazymumzysa barrysamways laniakea xxsenpaixsamaxx btsabc onceuponatime walterz scoop pharesim steemtruth sallyswitchblade agmalirsadi cervantes bigdaddy isaria blackbunny natureofbeing jameslashomb kjablonski jrcornel sirwinchester nexusfyre hannahwu elfkitchen ocd phelimint papa-pepper crypt wledge anna.art.time stan ashleynau steemit jacor atulmittal stellabelle voronoi sgtreport mindhunter fathin-shihab krnel full-steem-ahead schattenjaeger in strawhat chitty amylee kus-knee travelman cqf nerdfury ramengirl jenniferhosein justincottle nanzo-scoop frostfractal artguru timcliff cahlenlee yyyy paulie2coins steem2themoon aggroed sammosk bestofbest leesunmoo jeffberwick cryptoqu33n surfermarly allasyummyfood tribedownsouth lydiachan abdullar gardenofeden woz.software mynameisbrian hansikhouse bitcoinflood buzzbeergeek gringalicious joythewanderer sixexgames justyy teamsteem vitkolesnik rockchickjen justmousepixels jfollas suesa tygergamer gamerpool charlieshrem anritco kitcat stackin ana-edwards clayop htooms birds90 steemdice1 wony steemsports coldhair kafkanarchy markwhittam lemooljiang sweetsssj
You have skillz that killz )) nice post I enjoyed reading it even though some terminologies are unknown to me I sensed the whole idea thanks to your easy follow thru writing... I would like to ask if posts were left open to upvotes at any time .. Wouldn't that be democratic and solve the trending whales boost ...( boost that on the other hand has to be counter balsnced by minnowbooster, booster, minnowsupport, etc which relies mainly on bots and not real whales)? Wouldn't best call it "weekly trendings" as it is exactly that? Payouts should be weekly or monthly like a standard salary unless I am missing a point
Paulag, please just keep on digging! Your analyses are very insightful and I will keep a very close eye to your posts. I do hope you will keep sharing what you find on your quest for insight into the magic formulaes of Steemit...
I second that. @paulag I'm voting with 100% for whatever my vote is worth.
Great analysis! As your data suggests, it's all about the dollars and getting that whale vote. The trending page is really the whale page. I guess whales are people too and maybe they deserve their own page?
whales are people too, and don't get me wrong, they have either invested wisely or worked hard to get where they have got
Most of them have invested wisely and invested money early into the game. There was a time when 1 SBD was worth just 7 cents.
And most of these guys have invested their hard earned money and now they are enjoying their return on investment.
yea i believe i missed the boat on the 7 cent> I met someone that sold at that an the nxt day it flew up
Haha... I wish I know about Steemit last year I could have made some good money.
@paulag this is very interesting but you really should be careful.
I don't know if you saw @dwinblood's posts over the last few days but you may want to read this and get in touch with him. https://steemit.com/investigation/@dwinblood/i-m-going-to-do-a-little-investigation-into-the-down-voters-who-do-so-for-purely-dislike-and-other-reasons-as-well-as-their I think he'd be very interested in the data you've scraped.
Yeah, that's a good idea... it's really time some of all this skullduggery were dug up and simply put out in the open.
Wow never knew this happened. Very eye opening! Thanks you have lots of patience !
followed you :asalam maleko: :)
This is a really interesting analysis @paulag... apart from giving you many thumbs up simply for the sheer effort of the data scrape and information organization you also get resteem for that.
As a former data analyst and "keybreaker" type person something is massively obvious about your network chart and who's voting... those gray dots with black names are almost exclusively accounts that begin with "aa" or "ab," almost like those businesses in large cities that name themselves "A.A. Aardvark" services so as to be the first listing in the yellow pages.
A connected botnet? Owned and operated by the same person or group of persons as a "below the radar" co-op of some sort? WAY too off the charts to be "just a coincidence."
Whether it has any impact on what we're talking about here... that I can't say. But it definitely points to SOMEthing.
the grey dots are not really a big problem. I would 'assume' from the data that these are just voting bots...but many of they combined may be making a good reward curating.
I would be more concerned about the accounts that are voting for 3-4 of these authors time and time again and no one else....and also not posting....hmmmm that might be something.....but it would also be down voted......
I just have a suspicious nature... the person(s) who controls those accounts may actually control thousands of accounts... like dust, you don't notice it, but if enough grains of sand are poured into a bucket owned by a single person?
It'll be interesting to see what comes up... seems you and @dwinblood might be able to really help each other out on this one.
perhaps they exploit the very information which you have uncovered. and yes, obviously many of the names on the voter list are quite similar
If only the number of upvotes contributes to making the trending page, then it is sensible to make 20-25 bot accounts that each hold $1000 or so steem to upvote everything posted.
I'd like the system to reward engagement more than straight votes.
The incentives will drive people's actions; no conspiracy necessary.
It's not how well you write, it's who you know and what bonds you form.
True! :)
:P
Fact.
steemit sounds more like a mafia est. by the way you layed it where whales are like godfathers of powerful families rivaling each others for stakes while allegedly serving the community :D
Congratulations! This post has been upvoted from the communal account, @minnowsupport, by paulag from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, and someguy123. The goal is to help Steemit grow by supporting Minnows and creating a social network. Please find us in the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.
If you like what we're doing please upvote this comment so we can continue to build the community account that's supporting all members.
Hi @paulag! You have just received a 0.1 SBD tip from @strangerarray!
@tipU - send tips by writing tip! in the comment and get share in service profit :)
By upvoting this comment you support the service - thanks!
Great job, found this very interesting. One day I would like to be on the trending list 😊
This post deserves whale votes and should be on the top of trending page. Your BI skills is legit.
Thank you st3llar. I love data and BI so that makes these posts and analysis fun for me
Wow! Now that is some real analysis. I am happy you did this. I always wanted to know what it takes to be on the trending page.
I have had mild success on Reddit’s home page but I am still new here, and I am still learning how thing work here.
And I agree that there seems to be a lot of power that whales own and their votes are the deciding factor. It is however kinda strange to see the same names trending again and again.
Yes, you do sound like an investigative journalist.
Keep on working, and someday you will be a whale.
P.S: Check you my Giveaways Daily SBD Giveaway.
Thank you.
make friends with @joanaltres
Thank you. Now following you after realizing I enjoy everything you read. Your post about the Sherlock Holmes and block chain scam was awesome. Please follow me back
You are very new. Tim Cliff worked for more than 6 months before he got a Witness position. Your detemination and love for data will get you seen by more....
WOW, there is so much to learn. Thank you for the detailled insights. I confess, I need to read your post several times to understand all of it.
Imagine if you need to read it a few times, how long I spent looking at the data lol.
Glad you found the post of use
Yes, I really can imagine! Maybe in two or three years I can write similar posts with so many analysed data! Meanwhile I enjoy your posts!
Fascinating analysis as always. Thank you for the time you spend on these reports, they are always interesting to read.
This bit intrigues me :
Yes - this may be an issue for sure. I have seen similar patterns that I find 'curious'. But why risk a downvote battle.
I am sure the mechanics of getting on the trending page will be of increasing interest to potential corporate inhabitants of steemit as the numbers here grow.
Perhaps they are already here... Perhaps that doesn't matter.
Amazing how the "top" people on this network think that they can hide their cronyism.
But you know what... if it just becomes impossible for people to make real headway here because a few of the "in club" are keeping the gates firmly shut... all of the technology behind this is open source and can be forked. Golos did it. It will only be a matter of time before another decentralised crypto network will come along, perhaps one with fairer options for everyone.
This guy is giving it a shot.
(I'm not endorsing it, just sharing info)
Oh, very interesting! Just exactly what I predicted. Thanks for the heads up Matt!
I'm going to follow the development of this.
i like what i read ok going to golos, "this guy", following you and matt after i end this comment :). end of test.
Thank you @kingyus! I intend to try out Golos also. But it will be a challenge since I do not know Russian... yet.
Wishing you a good weekend.
wouldn't it be better to fix this one before a new one comes along
Can you fix it? Do you have the influence? Do you want to invest the time and energy into trying to change something that you have no immediate or direct control over.
I have participated in many online communities over the years. Give it a shot, you might succeed where others have failed. It requires a LOT of time and energy.
My artwork is my top priority. If Steemit works for me great, if not, I'll be looking for better options.
I will however make the most of Steemit and get to know people. I'll adapt to the circumstances. This is why I am interested in your Steemit research.
well, it likely took these folk quite some time\investment to have the privilege
Yes, some have.
If you go to any popular website, you will notice that the topics and people on trending pages change. On Steemit, rarely. Change is needed to keep the interest here.
indeed. but I do not fear some "glass ceiling" I feel there is plenty of room for advancement here with hard work and building relationships.
There are glass ceilings everywhere in this world, the internet is still a part of this world, because humans are behind it. The hardest glass ceilings to break are the ones we set for ourselves.
It took, the internet roughly 40 years of development to arrive at where it is today.
It took Facebook 13 years to arrive at where it is today.
It took Bitcoin eight years to arrive at where it is today.
Steemit has been around for only one year. So it is very early days, with much change and growth ahead of it.
Best bit is, you and I are here at the start of it. =)
Steem on @inquiringtimes!
Fuck. I can't even organize my closet. This level of analytics is unfathomable to me, and you probably do it for fun.
yep......I do enjoy this sort of stuff. Thanks for your comment, oh my closet aint that neat either lol
haha, yeah... it takes all kinds in life (-:
great analysis thanks for sharing ..upvoted resteemed and followed
Thank you kindly
you're most welcome
Thats a nice analysis, i too is observing for the meantime... some works some dont and taking notes of them
observation and keeping a record is good
Very interesting, thanks for sharing.
no problem.....load more interesting stuff to share
Thank you for this detailed report. Very useful information!
you are most welcome
Good job , and yea ........... you can get on trending as minnow .........
Good work! I've been trying to identify how the trending order is calculated (in the C++ code), but haven't quite manage to find it yet, so this is useful.
please do tag me in your findings
I will do :)
Nice work! What is TxVotes?
txvotes is a table in the sql database
I resteemed this due to the sheer effort you put into putting this together. I haven't had a chance to completely digest this and I personally never really worry about whether I am on the trending page or not. :) There is some other good data in there though that might mesh with some of my own current research.
Hi and thanks for the comment. Let me know if you need any assistance with your research, it looks very interesting and I would recommend people hop over and have a look
I just wrote a post detailing what I am doing. I am currently focusing on @bernie and his network, if you are so inclined feel free to look into @steemed and their network or @noganoo.
look at @elfspice's page "calibrae"
Hmmm... I'll keep an eye on that. Trying to determine if it is a fork of steem, or just a new fork of steemit.
Interesting analisys... although i always thought that its need just big payout, big number of upvotes, and a lot of comments... I dont see a mistery there... Also the faster you get big payout, the best the chance is to get on trending and hot...
Thanks for the great information!
tip! link
Cool analysis. Interesting to see that there is a bit of self interest going on with the voting. Social elitism is alive and well on Steemit. But such is inherent in the nature of man.
I would agree, this is the inherent nature of man. Glad you liked the analysis
I think your analyses are great, so I am very keen to see and read more of your posts! Steemit is a long game for most. It doesn't happen over night, but I think determination plays a central role in steemit......
Great article, thank you for sharing!
Powerful stuff and great analysis. I've only made it on the trending page a few times so far but doing so has a huge impact.
You saved me so much time researching this particular topic, @paulag. Thank you for the analysis of trending criteria. Keep producing content like this and you'll be a whale yourself before too long. Just takes one to notice, vote, resteem, etc...
I think I have found my calling now....i was doing posts on 'how to' with excel and power bi that didn't do too well. So I have pivoted from teaching to doing and I use these tools and my skills to do posts I enjoy just as much that suite the current Steemit audience.....
Good call. I'm working on doing that myself (discovering my forte as it were). A good gauge is always audience response. Inspiring.
Wow thanks for doing all this work! I'm a number nerd, too, but I wouldn't have a clue how to extract all this :)
Awesome analysis! You looked at the main trending page for your analysis, I would suggest doing a similar analysis on various tags and seeing where you come out. Also, add in some overlap to the promoted tab. I think there may be some consideration from promoting your posts as well!
Great analysis. I love data too, but this would have had my head spinning. I can see you put a lot of work into it. Resteemed, upvoted and following :D Thank you for this!
i never look at trending. i check hot lists for different tags sometimes, and maybe read something by a newer author, one less than 60 rep. i'm following 200 quality people now, so there's usually good stuff on my feed. i also check @ocd, @curie, and a few other curation projects for new people. ty
Thanks for sharing and doing this analysis! Really interesting!!
Really it's a great post . In a word it's a amazing post from all the sides.
thank you
It looks like one of the criterias to get in the "trending" page, is not to get upvoted by minnowsupport. :)
Minnow support wont get you on trending, but will help with the concept of social proof. Anyone with experience in growing a following online will get that!
Not going to lie. I had to skip down to the conclusion to get where you were going. I'm pretty sure Data Analysis sicked his dogs on my grandpa when they were growing up because I'm not allowed to go anywhere near it ^_^
But seriously, fantastic work and I'd love to know the rest of what you've learned
I find this data fascinating! thanks for compiling and sharing it! (and fun to see that I was on the trending page when I didn't even know it ;-)
Sure there you go now, a trending post!
That is a good post paulag. Well researched and insightful. It is indeed difficult for a minnow to get into the trending page and that might seem unfair. On the other hand, most Whales have been around for a long time and have been invested heavily in this community which speaks in favor of them getting a reward for that commitment.
I would agree that whales have either invested time or money to get where they are, and they will continue to do well as they have had time to grow their following. This means their posts get on the 'feed page' of more than most and are rewarded $$$
How witty you are! I really loved your analysis. It is your third post i am going to read. We should admit the people at the top of trending list are hardworking enough but they has not been kind to the people with low SP/ New users. Let's hope for good.
i think it's working! i can't remember what your post payout was, when i found this, but it's way up from that....
Keep digging! I love this information. When i first joined Steemit, I wanted a whale trending page, alongside with an actual trending......it's not Medium, I learned, but something else.
I love doing this stuff. Makes it easy
Amazing! I read this post gently. You informed me the best info. I am a new comer on steemit, and i am really happy to get a good and proper knowledge from yoir post. Thank you very much. Regard
ah thanks for the nice comment
Damn, Awesome job, very informative post..
Great info, thanks for sharing. Now following.