Finally! Yesterday @sjennon and I launched SPECTACLES, the Follower Tracking App we've been working very hard on for the past few weeks. It's far from finished, but it's a good start and we've already received a lot of positive feedback as well as many great suggestions.
Give it a try yourself on www.steemspectacles.com or read our introduction post.
Now that it's finally public, I figured it would be nice to also open up the source code for inspection and contributions. I love the idea of Open Source projects, and I'll warmly welcome anyone who wants to submit a change, no matter how trivial.
The project can be found here: https://github.com/stephanmullerNL/spectacles
The code
First of all, while have a couple of years experience as a web developer there's still a lot of things that I don't know or have little experience in. The stack used in Spectacles (Angular 4, Bootstrap and SteemJS) is quite new to me. If you see me make a mistake or think something could be improved, please let me know! I'm always looking to learn more :)
Angular 4
After working with AngularJS (up to 1.6) it was fun to finally explore the newer Angular. It took some getting used to, but I'm quite enjoying it so far. It has some interesting improvements, and although technically separate from Angular (but hard to see the two apart) I found it very nice to use TypeScript as well.
I completely neglected to add unit tests and e2e tests, mainly because I knew I was going to refactor everything very often but also because I'm lazy. I know that if I would like people to contribute I should at least make sure the basics are properly tested. Soon, hopefully 😬.
SteemJS
To fetch data from the Steem blockchain I use steemjs. It's the only complete API in JavaScript that can give me all the info I need, but I have to say it still leaves a lot to be desired. Its documentation is quite lacking, and the way the different kinds of data need to be requested are weirdly distributed.
Loading http://www.steemspectacles.com/@ned takes 157 websocket frames, which takes some time to load.
As a random example, to get the reputation of someone's followers you first need to fetch the user's follower names and then make a separate call to get the profiles of those users. To calculate their SP you need to do a separate call to get Steemit's global variables as well. This is pretty inefficient. For now I'll just have to make do, but it makes the app pretty slow sometimes.
Bootstrap 4
I chose Bootstrap because it enabled me to get a layout working pretty fast. I'm still not sure if I like it. Sometimes I feel like it takes more work configuring Bootstrap to look the way I want than it would take me to write it myself. For now I'll just stick to it though.
The Challenges
The main challenge I encountered was fetching the data from Steem using SteemJS. This challenge was twofold:
- Figuring out how to use SteemJS, which endpoints it provides and what data they return
- Loading a lot of data without making the app feel extremely slow
For the first challenge I found a lot of help on Steemit.chat, in the #dev and #steemjs channels. Thanks, everyone who answered my many questions! A special thanks goes to @jfollas for his excellent tutorials. You're the man 👊.
The second challenge was an interesting one. Take for example the app's dashboard which requires the following API calls:
- Your user profile
- Your follower/following count
- The user profiles for all your followers (could be thousands in case of whales)
- Your latest 100 posts and comments
- All the replies to your latest posts and comments (could be hundreds easily)
To make sure the app didn't feel too slow, I fetch some of the data before resolving a page (mainly your user profile and follower count). Then I render the page and start fetching all your posts and comments as well as all the names of your followers. Once those are in I can start fetching all the replies to your posts and comments, and the user profiles of all your followers. Those last bits are what I need to calculate the full statistics.
Promiseception aka room for improvement.
Because of the dependencies between calls I do some things in promises, but for the final calls I use Observables. That way I can already render the dashboard and show loading icons, and when the results are finally in it inserts them into the page. I'm pretty sure I can also improve the performance of rendering all the data onto the screen, but I haven't yet had the time to look into that.
In short, right now it all feels like a big mess. Again, if anyone knows a better way please educate me!
Next up
Spectacles is far from done! Here's some of the things that we want to add:
- Stats about the people you follow
- Stats about people who don't follow you but still interact wit you
- Stats about when people started (or stopped) following you
However, for some of those the SteemJS API is not sufficient. So one of the things I'll be working on in the near future is a new API. That might sound ambitious, but I think I should be able to make a basic, read-only API that gives a lot more useful information. Stay tuned for updates on that!
Contributing
Again, feel free to fork Spectacles and create pull requests. If they're going to be big maybe it's better to shoot me a message first, but I'm open to all kinds of contributions! I'll make sure to make the project easier to work on, by creating a better folder structure, adding documentation and tests and a detailed contributions.md file.
Questions?
I can be reached here in the comments, on steemit.chat in #spectacles (or through private messages) or if you want on [email protected].
Stay tuned for more updates on Spectacles itself and on the development process!
Thanks for this one, ^_^
Amazing, Excited to use this tool.
Let's work together as open source.
Thanks to @sjennon and @pilcrow and special thanks to @firepower for letting us know through his post.
Keep up the great work.
Yeah, it was awesome of @firepower to promote us that way. Thanks for your comment!
@pilcrow Amazing would love to see "Recent Followers" for last 24-48 hrs or like last 7 days
and "Recent Unfollowers".
Unfortunately that's near impossible to do with the current API. The only site I know that does that is steemdb and they use a custom database for it. We really want to add that but it will take a lot of work. I'll let you know when I figured it out! Thanks for the feedback :)
Very nice!
Ghost Followers are those who didn't "engage", does that cover voting as well or just commenting, I wonder?
Yes, it covers both voting and replying on your last 100 posts and comments.
Good job mate. I'll keep my eye on this one.
Thanks!
Thank you a lot! I used it already and it works just fine!
This is great! Looks good as well and so very simple! Checking it out it immediately gives the urge to give back you have received. Also, ranking your followers by loyalty and influence is awesome, that can really help improving the impact you have.
Great work in developing this, I'm certainly going to use it as I'm on the road to becoming a little bigger of a minnow :)
Upvoted!
Good job @sjennon & @pilcrow ! Spectacles is a really promising project. I just began to use it today, what a good interface ! I will continue to follow it, you have all my encouragements (Okay, that's not a big deal but... 😊)
Thank you, really glad to hear our little project is appreciated so much :)
Thank you so much! We appreciate the support <3
upvoted, and followed. This tool is awesome. Thanks so much for putting this together. I look forward to seeing what improvements you make to this tool. Love it, Love it, Love it!
Cheers! @rebele93
Thank you for the kind words! 😊
I just learned about this today and I'm already using it. It was really funny to see that my fellow expat friend in Panama is my top follower. :) I know you guys worked hard to create it. I can see how it is useful to the community! Congrats on getting it up and going! Looking forward to using it a lot in the future!
Thank you very much! Great tool!
Spent some time on the website and I've got to say it's one of my favorites so far ! Really good looking and straight to the point. It's kinda scary seeing how many of my followers are ghosts/dead but it helps to bring me back to reality :P
You guys did an excellent job ! Congratulations !
Awesome, thanks so much for your kind words!
thanks man, i might have to blog about this, new follower / peace !
This is cool, especially you sharing this info.
I'm not a JS dev, at least not proficient, but I do understand where your challenges lay.
Good work, I'll keep following you
Glad to hear my post is appreciated by someone who isn't a hardcore JS developer too :)
I've no clue what development mode in angular really does... but I'm pretty sure you should turn it off. :P I would bet that prod mode is faster or more stable or whatever... Also there's a console log still in the code. But hey... no project without accidentally commited debug code! :D
One advice in terms of UI: I would add some sort of loading animation to the username input as it takes some time (as you mentioned) for big users.
All in all... Great job!!!
Oh and.... Thanks for being my most loyal follower! ;)
Whoops, I guess I uploaded the wrong version to the production server. Thanks!
And yeah, a spinner on the search bar is a good idea. I think I'll add that tomorrow.
Congratulations! This post has been upvoted from the communal account, @minnowsupport, by Phtephan from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, someguy123, neoxian, followbtcnews/crimsonclad, and netuoso. The goal is to help Steemit grow by supporting Minnows and creating a social network. Please find us in the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.
This post has received a 0.52 % upvote from @drotto thanks to: @banjo.
Hi, Is it down? Thanks