We've been working hard on improving our infrastructure over the last 6-7 months and many improvements have already been made. In early February we migrated to AWS and now run our entire stack with Elastic Beanstalk in docker containers. Everything is now infinitely scalable and has moved from a traditional best-efforts hosting solution to high availability infrastructure. Our traffic has multiplied at an exponential rate. But, there's still room for improvement.
Join us in testing these optimizations!
We'd like you to join us in helping test a new feature which will bring some great performance optimizations to steemit.com. In the interest of catching potential bugs before they get rolled out to production, we'd like to get as much real-world exposure to these changes as possible. You can now experience these changes live at https://steemitstage.com.
Keys
Our staging environment includes the same security measures taken for steemit.com and is considered pre-production, it's designed to be as close to the 'real' steemit.com as possible in order to fully vet new code before it makes it to steemit.com. If you're still worried about using your keys, you're welcome to just use your posting key. Even if you don't login, just browsing the site without logging in and providing feedback to us is still useful.
What's changed?
We're implementing a new custom designed proxy and caching layer in front of steemd, which is the back end blockchain service that the front-end website talks to in order to perform all actions on steemit.com. Most read-only requests to steemd will now be cached in an elasticache/redis database and served directly from it instead of having to communicate with a steemd cluster for requests that are already cached. This will most likely improve performance of general site usage dramatically.
In addition, historically websocket technology has been used by the front-end in order to speak to steemd. Websockets certainly have their advantages over traditional HTTP, but when it comes to large-scale infrastructure it's much more difficult to properly load balance and distribute active connections. Further, traditional HTTP/JSONRPC calls are more common and generally easier for 3rd party developers to use and develop with. Because of these reasons, our proxy/caching layer will still talk to steemd using websockets but everything else (the front-end plus new services being developed) will talk to the proxy/caching layer now using HTTP/JSONRPC. In order to make this possible some new updates were made to the steem-js javascript library to be able to utilize HTTP/JSONRPC in condenser (front-end app that is steemit.com).
If you're still curious, since our code is open source, you're always welcome to look over it yourselves. The proxy/caching layer is called 'jussi' and is available at https://github.com/steemit/jussi. Steem-js is at https://github.com/steemit/steem-js and the front end (steemit.com) is at https://github.com/steemit/condenser.
Feedback
Please leave any feedback about your experience using steemitstage.com on this thread. Visually, the site will look identical - these are changes that are invisible to you visually, but you should feel an improvement in speed while browsing around and utilizing the site. If anything is broken, please let us know. You're also welcome to reach out to me (@justinw) on steemit.chat if you'd like.
Justin Welch (@justinw - Steemit DevOps Engineer)
disclaimer: steemitstage.com is not setup for high availability and could be broken or completely down at times
Sounds awesome. I'm actually amazed it took y'all so long to get a scalable AWS setup. That is one of the first things I would have done ;). Caching will definitely help website performance. I'd be curious to know what kind of specs the caching server has.
It is always cool to see the development and progression of open source projects such as Steem. Keep up the good work.
This type of transparency is excellent.
Is there any benefit for users to be testers on steemitstage.com? Maybe some kind of acknowlegement would get you more testers.
Check out the performance tools in the Firefox Developer Edition. They are very useful.
Thanks! I'm not sure what you mean by 'benefit' and 'acknowledgement' in this context. Can you elaborate?
Sure. I mean would the the users of steemitstage.com earn rewards or be acknowledged in another way for reporting issues, bugs, suggestions, etc. It could be as simple as listing the contributors?
benefit is blessing the platform we love, and advance knowledge of new features before they go live.... it sounds pretty beneficial
That's the spirit! :-)
Just trying it out right now, and the difference in speed is DEFINITELY noticeable! All the basics I tried out so far are working good, so... fingers crossed!
okay, I commented on a recent article, and the comment did go through... but the page kinda hanged without updating itself when the comment got posted.
Thanks for the feedback - I haven't experienced any lag in comments yet. Has it happened again?
just made a new comment, and it went up smoothly
Seems way faster, but I had a problem replying to a comment as well. "This page doesn't exist" was the response.
Thanks for communicating this and inviting testers. Great idea!
All this HF and optimizing will pay off in the long run .. Steem to the MOON
well done ur trying to ahead of performance issues. I will be blogging and doing my own testing to help out. I am a 15 years software developer in test. Thank you Thank
Thanks for the information sharing. I started some developments on current api, but these new developments will enhance further. I also use cloud in MS Azure. Now I'm using json-rpc api to scan changes, but block processing will make my solution much faster.
Sounds awesome! I think my only concern is you guys becoming too dependent on some of AWS's proprietary features, although I'm sure you've considered everything before making the move. When I joined this site, I thought it wouldnt have a lot of growth, seeing as it is a niche inside a niche, but I was pleasantly surprised when I saw news of 250000 users within months of 200k. For that kind of growth, good infrastructure is doubly important, so I wish you luck in maintaining and developing the site!
Fear not; we don't rely on any proprietary features of AWS, and we don't intend to. :)
The setup process for condenser is tedious and the vagrant setup is not working. I think if You can fix it more developers will start spending time on the project. I would like to improve the follower page to begin with
I submitted a PR today to get the README up-to-date. It includes instructions for using docker to both build and run condenser, in addition to corrections for building/running it the 'traditional' way. The
Vagrantfileshould be easily modifiable to mimic theDockerfile. https://github.com/steemit/condenser/blob/readmeupdate/README.mdAwesome. Good to know. I will try it out again this week.
Nice post!
Sure
@steemitdev I have been trying to open https://steemitstage.com/ without success. Is it offline now?
I would glad to help you, but then I try to open a page https://steemitstage.com/ I can see only HTTP error 503
Thank you. I shall be using steemitstage for the next few days.
Bug #1 - Voting from the feed view, the vote goes through, but the upvote icon doesn't turn blue. Also happens sometimes in post view.
Feels much more responsive overall.
Yes I have similar experience, the icon upvote does not trun blue directly :)
Hi @steemitdev! You have just received a 0.12 SBD tipfrom @cardboard. Noice!
@tipU - super easy steemit tips and sharing profit. Click here if you wish to learn more.
By upvoting this comment you support the service - thanks!
I'm so very glad to see these updates. Thank you and please keep them coming!
Edit: Browsed the site, did a Rewards claim, posted this comment... seems to be working nicely so far!
Edit2: Seems my first edit just kept spinning and never finished:
Edit3:
I got the spinning comment thing again (the comment does post, but the UI doesn't update) and was able to get the error from the console this time:
Would you like us to create github issues for things like this?
Thanks for the feedback Luke. Yes, I've been able to duplicate the sporadic issue and we'll troubleshoot to resolve it. Let's reserve github issues for problems that are already integrated into steemit.com. Reporting things here for problems with the staging site is completely sufficient.
I'm going to publicly disagree with you. :)
Bugs on staging condenser are absolutely eligible for issues:
https://github.com/steemit/condenser/issues
Please include three things:
exact steps to reproduce
what you expected to happen
what actually happened
(optional) screenshots, supporting explanations, theories, et c. :)
Thank you. That was going to be my next question as I see there are quite a few Github issues for the current site as well. Good luck to you all! Building/maintaining two different front end infrastructures as you complete this migration is no easy task.
Indeed - building and maintaining a completely unique blockchain-based decentralized social media site certainly has it's challenges :)
I can second this. Also, after it's submitted and done, the edit box still shows up on every comment.
I'm just going to leave this right here:
https://steemit.com/steemit/@eveuncovered/what-needs-to-happen-in-order-for-steemit-to-go-viral
Couldn't open the web page. Giving me an error:
This is what I keep getting also when I try to navigate to any tabs or pages. Going to try to login first and see if that fixes it.
What are the steps to reproduce that?
This will really work out fine. Thanks for sharing.
Interesting, especially when my name is Jussi. Anyway I learning day by day.
I have signed up for this. My code didn't work and no new emails. Hope I know what I am doing.
Great work Justin! I can't wait to poke around the test site!
Did a quick signup and while it seems more efficient than the current setup, the confirmation email gives a non-existent confirmation code. The phone confirmation works fine, though. All-in-all, it looks great and is much more usable.
Excellent - thanks for the feedback. It's not actually intended to test real signups, which is why it would be producing an incorrect code.
well that makes sense :)
Great job guys. This is definitely important for the preparation of HF 0.20.0.
Neat!
Great idea and good to get some infos about the upcoming changes here :) Are the changes to the front-end (condenser) also merged there? That would be awesome and I would definitely check some things out to help!
Yes and no - the staging environment is generally considered pre-production, so features that are really close to being rolled out to steemit.com will often be seen there. Newer features that are not yet part of the main code base are often tested in our development environment at steemitdev.com, but these features may cycle out multiple times on a daily basis due to rapid development and testing.
Disclaimer: steemitdev.com will frequently be broken, or could potentially have vulnerabilities as the code has not yet been reviewed and validated. If you do use steemitdev.com, only use your posting keys there
Thanks for the input! steemitdev.com seems to be interesting too :) Will take a look.
Please don't use steemitdev like you would staging or prod. We frequently deploy work-in-progress, half-tested, broken, unaudited code there. You are strongly advised to use posting keys only on steemitdev.
Thanks for the hint! I'm just playing around with the upcoming UI changes. Won't use my keys there.
Please make sure you include a disclaimer about steemitdev.com: it will frequently be broken, or could potentially have vulnerabilities as the code has not yet been reviewed and validated.
It's a dangerous sandbox, an active workroom - use only your posting key on steemitdev.
Hey there, I have been I am about to write a serious of Post and GitHub commits to beef up the testing of code.
Interesting - I look forward to seeing this.
When creating a new account my confirmation code in the email was not found.
Posting comment is stuck spining on new site.
But the comment actually got posted after refresh page.
I will try it for couple days :)
Thanks for the updates :) tip! noad
Now following @steemitdev...
A couple of questions:
I inadvertently attempted to log in to steemitstage using my active key, and did get a security warning...
Have I compromised my active key? Should I be re-keying my wallet now?
After then logging in with my posting key, the first thing I looked for was the ability to edit my posts a week old. I did not find that.
Where, if anywhere, is that promised function on Steemit's list of functions to implement?
Finally, the interface does seem noticeably faster and crisper in response... However, it that simply a matter of being very lightly loaded presently, or does that responsiveness correspond to real UI improvements?
Thanks in advance for your responses, and thank you for the ongoing work making Steemit a good place to create content...
😄😇😄
First, thank you for helping us beta test!
Keys: we take the same security precautions on steemitstage.com that are in force on steemit.com. We don't store users keys anywhere, ever. The attempt to login with your active key on steemit.com would lead to the same security message you experienced on steemitstage.com.
As far as the interface being faster, this is mostly a result of the new technology in place that we're implementing. A low server load will also cause a bit more responsiveness, but keep in mind we also utilize auto-scaling technology on steemit.com and at times of heavy load additional instances automatically become available - so instances being bogged down shouldn't be extremely common. This technology will make those instances even less bogged down and we'll be able to support even more users with less (or the same amount) of infrastructure.
As far as the ability to edit posts more than a week old - yes this is an item on the official 2017 roadmap and as far as I know we haven't made any plans to not move forward with that piece of the roadmap. With that being said, it's also a change that would require a hard fork of the blockchain and is not a simple front-end interface change. Most likely this will (if witnesses and the community agree) make it into a future hard fork. Just because something has not been implemented yet does not mean it's not going to happen, or that we've forgotten about it - we just need to first prioritize what's necessary for steemit to scale to millions of users.
The biggest priorities right now are scalability (both of infrastructure/software and creation of user accounts), a mobile app, and communities. Development of these items is rapidly happening across multiple teams at Steemit.
Hello, Justin,
Thank you very much for your comprehensive response to my comments and concerns.
I'm glad to hear that the newly brisk response on steemitstage.com is primarily due to improvements in the software technology. I will continue to experiment with the new setup and pass along any additional observations and questions as they occur.
While I understand and appreciate the need for prioritizing resources, virtually all content creators with whom I have discussed the need for perpetual edit access are on the same page. As authors, we really need the ability to access our own work for correction and update purposes. I've written extensively about the necessity, and perhaps most comprehensively in this article:
Your comment that this change would require a hard fork is somewhat surprising. I'm not disputing your remark. The reason for my surprise is that all other sources (prior to yourself) have indicated that the necessary changes were already implemented in recent hard forks.
As a life-long coder myself, it seems quite mysterious to me that enabling post editing would be at all difficult. Editing used to be allowed for a month, and has now been restricted to a week. The fact that it is enabled at all and that the time window was changed once seems to strongly imply that changing the time window again ought to be almost trivial?
I will admit that my coding experience has not recently included network programming, and that I have as yet to try my hand at interfacing to a blockchain, so please do not mistake my questions as in any way being arrogant. I am just curious.
In any case, I'll close with a request. I would tremendously appreciate it if you could use your influence to make perpetual editing available at the earliest opportunity.
Thank you again, sincerely, for your kind response.
My apologies, I did some digging and it looks like the feature is actually listed in the changelog for 0.18.3. I'm not a blockchain developer so it's sometimes hard to keep up with individual changes to the back end (steemd). It was designed to require an active key to prevent defacement of accounts who's posting keys may have been compromised. So, this could be implemented in the UI but may require a little thought as to how to implement it with best user experience in mind.
No apologies necessary, my friend! Very kind of you to do a little digging! Thank you.
I care not a whit for "best user experience" at this point... I would gladly accept any means of access, no matter how byzantine.
And so, please correct me if I'm wrong, but:
If I am understanding you, it would be possible right now (given the right tools and documentation) to write a standalone utility that would give me the ability to edit any or all of my old posts, provided that I supply my active key as part of the process?
Yes.
There are several blockchain features that condenser could but does not yet support: things like escrow, delegating your Steem Power to someone else, powering down to a different account, powering down less than 100% of your steem power (we're close to releasing this one), editing old posts, et c.
Some of these will make it in, some won't; we're working on several huge projects right now and have to choose our battles for the moment as we scale up the team.
Thanks for commenting and reaffirming that this ability already exists, apart from a necessary UI tweak. And, it is a tweak, isn't it? Or is there some hidden complexity that makes this change way harder than I imagine it could be?
The extremely frustrating thing for me is that the promise of edit access has been dangled in front of me for six months now, but still remains out of reach. :O
My friends behind the curtain: if there's any way to work post editability into the mix sooner, rather than later, you'd have at least one wildly cheering fan out here, thanking you profusely.
FANTASTIC! Lets get this party started!
The loading time and speed of the comments loading is much faster.
@steemitdev
Hello,
Excuse me, I have a question, would you please answer it?
Can someone have 2 different accounts on steemit? One for English and another for Mother tongue!
Is this legal?
You can have as many accounts as you like.
Great work! Looks good.
A couple of observations from today:
I am logged in to steemitstage.com using my "posting" key.
Today I tried to leave a comment below an article, and received this error message:
"Missing Posting Authority creatr"
Is this correct behavior?
Also, when I have tried to up-vote articles and comments, the little circled (^) symbol does not turn blue until I refresh the page.
And now, trying to Post this Reply, I get this error:
"Transaction broadcast error."
I will cut/paste this using the standard UI so you get the message... ;)
UPDATE I tried again, and this posted... Now trying an edit...
These are good reports. Can you reliably reproduce the issues?
Unfortunately, these seemed to be intermittent yesterday, and then steemitstage.com simply stopped working at all...
At this moment I can't access it either via Safari or Chrome... :(
I'm new, but if there is anything I can to help I surely will!
Looks nice!
Is it possible to make "Search" in future not basing on google but on some inner algorithm?
Yes, this is most certainly something on the list. We've actually already built a blockchain indexing service to query for such data (sbds).
Great to hear that! Thanks!
Worthy cause ! Lets :)
Theres even a vacant Test catagory in the tags
I have no idea what any of that means, but I am happy there are some really smart people behind Steemit :)
Me too.
nice post, thanks
That is great things and sure i will check that site and use to help steemit community. You are doing great work for community and we can help by using and checking it be we all are one family of steemit and we would like to see important in site. If something happens good wed
Awesome!
Thanks for the information!
@benjojo i think this helps matters. Perhaps, some answers are here!
Without any reasonable doubt , together we will achieve greatness , its just amazing hpw things in the virtual world are moving fast.... great work @steemit
Day by day i just get amazing posts and see cool stuffs here. Am so loving it here