As you already know, steemit.com has been under a DDoS attack yesterday. Now the dusts gently settles over the events, but it has been a convoluted day. I ended up using busy.org to post, or even my own "shell", or code written for another project.
What brought my attention, though, this morning, was a sentence from @steemitblog DDoS announcement. Here it is:
The site has been getting requests on the order of a hundred thousand per second from someone using a botnet spanning throughout dozens of countries.
Hmmmm... Let's try some math here.
Steemit Traffic Capacitiy
Right now, steemit.com has a daily volume of about 35,000 people. At this volume, I expect the traffic to be around hundreds per second, or probably thousands, during peak time. Assets are probably loaded from other sites, so what the Node.js app serves (I know from GitHub that steemit.com is a React + Node.js package) is just the content shell.
I don't know the setup, but for the good of the Steemit devs, I hope they're using some sort of load balancer. They have Node packages for that, or they may be using nginx (the latter approach is even better, as it allows to serve static files - and one of these static files could have been a "we're down and we're working on it" kind of announcement).
Under these circumstances, my humble opinion (I only do web development for about 20 years) is that a traffic of hundreds of thousands of requests shouldn't be a problem. I know, there are requests and requests and the attacker most likely played with the keep_alive and other parameters of the request, intentionally piling up processes. But even in this case, the problem can be mitigated very quickly by adding more machines on the fly (I remember an announcement telling they're using Amazon hosting specifically for scaling).
Smart Media Tokens Impact
Now imagine that SMT really takes off. I don't imagine thousands of tokens and communities. But even with just a few dozens of active communities, the traffic will increase logarithmically. It will most likely go close to "hundreds of thousands of requests, spanning from dozens of countries". That's usually a success metric, you know: how many people are using your product. It is a good thing to have. And it's a must have skill to be able to handle that amount of traffic.
I know for sure that these growth pains are unavoidable. I remember very clearly the Twitter whale in the early days and how frustrating that was. But in the end, they make it work. The technology is mind boggling, but they made it.
I think for Steemit the challenge is even bigger, because there are also the content nodes, those powered by the blockchain and managed by witnesses. During the attack, some of the nodes were down too (that's why steem.supply stoped working). They might have shut them down on purpose, as part of their attack mitigation procedure, or - that's my hunch - they were simply flooded.
One More SMT Caveat
So, trying to end this on a positive note: if you plan to launch your own SMT, please take into account the fact that you should have your own infrastructure and you will need serious system administration skills on your team.
I'm a serial entrepreneur, blogger and ultrarunner. You can find me mainly on my blog at Dragos Roua where I write about productivity, business, relationships and running. Here on Steemit you may stay updated by following me @dragosroua.
https://steemit.com/~witnesses
If you're new to Steemit, you may find these articles relevant (that's also part of my witness activity to support new members of the platform):
The whole point of SMTs (as far as I understand) is to use them on sites other than SteemIt.com. SteemIt is really just the first proof of concept site for the Steem blockchain. If it remains the primary front-end for Steem then I think Steem and SMTs will have failed.
If each SMT has their own front-end site, then sure - one or a few of them could be DDOSed, but it wouldn't be all that big of a deal since there will ideally be so many different sites running on Steem.
Overall I think the fact that the majority of people still use SteemIt.com as their only interface to Steem (and that many people don't know the difference between the two) is one of the biggest problems facing the platform right now. The DDOS the other day may have been a very good thing in that it spurred so many posts and so much learning about how the system works and that SteemIt.com is only a front-end website.
Yes, and I agree that it had an unexpectedly good result, by the amount of the educational stuff generated.
I think STEEMIT will be just fine and even though there were issues with the site last night @dragosrousa I was not overly concerned or worried. I watched @jerrybanfield and his posting on how to access your STEEMIT account when things like the DDoS attack happens and I felt very confident in the other ways to access my account..............
It will be fine, if we all do our part to build it. Optimism is great but only when paired with pragmatism - and in this case, the DDoS on Friday was a clear sign that Steemit Inc needs to learn from this experience and develop new protocols to handle high-traffic situations and malicious actors.
A simple 404 page, much like the Twitter whale, would be a great start.
Just trying to remember, but didn't steemit add new computers about 2 months ago to give more capacity and additional cloud space as well?
Thanks for posting this! I was wondering why I kept having issues accessing steemit yesterday. Steemd was also down for a good part of the day.
We need to keep in mind the distinction between steem (the blockchain) and steemit.com (the website/frontend).
A DDoS attack on steemit.com can be handled using already existing tools, and also you're still be able to use busy.org, chainbb, esteem or whatever frontend appears in the future.
SMT does not run on steemit.com, but on the blockchain. You would still build your own frontend where the new coin adds functionality.
Steemit.com would (probably) show the new entries as a post, but I'm not yet sure how that works. I could even see an SMT without any activity showing up on steemit.com.
This means that even an increase in activity because of new SMT's would not be a real big issue, since the blockchain is supposed to able to handle all that traffic.
Here is where it gets interesting. A DDoS attack on steemit.com would not interfere with SMT's, the SMT would not be bothered about the fact steemit.com is under a DDoS attack.
Where it gets hairy is a DDoS attack on the nodes/witnesses. When they go down, everything will go down. Attacking the backbone is the most dangerous situation we face.
That's not compulsory. It's advisable, but not compulsory. We don't know yet how many SMT-powered websites will run on separated websites. The cheaper approach is to start it where you already have a savvy and experienced community, so probably many will be started here, on steemit.com.
I'll dig into this matter a bit more. If you're right then I see no added value at all for SMT 's that use steemit.com as their frontend, unless you're able to also adopt the look-and-feel of steemit.com for your SMT-based app.
Why dont witnesses have to tun a mirror of the website?
It seems steemit.com is suffering from centralization issues, no?
A little bit, yes.
Well smt traffic and steemit website will be two different types of traffic as steemit is website related and smt is blockchain related, so the two are mostly independent right?
Great advice at the end though regardless. In order for smts to be taken serious they need to be professionally released.
We don't know that yet. My hunch is that steemit.com will continue to show all the content saved to it and the wallet will contain all the potential SMTs. So a drastic increase in traffic can come from any part: the separated sites of the SMT-powered community, or Steemit.com (via search engines, etc).
Ah thanks for the information!
So in layman's terms you are saying steemit needs to be able to handle a lot more traffic?
yes
I dont think there will be a few dozens SMT's, not at all... i think there will be just few and the reason is that to complicated to create your own token and just few have the skills to do it.
But i do think that SMT's will increase in time the value of Steem!😉
I really hope this kind of attacks will stop in the future. Have a great weekend Dragos!
Did you read the White Paper or any post about it? it is not complicated at all ...
Yes i have read the whitepaper! All of it! And at least for me it's not that easy. I understand their use and their concept, how they will be used and so on but to create one and take care of it i think its just for some of the users (at least for the moment)!
Or maybe i am really stupid and i dont get it!😂😂😂
To create one you have a box with some options, name, quantity, do u want this or that, nothing more. To be honest first time was scary, I also told to myself, "i do not understand this" but, I gave it a second try. There is a lot of info explaining the theories but to create a SMT, just that, is very simple operation.
As I told u before, it is nor complicated at all and from ur answer I can realize you ar not stupid at all ;) give it a second chanse to the WP and you will see. You can also read the series from @dragonroua about it, very easy to understand :) First chapter here.
They will increase the value of STEEM, but the contradiction is that if they do work, they will increase the traffic logarithmically and that will create scaling issues.
I can imagine that. I just hope that the platform will be ready to handle much more traffic as if not it will be a total drama around!
@dragosroua I just saw this comment in other post... It is right? Just a question because I know you are a witness
And here there is a post that was downvote about it ... I am juts looking for an expert opinion.
https://steemit.com/steemit/@sircork/steemit-is-undergoing-a-ddos-service-denial-attack-but-your-wallet-is-safe
I don't know anything about it. I don't follow the Condenser development very closely, but from what I know about the codebase, there isn't such a "feature" worked on. So I highly doubt it was an "inside" problem.
Good to know. I got a bit scare about it ...
I have wondered how steemit will scale. Running a site can't be cheap. Are they making enough to cover their costs?
Its a progessionnel diagnostic , they may attack exchanges like bittrex or poloniex etc
What do you mean ?
Sir , nicely written
Thanks for giving updates
Yesterday we were very curious
Thanks for the information sir.
Great read , Dragos , thanks !
This is exactly my concern. They are planning to land aircrafts on a road, which is not ready. First the basic infrastructure has to be strengthened and then only SMT should be worked upon.
Thank you so much for continuing to publish this important information.
upvoted
Thanks for the info. I had no idea what was happening yesterday.
While I regard DDoS as negatives for the web, I think this one was good for Steemers, and even for Steemit, as you point out it shows what needs to be done to grow.
My fear, particularly if SMTs sell like hotcakes, is that Steemit will just (continue to) be neglected to death. I have noted many user suggestions that seem to have not even been noticed by Stinc, and I don't think that's a good thing. Even bad ideas should be addressed, so that people that had bad ideas can find out why they were bad, and maybe come up with good ones.
Overall, my worst concern at the moment is the vulnerability of the witnesses to simply being replaced in a VP based Sybil attack, where Zuckerberg or someone just buys a bunch of Steem and votes in corrupted witnesses, and takes over Steemit.
The only way I can think of to mitigate this vector is to weight VP with something besides SP, or nothing at all. Should SMTs prove a valuable business, the incentive to takeover Steemit will grow stronger, thus increasing the danger of that happening.