So you want to play with Steem?
Let's prepare a proper environment for our playground.
First, the hardware.
If you’ve read the previous episode of Steem Pressure, you know that Steem is growing (and that’s awesome!). Today, we’ll choose proper hardware to run our nodes.
Video created for Steem Pressure series.
Network
A low-latency network with at least 100Mbps bandwidth in both directions is a recommended minimum, even if you are running “just” a seed node.
Please note that it is not enough to have a fast network card or an ISP that claims to provide 1Gbps connection. You need a decent network connectivity across the globe.
CPU
Virtually any modern, x86-64 CPU with at least 2 cores will be ok.
Currently, you will not benefit from multiple cores as much as from a faster single-core performance.
Most likely a CPU with 4 cores at 3GHz will perform better than a CPU with 8 cores at 2GHz.
RAM
A good choice for a consensus node is 24-32GB of RAM.
For a full node, it is 128-256GB of RAM.
Of course, if you can afford to put everything in RAM, just do it.
Storage backend
Size
Currently, we need over 65GB of data to hold the block_log file itself.
Additionally, you’ll need over 25GB to keep shared_memory.bin for the simplest node, up to 260GB for a fully featured API node with all the bells and whistles.
So I would say for now, it’s 100GB for a consensus node and 400GB for a full node.
That’s true at the time I’m writing this post, so by the time you read it, these parameters may no longer be enough.
Performance
Your storage backend needs to be not only large enough to hold your data but also fast enough to deliver data on time. If your node runs out of RAM and starts using swap (in case of tmpfs setup) or if it runs out of buffers/cache, the workload will put a lot of strain on your storage. You need to make sure that it can handle a lot of random reads and writes with minimum latency. Low latency is the key. That renders most of HDD setups unusable.
Trivial benchmark
Run dd if=/dev/zero of=tst.tmp bs=4k count=10k oflag=dsync in a directory where you are going to keep your shared_memory.bin file.
It will be either the value of shared-file-dir, or if undefined, the value of data-dir (-d)
The result will be a very rough guesstimate on how well your storage backend will be able to handle workload caused by steemd accessing its shared memory file.
Is your storage backend fast enough?
Check how long it will take to complete that command. Less is better.
- Less than 1 second is perfect.
- Up to 10 seconds is good.
- Up to 30 seconds is so-so.
- Up to 60 seconds is uh-oh.
- Anything more is a no-no.
Of course, the answer is not definitive, accurate, or even always true ;-)
Even if it is a minute or two, you still can play with your steemd toy without a purpose and waste your time.
Or maybe you should do something more productive and use your disks to play the “Imperial March” instead?
(Shoutout to @lordvader)
Oh, and by the way:
Blindly copying & pasting snippets is dangerous.
DO NOT EVER RUN ANYTHING YOU DON’T FULLY UNDERSTAND
So what does that command do?
It writes zeros to a temporary file in 4096-byte blocks. It does that ten thousand times, each time synchronizing I/O to get caching out of the equation.
Usually your drive, even if it is an old HDD, can write large chunks of data in bursts very effectively, but that’s not what will be happening with your shared_memory.bin file.
That command writes only 40MB of data.
Shared memory file for a consensus node is 600x bigger than that.
Oh, and that’s not all.
The shared memory file for a full API node is 10x bigger than the one for the consensus node.
Dedicated or virtual?
It doesn’t matter, but keep in mind that high quality does.
With VPS it’s more tricky because it’s usually hard to tell what real performance you can expect, how resources are shared, how much baseline resources are guaranteed, if the “SSD” drives are local or network attached, what your I/O limits are, etc.
What’s next?
In the next episode, I will show you step by step how to set up a machine from scratch for a basic consensus node, how to configure it, and how it performs in the real battlefield environment.
If you believe I can be of value to Steem, please vote for me (gtg) as a witness on Steemit's Witnesses List or set (gtg) as a proxy that will vote for witnesses for you.
Your vote does matter!
You can contact me directly on steemit.chat, as Gandalf
Steem On
I remember when I first got my interest in Steemit witnesses. Not running one myself though, as it was something completely outside of my knowledge and skills. I noticed how quite many witnesses at least at that time seemed to have low end hardware for witnesses.
In general, I think people who try to run Steemit witness with as little as possible are not showing they care for the platform enough.
Very true. I hope that my series would help to improve the quality of various nodes across our platform, including witness nodes. There are many people around that treat witness position as a classical PoW mining: fire and forget. It's so wrong. But it's not their fault. It's ours if we vote for them.
Well I can imagine it's same with witness votes for most people: vote and forget.
Users (including me) have kept some witnesses with the votes just as long the limit of 30 votes is not full yet.
For some, it seems that's the first moment when they even consider removing some votes from other witnesses.
But thank you, I appreciate the post series already.
Luckily neither you or I need to run a full Steem node then :)
I am pleased by your decision to give me one of your inhabitant's "shout outs". It is further evidence that you are indeed one of the wisest of your planet's inhabitants. There will be a place for you when the Empire arrives next week.
Dennis! Get in here! How do I edit out that last par...
Well I don't understand much of this at all but that video was AWESOME!
Also of curse I have been voting for you as witness for over a year and a half!
SteemOn Gandalf~*~
Thank you for your support! :-)
No, really thank you. You deserve the support for witness you have done a lot for this community and thats just a little bit of what I even know about. Also you are a kind and good person in RL on top of your contributions to this platform.
Glad to know you and have you as part of the community!
Upvote and resteem. GTG, almost a post which I can practically make use of. Keep on trying!
Thank you :-)
Keep up the good work!
Thank you for noticing my blog, it means a lot to me.
I voted you as a witness, @surpassinggoogle told me that you have done a lot for the community so thank you so much!
Thank you :-)
Gtg, Thank u for this amazing tutorial! I am really looking forward to see your next episode! Do u know what I really like about this post!? Your also explain people that they should never run smth they do not understand! Running smth u do not understand can be really bad for u.... Thank u one more time!
Ps: Może cholerka dodam, że nie zrozumiałam aż tak wiele, lecz próbuję :D
True, that part is valid in various scenarios, not only those related to Steem.
I'm glad you like it. :-)
Tak widziałam, że Paulinka jakoś próbuję też zrozumieć tą czarną magię ;D Nie chcę byc gorsza. Pozdrawiam @gtg @highonthehog
upvoted. and i have voted you as my witness...
Thank you :-)
Thanks for the info, agreed, if you don’t what the snippet does, first find out.
a very curious article. Thank you very much for the information. very useful information.
@gtg, thank you for this series! I love learning about what goes on behind the scenes. And, yes, voting for the right witnesses is so important. Knowing what they have to do helps us figure out who we may want to vote for. I am not done voting yet. 😳
awww this is awesome! Thanks for this informative post. I had no idea Got to Go! :)
Sueprize, huh? :-)
hahaha got to go got some JOKES! :)
I want to follow this and there's a fair chance I'll forget so here's a comment so I can find it again.
Also, thank you for doing this!
My pleasure.
From a few days I read through all guides on the subject of "Witness" and found out that it takes a lot of effort and knowledge. I have been thinking about it and decided to try it sometime in the future, if I have grown up on the platform and gain the necessary skills to trustingly run a Witness-Server.
You seem to definitely have these skill and I've been looking at your Witness thread, which is now 2 years old. Why don't you update this one? You're a place 4 Witness and that would help people understand your work better. My voice is not worth much, but I would probably vote for you if your Witness thread was more detailed what you did, do and have.
Thank you, probably you are right. I'm really bad at making MVP so in this case, instead of low quality update, I chose to not update at all.
But it's on my (rather long) TODO list.
I believe however, that those stakeholders that take this seriously, will take some time to lookup some info about witnesses to perform good choices.
I looked trough your posts just because the Witness post was not that informative, but I am sure you know what you are doing, else you would not the 4th witness anyway. But I fully understand that things like that bother you much. Reminds me somehow to Nicola Tesla
Oh wow, that is one hell of a compliment.
Nonetheless I'll do my best to update it ... someday.
a topic I really wonder.
you have given wonderful information .. I am waiting for more curiously :)
This post is so well crafted; that while it's above my tech-awareness, I read it completely, gleaning an understanding of how many facets are involved, behind the scenes, to keep the platforms secure.
For this reason, I've found a new witness to add to my list.
Best regards!
Peace.
Thank you :-)
✌!
What is the difference between a consensus node and a full node? From what I have been able to find so far, it looks like you only need to run a full node if you want to host a site like Steemit or DTube. Everyone else just needs to run a consensus node. How wrong am I? ;)
You are right to some extent. Consensus node can do all it needs for Steem platform to run. That's
steemdrunning with "low memory" settings, keeping in memory only things that are absolutely needed for Steem platform to work (that's getting newly produced blocks from the network, broadcasting transactions etc.). If you are looking for more, like the ability to respond to certain API calls you can add plugins for tags, follows, account history. For some callssteemdwith "low memory" is ok, but to be able to use all the API calls you need to connect to full node that has everything prepared to serve your requests.I'm following Steem Pressure series, thanks for bringing that to us @gtg! As I said in the previous episode I was waiting for a series like that for a long time. @gtg I wonder if you authorize me to translate Steem Pressure episodes to Portuguese language? It would be very useful to our community since we have a growling number of newcomers every day, and a few old steemit dogs who are thinking to start run nodes. Just an idea. Thanks man, cya!
Thank you for following and for your offer, but please do not translate my posts. There are few reasons why I would like to avoid translations.
Hey @gtg, yes these are two good reasons...I agree with you! Don't worry I'll not translate it. Thanks man! steem on!
I already voted for you as a witness!
(I hope this does not take long, and will soon be available. I hope that soon you will have time to show how it works, those who need it :))
although I'm not very much at it: 0 hahahah
know! you're cool! and thanks for the support always!
Thank you :-)
WHat an awesome write! With instructions like these I am quickly running out of excuses. Thanks so much for your awesome, benovolent piece.
Good support and educative
@fenmark
Thank you :-)
Thanks for this awesome infromation.it seems like with slow conection I can't mine steem??§§ I can afford just 8 Mgb isn't it enough to solve some bloacks everyday??
Please read other comments here, I already clarified it. TL;DR: there's no such mining on Steem as most people think.
These informations are really helpful.More people are sufferung form these bandwith problem.So,you got solve and system req.Thanks mate! and Good luck also
Actually that's not related. Bandwidth issues that some people are experiencing are by design limitations that prevent spamming the blockchain. We are trying to figure out more user friendly algorithm (while still preserving that precious feature) but it would take some time.
Each Steem user has Steem Power that represent their shares in the platform. It is used not only for determining how big is your impact while deciding about rewards allocation but also how much of a whole platform bandwidth is available for you to use.
So when the platform is heavily utilized, those who have very little Steem Power have to wait a bit before they can transact (post, vote, etc). One simple solution (other than waiting or making less actions on site) is to increase your share by getting more Steem Power.
@gtg
Words cannot explain your awesomeness. I love how you took time to explain everything. I appreciate you from the two sides of my kidney. 👍👍👍👍
ah ah ah ha @kdkwrites how about the third side of your kidney? Your feedback ket me rolling. I just have to give you an upvote.
Thank you, take care of your kidneys ;-)
amazing post..... good luck and thank you so much for information
This is my first comment to your self @gtg. Very informative details entire. Need to learn more about witness post. But have some idea how to create witness project. Waiting to learn more.
Yeah..Steemit is growing more faster then would be awesome in blockchain @gtg. You've nicely explained what's choose proper hardware to run of your nodes. I interesting to learn how to create witness category. Nice to decided I'm following you. Excellent post.
Thank you. Please also consider voting for witnesses, you can chose 1 to 30.
Nice post @gtg
Good luck for your steam 😉
I am waiting for your support. For me last post 😊
Thank you so much. Good evening 😊
At first read, it does sound like Greek, but here's my fifth read and it is making sense; my little techie sense kicking in.
And like you said in a reply, I can bring on quality to steemit by producing quality content and voting for witnesses.
Well, I can handle that without feeling lost in Steem!
Good luck! :-)
Thanks!
Ohhh woww, i didn't know that is possible to mine Steem. One my friend has a Bitcoin farm and he argued with me that it's possible with Steem. I will show him tomorrow your post! Hahaaaa i can only imagine his face 😁😂😂😂. Thanks a lot my Polish friend for such a priceless info!
Please take a look at my comment or yet another clarification :-)
Because "Blogging is the new mining"(TM), you've just "mined" some Steem with your comment ;-)
I read it several times but still don't understand all this technical stuff 😁😂😂😂. So i suppose, everybody can mine Steem by writing posts and comments?
Maybe that's not a strict definition of mining but in the end you receive parts of STEEM for that, so pretty much yes ;-)
(Most of the newly created STEEM goes to authors and curators)
Ohhh now it's getting clear. Thanks a lot for explanation! 😉😉😉😊
My pleasure :-)
Too many technical terms that might be a little off for non tech like me but in all I got the message. Looking forward to the next article. There is so much to learn from you.
Thanks @gtg
@iamjimmy
gdybym tylko cos z tego rozumiala .. ;)
... to byś siedziała przy komputerze zamiast zwiedzać Afrykę ;-)
Ale przynajmniej video fajne, nie? :-)
kingsajz :)
ech .. i tak siedze przy kompie.. zwiedzanie bedzie w marcu.
Best post, but can you mining steem?
You can take a look at my answer to similar question.
Definitely because Steem is growing faster and faster! Steem On!
thx, for this very intrresting info
Steem is the best growing digital currency. No rushing,steady moving upwardly everyday. I think it deserves proper monitoring and must be maintained carefully. Any steemian with lot of steem is a wealthy man or woman already.
I'm curious if it makes financial sense for people to buy machines specifically to "mine" Steem.
Do you have an approximate cost of a machine to run a node?
You can take a look at above requirements and calculate cost of such setup by yourself.
Steem is a DPoS, not PoW, so there's no classic mining here.
Hardware is cheap. It’s people and their skills that cost money. You can be technically a witness just by running a Steem witness node, but block production rewards depends on how high you are on the witness list and that depends on power of votes that were cast on you... and votes depends on your engagement in the platform. So it's not really a passive income where hardware works for you.
But Steem is special in many ways.
As they say "Blogging is the new mining"(TM) ;-)
A successful blogger can earn more rewards than a successful witness.
Would've loved to participate. Too bad I'm no techie🙁
On Steem there's no need to be techie, you can participate by creating high quality, original content. Also, please consider voting for witnesses.
@gtg Yeah I vote for a few. But which others do you suggest?
No, you are not voting for witnesses currently.
helpful informations here
I hope so :-)
yeah... just carry on with your job..you will get more support that I believe
is important that nodes are of great quality thats what will make steem explode...
@gtg what can I say than you are awesome
Hey @desmonddesk. You gonna exhaust the nodes on that kid. I hope @gtg has enough RAM o supply him
Hahaha. I am sure @gtg got him.
I like that awesome gif, so I'll upvote it lol
nice post!
Seriously? That's why you wrote two "nice post!" comments and upvoted yourself, but wasn't nice enough to upvote this nice post?
This is 4 generation hardware material
And still you think that your comment is worth more than my post, right? :-)
Very cool
Thanks god..
I got to know today..!
Thanks for the information
This play ground is full of heavy arsenal. So much input @gtg
Thats what you get to see over and over again on steemit.
O boy!!!!
Hello,
Do you guys review products or promote other author's posts? for a fee of course.
How is that related to this post? :-)
I can do a review of a product for a fee if it is on a topic. So for example if someone is going to give me
Intel Core i9-7980XEorIntel Xeon Gold 6154I will be happy to feature it in Steem Pressure series :-Dnice post!
Right now I have a decent network all day providing less than 100kb/s.
How do you think I'd be able to match the steem pressure toy.
Besides, how about 50GB for a consensus node and 300GB for a full node. Wouldn't that still be okay?
I have a remote desktop server plan with Aweber and Google Cloud is there any positivity to make use of either option @gtg?.... Oooops! You just gained another follower.
You better save your (and ours) bandwidth and avoid posting copied content on Steem. @cheetah was after you multiple times. That's not good sign for your future. Original, high quality content is the key.
Good luck.
Very good
Many thanks,gtg,for all the information on steem pressure! More grease to yoyr elbows!!