A load balanced, SSL-secured, multi-region steem api cluster brought to you by @xeroc and @jesta.
After a few conversations with @xeroc, we decided to join forces to start the first cluster of community API servers for anyone to use in their projects. This API cluster is the same kind that powers piston, steemstats, and even steemit.com.
Our goal is to provide infrastructure to the community (and it's projects) that is reliable and resilient.
steem.ws - the public steem api cluster
- Status Page: https://steem.ws
- Websocket Host:
wss://node.steem.ws
steem.ws is run on top of multiple levels of load balancing - latency based DNS routing to the appropriate data center, and nginx based load balancing from the server to the API. It also has automatic failover (within ~30s) if the host you're connecting to goes down.
The initial members of this cluster are the servers that power piston.rocks and steemstats.com, both setup with the same full featured configuration required for steem based applications.
Currently the cluster has servers located in 2 regions, the US and the EU, and will hopefully be expanding into more regions in the near future.
These servers have the following plugins enabled:
database_apilogin_apimarket_history_apitags_apifollow_api
Get Involved
This is a community run project, so we want others to get involved! There are a number of ways you can help out with the project:
- By contributing server resources to this steem cluster.
- Analyzing our networking setup and recommending improvements.
- Helping us find a way to distribute SSL certificates to all hosts securely.
- Creating applications to help monitor the status of the service + metrics.
- Helping coordinate upgrade cycles when new steem features are released (reputation?).
- Building projects for the steem community.
That last one really doesn't help steem.ws, but what helps steem helps all of us!
If you fit into any of these categories, please reach out to one of us via steemit or through our new #steem.ws channel in steemit.chat.
Disclaimer
The use of this API server is on a best-effort basis. Do not rely on it's accuracy and availability. We may, in our sole discretion and without cost to you, with or without prior notice and at any time, modify or discontinue, temporarily or permanently, any portion of our Services. You agree that we are not responsible or liable for Third-Party Content and that you access and use Third-Party Content at your own risk. Your interactions with Third-Party Content are solely between you and the third party providing the content.
Has anyone written / come across a sort of "dummies guide to Steemit apps" type of post...?
I know there are HUGE opportunities awaiting for different sorts of applications that could be built out on and integrated with the Steem blockchain - yet not have a background in coding or programming, I often get lost reading posts like this talking about technicalities...
Of course, its an entire area of expertise in its own - however, the community could surely benefit from some guidance in "APIs 101" or the sort that help newbies grasp the fundamentals required to navigate their way and become literate enough to engage in such conversations. For instance, something like these two short vids that help understanding what an API is:
Of course, such might be my own ideal and curiosity. However, as this post is top on the trending charts - there's probably only a small fraction of people who see it that could actually understand it...
I think it could be very valuable to the community to create some educational content to fill in the gaps of knowledge to grasp stuff like this - as the more people who do learn it, the greater the human resources accessible in the community - the more people who will have that fundamental knowledge to bridge the gap between ideas they might have for applications to integrate into the Steem ecosystem and actually getting them developed out...
I absolutely agree, education of developers is crucial for developing a community around a platform like this.
I question whether we are at the proper place to really do that yet though. Everything right now is incredibly immature (no offense to anyone working in this space) and things are changing faster than guides and tutorials can be written. Hell, some of us are talking to each other for the very first times. If I would have written a guide about the APIs used behind Steem last week, they'd already need the addition of at least 4 new sections.
I believe most of what you see being built today is the infrastructure and prototypes for what will someday be the toolkit for new developers. We don't have our jQuery, WordPress, or Bootstrap yet. They're being built though, by a lot of the developers you see posting about new things.
As it stands right now, only the bravest (insane-st?) of developers dares venture into this ever shifting codebase. Someday (and hopefully soon) we will hit a more stable period, can all take a collective breathe, then write some damn documentation and guides on getting involved.
But for now, it's go time. Competition I'm sure will be fierce over the coming weeks/months, and the lead that Steem has taken needs to be pushed even further before the competitors start coming online.
Great comment, great thoughts, and I wish everyone wrote things with as much thought as you have. I'll end my mini-rant on that note :)
excellent insight. thanks for expanding/sharing on this. :-)
@jesta could you reach out to me on steemit.chat?
I would like to chat with you about something I think you would like hear. :)
Are you just
fuzzyon steemchat? I went looking for fuzzyvest and found no matches. Just wanted to make sure I was hitting up the right person!@jesta Has this been discontinued?
Hmm I see that on: https://steem.ws it says operational though..
Time for bed I think.
This post received a 4.5% upvote from @randowhale thanks to @wolo! For more information, click here!
It's been offline for a while, not sure why that page didn't update it's status... wonder if that service checking it got stuck or something.
I'm hoping to bring the node back online at some point in the future - steem just outgrew the server I had it on and July's been a busy month for me. I'd recommend steemd.steemit.com for the time being as it's on much, much better infrastructure than I could possibly afford.
Yep.. Precisely what I did in the end ;)
I really liked the load balancing idea for steem nodes.
Thanks for updating me regardless.
I will keep an eye on your posts so I know when to switch back.
Great stuff, thank you guys!
You beautiful glorious amazing people! I am wildly pleased and can't wait to try out what you've built.
Thanks! Do you know the best place to find documentation on the api methods and what input parameters to use to make the calls?
try piston docs https://piston.readthedocs.io/en/develop/lib.html
Thanks!
I have looked, I don't think follow_api is in that documentation, I could be wrong of course. I am trying to build a simple bot that autofollows upvoters, commenters and new followers. I know that the API is pretty simple, but it's far from ideal how useful it is at the moment. I recall reading that it requires a lot of requests and procesing on the client side to get it to dish out a complete dataset.
Currently, there isn't a way to interface with the follow_api through Piston. Maybe you can do it through the cli_wallet?
We definitely need Asia :) Good job to you guys !
Yes we do :)
I've been waiting to see if someone else wants to contribute a server in the asian region, and if no one has resources available, maybe I'll spin up an AWS instances myself.
I'm working on a project in China / Korea/ Japan that will help steem scale a lot.
Hope we will have contributers for some strong nodes here;)
That's awesome, look forward to see what you're up to!
@jesta I've some very rich friends in china since I've been doing business with china for years, but they are afraid to take any risk and being involved in something that could be illegal soon or later.
So what's the solution to make Steem scale in china and other pro censorship countries ?
the blockchain is decentralized so cannot be censored, but chinese gov can stop people who use this. And force them not to. The Law is the Law.
That's quite simple to understand. What you post on blockchain will stay, OK that's right, but first you need to be able to post (if they dont block all IP / nodes/ access), then if you do, it can be illegal for you soon or later if they decide that it should be.
People can go in prison for doing this, so Nobody can stop the blockchain that's right, but people can be stopped very easily.
So I'm really trying to work on this but it will be very difficult i think for Steem to Scale in China.
In Japan that should be a bit easier.
any suggestion would be great, anyone can contact me in the steemit.chat @ekitcho
Excellent!
You guys rock, thanks for this! Now if only I could catch up to your dev level Johnny Mnemonic style so I can help more =P
what I would give for a neural implant that lets me run SSH scripts to unlock server secure storage. Seriously, how complicated would this be to have a silicon based neural trigger system. The device fires off signals that wake you up, and you just have to activate the script, and then it triggers a reverse function, and you go back to sleep, instantly.
+5%
good stuff @jesta ! 8]
good news!
Great initiative! I just switched to steem.ws on steemroll.io. Good stuff.
switched my bots to it :D
Solid informative post.
laykni me))))))
@faddat and I have servers we can contribute.
Wow, @jesta! Steemit and I thank you for everything you do! You've got my vote! (in about 20 minutes lol)
Oh nvm, just voted right now. haha
Love it. Now all I need is some time :)
Bravo! Thanks for your great work, @jesta, @xeroc.
Steemit is growing fast
Great post
Amazing post @jesta
This is phenomenal! This takes a lot of effort away from beginners wanting to play around. Awesome job!
Thank you both. This is very much needed, one of the biggest key pieces in our infrastructure.
I am using this in screem! Works great.
Hopefully we can have one for the Philippines too.
Thank you to you both. You're both incredible.
👍nice information, thanks @jesta
Wow I've got so much to learn. I ran a full bitcoin node on a raspberry pi a few months ago, but unfortunately, the memory crashes destroyed my dreams of my very own 80 GB blockchain on a raspberry.
But man, you guys are real blockchain engineers. I've got a lot to learn.
wow steemit was awesome!
Good work, guys! We definitely needed this. As for Steem Mobile - eSteem, user has choice to select node, now that list of nodes are increased. Really cool! I back this project and let me know if there is anything I can contribute with. Thanks!
I haven't even considered the app you guys were building, that's awesome. We're happy to be helping!
Hey @jesta and @xeroc
On vacation right now but just sat down to work on a Steemit Pond update for a bit and piston.rocks was down. I decided to check steemit instead and I'm super glad I did because I saw this post. Great work! This could prove to be a very valuable tool for developers. I'll try to contribute where I can when I return from vacation.
Looks like it had a few connectivity problems about 2 hours ago. Look forward to chatting with you and enjoy the vacation!
Sokets it's good)
If SteemBlubber was real i'd vote you to the Moon!
This is absolutely fantastic! If I was located in Asia, I'd love to get involved and help you guys out. As it stands, I can only contribute some minor help here in the States.
Great stuff, guys!
for good mood =)
hey this is really great! awesome work!
Thanks for sharing post @jesta
nice! useful
I don't know if someone has already suggested a secure way to distribute the SSL certificates, but I think the way to do it is to have several people running control terminals that have the certs securely stored, and then propagating them to secured memory through SSH connections. The users get email notifications telling them when a server has restarted, and they then need to run a script that securely loads the certs into memory on the server.
I in fact have had a somewhat similar quandry with running a simple server box (in this case, my laptop) and I made scripts on my main workstation that push shell commands to perform storage unlocks. I think this probably would be the way to do it on these servers. Store the certificates on a secure partition, and have it automatically unmount when the server restarts, and require manual unlock to re-enable it. Since you have a cluster running, yes this requires some attention, but with sufficient, geographically distributed administrators (trusted, of course) they can fire off an SSH command to unlock the certificate stores and keep the secure storage from being unlockable from the server.
I of course am no expert on this but I puzzled through this problem for my own specific application. My goal was that it would be impossible to physically compromise the security of my secure storage partition, because I was physically monitoring and ensuring the controlling system was secure.
Good points in this post. Thanks for bringing this to our attention. Investing in cryptos is still a risky thing. I was researching a way to find more info about the current cryptocoins on the market. I really advice people to take a look at: https://www.coincheckup.com This site did all the research for you. It's truly amazing. Check: https://www.coincheckup.com/coins/Steem#analysis To check Steem Detailed analysis
Sadly the project seems to be abandoned.
http https://steem.ws http: error: SSLError: hostname 'steem.ws' doesn't match either of 'api.juicedb.com', 'js.juicedb.com', 'juicedb.com', 'ping.juicedb.com', 'slack.juicedb.com', 'socket.juicedb.com', 'www.juicedb.com' while doing GET request to URL: https://steem.ws/`There's still a wallet based node being hosted there (used for Vessel, cli_wallet, and account information):
https://wallet.steem.ws
But yes, the full node itself is no longer in service (it outgrew all of the servers I had available).