The goal of this proposal is to make HiveSQL free to use again in order to enable as many developers and users as possible to benefit from this valuable tool and develop applications and services for the Hive ecosystem.
In the following document, I may have to mention It Which Must Not Be Mentioned. In order to spare the faint of heart, I'll do my best to avoid it and replace it with Hive. In case that is not possible, I will just strikethrough.
A bit of history
HiveSQL has been one of my first major projects for the Steem Hive blockchain. I initiated it in 2016, at the very beginning of the Steem blockchain life.
At that time, HiveSQL was free to use! Yes, totally free... except for me ;)
Over time, with more and more users and applications relying on HiveSQL for their proper functioning, and with the volume of data steadily increasing, operating costs started to climb.
HiveSQL has regularly undergone several code reviews [1][2][3] and infrastructure upgrades [4][5] to be able to maintain the performance provided at the best level.
A few numbers for the techies
HiveSQL has been running uninterrupted for 4 years now and its database size is over 2TB.
Every day, HiveSQL processes around 11082 requests per minute (graph), daily servicing an average of 25 applications (graph). Among them, well known apps likes Hive Keychain, HiveDapps.com, 3Speak, ...
Hive SQL database indexes are extremely harnessed and are the key to the performances it delivers. Every minute, up to 2 millions index scans (graph) are performed.
Failing to stay free
Inevitably, being able to sustain such hammering had an impact on the costs and at the end of 2017, I launched a call for help and support in order to continue to provide this service with the same top quality level.
The call was not heard and the only solution I found to make HiveSQL survive was to switch to the actual subscription model. Unfortunately, DHF and its proposals system did not exist at that time.
Why a proposal?
My wish has always been to keep access to HiveSQL public and free to everyone. I'm a bit of a utopian and would like to try again to find a way to fund this project while remaining true to the idea of providing it for free.
Others reasons are:
1. To cover costs
Let's call a spade a spade: I also need to cover costs.
For more than 4 years, HiveSQL has proven to be reliable, responsive and highly available.
Providing high-quality services is not free. Setting up, monitoring and maintaining such an infrastructure requires a lot of attention. Maintaining and monitoring the processes that inject data from the blockchain into the database also requires its share of work.
Although costs are managed effectively, they keep growing.
I have tried to keep the subscription fees affordable, but they are no longer enough to cover these costs and increasing the price of subscriptions is not a viable solution.
It is slowly becoming unsustainable for all and I cannot continue to provide this service at loss.
2. Bring back Less fortunate devs and users left behind
It really pissed me off when HiveSQL moved to a subscription model because I saw a lot of developers and users who couldn't afford the subscription get left behind.
This led me to offer a daily subscription, but despite that, it was not enough to slow down the slaughter.
I would really like these people to be able to take advantage of the enormous potential that HiveSQL provides to them in terms of support for applications, analytics or any other use case that they have.
3. Grow the hive ecosystem
HiveSQL is a powerful, fast and efficient tool.
By making HiveSQL available to a maximum of people already immersed in Hive or, even better, to people from outside our ecosystem, I am convinced that we can make it grow.
Knowledge is power and data brings knowledge.
If we give users and application developers free access to the blockchain data, in a way they can easily retrieve and manipulate it, we are providing them with the perfect tool to develop their business and services and in turn to onboard more users.
Request for Funding
This proposal is not intended to reimburse the expenses that have been incurred so far. It is strictly to cover current and future costs in terms of infrastructure and human resources so that HiveSQL can continue to run and enable developers and users to use it for free.
Here are the costs that have been budgeted, taking into account the past experience of several years and my plans for the future:
1. Infrastructure and services ($1900/month)
- SQL Server (including license)
- Dedicated Hive private nodes (AH and HM)
- Firewall
- Security management and monitoring tools
- Performances monitoring tools
- Backup services
- Language identification services
2. DevOps ($800/month)
- Infrastructure maintenance and updates
- Code maintenance
- Support to users and developers
- New features implementation (like Sentiment Analysis)
3. WebOps ($300/month)
- HiveSQL.io website update and maintenance
- Documentation creation and maintenance
- Tutorials creation and maintenance
Considering the elements mentioned above, I'm asking for 100 HBD per day for the remaining days of 2020 and the next two years.
Make HiveSQL free again!
Support This Proposal:
- Vote for this proposal on Peakd.com
- Vote for this Proposal on Hive.blog / https://wallet.hive.blog/proposals
- Vote for this Proposal using HiveSigner
Thank you for your support!
I heard a lot about hivesql, is it opensource, I mean the codebase to sync, indexing, etc.?
HiveSQL is more than a simple synchronizer.
It's made of multiple micro-services running simultaneously. It also relies on paid third-party services (ex: language analysis, performance optimizer).
Hard to "opensource" such an infrastructure.
Opensource is always a choice. Setup could be opensourced, documenting process to setup similar environment could be opensourced, paid 3rd party tools could be disclosed (names, why, how they are used, etc.) Micro-services could be explained, etc.
Yes, but all of the processes you describe here are very time consuming and useless if they are not regularly updated.
I didn't say it can't be done. I said it's "hard" to do.
You might be right, as I don't know your setup. Directly asking for 2+ years without first having things documented in case you shutdown or something happens to you, is quite risky to entire community. But if there is already documentation that proofs that anyone could start their own instances, doesn't mean they should do it (given the cost of infrastructure is high as you wrote). It would give a lot more confidence, imho.
I made the proposal a long term one to avoid having to campaign for its funding every N months.
This does not represent a risk to the community because the proposal is not funded in its entirety at one time.
Small payments are made on a regular basis and can be interrupted at any time if the work or services provided no longer meets the expectations of the community.
And what if you just open source the code not the infrastructure so any one could set a hivesql server tracking certain tag or certain community? Opensource is a great choice because anyone could contribute to the code and audit it.
What are the cons on opensource hivesql?
Your post has been voted as a part of Encouragement program. Keep up the good work!
Dear reader, follow and support this author, Install Android: https://android.ecency.com, iOS: https://ios.ecency.com mobile app or desktop app for Windows, Mac, Linux: https://desktop.ecency.com
Learn more: https://ecency.com
Join our discord: https://discord.me/ecency
Would love to support, just wanted to know if it's going to be open sourced?
Posted using Dapplr
I have already considered it, but as explained in my reply to @sanat, HiveSQL is more than a few lines of code.
Moreover, open-sourcing would also require rewriting it as it uses private libraries that should therefore be removed.
So this is not currently planned, but having the proposal funded could help.
Ok, I can understand. IMO, open sourcing this can be big and you don't need to do it all at once and complete. You can open source it in modules which the community can use and work upon.
Posted using Dapplr
Done voting for the proposal 138 ..thank you @mers for tagging me.
Thank you for your support @lilymay, I really appreciate it! 👍
You are welcome sir :-)
Just seen this. Thanks for your work @arcange. Definitely happy to support this proposal (even though it's supported anyway).
Peace and Love
All support welcome. Thanks @atma.love ❤️
Considering your experience/credentials, the quality of the service and its utility, the proposal is absolutely reasonable and in my opinion should be supported.
Thank you for your appreciation and support @guiltyparties
Happy to support it!
Btw the links didnt work for me.
Please retry. It should work now.
Deploying a proposal is a tricky process as you have to create the proposal post (without knowing the proposal id) then create the proposal, then update your post links with the correct id 😅
Thank you for your support @dalz
Done!
Thank you for your unfailing support @traciyork, really appreciated!
I was a subscriber back when it was SteemSQL and I can attest to the stability of what you offer. Furthermore, what you offer allows running some substantially powerful and long queries, the kind that you couldn't query the blockchain itself (not without some tricky engineering a whole lot of Hive Power).
I would love to see @blocktrades and other heavy hitters in the Hive ecosystem support HiveSQL because the possibilities it allows from a dApp perspective are severely underrated and worth paying for. Not having to manage hardware, servers and sync your own database is worth paying for alone. I will be disappointed and disheartened if the Hive community doesn't support this proposal and make it happen.
Just letting you know @arcange that the second link for the proposal on Hive.blog appears to be broken as well.
Thank you for your feedback about your experience with HiveSQL (formerly SteemSQL) @beggars 👍
PS: hive.blog link fixed. Thanks for notifying me.
Love HiveSQL, I've subscribed a few times for specific instances but could not keep up the sub, sorry! I think it's a fantastic tool that could be used by devs and others, like me. Totally supporting your proposal!
Thank you for sharing your feedback about your experience with HiveSQL.
Your support is much appreciated!
You bet. Glad I can help in any way. You do a hell of a lot for the Hive ecosystem. Would like to see you rewarded and supported ..!!!! AND I just saw your latest POST! FANTASTIC! 👍
Things on the Internet are either free or out of reach.
The problem with free is that free things tend to be abused.
I've asked myself: are there any queries that I could perform on HiveSQL that would be worth the $40 a month subscription. So, the sad truth is "No"
IMHO, HIVE would be a happier and more sustainable ecosystem if we had more middle ground structures that minnows and dolphins could afford. I don't think you have to aim at making HIVEsql free. There needs to be some ways for low value accounts to small amounts of computer time on the machine to make periodic queries.
I'm new here so i could be missing details, but if this proposal is rejected and the service continues to exist as a subscription model, a smaller pool of funds than what is in this proposal could be used as a community subscription or pool to be used. Restrictions would be required to keep usage low/prevent abuse. This same idea, but skewed in cost/usage in favor of the service provider, could be used as a temporary subsidy.
As much as I'm concerned about the closed source nature of this, I think that the benefits it can provide are worth it because:
So, it's a yes for me.
Thank you for your unfailing support @eturnerx, really appreciated!
I support this. If you're doing ruby/rails development, you can access HiveSQL through this library, which wraps the tables in ActiveRecord:
https://github.com/inertia186/hive_sql
Thank you for your support and for the integration with Ruby
I support this!
Thank you @gregory-f, Merry Christmas to you 🎅
BTW, the current HiveBuzz proposal will expire at the end of December (in a few days).
Do you mind to support its proposal renewal for 2021 so the team can continue with its work next year!
Thank you.
I'm already supporting the new proposal. Hope you have a Happy New Year!
Oh, my bad. sorry for bothering you with my request.
All the best for 2021! 🥂
Not a problem, sometimes I need to be reminded.
I will support this proposal to make it easy for all developers to access the information
Thank you @hiro-hive
Done! 👍
Thank you @doze
You're welcome :)
Happy to support something usefull... you have my vote 👍
Thank you @pixelfan
Do you mind me asking if you use the funds from your frequent posts to support HiveSQL costs?
I'm not sure if it's going to get voted through, but maybe you can achieve your goals and receive some reimbursement from these posts.
Also I suggest using some H-E tokens like Leo, Palnet or Neoxian to bootstrap the earnings.
Voted!
Posting to finance a project, it could sometimes work at the very beginning of Steem, when posts received payouts with several zeros.
It doesn't work anymore.
Anyway, thank you for your vote @sk1920.
We need to move out of stagnation, I hope this will help other developers to create some more commotion in our ecosystem.
Voted
I also hope this will inspire developers and users.
Thank you @jocieprosza
Voted!
Thank you for your support @fbslo, really appreciated! 👍
I support this 100% and if I had more of the technical skills would be very interested in helping.
Thank you support and your help proposal @darkflame
Having HiveSQL free to use might be a good opportunity for you to play with and improve your tech skills 😉
Yes that sounds great I would like to do this. There are a few things I want to make my website do using the Hive blockchain, so I have many questions. Thank you for your replies!
If you have any question, feel free to use the support channel.
Voting for this. I'd really like to experiment more and I think this is a key part of that.
Thank you @brianoflondon. Hopefully you will be able to play with for free.
I support this.
Thank you @vimm
I'll support you for sure!
Posted using Dapplr
Thank you @amico
My pleasure! 🙏
Posted using Dapplr
Done!
Thank you @papilloncharity 👍
Only a pleasure!
I hope it goes through, I am pretty sure my vote proxy will be voting for it.
Thank you for your support @bashadow, appreciated!
Voted! I hope it can be free!
Thank you @drniche
Don't get me wrong but hivesql is a centralized closed-source database. You leave at any moment and hivesql will be gone too.
No problem. These are expected and sensible questions.
Access to all data from the database is fully open, ask those who already use it.
The rest is
closedsecured for stability and security reasons.When I first publicly opened it to the internet, some people called me crazy and predicted he wouldn't hold out for long.
They were right: you can't imagine the number of intrusion attempts that the infrastructure undergoes every day.
The real added value of HiveSQL is its design, infrastructure and the professional service that comes with it.
Congratulations @arcange! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @hivebuzz:
You mentioned that you seen a lot of smaller users unable to use HiveSQL once it became a payed subscription. That makes me wonder a few things:
These are all sensible questions.
Sorry I missed this one - have just reblogged!
Do I only have to vote once? Ie on Peaked OR Hivesigner? But not both, correct?
Also, can you tell me more about HiveSQL? I was not aware of it. It's been a long time.
One vote is enough. For more information about HiveSQL, you can visit https://hivesql.io
I put in my vote to support it :) ...
Thank you @wrestlingdesires
That's an easy yes.
Great service that adds a lot to the eco system.
Thank you for your support @niallon11
Bonjour @arcange je trouve cette proposition très intéressante, j'ai donc voté pour elle. Cependant, j'ai une petite question. Je ne sais pas si j'ai le droit d'organiser un giveaway (des récompenses) pour toutes personnes qui vote pour votre proposition.
Merci de votre éventuelle réponse.
Bonjour @challenge-fr, désolé de le dire comme cela, mais je ne vois pas ton soutien à la proposition.
Il ne s'agit pas de voter pour le post de la proposition mais bien de cliquer sur le bouton "support" sur la page de la proposition. Peux-tu vérifier et réessayer.
D'autre part, en ce qui concerne l'organisation d'un giveaway, tu es bien sûr libre de le faire pour autant que cela n'engage que toi ou des comptes que tu contrôle.
Je t'en serai en tout cas fort reconnaissant.