Introducing the Acela Core - Upgrading Existing Web2 Apps into True Web3 Dapps
Hello, Community!
Welcome to another 3Speak update! In this post will talk about Acela Core, the new video platform backend we are developing for 3Speak.tv and SPK Network apps. As we develop this backend, it will play a critical role in bridging the gap between web2 like platforms and web3 land. We intend for this blog post to be a rough overview as each topic mentioned can be expanded out to dozens of individual posts. Over time we will gradually release more updates and information about what this new backend would entail.
Think of it similar to a layered cake. At the very bottom, the first layer of the cake is HIVE and other web3 tech. Then, one level up is a barrage of indexers to make the base layer data useful (SPK Network union indexer is a perfect example). Then, finally, the icing on the cake, the Acela Core, will go the remaining extra mile to provide platforms will a high end web3 experience without needing to write a ton of backend code.
Creating a platform is hard, you need:
- Account management (authentication)
The authentication side of the backend combines various authentication and identity systems such as HIVE keychain, metamask, SSO, and more into a single unified backend. This doesn't just apply to 3Speak, but other apps as well wanting to utilize this backend. A portion of this is simply linking your HIVE account to a username/password login and using that as the primary logins means. But also being able to create a HIVE account without needing to know about HIVE at all (web2 like experience). On the more complicated side of the spectrum is HIVE proxy accounts. With this, we can bridge operations from non HIVE accounts onto chain via a set of dedicated proxy accounts maintained by 3Speak or others. While this is a later step in the development process, we are actively looking at building it into the backend.
The authentication backend would have a simple set of APIs for handling login, posting, voting, account creation, etc. All can be easily accessed via regular HTTP calls with proper authentication token created during the login process.
- Storage layer
Large scale projects involving thousands of files and hundreds of TBs of data tend to require a dedicated service for handling data storage, management of data storage and integrations with other services. To begin with, storage on IPFS isn't easy. You can't just put a file on IPFS and expect everything to work 100% of the time. Thus, we are building a dedicated service to handle uploads, communication with a dedicated IPFS-cluster to manage part of the IPFS storage and a other misc functions. Additionally this storage management service would handle SPK Network POA (Proof of Access) integration at a later date once available.
- Indexing
By default in HIVE, there are only a handful of indexing systems available. Many of which are focused around providing their own forms of social indexing, and all of them do not have support for offchain posting. You can't just execute an API call on any random normal HIVE node and to do a massive complex query. That is why we are building the SPK Network's union indexer to turn HIVE on chain data and most importantly offchain data into a web2 like database. By indexing all this data we can do things like full text search. And we can even do advanced recommendations to improve the trending or user feed. We can mix both on and off chain content into the same feed. As time goes on we will publish more detailed blogs on such topics.
https://github.com/spknetwork/union-indexer-node
- Write capabilities
Currently most write operations such as posting a video, commenting, voting and more are done by first granting posting authority to @threespeak then using an API to interact in a web2 like fashion. The API translates incoming requests into on chain actions on the behalf of the user. This gives us lots of flexibility to do things like scheduled posts, login via email/password etc. We will be keeping this largely the same in the new backend, but modernized to significantly usability and overall quality of the backend. Additionally, HIVE proxy accounts will come into play where instead of posting directly to an account on a 1:1 basis, we can directly post onto a shared proxy account for hundreds or even thousands of users. Utilizing the union indexer to differentiate between users.
We also acknowledge login options exist such as HAS (Hive Authentication Service)/Hive Keychain are available where the user can directly sign for actions. We will be working that into the Acela Core where it fits/is necessary.
- Video Encoding
Generally speaking, all videos on the 3Speak site need to go through some form of encoding. Whether that reason is to create multiple resolutions of the same video or to reduce the size of the original video. It's widely considered necessary. The new backend will provide an interface to communicate with a dedicated SPK video encoder cluster for video encoding needs. That cluster is either operated by the a platform, or a remote 3rd party of users who form their own encoding clusters. The video encoding part of the backend is heavily tied into the auth, and most importantly storage.
- Health checks
We will be creating a dedicated service for handling healthchecks of the backend. To start this covers mundane things like an API being down and notifying developers. But also more complicated tasks such as verifying integrity of all stored video content and post metadata. Healthcheck system status will be available on a webpage and through a discord bot for more frequent updates in the future. This will play a critical role during the development process and maturity phase of the backend. Even more so important for platforms that might not have any idea what is going on during the initial phases of setup.
This is why we are developing the Acela Core for platforms like 3Speak and others to leverage without spending significant time developing a customized backend. It's also entirely open source and able to easily fork and build your own copy. Why build your own web3 infrastructure when the Hive and SPK Network community already provides it for free? Not only will these changes be important to the advancement of the 3Speak.tv platform, but also the entire web3 content and video community.
Ending notes
If you'd like to know more about the Acela Core backend in depth and discuss your use case. Please reach out to us on our Discord Server or contact @vaultec directly at vaultec#0001 to get in touch
Stay tuned for our upcoming witness blog post that should be available in coming days!
The above graphic is an in progress and incomplete representation of the new backend architecture.
Legend for the above graphic:
Green = Available/Meets MVP requirements
Yellow = In progress/partial
Gray = Not started/unknown/TBD
~~~ embed:1626722373800980482 twitter metadata:MTE2Mjc4MzQ5NDkxMDU5NTA3Mnx8aHR0cHM6Ly90d2l0dGVyLmNvbS8xMTYyNzgzNDk0OTEwNTk1MDcyL3N0YXR1cy8xNjI2NzIyMzczODAwOTgwNDgyfA== ~~~
~~~ embed:1626772797820116995 twitter metadata:MTA2NjgxNjU2MTAyODE4MjAxNnx8aHR0cHM6Ly90d2l0dGVyLmNvbS8xMDY2ODE2NTYxMDI4MTgyMDE2L3N0YXR1cy8xNjI2NzcyNzk3ODIwMTE2OTk1fA== ~~~
~~~ embed:1627005283917606912 twitter metadata:MTM4NTY2MjQ1fHxodHRwczovL3R3aXR0ZXIuY29tLzEzODU2NjI0NS9zdGF0dXMvMTYyNzAwNTI4MzkxNzYwNjkxMnw= ~~~
~~~ embed:1627096028020371461 twitter metadata:MTMxMTIxODY4OTkyNDkzNTY4MHx8aHR0cHM6Ly90d2l0dGVyLmNvbS8xMzExMjE4Njg5OTI0OTM1NjgwL3N0YXR1cy8xNjI3MDk2MDI4MDIwMzcxNDYxfA== ~~~
~~~ embed:1627205330248540160 twitter metadata:MTQxNjY4Nzc0NTA3NDgxOTA3M3x8aHR0cHM6Ly90d2l0dGVyLmNvbS8xNDE2Njg3NzQ1MDc0ODE5MDczL3N0YXR1cy8xNjI3MjA1MzMwMjQ4NTQwMTYwfA== ~~~
~~~ embed:1627356560426172417 twitter metadata:MzgzMDQ4ODY1fHxodHRwczovL3R3aXR0ZXIuY29tLzM4MzA0ODg2NS9zdGF0dXMvMTYyNzM1NjU2MDQyNjE3MjQxN3w= ~~~
The rewards earned on this comment will go directly to the people( @seckorama, @threespeak, @depressedfuckup, @jomancub, @flquin, @fermionico, @muneeb487, @funtraveller, @hiro-hive, @bo022, @rubencress, @dynamicrypto ) sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.
WOW! This is such an exciting update! It's amazing to see the development of Acela Core and how it will bridge the gap between web2 and web3 platforms. The layered cake analogy is perfect, and it's great to see the thought and effort put into the authentication, storage, and indexing layers. The fact that the authentication backend will combine various authentication and identity systems into a single unified backend is impressive, and the storage layer's dedicated service for handling uploads and communication with IPFS-cluster is exactly what large scale projects need. Indexing is also critical, and the SPK Network's union indexer will be a game-changer, allowing for full text search and advanced recommendations to improve trending and user feed. I can't wait to see more detailed blogs on these topics as the development process continues!
This all looks very interesting & exciting. But you all know I'm not a coder but I know enough to get me into trouble. But with my tin foil hat getting a whole lot more fancy and bigger lately I look at that fancy diagram and all I can think of is the presnetation of a multitude of attack vectors and points of failure in a very tightly dependent system.
You are going to need a much bigger developement team and a lot of ongoing maintainers.
And money.
I do hope you guys can pull it off and make it work. Thats one hell of an undertaking.
I wouldn't consider it all to be incredibly tied in a way that'll fail. The bulk of the work being done by this piece of software is separated out to numerous micro services. For example union indexer, encoder, ipfs-cluster, etc. Just the difference is we have a core backend that hooks all this stuff together compact and easy to run way.
On more sensitive topics like account creation/authentication, We (me) are looking at building a decentralized multisig to handle account creation and other ways to further decentralize/derisk our account creation proxy. Especially for proxy accounts where you can "select" a provider to be the account you ultimately post on chain... Of course, I am talking about further into a future. One step at a time right?
Baby steps are awesome as long as it isnt into a mud pit. LOL!
Susan? Susan Wojcicki? Is that you? Long time no see! I mean, it's been a whole day since you left Youtool. Ahahahaha! Just j/k.
I hope.
All in all this sounds like a serious plan to tackle the tasks ahead. I bet you could just tack this plan right onto the BTC ordinals fever ongoing and put paid to that parcel.
Thanks!
Hi, I selected your post to be included in HiveMagz (Hive Magazine) today, if you don't like this you can let me know, if you want to see the magazine that I made you can visit my post. Thank You
@threespeak! The Hive.Pizza team manually upvoted your post.
Please vote for pizza.witness!
Congratulations @threespeak! Your post has been a top performer on the Hive blockchain And you have been rewarded with the following badge
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out our last posts:
Support the HiveBuzz project. Vote for our proposal!
Thanks for the update, guys. :) Following along with interest.
Felicitaciones, que bueno que sigue creciendo, gracias por la información y gracias por su trabajo
Good development. I've had issues with how to upload a video using 3speak. I don't know if I can get help.
https://leofinance.io/@manniman/re-taskmaster4450le-2d3kwm6ik
The rewards earned on this comment will go directly to the people ( @manniman ) sharing the post on LeoThreads.
🎪
Thanks for sharing this .keep it up