About a week ago I noticed that the memo messages that you can attach to a transaction can hold a lot more information that what you would expect.
You can store about 2048 characters per memo. that's 2 kb. If you don't really know what this represents, well, so far I've only written 286 characters.
Right now you're probably like "yeah 2048 chars is nice but posts can hold so much more". I actually don't know the size limit of a post, but I know that you can only send one post every 5 minutes, and that's just not gonna cut it. Memos don't have such limitations which gives us a lot more flexibility.
What is a file :
Another thing you might say is : "Yeah but memos can only hold text information". I'll let you in on a little secret :
Files are nothing but characters
If you open with an image it will probably look like this :
Lots of random (and often unreadable) characters. But it's actually just information about the image, it's looking like that because most image format are compressed so we try to use as little information as possible to represent the pixels. If you open a .pdf you can notice that it's already way more structured :
So what software does when it reads a file is simply read it and parse the information on it to produce the desired effect, an image viewer will read the pixels infos and display them, a pdf reader will display the pdf etc. You could produce a pdf reader if you want, you just have to know the format of the file and write a program that does that.
Memos :
To write data to memos we need to store it as readable characters, so we use base64 to encode the data. Base64 is made specifically for the purpose of representing binary data with readable characters so it's perfect.
Since a memo holds 2kb of data, to store 100kb, we'll need around 50 memos.
The storing scheme is very simple :
File -> binary data -> base64 encoded data -> split in x subtexts of 2kb -> send to memos
When we send the memos, we retrieve its unique id and store it. And when the complete file is uploaded to the blockchain, we send one last memo that holds the ids of all the previous memos and the filename.
It looks like this : "data/steem.png:3294:3295:3296:3297:3298:3299:3300:3301:3302:3303:3304:3305"
If you're curious you can see it on my steemd or if you're reading this a long time in the future :
Retriving the data
Retrieving the data is pretty easy, we just look backwards (starting from the most recent memo). For the filename.
Once we find the filename, we parse the header file to know on which memos the data is stored. We read them, put them together again, decode using base64 and then write the binary data onto a file. And bam ! you got your file back.
Since we look for the most recent memo header, if you write a file named "steem.png" and then another file with the same name, we will retrieve the first one.
Conclusion
Obviously this puts your data out in the open for anyone to see, so don't put anything that you would not put in public.
Since we are sending transfers to ourselves, you can store data with 0.001 sbd. This also means that an attacker needs your active key to write on your storage. And if someone got your active key, you got bigger problems than that.
The only limit is your bandwidth. Right now with 1067 sp I got 1.37gb, so I can store quite a lot of data on there, It would be interesting to see if we could write a front end that writes the images to the blockchain. Because then we would have completely decentralized posts. that can be red by anyone, anywhere, with anything.
Anyway, it's been very fun to work with memos, I will definitely experiment a bit more with this idea and try to implement a database.
That would allow some services to work completely on-chain without any centralization. Perhaps even some form of very basic smart contract where bots parse the memo code and can interact with it to change it and execute its code.
I will also look into encrypted memos. This would provide privacy to our storage but it's pretty risky because if someones manages to get your memo key then everything that you stored becomes available to everyone and you have can't erase it.
If you're curious, you can find the code here : https://github.com/drov0/MemoBox
Posted on Utopian.io - Rewarding Open Source Contributors
Obviously it's not a good idea to store nudes (or other sensitive files on the blockchain), even if they're encrypted by # in the beginning of the memo, I mean even Apple's iCloud got hacked and Jeniffer Lawrance victimized.
By the way, can you write binary data in memos, or will you have to use alpha-numeric-symbolic chars to encode non-text files on the blockchain?
Technically apple's icloud was another issue. And Nah you can't put binary data, I tried, you have to encode it.
Maybe an idea for the next hard-fork :P
Or prevent lunatics like me from being able to put random junk on the blockchain for free and without any timeouts :p. I don't have a lot of sp but someone like you could put 41gb on the blockchain every x times (I haven't found how bandwidth regenerates) ^^ And all this has to be stored somewhere. So this puts extra weight on the witnesses shoulders.
Well, since each memo costs 0.001 SBD, uploading 1GB of data will cost 500$ or so, so I doubt anyone would upload junk via this method.
Not really, since you can send transfers to yourself :D look at my wallet : https://steemit.com/@howo/transfers.
And even if it was not the case, it's not that hard to have another account to send you back the 0.001 sbd each time you send him a transaction.
Damn, forgot about that.
Well, if people will start using it that way, maybe a cooldown will be introduced, like the 5 minute minimum between posts.
Thank you for the contribution. It has been approved.
You can contact us on Discord.
[utopian-moderator]
@originalworks
To call @OriginalWorks, simply reply to any post with @originalworks or !originalworks in your message!
Hey @howo I am @utopian-io. I have just upvoted you at 10% Power!
Achievements
Community-Driven Witness!
I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!
Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x
This is an interesting idea for a database project. I wonder if in the future Steemit could support a paid service for something like this.