Repository
https://gitlab.com/vaultec/dtubepermanente
Weekly permanente dev update 1/12/19
In this series of future posts, i will post progress updates towards the development of permanente. This includes bugs, release dates, project changes, future improvements, new technologies that are being utilizes. I will also be developing all the modules permanente will be utilizing. Such as an IPFS distributed database, which any solution I have seen is not up to the requires that I need. OrbitDB comes close to something I can use but it is written in javascript/nodejs. These modules will be full developed to be used in external applications. An entire set of tools will be created for usage in permanente and external projects (more information in future posts).
Overview
- [Done] Database migration. Calling it permDB for now.
- [WIP] P2P networking.
- [WIP] Contract system.
- [WIP] PermDB (description TODO in future post)
- [Bug-fix] Database json serialization.
- [Not done] Logging support.
view roadmap here
P2P networking
This will serve as the main communication method for all permanente clients to utilize. Permanente (full instance) functions as an P2P server/client. If API is enabled clients can connect and utilize P2P functions of a full Permanente instance. Full instance meaning port forwarded + API + P2P. Contracts will be stored via DHT. In order to accept a contract, a node wanting to accept a contract, must send a message to the responsible node selling the contract. Once connection is established contract can be accepted. The node selling the contract will remove access for future peers to buy the contract. More details below on contracts. As mentioned above P2P capabilities are a work in progress. I would rather be using libp2p, however, they do not have a java implementation as of writing this. Using this off chain communication system reduces lag of contract agreement and prevents unneeded data to be stored on chain (contract hash can be referenced)
Specs
After researching a reasonable amount of P2P libraries for java. I have nominated TomP2P as the golden choice for P2P library. However, it lacks by default encryption support. Encryption can be added on application level, all P2P traffic is cryptographically signed. See security.
According to what is posted on TomP2P documentation. It can automatically manage port forwarding (UPnP), hole punching and firewall configure (if a peer is firewalled, has no control over actual firewall). Protocol is RPC like, TomP2P provides an interface into other peers via specifying raw bytes or utilizing object serialization. I do not know if sending objects through P2P works. I will have to experiment to figure out what the P2P system should look like.
Security
P2P encryption will be accomplished with secp256k1 over top TomP2P direct messaging capabilities. DHT records are signed with owner private key and can only be overwritten with that key. Allowing secure records to be published on DHT such as contracts, secp256k1 public keys, contract transaction memo/account name. PeerIDs are DSA public key. The key pair for DSA can be generated from a password or seed. Config.json will contain a a P2P seed. The hashing algorithm to improve security and reduce potential for attack is argon2. I have specifically choose Argon2 because it has won the Password Hashing competition. Argon2 settings are as follows; Iterations: 10, memory cost: 65536, parallelism: 4, hard coded salt. This brings the hash time to around 750ms (estimate). Which is well above the reasonable amount of time it would take for any GPU cracking device to break the key. That is not even including the time required to generate a DSA key pair out of the generated hash. Generated keys from P2P seed can be used in other areas of P2P related functions. Argon2 will be utilized in various areas related to security of permanente.
Contracts
See contract.java
Contracts should be built on top of IPLD objects. This is a major issue as IPLD is lacking java documentation and development. I will have to work around this and find other mechanisms to reference contracts, videos and other objects on the chain. Contracts are stored on DHT and signed by owner. They should also contain signature data to verify owner. These must be unspoof-able, strong data structure. Contracts should be verifiable. Nodes with active contracts will be checked by other nodes. Contract cost also must be verified.
Verification
It is required that nodes should check contract validity at all times. Contract rate per mb is in mSTM (0.001 Steem). Contract price is based off of total size. Size is calculated by adding the total size of every IPFS sub object that is in the contract root. Contracts are signed with DSA owner private key. Support for steem posting key for signature a possibility.
Final notes
Where are the IPFS Video formats in the last post at? Well due to lack of documentation and really my lack of understanding with IPLD and advanced sections of IPFS, I cannot create a working format (that is up to what i want) for Dtube videos and future formats. Keep in mind I am a single developer working on a massive undertaking. There will be more information in future updates. I may extend posting period to around 2 weeks. In the future permanente will not just be a video storage system. including new formats such as dImage, dAudio and D-Ads; More details in future posts!
Contact
You can reach me via partiko messenger or discord
See also:
dtube community improvement overview
#permanente #dtubearmy @dtubearmy #onelove #ipfs #WeAreDtube
Hello, @vaultec. This is a huge update and I appreciate that you published it via Utopian. You gave a very insightful overview of the project on this post. Well done!
It seems this is your first time contributing via Utopian. I have noticed that this project is hosted on gitlab. Unfortunately, Utopian only supports projects hosted on GitHub. However, Utopian also support an official mirrored project to Github, which mean, Utopian will support this project if the code also exists on Github.
Again, I appreciate this quality update and hope to see more in a short while. Thanks for using Utopian.
Need help? Chat with us on Discord.
[utopian-moderator]
Thank you for your review, @knowledges! Keep up the good work!
This is the future brother ....you are the wizard to see this thing through too ... Gray is right ... would be good to do a podcast and keep putting the word out there to people listen, understand and get excited about it! Great post !
I'd love to get you on a podcast to talk about this update at some point. Great work.
We should do one for sure ... times are changing
Congratulations! Your post has been selected as a daily Steemit truffle! It is listed on rank 25 of all contributions awarded today. You can find the TOP DAILY TRUFFLE PICKS HERE.
I upvoted your contribution because to my mind your post is at least 3 SBD worth and should receive 119 votes. It's now up to the lovely Steemit community to make this come true.
I am
TrufflePig, an Artificial Intelligence Bot that helps minnows and content curators using Machine Learning. If you are curious how I select content, you can find an explanation here!Have a nice day and sincerely yours,
TrufflePigCongratulations @vaultec! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPTo support your work, I also upvoted your post!
Do not miss the last post from @steemitboard: