Computation Bytes
Welcome to the first in a series of post about NFT building that I hope to get some discussion on; Continued from Building NFTs on Hive. Non-Fungible Tokens are a tricky problem to find solutions for, especially because several exist. Most of what a solution hopes to find is the most economical way to provide security for these assets. Security in this context is managing permissions and ensuring data maintained.
Economics here are the two aspects of computational efficiency: computation space and computation time. In a distributed ledger that will maintain NFTs and other assets, both are extremely important for scaling. In my view the aspect that should be prioritized is computation space. As long as the computational speed is enough to keep real time, then computational space represents bottlenecks in computer RAM as well as network backups.
Layer 2 networks built on blockchains are Infinite Turing Tape Machines... an infinity anywhere in an equation needs to have an evaluation shortcut. For example, if in 100 or 100000 years somebody needs to audit a transaction/calculation that occurred 99 or 99999 years from now, then replaying the tape/blockchain from its present day genesis is a huge issue in terms of computational complexity. Having backups that would allow replay from a short period before the transaction becomes more economical, and the smaller those backups are the more frequent and easier they are to retrieve and process.
The Layer 2 software that I've built actually uses these frequent backups to come to a consensus on it's data. Every 5 minutes an entire state backup is performed and it's content is stored on IPFS which provides a hash of that data. If an NFT was an entire photo or video stored on the layer 2 this would get extremely unwieldy. But just as important, if you are storing 10000 1kb NFTs it also get's cumbersome.
The First Spec
I'll be going over the basics of three types of NFTs and the first one is by far the most straight forward: IPFS.
This is a fairly common type of NFT in today's economy, an artist uploads a file to IPFS then takes the locator hash and builds an NFT to track the ownership of that particular asset. Given that it's so straight forward it should be relatively easy to track. So let's define what needs to be tracked and any concerns we might have.
| Aspect | Explanation | In Practice |
|---|---|---|
| Type | IPFS | 0 |
| Item | Location | "QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5" |
| Handling | File Type / Executable | "jpg" or "js" |
| Author | Builder | "disregardfiat" |
| Royalty | 10% | 1000 |
| Owner | Current rights holder | "dlux-io" |
| Created | Block number | 50000000 |
| Modified | Block number and reason | 51000000 |
| Collision | Ensure uniqueness | "Qm:WCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5" |
Now let's see this in database records:
//Set => ./sets/
"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5":{ //51bytes = 124 bytes average
"a":"disregardfiat", //upto 27 bytes... average 19
"c":50000000, //13-14 bytes
"o":"dlux-io",//upto 27 bytes
"r":1000, //6-9 bytes
"t":0, //5 bytes
"f":"jpg" //8-10 bytes
}
//Item => ./nfts/dlux-io/
"Qm:WCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5":{ //52bytes = 68 bytes with out extras(rent/license/etc)
"m":51000000, //16-17 bytes
}
The first thing most of you will notice is the NFT is split between two records, this is done for a number of reasons, most importantly for search indexing, but also facilitates collision/uniqueness, and much smaller records for other types of NFTs.
By storing the whole record as the item name it prevents the same item from being claimed twice. In the user's directory the file name has a ':' in it which keeps the way larger set records are stored more uniform across the platform. All IPFS locators currently start with 'Qm' which specifies the type of hashing algorithm. It's easy to reassemble this at runtime. Now I can easily find out what somebody owns, or who owns what.
Across multiple layer 2s I can use the created block number to find who signed an item as an NFT first. I can also use the modified field to load specific back ups to compile an NFT history and get the specific transactions that modified the NFT from the layer 1 blocks. The file type let's a client side program load IPFS assets into the web in an easy way such as <img src="https://ipfs.ip/ipfs/QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5?.jpg">
All in all this NFT comes out to ~190 bytes... Assuming a Layer 2 has 10M tokens(Like 10M DLUX) and each byte costs ~0.01 tokens to write, the chain would be limited to roughly a gigabyte. Keeping Gigabyte backups would quickly get painful and a change only system like Time Machine by Apple would likely be employed. In actuality these fees would need to go to the nodes that run the layer 2 to pay for their systems to keep up. Really the only way to ensure the data remains.
The Second Spec
There's a completely different type of NFT going around and this one is based on layering partial images onto a canvas and owning a unique subset of these layers. These would be the Bored Apes and Crypto Punks of the world.
These particular types of sets are a little heavier in general than an IPFS link, but individually can be much smaller. For example defining a few bases and layers only cost that many bytes, but the number of ways they can be put together is much like the alphabet and grows geometrically.
| Aspect | Explanation | In Practice |
|---|---|---|
| Author | Account defining set | "disregardfiat" |
| Script | Script hash | "QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5" |
| Base | Array of base images | ["QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5"] |
| Characteristics | Array of layers | See Below |
| Issued | Counter | 1 |
| Max | End of counter | 4096 |
| Royalty | 0.01% | 1000 |
| Name | Name of Set | "punks" |
| Link | Hive Locator for announcement/dApp | "author/permlink" |
| Uniques | Array of UniqueIDs and Owners | ["base64_owner"] |
| Type | Layer | 1 |
| Created | Block number | 50000000 |
| Modified | Block number | 51000000 |
| Characteristics | Per item | 165 |
Now in Code:
//Set ./sets/
"punks":{ //5 plus set name bytes
"a":"author", //the account that pays the set fee, --23 bytes
"s":"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5", //build app hash --53bytes
"i":1, //issued counter for IDs -6bytes
"m":4096, //max issue -6-10bytes
"n":"punks", //name of set, 7+ bytes
"r":100, // 8 bytes
"t":1, // 5bytes
"f":"png", //can be put in script
"l":"author/permlink", //count bytes ~50
"b":['ipfsNeuterpng', 'ipfsMalePNG', 'ipfsFemalePNG'], //base
"u":['cFromNFTsBase64_owner'], //unique
"c":[
[['ipfsLayer1PNGa', qtyINT, 0, mintedInt, primeInt],['ipfsLayer1PNGb', qtyINT, 1, mintedInt, primeInt]],
[['ipfsLayer2PNGa', qtyINT, 0, mintedInt, primeInt],['ipfsLayer2PNGb', qtyINT, 1, mintedInt, primeInt],['ipfsLayer2PNGc', qtyINT, 2, mintedInt, primeInt]]
], //characteristics of layers ... count bytes...
}
// Item ./nfts/dlux-io/
"punks:165":{ // 8-20 bytes
"m":50000000, //12-13 bytes
// above"c":165 // ~15 bytes set dependent (165=3*5*11, which is female, layer1a, layer2a)... this will draw out traits : c%primeInt == 0
}
Here you can see just how quickly the benefits add up. With a short named set like "punks" and an average account length of 12, each additional item is only adds the base64 bytes of the uniqueness with the owner account twice plus the individual modified time for auditing. Maybe ~50 bytes each. Given a large enough set this smaller size will quickly dominate the space requirement, and maintains our two way index. Where we could have stored 6 million IPFS NFTs we can now approach 20 million Layered NFTs
Reconstruction of the layers via the uniqueness code requires a log n of items per layer, and is about as computationally expensive as an array search.
Minting NFTs like this with a psuedorandom seed from blockchain data will have the most collisions in the middle of minting and return to no collisions at the tail, also trending on a log n of layers. A decent trade off for saving so much space. Several of the items in characteristics can be logically ommited but are named here for clarity, like primeInt. Having a qtyINT also let's the artist define the rarest traits... but a check at the definition stage will need to be performed to ensure the set won't fail uniqueness with the given parameters. For instance having 27 mint tokens for an set of the alphabet(26 items).
Moving NFTs
These type definitions can easily be built and distributed allowing a layer 2 to compute NFTs of any specific type.
Once a type definition is available on another layer 2 you should be able to move an NFT from one chain to another. This becomes especially handy if a layer 2 decides that each NFT must pay a rent for the space it's using on the blockchain. Maybe each NFT pays 0.01 rent for every 0.1 mint... every period of time. While a cold storage layer 2 might charge 0.001 rent for the same period... but auctions and account transfers aren't available, and only moving the item to a hot layer 2 will allow item modification.
I personally can't see saving much more than 30 bytes by doing this as the uniqueness locator would just be modified to the account of another layer2... maybe the swap account for hive engine... however... moving an IPFS NFT may save quite a bit more.
I'm inviting discussion of any aspect of NFTs, especially related to moving them around or if there is any part of creation, tracking, and storage that I might be over looking.
Reading through your posts, still at the beginning.
How can you be sure that the state backup will always remain available on IPFS if it's not incentivized?
It actually doesn't, it stay long enough that anybody who need to restart can, but IPFS is nearly built in memory management.
Any idea how long it's supposed to remain on there?
It seems to be up for at least a week, I haven't tried pulling anything too old because it's out of focus. If you're building a service with this you might want to store at least a daily snapshot of the state, most of the memory management happens once daily, and even if it was a gig, it'd take ~3 years to fill a consumer level 1TB harddrive.
Could always implement an incentive mechanism to keep selected backups. :)
The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.
Interesting discussion.
NFTs are great technology. It's worth getting interested and investing some money, because you can earn good money. I recommend this post on this topic: https://nftmonk.com/nft-royalties-passive-income-stream-in-a-volatile-market/