While preparing for the WandX mainnet release after our prototype release, we were exploring scalable smart contracts for Order Management. While planning and thinking about our orders storage, we came across a few issues -

Array size: The size of the array to store orders was too large
Contract upgradability: If the contract needs to be upgraded or migrated, how can the orders data be migrated?
Gas costs: Array size increases rapidly as the contract starts processing more and more orders, in that case how can we keep gas price as low as possible for state changing transactions?
Contract storage: How can we maintain order history, keeping in mind that the contract may have to run for years?

Sample soldity code

We started exploring standard patterns in smart contracts to resolve the above issues, shortlisted a few options, but none were satisfactory. So we turned to a traditional database related solution of a History Table and Transaction Table concept. Still there is one limitation with History Tables which is the number of records and size of a single table; we came up with a solution for this by including partitioning or sharding.

Hence, we combined the concept of History/Transaction tables with Partitioning/Sharding and came up with new concept for Smart Contracts called “The Split Vault Algorithm”.

What is the Split Vault algorithm?

Consider the array as a Vault, which stores some information in it. Each vault can have operations like open,close, and seal. In a smart contract it will be a single giant vault with all orders.

Instead of having one array with all orders, we will have to split the same array into two or more based on the size and the number of orders and offload all the arrays as individual vaults. The keys we can use to split the storage are depends on the use case and the need. At WandX, we’ve used “time” as the key.

The main contract can refer to these vault addresses and retrieve the data when needed. Also, we can maintain an active reference for the current vault for easier access and a decent performance.

Benefits

Offloads the most of the orders to a different contract while keeping the references to the contracts for data retrieval when needed
Upgrading contracts becomes less painful
Faster Order Search
Limited and manageable array sizes without while minimizing gas costs

Originally posted on WandX blog. Checkout WandX marketplace here - https://www.wandx.co/

Managing the storage problem in Solidity — the Split vault algorithm

Sample soldity code

Benefits