Scaling blockchain networks is incredibly important as the demand on these systems is exponentially growing. To better understand what sharding is and how it applies to databases we first need to understand the basics.
What is Sharding?
‘Sharding’ is a technique specifically for database partitioning. It dramatically scales the Ethereum Blockchain and allows it to process more transactions per second. Sharding is the horizontal partitioning of databases through separation into rows. The conceptualization of shards (i.e. the rows) draws primarily from characteristics. For instance, a singular shard may be responsible for storing an address’ state and transaction history. Alternatively, it may be possible to divide shards concerning the type of digital asset inside them.
Transactions with the involvement of that digital asset might be possible by way of a collection of random shards. For example, take into consideration a rental real estate transaction that involves multiple shards. These shards are correspondent to various entities in the transaction. This ranges from customer names to digital keys. These are arranged into a small lock made available to the renter when the time comes for the rent payment.
The importance
The promise of decentralization and immutable record make blockchain the significant innovation that it is. While Bitcoin and Ethereum are its most prominent applications, financial technology (FinTech) companies quickly understood its value outside of cryptocurrency. Multiple companies - some outside of FinTech - caught on to the promise of blockchain. Moreover, they are working on upsetting the industry verticals they are in with the use of this technology.
For blockchains to become a truly universal entity they must address the following:
- The scalability issues
- The latency issues
- Low-level throughput
Nodes
All blockchains that are public and lack permission have nodes. In other words, computers on the network that maintain a similar version of the information. Each and every node has complete information about the entire blockchain. This ensures the stability of the chain, like a cyber attacker being unable to destroy it by taking out a central server.
A majority of public blockchains possess secure means of guaranteeing data integrity. There is no way for an existing block to go through deletion or experience modifications. The addition of a new block will update a blockchain, and in a public blockchain, any node can do so. Therefore, there is a need to preserve the stability and the order of the transactions. Blockchain is able to pull this off with the help of a ‘consensus mechanism.’
Mechanisms and Miners
The more common consensus mechanisms use ‘proof of work’ (PoW) algorithms. A ‘miner’ will add a new block, but only after solving complex cryptographic puzzles.
‘Miners’ are actually a combination of three essential things. These are:
- Special-purpose software
- Hardware with a powerful design
- The user
The miner has to complete a large mathematical operation at high-speed and in a competitive environment. Not only that, the PoW algorithm requires that the majority of participating nodes approve of the transaction. In a network that’s decentralized, no one can infiltrate the computing power of most of the node. This basically makes hacking blockchain economically impossible.
Every node contains information on the blockchain and the PoW consensus algorithm makes the chain secure. Overall, it’s pretty solid. Having said that, the design principles impact scalability and transaction throughput negatively. As the blockchain grows, so too does the amount of node on each node. Participation of every engaging node in transaction validation slows transaction processing considerably.
The solution
Again, this is where the concept of sharding comes into play. In terms of blockchain, each node has only a portion of the data on the blockchain with the execution of sharding. Nodes that preserve a shard maintain information only on that shard in a mutual manner. So, at least within a shard, there is still the preservation of the decentralization. However, each node does not load the information on the entirety of the blockchain. This, essentially, aids in the scalability.
Algorithms
PoS possesses specific nodes that take responsibility for transaction validation. These nodes are ‘stakers’, as they stake some of their crypto tokens for transaction confirmation. Upon successful validation, the staker may earn part or all of the transaction fees. The more crypto tokens a staker obtains, and the longer the duration of the stake, the higher the number of transactions the node validates.
PoS algorithm has some notable advantages over PoW:
- Only specially chosen nodes, not the entire network, confirm the transaction.
- Seeing as how there’s no mining, there is no need for expensive special-purpose hardware. The energy requirements are much lower.
- Identifying validators with high loyalty is an easy task. Pick the ones who stake a higher number of crypto tokens and longer duration. A blockchain that uses sharding identifies stakers in each shard, who’s responsible for the transaction validation. Because the validation is done by legal stakers, the blockchain’s security is still very high.
Network Sharding
This protocol uses PoW to divide up the validation of transactions across the network. It chooses and continually updates a ‘directory service committee’ (DS Committee). This is a group of randomly chosen nodes that assigns transactions to specific shards and validates blocks of transactions. These transactions were proposed by those shards.
A transaction on this platform involves a user initiating a transaction, which is then sent to a shard. The shard authenticates the transaction, grouping it with other transactions to form a ‘microblock.’ The shard reaches a consensus on the validity of the microblock and is sent to the DS Committee. From here, they combine the microblocks into the ‘final block.’ The DS Committee will run a final consensus on this block before adding it to the blockchain.
The blockchain protocol that confirms each block to update the state of the chain is the ‘Elliptic Curve Based Schnorr Signature Algorithm’ (EC-Schnorr). It also uses a type of practical Byzantine fault tolerance for each shard. This allows shards to agree on blocks that they propose to the DS Committee, and for the Committee to reach a consensus on the final block.
With network sharding, developers believe that the network will match Mastercard and Visa in transaction processing speed. The fees will be lower and there’s no reliance on a central authority to manage the network.
Computational Sharding
This type of sharding refers to the ability of the network to run DApps at scale. Moreover, with more than one instruction being run across the network concurrently. This is unlike other DApp and smart contract platforms, like Ethereum. These only allow one instruction at a time, requiring the repetition of tasks across network nodes.
How this method is done is by coordinating the data dependencies between operations. It allows a shard to execute independent operations, or even a portion of the nodes in a shard, once the operations have what they need to run.
