What's up with the Steem Smart Contracts tool?

Hi Everyone!

For those how have been following the Steem Smart Contracts repository you might have seen that I recently merged a major branch to master, indeed I've been working on rewriting the entire architecture of the tool for a few weeks.

Why rewriting the architecture?

A blockchain is built in a synchronous manner meaning that blocks are produced one after the other, same goes for the transactions, the problem with that is that when you do synchronous actions, you actually block the process of your application while the block/transaction is processed. The tool has been designed to embed the block production, the database as well as the JSON RPC server which means when a block is produced (hence blocking the process) nobody can interact with the other major components of the tool (retrieve data from the database for example). NodeJS is mono-thread by design (though it can now handles multi-threads via workers in its latest version) which means that you have only one big process that will handle all your actions one after the other. (produce block -> update database -> handle JSON RPC request -> etc...)

The new architecture

The tool has been revamped to use the child_process feature of Node JS, you basically now have one main process (still) but each big component of the tool is now running inside its own process. Each big component is now named plugin, you have the blockchain plugin, the database plugin, the JSON RPC plugin, etc..., and they all communicate with each other via the Inter Process Communication (IPC) protocol, which is a messaging bus that is available in all the running processes (main and child processes). The main process now handle only one basic but very important function, the routing of the messages between the plugins.

What does this new architecture change for the users?

It basically doesn't change anything for the end user except the fact that they won't have to wait the end of the production of a block to retrieve data from the JSON RPC server. One major thing that changed though is how the smart contracts are written, since the plugins are all decoupled from each other, they don't share the same memory during the execution of the tool, this leads to the usage of asynchronous methods to retrieve information from a plugin, for example, to retrieve a user from the database from a smart contract you'll have to send a request (via IPC) to the database plugin and wait for the database plugin to send the response back. (this is extremely fast but still requires to use the "await" keyword in the smart contracts) (the wiki has been updated accordingly https://github.com/harpagon210/steemsmartcontracts/wiki/Database-API and https://github.com/harpagon210/steemsmartcontracts/wiki/Smart-Contracts-API)

Other changes

Another new feature was brought with the last major pull-request, the ability to replay the sidechain from a blocks.log file. This is a "basic" feature for the tool but it was missing and it will save some time to those who want to setup their own node.

Why setup a node? I'm currently working on implementing a block production reward system, this will also introduce fees in the system, the owners of smart contracts will have to buy tokens to be able to use the storage and the bandwidth of the sidechain. This will be a basic system where the owners of smart contracts will have to pay for the storage they use (number of tokens / byte used) as well as the number of transactions related to their smart contracts (number of tokens / transaction)

This will be a major move for this tool and it will introduce a utility token that will be independant from Steem.

More about the block production in another post!