Creating a tracker # 2 Creating a plugin for RDBMS linkage

Hello, we’re the eosBLACK team.

We are working on a series on the topic of making trackers using a development journal. In the previous post, we introduced “Making a tracker: Understanding the role and structure of the plugin”. We will review how to create a plug-in for RDBMS linkage in this posting, the second in the series on tracker making.

Why we started development

EOSIO provides a blockchain data storage method using MongoDB, called eosio::mongo_db_plugin. When this method is used for tracker making, the block data can be viewed using MongoDB instead of nodes. However, you may wonder why we tried to take our time to make a separate plug-in using MySQL DB.

MongoDB has the advantage of being intuitive and flexible enough to store any data. However, it is inconvenient to divide the data as desired and retrieve only necessary data as there is no joining function due to the properties of MongoDB. Because the B-Tree index is used, MongoDB also has the disadvantage of performance degradation with data size increase when new data is entered.

Even though MongoDB is suitable for simple data input and retrieval, it is not suitable for storing 24 million blocks and hundreds of millions of action data continuously. That is why we have selected SQL family databases. In particular, we have created a plug-in for MySQL that is widely used in web services.

Ultimately, we have considered the system with the above structure and the plug-in was developed to build a separate DBMS so that the stored data can be applied in various ways using the web, mobile, console, and so on.

Creating a MySQL Plugin

As explained in the previous article, create a new plugin directory and default files using eosio-make_new_plugin.sh under the EOSIO/plugins directory to create the directory and files. If the name is not specified, a default template will be created, so assign an appropriate name which allows for the task to be performed twice.

The newly created plug-in may be located under eosio/plugins, but it is recommended to locate it outside of the EOSIO source if possible. It is recommended to place it outside considering various circumstances, as an option is available for building together even though the plugin is outside of EOSIO since the eosio 1.2.x version.

Once the plug-in is created as described above, basic preparation is complete. The following section describes how to build.

After EOS 1.2.x

Before EOS 1.2.X

Configuring the MySQL plugin

Tables should be defined and created to save the data needed for tracker creation in the DB. The names of the required tables are as follows.

Please refer to the published github source for the detailed table schema.

Source: https://github.com/eosBLACK/eosio_mysqldb_plugin

Create account_table, actions_table, blocks_table, and transactions_table as a separate class and process using a method. The following source code shows the contents of the init function of the mysql_db_plugin_impl class, which creates threads to handle the data received by connecting the database, and performs the necessary operation for initial setting.

Now, set up the event that receives the data from the node as follows, as explained in the previous post.

Connect each of the four signals that you set up to the functions as shown below, and the data suitable for each event will be received.

Once we save all data, it will be the complete. However, since we have to worry about the data capacity, we will save only the data stored in the irreversible block in the SQL DB. If you want to process only the data placed in the block, you can choose to receive only the accepted_block and irreversible_block event.

At the time of writing this article, the total number of blocks in the EOSIO main net exceeded 24 million. The structure of EOSIO consists of Block Transaction = 1:n, and Transaction: action = 1:n once again. In addition, since the action has a series of inline actions, the size of the action data becomes several times the number of blocks. As a matter of fact, more than 1,000 transactions can be included in a block. It is therefore easy to imagine that each transaction might contain a considerably high number of actions.

For this reason, the applied_transaction event is configured to handle transaction_trace information and action information.

So far, we have explained the configuration of the MySQL DB plugin based on the source. Please refer to the source document for details, based on this article.

We can then try building and running the source code.

Finish

You are now ready to import the block data to create a tracker. In the following post, we will explain how to call an actual node, retrieve the block data, save it in the DB, and use it in the tracker.

Thank you

eosBLACK team

eosBLACK Contact:

eosBLACK Homepage
eosBLACK Koreos
eosBLACK Medium
eosBLACK steemit
eosBLACK Facebook
eosBLACK twitter
eosBLACK Telegram(Korean)
eosBLACK Telegram(English)

White Paper (Chinese)
White Paper (Korean)
White Paper (English)