How Attention-Based Models Outperform Traditional Bitcoin Fee Estimators

Table of Links

Abstract and 1. Introduction

2. Preliminaries
3. Problem definition
4. BtcFlow
5. Bitcoin Core (BCore)
6. Mempool state and linear perceptron machine learning (MSLP)
7. Fee estimation based on neural network (FENN)
8. Experiments
9. Conclusion, Acknowledgements, and References

8 Experiments

The datasets, experimental evaluation metrics, and transaction fee estimation solutions are all introduced in this part. Following that, we run a performance analysis on the experimental data.

8.1 Experiment settings

8.1.1 Datasets and implementation

We constructed datasets by picking 6 different block intervals at random via Blockchain Explorer[11]. Each dataset has 225 blocks, the first 180 blocks are used for training (about 400,000 transaction instances) and the last 45 blocks for testing (see Table 4). In terms of implementation, the hidden units in the sequence processing module in the feature extraction layer are set to 64, and the sequence length is set to 3. FENN’s prediction layer is a fully linked three-layer neural network with hidden units 64, 8 and 1, respectively. The Adam optimizer is used to optimize parameters using stochastic gradient descent (SGD) with a batch size of 1000 while training models. All of the algorithms are written in TensorFlow, and all of the tests are run on a single NVIDIA P100 12GB PCIe GPU.

8.2 Evaluation strategies

During test, RMSE and MAPE are calculated to evaluate the predictive error. Higher feerate transactions tend to confirm earlier than lower feerate transactions, hence in the fee estimate problem, the lower bound fee is usually returned. Compared to MAPE, RMSE concentrates more on avoiding high abnormal values, i.e. abnormal transaction fee values, which are outliers, have high impact on the error values.

Because of the SegWit upgrade in Bitcoin, a vByte was created to signify transaction size. It is roughly equivalent to four weight units. Typically, transaction feerate are expressed in sats/vByte. As a result, models with predicted feerates need to be converted to transaction fees using Eq.22. The transaction feein BtcFlow is the integer component of the value according to its official documents.

8.2.1 Compared methods

– BCore: We use the latest configuration in BCore (which is the same in V0.15 - V0.21), with a bucket interval of 5% and three alternative block period modes.

– BtcFlow: In order to simulate block generation speed, BtcFlow offers three distinct probability parameters: ‘Optimistic’, ‘Standard’, and ’Cautious’. The ’Standard’ mode is selected, with p = 0.8.

– MSLP: It is a one-layer neural network with a linear activation function.

– FENN variants: It includes LSTM models, attention models, and variants with various feature compositions.

– LSTM mechanism: LSTM in Eq. 18.

– Attention mechanism:

• Adv: Additive attention in refAdvEquation
• Self: Self-attention in Eq. 20
• Wht: A combined LSTM and a simple weighted attention in Eq. 21
• LSTMadv: A combined LSTM and additive attention

– Feature compositions on Adv:

• Adv Tx: Transaction features only
• Adv BloTx: Transaction features and network features
• Adv MemTx: Transaction features and current mempool states

8.3 Result analysis

We test on the genuine data to demonstrate the effectiveness and efficiency of FENN transaction fee estimation solution.

8.3.1 Estimation results comparison

Table 6 and Table 5 show an overall evaluation of performance over various confirmation time. FENN variants outperform earlier work across all datasets evaluated by RMSE and MAPE. Meanwhile, the models using the additive attention mechanism, Adv and LSTMadv, outperform other FENN models evaluated by MAPE. Furthermore, Adv has the best RMSE performance for all of the accessible datesets according to Table 5. In other words, Adv outperforms the other models when it comes to dealing with this estimation problem.

Besides, previous work models perform poorly, with BtcFlow being the worst of them all. Table 7 demonstrates that each existing model has a significantly higher estimation feerate than the lowest confirmed feerate and the median feerate in the target block, contradicting its feerate processing contradicts its assumption of strictly feerate processing priority. In the following section, we will study the effectiveness of our feature framework in FENN.

8.3.2 Impact of different features in Adv

We examine four different feature compositions (Adv Tx, Adv MemTx, Adv BloTx, and Adv) in the FENN framework to establish the efficiency of our feature composition. According to Fig.5 and Fig.6, the FENN framework’s Adv Tx has the poorest performance, and the accuracy can be improved by introducing mempool states and network features. Specifically, The accuracy of model Adv MemTx is increased when mempool states are incorporated into the Adv Tx feature structure, as measured by RMSE and MAPE.

Meanwhile, a same conclusion concerning the effectiveness of network features can be drawn based on the superiority of Adv BloTx to Adv Tx under RMSE, which is due to its ability to capture blockchain network trends. While network features exhibit a variety of effects evaluated by MAPE, as seen in Fig. 8. For example, when the block time varies substantially on the datasets S4, S5, and S6, Adv BloTx can improve Adv Tx’s accuracy by approximately 100%. While network features can have a negative impact on MAPE on S1 and S2 with a steady block time, these issues can be addressed by introducing mempool states, as demonstrated in model Adv. Furthermore, network features can have a modest favorable effect on Adv MemTx when compared to Adv performance on RMSE and MAPE, with the exception of one occurrence on S4 under RMSE. In conclusion, the FENN algorithm benefits from both mempool states and network features, and combining the two parts results in stable outperformance for Adv.

Finally, we compare Adv Tx against MSLP, which has already been proved to be the most effective in the existing work in Table 6 and Table 5. The effectiveness of introducing transaction details in this transaction fee estimate issue is demonstrated by the superiority of Adv Tx. In conclusion, FENN demonstrates the effectiveness of introducing transaction features, network features, and mempool states.

8.3.3 Time efficiency of FENN variants

We conduct experiments to illustrate the efficiency of our proposed FENN framework algorithms. Table 8 indicates that all FENN variations can complete the training process in one block interval, demonstrating that our framework can handle continuous Bitcoin blockchain data for model updates. Moreover, compared to LSTMembedded algorithms, the training time for Adv and Self can be reduced almost 50%.

8.3.4 Training frequency in Adv

In prior experiments, Adv has proven to be useful and efficient. Another essential characteristic of Adv is the ability to adapt to new information. We undertake a set of tests to see study its performance with different update frequencies. In our research, we present six different update policies (namely, 1,3,5,9,15, and 45), which imply retraining models at different block intervals. Fig. 5 and Fig. 6 show how Adv performs in terms of accuracy. As we can see, the accuracy of Adv falls as the updating block interval grows. The best frequency policy is one block. Furthermore, when we compare the 3-block technique to the existing work (BCore,

MSLP and BtcFlow), we discover that it still outperforms them, implying that our FENN has the ability to incorporate more details in future work.

9 Conclusion

This work begins by documenting and analyzing previous transaction fee estimation research. Then we proposed a new neural network-based framework to analyze complex interactions from a wider range of sources, including transaction details, network features, and mempool states, in order to address the issues of inferior estimation accuracy and limited knowledge used in previous work. The effectiveness and efficiency of our suggested architecture have been demonstrated on genuine blockchain datasets.

Acknowledgements The authors are thankful for the support from Data61, Australian Research Council Discover grants DP170104747, DP180100212, DP200103700 and National Natural Science Foundation of China grant 61872258.

References

1. Al-Shehabi, A.: Bitcoin transaction fee estimation using mempool state and linear perceptron machine learning algorithm. Master’s thesis, San Jose State University (2018)

2. Antonopoulos, A.M.: Mastering Bitcoin: unlocking digital cryptocurrencies. ” O’Reilly Media, Inc.” (2014)

3. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

4. Chaum, D.: David chaum on electronic commerce how much do you trust big brother? IEEE Internet Computing 1(6), 8–16 (1997)

5. Dang, H., Dinh, T.T.A., Loghin, D., Chang, E.C., Lin, Q., Ooi, B.C.: Towards scaling blockchain systems via sharding. In: Proceedings of the 2019 international conference on management of data, pp. 123–140 (2019)

6. Dinh, T.T.A., Liu, R., Zhang, M., Chen, G., Ooi, B.C., Wang, J.: Untangling blockchain: A data processing view of blockchain systems. IEEE Transactions on Knowledge and Data Engineering 30(7), 1366–1385 (2018)

7. Dinh, T.T.A., Wang, J., Chen, G., Liu, R., Ooi, B.C., Tan, K.L.: Blockbench: A framework for analyzing private blockchains. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 1085–1100ACM (2017)

8. Easley, D., O’Hara, M., Basu, S.: From mining to markets: The evolution of bitcoin transaction fees. Journal of Financial Economics 134(1), 91–109 (2019)

9. Eyal, I., Gencer, A.E., Sirer, E.G., Van Renesse, R.: Bitcoin-ng: A scalable blockchain protocol. In: 13th {USENIX} symposium on networked systems design and implementation ({NSDI} 16), pp. 45–59 (2016)

10. Eyal, I., Sirer, E.G.: Majority is not enough: Bitcoin mining is vulnerable. In: International conference on financial cryptography and data security, pp. 436–454. Springer (2014)

11. Felbo, B., Mislove, A., Søgaard, A., Rahwan, I., Lehmann, S.: Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. arXiv preprint arXiv:1708.00524 (2017)

12. Fu, R., Zhang, Z., Li, L.: Using lstm and gru neural network methods for traffic flow prediction. In: 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp. 324–328. IEEE (2016)

13. Gers, F.A., Eck, D., Schmidhuber, J.: Applying lstm to time series predictable through time-window approaches. In: Neural Nets WIRN Vietri-01, pp. 193–200. Springer (2002)

14. Gilad, Y., Hemo, R., Micali, S., Vlachos, G., Zeldovich, N.: Algorand: Scaling byzantine agreements for cryptocurrencies. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 51–68 (2017)

15. Hileman, G., Rauchs, M.: Global cryptocurrency benchmarking study. Cambridge Centre for Alternative Finance 33, 33–113 (2017)

16. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)

17. Jack, W., Suri, T.: Mobile money: The economics of mpesa. Tech. rep., National Bureau of Economic Research (2011)

18. Karim, F., Majumdar, S., Darabi, H., Chen, S.: Lstm fully convolutional networks for time series classification. IEEE access 6, 1662–1669 (2017)

19. Kasahara, S., Kawahara, J.: Effect of bitcoin fee on transaction-confirmation process. arXiv preprint arXiv:1604.00103 (2016)

20. Li, J., Yuan, Y., Wang, S., Wang, F.Y.: Transaction queuing game in bitcoin blockchain. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 114–119. IEEE (2018)

21. Luu, L., Narayanan, V., Zheng, C., Baweja, K., Gilbert, S., Saxena, P.: A secure sharding protocol for open blockchains. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 17–30 (2016)

22. McNally, S., Roche, J., Caton, S.: Predicting the price of bitcoin using machine learning. In: 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), pp. 339–343. IEEE (2018)

23. Mukhopadhyay, U., Skjellum, A., Hambolu, O., Oakley, J., Yu, L., Brooks, R.: A brief survey of cryptocurrency systems. In: 2016 14th annual conference on privacy, security and trust (PST), pp. 745–752. IEEE (2016)

24. Nakamoto, S.: Bitcoin: A peer-to-peer electronic cash system. Tech. rep., Manubot (2019)

25. Ruan, P., Chen, G., Dinh, T.T.A., Lin, Q., Ooi, B.C., Zhang, M.: Fine-grained, secure and efficient data provenance on blockchain systems. Proceedings of the VLDB Endowment 12(9), 975–988 (2019)

26. Salah, K., Rehman, M.H.U., Nizamuddin, N., Al-Fuqaha, A.: Blockchain for ai: Review and open research challenges. IEEE Access 7, 10127–10149 (2019)

27. Schwartz, D., Youngs, N., Britto, A., et al.: The ripple protocol consensus algorithm. Ripple Labs Inc White Paper 5(8) (2014)

28. Sharma, A., Schuhknecht, F.M., Agrawal, D., Dittrich, J.: Blurring the lines between blockchains and database systems: the case of hyperledger fabric. In: Proceedings of the 2019 International Conference on Management of Data, pp. 105–122 (2019)

29. Srivastava, N., Mansimov, E., Salakhudinov, R.: Unsupervised learning of video representations using lstms. In: International conference on machine learning, pp. 843– 852 (2015)

30. Sundermeyer, M., Schl¨uter, R., Ney, H.: Lstm neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association (2012)

31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008 (2017)

32. Vukoli´c, M.: The quest for scalable blockchain fabric: Proof-of-work vs. bft replication. In: International workshop on open problems in network security, pp. 112–125. Springer (2015)

33. Wang, S., Dinh, T.T.A., Lin, Q., Xie, Z., Zhang, M., Cai, Q., Chen, G., Fu, W., Ooi, B.C., Ruan, P.: Forkbase: An efficient storage engine for blockchain and forkable applications. arXiv preprint arXiv:1802.04949 (2018)

34. Wood, G., et al.: Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151(2014), 1–32 (2014)

35. Xu, C., Zhang, C.: Towards searchable and verifiable blockchain (2019)

36. Xu, C., Zhang, C., Xu, J.: vchain: Enabling verifiable boolean range queries over blockchain databases. In: Proceedings of the 2019 international conference on management of data, pp. 141–158 (2019)

37. Xu, Z., Han, S., Chen, L.: Cub, a consensus unit-based storage scheme for blockchain system. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 173–184. IEEE (2018)

38. Yaga, D., Mell, P., Roby, N., Scarfone, K.: Blockchain technology overview. arXiv preprint arXiv:1906.11078 (2019)

39. Zhang, C., Xu, C., Xu, J., Tang, Y., Choi, B.: Gemˆ 2-tree: A gas-efficient structure for authenticated range queries in blockchain. In: 2019 IEEE 35th international conference on data engineering (ICDE), pp. 842–853. IEEE (2019)

40. Zheng, Z., Xie, S., Dai, H., Chen, X., Wang, H.: An overview of blockchain technology: Architecture, consensus, and future trends. In: 2017 IEEE international congress on big data (BigData congress), pp. 557–564. IEEE (2017)

41. Zhu, L., Laptev, N.: Deep and confident prediction for time series at uber. In: 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 103–110. IEEE (2017)

42. Zhu, Y., Zhang, Z., Jin, C., Zhou, A., Yan, Y.: Sebdb: Semantics empowered blockchain database. In: 2019 IEEE 35th international conference on data engineering (ICDE), pp. 1820–1831. IEEE (2019)

[11] https://www.blockchain.com/explorer