Mine Intelligence Instead of Bitcoin: The Future of Green Computing

Numerous articles have been written about the enormous energy demands and environmental impact of Bitcoin mining. The amount of energy currently used for mining approaches the energy consumption of an entire country, and most of this energy comes from fossil sources. We have known this for a while. However, Sabine Hossenfelder recently made a video on the topic, revealing that Bitcoin mining can also be beneficial from a sustainability perspective.

The essence of the video is that in many countries, there is waste energy that cannot be used for anything because there is no electric grid infrastructure. This waste energy is ideal for Bitcoin mining, and the money generated from the energy can be used for good purposes. This is the highest level of green mining, as it uses energy that would otherwise go unused.

Mining Bitcoin is very cool, but couldn't we use this energy for something even more useful?

Everyone knows that there is only one thing that consumes more computational capacity than Bitcoin mining, and that is artificial intelligence. Why not mine intelligence instead of Bitcoin?

Training and running AI algorithms are just as computation-intensive as Bitcoin mining, and those who use waste energy for this can also earn income. However, in this case, the 'byproducts' are useful algorithms and knowledge. Imagine using the heat from volcanoes to find solutions for sustainable fusion energy production or searching for new materials to build more efficient batteries. With such a solution, mining would not only be non-polluting but could also help solve the problem of climate change.

The essence of Proof of Work (PoW), which underlies Bitcoin mining, is to solve a problem that is very difficult to solve but much easier to verify. The same is true for artificial intelligence. Training requires much more computational capacity than running the network. For a system like ChatGPT, for example, training costs millions of dollars in computational capacity, while each run costs only a few cents. Based on this, it seems that mining artificial intelligence is an ideal candidate to replace PoW; however, the situation is a bit more complicated.

In the case of PoW, verifying the result is straightforward. In contrast with a neural network, it is difficult to prove that the training actually took place and that it was done with the intended dataset.

Neural networks are trained using backpropagation. In a nutshell, this involves providing the network with inputs and the expected outputs, then calculating the error and adjusting the network accordingly. After the appropriate steps, ideally, the network's error is minimized, and it will produce the correct outputs for the given inputs. This is how ChatGPT was trained, partly using a dataset gathered from the Internet and partly with manual fine-tuning (RLHF). This backpropagation process needs to be outsourced in a way that it can be proven that the miner correctly ran the algorithm on the data.

The most straightforward solution for this is for the miner to create a log of every step, which includes the current weight matrix of the network, the inputs and outputs, and then the modified weight matrix. This way, the entire sequence of steps can be tracked and verified. Of course, if all the calculations had to be redone, the system would not make much sense. Therefore, it would be sufficient to verify a certain number of randomly selected steps. This is feasible because each step modifies a given weight matrix according to the data, allowing for the verification of individual steps.

Based on the above, a new type of blockchain concept is emerging that uses a consensus algorithm based on artificial intelligence training instead of PoW. There are two types of mining possible in the network. One type of miner trains a model (trainer), while the other type verifies the result (validator). If someone wants to train their model using the network, they upload the model and the dataset and deposit a certain amount of crypto that the trainer receives if the training is successful. The trainer sends the trained model to the requester upon completion of the training. Then, using an on-chain (provable) random number generator, the validators select a specified number of training phases from the log and verify them. If the verification is successful, the trainer receives the deposited amount, and the validator also receives a fee for the validation.

In broad terms, this is how such a network could operate. Validators and trainers would stake an amount that they would lose if they tried to cheat. Anyone could freely upload a model and training set and offer an amount of crypto for the training, similar to how transaction fees currently work on existing blockchains. After the training, the trained model and the training log are uploaded to a storage. The validators randomly check sections of the training log, and if they find it correct, the trainer receives the amount offered for the model training.

Theoretically, validators could be replaced by zk-SNARKs, which is a type of zero-knowledge proof. The essence of SNARK is that it allows one to verify that a complex computation was performed correctly with a simple calculation, without having to publish the computation itself. In this case, the role of validators would be taken over by a smart contract. The blockchain would automatically generate the indices of the sections that need to be verified. The trainer would need to present a valid SNARK for these sections. If successful, the smart contract transfers the staked amount to the trainer. The problem with this method is that generating SNARKs requires a lot of memory and computational capacity even for simpler algorithms, so it might not be practical for complex models. However, proving machine learning models with zero-knowledge proof (ZKML) is an active research area, so an efficient solution may be developed for this problem.

Although many details still need to be worked out regarding the concept, I believe that a specialized blockchain developed for training artificial intelligence is very promising and would benefit all of humanity.