A Game of Chess: Pitting ChatGPT Against Stockfish

Before I elaborate on my experiment, let me explain my proficiency in Chess. I am no advanced-level player. I know a few openings and sometimes fumble with chess notations. I mostly play blitz games against Stockfish.

A few months ago, I started playing against Stockfish Level 8, which is the highest level of difficulty offered at Lichess, the open Chess platform that comes with the most beautiful UI/UX.

As a player with limited chess knowledge and skills, I always found beating Stockfish Level 8 a daunting task. For those who are not aware of Stockfish Level 8 prowess, here is more. You need to be a super GM or someone with an ELO rating of 2600+ to achieve that feat. Again, I am not sure of what I said because Stockfish Level 8 usually plays with an accuracy of 99 percent with an average centipawn loss of almost 20.

In the past, I have challenged Stockfish Level 8 in a variety of ways. Once, I gathered a group of 5 Chess enthusiasts comprising all good players, and still, Stockfish Level 8 defeated us in 22 moves. My best performance against Stockfish Level 8 is being checkmated after 47 moves. Lol.

With that said, I thought, why shouldn’t I employ the power of ChatGPT? In the experiment, I have summoned the powers of ChatGPT 3.5 against Stockfish Level 8. The game had no time limits, and I was assigned white pieces.

My very first prompt to ChatGPT was the following:

hey, I am playing the game of chess against the chess engine stockfish level 8, which is the hardest level of difficulty offered. I am playing with white pieces so suggest me moves. Guide me throughout the game and now suggest me the opening.

Here is what it responded with

Sure, here’s a classic opening you can use as White: the Queen’s Gambit. It’s a solid opening that focuses on controlling the center of the board and developing your pieces efficiently. Here’s how you can start:

d4: This move is the start of the Queen’s Gambit. It immediately fights for control of the center and opens lines for your pieces.

Now, let’s see how Stockfish responds. It might play:

1… d5: Black also aims to control the center.

Now, you can continue with:

c4: This move is the Queen’s Gambit proper, offering a pawn sacrifice to gain central control.

Stockfish might accept the gambit with:

2… dxc4: Accepting the gambit pawn.

And then you continue with:

e4: This move aggressively reinforces your center and gains space on the board. It also threatens to regain the pawn on c4 with e4xd5.

Now, you’ve established a strong central presence and initiated an aggressive game. Keep developing your pieces and look for opportunities to exploit weaknesses in Stockfish’s position. Let me know how the game progresses, and I can provide further guidance as needed.

Sure, here’s a classic opening you can use as White: the Queen’s Gambit. It’s a solid opening that focuses on controlling the center of the board and developing your pieces efficiently. Here’s how you can start:

d4: This move is the start of the Queen’s Gambit. It immediately fights for control of the center and opens lines for your pieces. Now, let’s see how Stockfish responds. It might play:

1… d5: Black also aims to control the center.

Now, you can continue with:

c4: This move is the Queen’s Gambit proper, offering a pawn sacrifice to gain central control. Stockfish might accept the gambit with:

2… dxc4: Accepting the gambit pawn.

And then you continue with:

e4: This move aggressively reinforces your center and gains space on the board. It also threatens to regain the pawn on c4 with e4xd5. Now, you’ve established a strong central presence and initiated an aggressive game. Keep developing your pieces and look for opportunities to exploit weaknesses in Stockfish’s position. Let me know how the game progresses, and I can provide further guidance as needed.

I inadvertently played d5, a move not suggested by ChatGPT. The move by Stockfish was f5. Curious to know the implication of this, I updated ChatGPT about my move.

Here is the guidance from ChatGPT.

It seems Stockfish is opting for a somewhat unorthodox response with 1… f5, which is known as the Dutch Defense. This move aims to control the e4 square and potentially fianchetto the bishop on g7 to create a solid pawn structure on the kingside. Here’s how you can respond:

e4: This move challenges Black’s pawn on f5 and aims to open up lines for your pieces. It also gains more control of the center.
Now, Stockfish might respond with:

2… e6: This move supports the pawn on f5 and prepares to develop the bishop to d6 or e7.

You can continue with:

Nf3: This move develops a knight and prepares for further centralization of your pieces. It also indirectly supports the e4 pawn.
At this point, you have a solid position and are ready to continue developing your pieces. Look for opportunities to expand your control of the center and prepare for potential pawn breaks to create imbalances in the position. Let me know how Stockfish responds, and I’ll be here to guide you further.

Clearly, both responses showcase ChatGPT’s power to analyze the information and offer insights.

Sadly I failed to follow through the advice and played some other move. However, this could not be the end of the experiment. Thus, I made up my mind to start a game afresh. I quickly told ChatGPT this:

Me : I have begun a new game. suggest me an opening

ChatGPT : Certainly! Let’s try a different opening this time. How about the Italian Game? It’s a classical opening that focuses on developing your pieces rapidly while exerting pressure on Black’s position. Here’s how you can start:

e4: This move controls the center and frees your bishop and queen for future development.

Now, Stockfish might respond with:

1… e5: This is a symmetrical response, aiming to control the center and mirroring your moves.

You can continue with:

Nf3: This move develops your knight and attacks the e5 pawn.

Now, Stockfish might continue with:

2… Nc6: This move develops a knight and defends the e5 pawn.

You can now play:

Bc4: This move develops your bishop, aiming it at the vulnerable f7 square. It’s a key move in the Italian Game, often leading to aggressive attacking possibilities.

I began with white pieces and played e4. My idea was to control the central block of squares, and ChatGPT had already suggested that.

Stockfish moved a pawn to d6. ChatGPT inferred the motive as developing bishop and predicted the next step would be using Pirc Defense or the Philidor Defense. It suggested D4 as a response, but I did not find any merit in this move. Another suggestion was Nc3. I found this move to be better as it brings the knight to the front, and thus, e4 gets secured. Stockfish responded with e5, and ChatGPT suggested the development of the Pirc Defense, as in the last attempt. I played Nf3. But to our surprise, Stockfish came up with c5. As I have played so many times against the Stockfish, I have witnessed how it quickly develops bishop and queen. The motive is to checkmate you in the next 4 or 5 moves. I was suggested d5, and I found it to be a typing error as the pawn could move to d4 at most. Here, the game position was this:

No brainer for the above-average AI assistant. cxd4 was the obvious choice, and ChatGPT anticipated this move as expected. Like a reflex action, I would move the knight to b1. However, ChatGPT had something else in the mind. It suggested me Nxd4. And I did it with zero regard for a less of a sacrifice. I paid a little more price as I lost the knight to a pawn. In order to salvage this situation, I was suggested Bb5+, which is nothing but forcing a check on Stockfish’s king. Left with no choice, Stockfish had to bring in its bishop to d7. Though not among ChatGPT’s suggestions, I played Qxd4.

After this move, I updated ChatGPT with the moves I had played till then. And I was reverted with:

After this, I castled and asked about the next move. ChatGPT gave a response that was somewhat similar to its previous one. I was disappointed after casting. I could sense more trouble as I had lost a piece more than Stockfish.

Finally, the position was this:

The suggestions were not clear. They were repetitions and thus stopped my experiment.

This was the battle between a generalist (ChatGPT) and a specialist (Stockfish). In my opinion, ChatGPT fares well initially and excels at analyzing the moves, but the analysis lacks substantiated knowledge. It fails to advance you to the mid-game.

If you are thinking of hiring a bot or AI tool as your chess coach, then nothing could help you. Here is the score line according to me:

Stockfish Level 8 vs ChatGPT 3.5 : 1–0

Also published here.