Research Evaluation After a month of working, the full research report is finally done! Being that it’s 40 pages long in total, you can find it on GitHub either in rendered Mark-Down or PDF format. I recommend reading the main paper, but I’ve included a quick summary below. rendered Mark-Down PDF format Summary This report evaluates a six-month live trading experiment in which a large language model (ChatGPT) managed a micro-cap equity portfolio under strict, forward-only constraints. Rather than optimizing for performance, the experiment was designed to study decision behavior: how the model reacted to gains and losses, how often it re-entered positions, how it handled risk through stop-losses, and how its decisions evolved over time. All trades, portfolio states, prompts, and outcomes were logged and preserved to ensure transparency and auditability. decision behavior Overall analysis was mixed. While the portfolio failed to outperform major benchmarks over the full period, the results revealed several consistent behavioral patterns. The model demonstrated persistence in certain position-level theses, frequently re-entering previously traded securities even after poor realized outcomes. At FIFO lot and pure PnL level, removing a single extreme outlier (ATYR) materially altered aggregate metrics such as profit factor and expectancy, highlighting the sensitivity of short-horizon micro-cap strategies to tail events. These findings reinforce the importance of the structure and stability of decision-making. Importantly, this analysis reflects a single experimental run and does not attempt to generalize performance claims. Instead, the value of the experiment lies in what it reveals about how large language models behave when placed in environments with irreversible consequences and limited human guidance. The full report documents methodology, failure modes, and limitations in detail, and serves as a foundation for future experiments focused on understanding LLM decision-making under risk. Future work will extend this framework to additional market regimes and repeated trials to better separate behavioral tendencies from noise. LLM Investor Behavior Benchmark — LIBB LLM Investor Behavior Benchmark — LIBB After noticing a lack of a flexible, open-source standard research library for LLM experiments, I changed my focus from a generic LLM trading benchmark to powerful library for researchers. Just create your prompts and schedule and LIBB can automatically create the predictable filesystem, handle all portfolio processing, analyze sentiment, behavior, and performance, and graph your collected data. Because of its simplicity and flexibility, use future experiments will use LIBB for experimental setup. Code Example from libb.model import LIBBmodel
from .prompt_models import prompt_daily_report
from libb.other.parse import parse_json
import pandas as pd

MODELS = ["deepseek", "gpt-4.1"]

def daily_flow(date):
    for model in MODELS:
        libb = LIBBmodel(f"user_side/runs/run_v1/{model}", run_date=date)
        libb.process_portfolio()
        daily_report = prompt_daily_report(libb)
        libb.analyze_sentiment(daily_report)
        libb.save_daily_update(daily_report)

        orders_json = parse_json(daily_report, "ORDERS_JSON")

        libb.save_orders(orders_json)
    return from libb.model import LIBBmodel
from .prompt_models import prompt_daily_report
from libb.other.parse import parse_json
import pandas as pd

MODELS = ["deepseek", "gpt-4.1"]

def daily_flow(date):
    for model in MODELS:
        libb = LIBBmodel(f"user_side/runs/run_v1/{model}", run_date=date)
        libb.process_portfolio()
        daily_report = prompt_daily_report(libb)
        libb.analyze_sentiment(daily_report)
        libb.save_daily_update(daily_report)

        orders_json = parse_json(daily_report, "ORDERS_JSON")

        libb.save_orders(orders_json)
    return With only one user created function, LIBB allows a simple workflow to automatically create identical file systems for different models, process pending orders, save future orders and raw reports, and analyze and log sentiment from the given reports. Example Graph LIBB also allows ease for graphing. By saving report sentiment over a month of back-testing, I can call a function and LIBB generates: Sooo.. What’s Next? Right now I’m focusing my attention on LIBB and making sure it provides a stable foundation for my future experiment over new IPOs. After that, I’ll begin developing the experiment’s workflow and create a blog finalizing rules and explaining the setup. As always, I post more frequent updates on both X and LinkedIn. X LinkedIn Let me know what you guys think of the report and the new project!

Can ChatGPT Outperform the Market? Final Report

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Can ChatGPT Outperform the Market? A Finance Nerd's Quest to Find Out

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Can ChatGPT Outperform the Market? A Finance Nerd's Quest to Find Out

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps