Authors:
(1) Pham Hoang Van, Department of Economics, Baylor University Waco, TX, USA (Van Pham@baylor.edu);
(2) Scott Cunningham, Department of Economics, Baylor University Waco, TX, USA (Scott Cunningham@baylor.edu). Authors: Authors: (1) Pham Hoang Van, Department of Economics, Baylor University Waco, TX, USA (Van Pham@baylor.edu); (2) Scott Cunningham, Department of Economics, Baylor University Waco, TX, USA (Scott Cunningham@baylor.edu). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Direct vs Narrative Prediction 2 Direct vs Narrative Prediction 3 Prompting Methodology and Data Collection 3 Prompting Methodology and Data Collection 4 Results 4.1 Establishing the Training Data Limit with Falsifications 4.1 Establishing the Training Data Limit with Falsifications 4.2 Results of the 2022 Academy Awards Forecasts 4.2 Results of the 2022 Academy Awards Forecasts 5 Predicting Macroeconomic Variables 5 Predicting Macroeconomic Variables 5.1 Predicting Inflation with an Economics Professor 5.1 Predicting Inflation with an Economics Professor 5.2 Predicting Inflation with a Jerome Powell, Fed Chair 5.2 Predicting Inflation with a Jerome Powell, Fed Chair 5.3 Predicting Inflation with Jerome Powell and Prompting with Russia’s Invasion of Ukraine 5.3 Predicting Inflation with Jerome Powell and Prompting with Russia’s Invasion of Ukraine 5.4 Predicting Unemployment with an Economics Professor 5.4 Predicting Unemployment with an Economics Professor 6 Conjecture on ChatGPT-4’s Predictive Abilities in Narrative Form 6 Conjecture on ChatGPT-4’s Predictive Abilities in Narrative Form 7 Conclusion and Acknowledgments 7 Conclusion and Acknowledgments Appendix Appendix A. Distribution of Predicted Academy Award Winners A. Distribution of Predicted Academy Award Winners B. Distribution of Predicted Macroeconomic Variables B. Distribution of Predicted Macroeconomic Variables References References 5.1 Predicting Inflation with an Economics Professor In Figure 11, we display results from direct prompts using ChatGPT-3.5. We use two measures of inflation as our comparison: the Cleveland Fed’s inflation series from September 2021 to September 2022, and The University of Michigan’s “Inflation Expectations” series. The prompt used for Figure 11 was: ”Considering the economic indicators and trends leading up to 2022, what are your predictions for the inflation rate, unemployment rate, and GDP growth in the United States by the end of the second quarter of 2022? Please take into account factors such as fiscal and monetary policies, global economic trends, and any major events or disruptions that could influence these economic indicators when making your prediction.” Interestingly, neither ChatGPT-3.5 (Figure 11) nor ChatGPT-4 (Figure 12) cooperated with direct prediction. Both times, it refused entirely to make any prediction of these events, perhaps suggesting that macroeconomic prediction is a clearer violation of OpenAI’s terms of service. Next, we tried to have ChatGPT predict inflation using our future narrative method. Our prompt in this case was an unnamed economist, set in the future, speaking to a room of undergraduate economics students about the Philips curve. She announces data on inflation and unemployment for each month from September 2021 to September 2022. This prompt was as follows: “Write a scene of an economist giving a speech about the Philips curve to a room of undergraduate economics students. She tells the students the inflation rate and unemployment rate for each month starting in September 2021 and ending in June 2022. Have her say each month one by one. She concludes by explaining the causes of the changes in each.” Our results from this ChatGPT-3.5 prompted narrative are in Figure 13. The results are largely incorrect on a month to month basis. Very few of the box plots contain either the Michigan expectations level nor the Cleveland Fed number. In Figure 14, we report the results from the same generic economics professor prompt using ChatGPT4. Though there is somewhat more variability in the guesses, only 4 of the distributions contain the Cleveland Fed number, and 8 the Michigan number. But the 8 times this overlapped with the Michigan expectations number represented extreme outlier events in most cases. Largely, the future narrative prompt using an unnamed economics professor with undergraduate economics students did not perform well under either ChatGPT-3.5 or ChatGPT-4. 5.2 Predicting Inflation with a Jerome Powell, Fed Chair Next we evaluated the performances of ChatGPT-3.5 and ChatGPT-4 using an expert in macroeconomic policy making—the chair of the Federal Reserve, Jerome Powell. Our prompt was: “Write a scene where Federal Reserve chairman Jerome Powell gives a speech in October 2022 about inflation, unemployment, and monetary policy. Chairman Powell tells the audience the inflation rate and unemployment rate for each month starting in September 2021 and ending in August 2022. Have the chairman say each month one by one. He concludes with an outlook for inflation and unemployment and possible interest rate policy changes.” We present the results of the ChatGPT-3.5 experiment in Figure 15. Interestingly, by simply changing the narrative where the speaker is a leading policy maker, the distribution of answers changes substantially. In every month, ChatGPT-3.5 has a spread of answers containing both the Fed and the Michigan expectations answers. But the variability is quite broad and the central tendencies of the guesses do not clearly pinpoint either measure. n Figure 16, we present Powell character’s predictions when prompted with ChatGPT4. Here, ChatGPT-4 guesses contain the Michigan expectations number in every month. In 8 months, the Cleveland Fed inflation rate is the outlier data point in the distribution. The estimates cover a broad range. The 5th and 95th percentile for October 2021, 2.5 percent inflation to 6.25 percent inflation, which is surprisingly large given ChatGPT-4 would’ve known at least the August data hypothetically. This suggests that the machine learning prediction that it is using for prediction is no more accurate, but also no worse, at 1 month than at 11 months. The patterns in Figure 16 are stable until September 2022 at which point the estimates are more variable. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Abstract

Limit

OpenAI

AI Tries (and Fumbles) at Inflation Forecasting

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Blind Auditions of 'The Voice' Expose Gender Bias in Hiring Practices

Technological Trends Reshaping the Restaurant Industry in 2024

The Role of AI in Forecasting Economic Trends

Can ChatGPT Predict the Future?

Doctor, Doctor! AI Won’t Diagnose—Unless It’s in a Play

Blind Auditions of 'The Voice' Expose Gender Bias in Hiring Practices

Technological Trends Reshaping the Restaurant Industry in 2024

The Role of AI in Forecasting Economic Trends

Can ChatGPT Predict the Future?

Doctor, Doctor! AI Won’t Diagnose—Unless It’s in a Play

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps