GPT-4 Technical Report

TLDR

We report the development of GPT‑4, a large‑scale multimodal model that accepts image and text inputs and produces text outputs. GPT‑4 is a Transformer‑based model pre‑trained to predict the next token, built with scalable infrastructure and optimization methods that enable performance prediction from models using only 1/1,000th the compute. GPT‑4 shows human‑level performance on professional and academic benchmarks, such as a top‑10% simulated bar exam score, and post‑training alignment improves factuality and adherence to desired behavior.

Abstract

We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4.