arXiv (Cornell University) · 2023 · 566 citations · 0 references
ComputingComputer ScienceCreative WritingArtificial IntelligenceComputational LinguisticsNatural Language ProcessingLanguage ModelingProblem SolvingCreative Writing ProcessLanguage Model PromptingLanguage Processing ModelsNatural Language InferencePromptingReasoning ModelsSearch Space Exploration
Language models are widely used for general problem solving but are limited by token‑level, left‑to‑right inference, which hampers tasks requiring exploration, strategic lookahead, or critical early decisions. This work introduces the Tree of Thoughts framework to enable language models to explore multiple reasoning paths and make deliberate, self‑evaluated decisions during inference. Tree of Thoughts lets language models generate and evaluate multiple coherent “thoughts,” enabling them to look ahead, backtrack, and choose globally optimal paths during inference. Experiments demonstrate that Tree of Thoughts boosts performance on tasks requiring planning or search, achieving a 74 % success rate on Game of 24 compared to 4 % with chain‑of‑thought prompting, and improving results on Creative Writing and Mini Crosswords. Code and prompts are available at https://github.com/princeton-nlp/tree-of-thought-llm.
Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving. ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices. Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. Code repo with all prompts: https://github.com/princeton-nlp/tree-of-thought-llm.