Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching
jinheon
Abstract
Recent advances in large language models have demonstrated remarkable reasoning capabilities through Chain of Thought (CoT) prompting, but often at the cost of excessive verbosity in their intermediate outputs, which increases computational overhead. We introduce Sketch-of-Thought (SoT), a novel prompting framework that combines cognitive-inspired reasoning paradigms with linguistic constraints to minimize token usage while preserving reasoning accuracy. SoT is designed as a flexible framework that can incorporate any custom reasoning paradigms based on cognitive science, and we instantiate it with three such paradigms - Conceptual Chaining, Chunked Symbolism, and Expert Lexicons - each tailored to different reasoning tasks and selected dynamically via a lightweight routing model. Through comprehensive evaluation across 15 reasoning datasets with multiple languages and multimodal scenarios, we demonstrate that SoT achieves token reductions of 76% with negligible accuracy impact. In certain domains like mathematical and multi-hop reasoning, it even improves accuracy while using significantly fewer tokens. Our code is publicly available: https://www.github.com/SimonAytes/SoT.
Community
Sketch-of-Thought (SoT) is a novel prompting framework inspired by human cognitive science that dramatically reduces token usage by an average of 75% across various reasoning tasks with minimal accuracy impact, all without requiring additional training or finetuning. We introduce three specialized reasoning paradigms (Conceptual Chaining, Chunked Symbolism, and Expert Lexicons) along with a lightweight router model for selecting the optimal paradigm based on query characteristics. The router model is available on Huggingface (https://huggingface.co/saytes/SoT_DistilBERT) and an easy-to-use Python package is available on GitHub (https://github.com/SimonAytes/SoT).
You can get started with Sketch-of-Thought (SoT) with the following code snippet. For further details on how to use SoT with our Python package, please see our GitHub repository.
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
import torch
import json
# Load the model directly from Hugging Face
model = DistilBertForSequenceClassification.from_pretrained("saytes/SoT_DistilBERT")
tokenizer = DistilBertTokenizer.from_pretrained("saytes/SoT_DistilBERT")
# Define label mapping
label_mapping = {
"chunked_symbolism": 0,
"conceptual_chaining": 1,
"expert_lexicons": 2
}
# Function to classify questions
def classify_question(question):
inputs = tokenizer(question, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits, dim=1).item()
# Reverse mapping to get the paradigm name
label_mapping_reverse = {v: k for k, v in label_mapping.items()}
return label_mapping_reverse[predicted_class]
# Example usage
question = "Alice has 5 apples. She gives 3 apples to Bob. How many apples does Alice have?"
paradigm = classify_question(question)
print(f"Recommended paradigm: {paradigm}") # Output: "chunked_symbolism"
Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper