Parameter
A learnable value in a neural network that gets adjusted during training to improve model performance.
AI-generated
Parameters are the numbers inside a neural network that determine its behavior. During training, these numbers are adjusted to improve the model's predictions. When people say "GPT-4 has over a trillion parameters," they mean the model has over a trillion adjustable numbers that collectively encode its knowledge and capabilities.
Parameter count is a rough proxy for model capability. Larger models generally perform better (up to a point), but they also cost more to train and run. Understanding parameter counts helps you compare models and understand the trade-offs: a 7B parameter model is faster and cheaper but less capable than a 70B parameter model.
Andrej Karpathy: "Intro to Large Language Models" - https://www.youtube.com/watch?v=zjkBMFhNj_g
Wikipedia: Neural network parameters - https://en.wikipedia.org/wiki/Artificial_neural_network