Synthetic Data
Training data generated by AI models rather than collected from real-world sources.
AI-generated
Synthetic data is artificially generated data used to train AI models. Instead of collecting real examples (which is expensive and raises privacy concerns), you use an existing AI model to generate training examples. For instance, GPT-4 might generate math problems to train a math-focused model.
Synthetic data is increasingly important because high-quality real training data is becoming scarce. AI labs have already trained on most of the publicly available internet text. Synthetic data generation is one solution to this "data wall." It is also used for safety training: generating examples of harmful requests and appropriate refusals to teach models safe behavior.
NVIDIA: What is synthetic data? - https://www.nvidia.com/en-us/glossary/synthetic-data/
MIT Technology Review: "The growing role of synthetic data" - https://www.technologyreview.com/