Originality and Decomposition of Generalization.

We are interested in well determining the effect of training data on generalization versus memorization. Because benchmarks are saturated, it has not been made clear yet, how much models are capable of deriving results from information that isnt duplicated (in some form), in the enormous training corpuses. This is confounded by the clear effect that nearby, or semantically duplicated data is a strong contaminate of benchmark data itself, so decontamination is very difficult, despite current efforts by large labs.

To help understand this problem we propose the following expirements.

1. Build two twin datasets, and inject them into training data at count N.

Dataset 1. Valid training data, related to the corpus text, via our methods of semantic duplication. The model can get this right over either memorization (extrapolation) or generalization.

Dataset 2. Identical in distribution items to dataset 1, so prompts are of the same benchmarkin training. However in this case solutions are selected at random (from the space of reasonable solutions to benchmark problems, i.,e. random multiple choice solutions). Since solutions are random, the only way for the model to recreate these solutions is via memorization.

By training on BOTH datasets at the same time, at rate given by count N per epoch, we observe that generalization is the difference in evaluation of Dataset 1 versus Dataset 2. Evaluation on Dataset 2 in this case is the model reproducing the random answer, the memorization artifact. If the model only memorizes then your delta will be approximately 0, that is the model will be right as frequently on dataset 1, as it is wrong (via memorization on Dataset 2). However if the delta is larger then 0, then we have shown a metric of generalization, since pure memorization always gives one 0. Behavior of model during experiment will be monitored by other nearby benchmarks. The behaviors of the model, and how these differences occur as training proceeds presents a method of understanding model generalization, and measures model capacity in a way that we believe is novel. A full study of these behaviors will occur, and be published, including likelihood, entropy, and complexity generalization.

2. Measuring benchmark duplication by inspection of computational graphs.

We propose to study benchmark alignment over computational graphs via methods like NTK (neural tangent kernels), and activation features to determine whether model computation can illicit a metric of semantic duplication, that is deeper then current methods. We hope to develop a functional continuous metric over model weights or computational graphs that decreases via semantic or notional similarity between benchmarks and a training point. Using such a definition, models who are trained on duplicates would show very little distance to contaminated benchmark points. If such a method is successful, we would 1) publish the work, 2) provide the method as an open source module for users to label and determine the contamination of their own benchmarks, over a model, against our large set of training data, held at Arb.

Ari Spiesberger - Arb Research Ai researcher, ML engineer, Scientist. Will direct project. Running research, directing effort and output, responsible for content.

Juan Velasquez - Arb Research AI researcher and Scientist. Co-author of Sem Dupes paper that motivates this work.

Vasudah Kowtha - Ex-Apple Audio Researcher, fellow at Berkman Klien center at Harvard. Will provide valuable input and ideas, help derive scientifically strong experiments, provide outside perspectives.