Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World

Michael Pichat, William Pogrund, Paloma Pichat, Armanouche Gasparian, Samuel Demarchi +4 more
2/25/2025
cs.CLcs.NEq-bio.NC

No confident code implementation found for this paper.

Abstract

How do language models segment their internal experience of the world of words to progressively learn to interact with it more efficiently? This study in the neuropsychology of artificial intelligence investigates the phenomenon of synthetic categorical restructuring, a process through which each successive perceptron neural layer abstracts and combines relevant categorical sub-dimensions from the thought categories of its previous layer. This process shapes new, even more efficient categories for analyzing and processing the synthetic system's own experience of the linguistic external world to which it is exposed. Our genetic neuron viewer, associated with this study, allows visualization of the synthetic categorical restructuring phenomenon occurring during the transition from perceptron layer 0 to 1 in GPT2-XL.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons

87% match2024

The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition

86% match2025

Neuropsychology and Explainability of AI: A Distributional Approach to the Relationship Between Activation Similarity of Neural Categories in Synthetic Cognition

85% match2024

Intra-neuronal attention within language models Relationships between activation and semantics

84% match2025

Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces

83% match2025

Neuropsychology of AI: Relationship Between Activation Proximity and Categorical Proximity Within Neural Categories of Synthetic Cognition

82% match2024

Cite this paper

@article{pichat2025synthetic,
  title  = {Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World},
  author = {Michael Pichat and William Pogrund and Paloma Pichat and Armanouche Gasparian and Samuel Demarchi and Martin Corbet and Alois Georgeon and Theo Dasilva and Michael Veillet-Guillem},
  year   = {2025},
  eprint = {2503.10643},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.10643v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.10643.svg)](https://codeofpaper.com/paper/2503.10643)

Discussion