CONVERGENCE OF THE PARTITION FUNCTION IN THE STATIC WORD EMBEDDING MODEL
Mynbaev K. Assylbekov Z.
2022L.N. Gumilyov Eurasian National University
Eurasian Mathematical Journal
2022#13Issue 470 - 81 pp.
We develop an asymptotic theory for the partition function of the word embeddings model word2vec. The proof involves a study of properties of matrices, their determinants and distributions of random normal vectors when their dimension tends to infinity. The conditions imposed are mild enough to cover practically important situations. The implication is that for any word i from a vocabulary W, the context vector ci is a reflection of the word vector wi in approximately half of the dimensions. This allows us to halve the number of trainable parameters in static word embedding models.
asymptotic distribution , neural networks , partition function , word embeddings , word2vec
Text of the article Перейти на текст статьи
International School of Economics, Kazakh-British Technical University, 59 Tolebi St, Almaty, 050000, Kazakhstan
Department of Mathematics, School of Sciences and Humanities, Nazarbayev University, 53 Kabanbay Batyr Ave, Astana, 010000, Kazakhstan
International School of Economics
Department of Mathematics
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026