Mathematical Approaches and Algorithms in Big Data Architecture and Hybrid System Efficiency

Aliaskarov S. Uskenbayeva R. Serbin V. Bekmurat O. Bazarbayeva U. Bakhtiyarova Y. Sansyzbay K.
September 2025 Ital Publication

HighTech and Innovation Journal
2025 #6 Issue 3 1013 - 1034 pp.

This article presents a formal demonstration of a hybrid big data processing architecture that combines the fault tolerance and storage robustness of Hadoop with the speed and in-memory processing capabilities of Apache Spark. The proposed architecture is evaluated through test execution and performance benchmarking in real-world data centers across three regions in Kazakhstan. The model integrates distributed resource management components, Directed Acyclic Graph (DAG)-based scheduling mechanism, and Resilient Distributed Datasets (RDDs) to enable dynamic workload distribution and rapid failure recovery. The results demonstrate that the hybrid system consistently outperforms standalone Spark and Hadoop architectures under variable workloads, illustrating enhancements in execution time, task recovery, and resource utilization. Quantitative performance metrics allow for a structured comparison of architectures and help optimize deployments for diverse scenarios. The proposed hybrid architecture shows significant improvements, reducing average execution time by up to 38% and increasing resource efficiency by 25% compared to standalone Spark and Hadoop systems.

Apache Spark , DAG , Fault Tolerance , Hybrid Big Data Architecture , RDD , Scalability

Text of the article Перейти на текст статьи

International Information Technologies University, Almaty, 050040, Kazakhstan
Satbayev University, Almaty, 050013, Kazakhstan
Kazakh National Pedagogical University named after Abai, Almaty, 050012, Kazakhstan

International Information Technologies University
Satbayev University
Kazakh National Pedagogical University named after Abai

10 лет помогаем публиковать статьи Международный издатель

Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026