کنفرانس مهندسی برق ایران

صفحه اصلی / سی و چهارمین کنفرانس بین المللی مهندسی برق

Robust Preprocessing Pipeline for Reaching High-Performance Machine Learning on the TON-IoT Dataset

نویسندگان :

Mohammad Hossein Esfahani¹ Ali Sadr²

1- دانشگاه علم و صنعت ایران 2- دانشگاه علم و صنعت ایران

کلمات کلیدی :

Data Preprocessing،TON-IoT،Zeek،Yeo-Johnson Transformation

چکیده :

Abstract— Although the TON-IoT dataset has become a central benchmark for evaluating intrusion-detection systems in IoT networks, its raw network traffic presents a persistent challenge: the data are severely imbalanced, contain heterogeneous features, and exhibit irregular statistical distributions that hinder the reliability of both machine learning and deep learning models. Existing studies have attempted to mitigate these issues using isolated techniques such as feature pruning, oversampling, or standard normalization; however, these fragmented solutions do not provide a unified preprocessing strategy capable of consistently improving model robustness and class separability on TON-IoT. This paper proposes a cohesive preprocessing pipeline that systematically addresses all major deficiencies of the TON-IoT network component. The pipeline integrates Zeek-based network traffic analyzing, targeted feature selection, Yeo–Johnson transformation for distribution normalization, SMOTE-based balancing, and dual-stage normalization, followed by recursive feature elimination. To assess the proposed preprocessing pipeline, the processed dataset is evaluated using a Random Forest classifier and a hybrid deep learning architecture combining LSTM, GRU, and attention mechanisms; also impact of Yeo-Johnson in classifying with deep learning model has been evaluated. Results demonstrate substantial improvements across all performance metrics, clearer cluster boundaries in t-SNE projections, and a sharp reduction in both Type I and Type II errors. The Random Forest model achieves the highest overall accuracy, confirming that comprehensive preprocessing—not model complexity—is the dominant factor in achieving high-performance intrusion detection on TON-IoT.