HYBRID POSITIONAL EMBEDDINGS IN TRANSFORMER MODELS FOR CYBERATTACK DETECTION IN INDUSTRIAL CONTROL SYSTEMS

ICTACT Journal on Soft Computing ( Volume: 16 , Issue: 3 )

Abstract

Cyberattack detection in industrial control systems (ICS) is critical to ensuring the security and resilience of essential infrastructure, such as water treatment and distribution networks. However, existing anomaly detection methods often struggle with capturing complex temporal dependencies and differentiating between cyberattacks and normal operational variations. In this study, we propose a novel Transformer- based approach with hybrid positional embeddings for detecting cyber- attacks in multivariate time series data. Our method integrates learnable, sinusoidal, and rotary position embeddings, enabling the model to effectively capture both absolute and relative temporal relationships. This hybrid embedding strategy addresses key limitations of conventional Transformers in handling time-series data by improving the encoding of temporal dependencies. We evaluate our approach on two widely used cybersecurity datasets: Secure Water Treatment (SWaT) and Water Distribution (WADI), which simulate real-world industrial cyber-physical system (CPS) attacks. Our model outperforms state-of-the-art baselines, achieving high detection accuracy and robust anomaly identification. Additionally, an ablation study demonstrates the contribution of hybrid positional embeddings in improving cyberattack detection performance. This work enhances AI driven security frameworks for industrial systems by providing a scalable and effective solution for cyber threat monitoring in critical infrastructures.

Authors

Syed Minhaz Ul Hassan, Meena Chaudhary
Mangalayatan University, India

Keywords

Multivariate Time Series Data, Anomaly Detection, Transformer Models, Learnable Positional Embeddings, Rotary Position Embeddings

Published By
ICTACT
Published In
ICTACT Journal on Soft Computing
( Volume: 16 , Issue: 3 )
Date of Publication
October 2025
Pages
4018 - 4029
Page Views
140
Full Text Views
7