Layer Regression: State-of-the-Art Adversarial Attack Detection Made Simple

Layer Regression (LR) is a fast, effective, and easy-to-implement adversarial example detection method that achieves state-of-the-art performance across a wide range of adversarial attacks and neural network architectures in our experimental evaluation. It also demonstrates strong detection performance against adaptive attacks specifically designed to evade the detector. A key advantage of LR is how easily researchers can integrate it into existing models and evaluate it against newly developed adversarial attack methods. Since LR is trained using only clean data, it requires no attack-specific training and can be used as a practical detection baseline for new adversarial attacks.

In our paper, Universal and Efficient Detection of Adversarial Data through Nonuniform Impact on Network Layers, published in Transactions on Machine Learning Research (TMLR), June 2025, we evaluate LR against widely used attacks, including PGD, APGD, BIM, AutoAttack, ANDA, PIF, VMI, and VNI. LR achieves an average 0.99 AUROC on ImageNet across seven individual attacks and six image classification architectures. It also achieves approximately 0.80 AUROC against a detector-aware adaptive PGD attack under the evaluated setting, with a detection processing time of just 0.4 milliseconds per sample in our benchmark.

Paper: arXiv:2506.20816
Publication: TMLR / OpenReview
Code: GitHub – Layer Regression

Why Is Adversarial Attack Detection Important?

Deep neural networks are vulnerable to adversarial examples: inputs deliberately modified to cause incorrect predictions while satisfying constraints on the perturbation magnitude.

Consider an image classification model that correctly recognizes an image. An adversarial attacker may introduce a small perturbation that causes the model to produce an incorrect classification, potentially with high confidence.

Broadly, adversarial defense methods follow two different objectives:

1. Adversarial robustness: Improving a model's ability to make correct predictions despite adversarial perturbations.
2. Adversarial attack detection: Identifying adversarial inputs so they can be flagged, rejected, or handled separately.

Although related, these objectives are not identical. A detector does not necessarily need to recover the correct prediction. Instead, it aims to distinguish clean inputs from adversarial ones.

A major challenge for adversarial detection is generalization. Methods trained on specific attacks may struggle with previously unseen attack strategies, while more computationally intensive detectors may be difficult to deploy or repeatedly evaluate.

This is particularly relevant for researchers developing new adversarial attacks. Evaluating a new attack should ideally involve not only its effectiveness against the target model but also whether existing detection mechanisms can identify it.

Layer Regression addresses this need by providing a fast, clean-data-trained detection method that researchers can readily evaluate against attacks not included in its original training procedure.

How Does Layer Regression Work?

Layer Regression is based on the observation that adversarial perturbations can have nonuniform effects across different layers of a deep neural network.

Many adversarial attacks introduce relatively small modifications at the input while producing substantial changes in the model's deeper representations and final predictions. This can disrupt the relationships normally observed between early and deeper neural network features.

LR uses this inconsistency as a detection signal.

Instead of analyzing input perturbations directly or requiring a separate large neural network, LR trains a small multilayer perceptron (MLP) to predict deeper-layer features from selected intermediate representations.

The method consists of three main steps:

1. Feature extraction: Select and combine outputs from intermediate layers of the target neural network.
2. Layer regression: Train a lightweight MLP to predict a deeper feature representation using only clean training samples.
3. Adversarial detection: Compare the predicted representation with the actual representation. A large regression error indicates a potentially adversarial input.

For clean inputs, the regression network learns to predict the expected deeper-layer representation accurately. For adversarial inputs, the relationship between earlier and deeper features can be disrupted, increasing the prediction error.

The regression error therefore serves as the adversarial detection score.

Figure: Overview of Layer Regression. Selected intermediate features are used to predict deeper-layer representations. The prediction error serves as the adversarial detection score. Adapted from Figure 2 of our TMLR 2025 paper.

Why Is LR Easy to Implement?

One practical advantage of LR is its relatively simple design.

• No target-model retraining: LR works alongside an existing classifier without modifying its training procedure.
• No adversarial training dataset: The regression model is trained exclusively using clean samples.
• Simple architecture: The detector is a small MLP with two hidden layers.
• Minimal inference overhead: Detection requires only intermediate feature extraction and a forward pass through the regression model.
• Public implementation: The source code is available on GitHub.

These properties make LR particularly convenient for researchers who want to incorporate adversarial detection into an existing experimental pipeline without implementing a complex defense framework.

Evaluating New Adversarial Attack Methods with Layer Regression

A particularly useful application of LR is evaluating newly proposed adversarial attacks.

In adversarial machine learning research, new attacks are often evaluated based on attack success rate, perturbation constraints, transferability, and performance against existing defenses.

However, evaluating an attack against a defense can introduce additional complexity if the defense requires adversarial examples for training or must be retrained for each new attack.

Layer Regression does not require attack-specific training. Once its regression model is trained on clean samples for a particular target network, it can be evaluated directly against adversarial examples generated by different attack methods.

A typical evaluation workflow is straightforward:

1. Select a target neural network and clean training data.
2. Train the LR detector using the target network's intermediate feature representations.
3. Generate adversarial examples using the attack method being evaluated.
4. Compute LR detection scores for clean and adversarial inputs.
5. Evaluate detection performance using AUROC or other appropriate detection metrics.

This makes LR useful as a reusable adversarial attack detection baseline, particularly when comparing multiple attack methods against the same target model.

An important distinction is that evaluating a new attack does not guarantee successful detection. Its effectiveness must still be measured experimentally, especially when the attack explicitly targets the detector.

State-of-the-Art Adversarial Detection Performance

We evaluated Layer Regression extensively on image classification tasks using ImageNet and CIFAR-100, considering six different neural network architectures.

The evaluated target models include:

• Convolutional neural networks: VGG19, ResNet-50, and InceptionV3.
• Vision Transformers: ViT, DeiT, and LeViT.

We tested seven individual adversarial attack methods:

Attack	Description
BIM	Basic Iterative Method
PGD	Projected Gradient Descent
APGD	Auto-PGD
PIF	Patch-wise Iterative attack
ANDA	Transferability-oriented adversarial attack
VMI	Variance-tuned Momentum Iterative attack
VNI	Variance-tuned Nesterov Iterative attack

We additionally evaluated AutoAttack, an ensemble-based adversarial attack benchmark, separately.

Detection Accuracy

Across the main comparison with six computationally efficient baseline defenses, Layer Regression achieved:

Dataset	Layer Regression AUROC	Best baseline average AUROC
ImageNet	0.99	0.66
CIFAR-100	0.97	0.65

The evaluated baselines included JPEG compression, randomization, pixel deflection, denoising, wavelet denoising and super-resolution, and feature squeezing.

LR substantially outperformed these methods in average detection AUROC across the evaluated architectures and attacks.

We also compared LR with two more computationally expensive detection methods: EPS-AD and VLAD.

In this separate evaluation, both LR and EPS-AD achieved approximately 0.99 average AUROC, while VLAD achieved approximately 0.88. However, LR required substantially less computation than either method.

These results demonstrate that LR can achieve highly competitive adversarial detection accuracy without relying on expensive inference procedures.

Fast Adversarial Attack Detection

Adversarial detection methods must be not only effective but also computationally efficient, particularly when they are used in real-time applications or repeated experimental evaluations.

Layer Regression was designed with computational efficiency as a central objective.

Unlike approaches that require multiple transformations, repeated model evaluations, or large generative models, LR relies on a compact MLP and existing intermediate features from the target model.

In our experiments, the average detection processing times were:

Detection method	Processing time per sample
Layer Regression (LR)	0.0004 s (0.4 ms)
VLAD	0.1431 s
EPS-AD	2.9998 s

All processing times were measured on a workstation equipped with an NVIDIA RTX 4090 GPU, AMD Ryzen 9 7950X CPU, and 64 GB RAM.

LR achieved detection performance comparable to EPS-AD while requiring substantially less processing time.

Figure: Adversarial detection AUROC versus processing time for Layer Regression and existing defense methods. LR combines high detection accuracy with low computational overhead. From Figure 3 of our TMLR 2025 paper.

This efficiency is particularly useful for researchers who need to evaluate numerous adversarial attack configurations across multiple models and datasets.

Detection Performance Against Adaptive Attacks

An important challenge for any adversarial detector is the possibility of adaptive attacks, in which the attacker is aware of the detection mechanism and explicitly attempts to evade it.

In addition to conventional attacks against the target classifier, we evaluated Layer Regression under a detector-aware PGD attack.

The adaptive attack jointly optimizes two objectives:

• Causing the target classifier to make an incorrect prediction.
• Reducing the Layer Regression detection score to evade detection.

This creates a substantially more challenging evaluation scenario because the attack is optimized against both the classifier and the detector.

To investigate this setting, we evaluated a PGD-based adaptive attack against a Vision Transformer across several perturbation budgets.

Perturbation budget (ε)	Detection AUROC
4	0.81
8	0.78
16	0.79
32	0.80
64	0.81
128	0.81

Layer Regression achieved approximately 0.80 average AUROC under the evaluated adaptive attack.

The method also uses inference-time randomization of the concatenated intermediate feature vectors as a mechanism for addressing adaptive attacks.

Although these results demonstrate promising detection performance, adaptive attacks remain an open challenge. The reported results characterize the specific evaluated attack configuration and do not establish guaranteed detection against every possible detector-aware adversary.

Performance Against AutoAttack

We additionally evaluated Layer Regression against AutoAttack, an ensemble of adversarial attacks commonly used to assess adversarial robustness.

LR achieved an average 0.98 AUROC across six image classification architectures in this evaluation.

It is important to distinguish this experiment from the detector-aware adaptive PGD evaluation: AutoAttack was not explicitly optimized to evade the LR detector.

Adversarial Detection Beyond Image Classification

Although our primary experiments focus on image classification, Layer Regression is designed around a general property of deep neural networks rather than a specific data modality.

We also investigated its applicability to video action recognition and speech recognition.

Video Action Recognition

We evaluated LR using MViT and X3D action recognition models on the Kinetics-400 dataset.

The experiments considered two adversarial attack methods:

• PGD-v: A gradient-based adversarial attack targeting video action recognition.
• Flick: An adversarial video attack involving temporal perturbations.

Across the evaluated configurations, LR achieved an average detection AUROC of 0.93, compared with 0.91 for VLAD.

These results demonstrate that the internal feature regression principle can also support adversarial attack detection in video understanding systems.

Speech Recognition

We further evaluated LR on a wav2vec speech recognition model using the LibriSpeech dataset.

Under the evaluated FGSM attack, Layer Regression achieved an AUROC of 0.99.

The results show that LR can identify adversarial audio inputs by analyzing inconsistencies in neural network representations, extending the same detection principle beyond computer vision.

When Should Researchers Use Layer Regression?

Layer Regression is particularly suitable for researchers and practitioners looking for an adversarial detection method that offers:

• High detection accuracy: Strong experimental results against widely used adversarial attacks.
• Fast inference: Low computational overhead suitable for repeated evaluations and potentially real-time applications.
• Easy implementation: A compact MLP-based detector that can be integrated with existing neural networks.
• Simple evaluation of new attacks: No attack-specific retraining is required when evaluating newly developed attack methods on a target network with an already trained LR detector.
• Adaptive attack evaluation: Demonstrated detection performance against a detector-aware PGD attack.
• Applicability across domains: Experiments spanning image classification, video action recognition, and speech recognition.

For adversarial attack researchers, LR provides a particularly convenient way to incorporate detection-based evaluation into experimental comparisons without introducing a complex training or defense pipeline.

For researchers developing detection systems, it offers a strong and computationally efficient baseline based on intermediate neural network representations.

Conclusion

Layer Regression demonstrates that effective adversarial example detection does not necessarily require complex defense architectures, expensive inference procedures, or attack-specific training.

By learning the expected relationship between intermediate and deeper neural network features using clean data, LR provides a simple but powerful mechanism for identifying adversarial inputs.

Our experiments demonstrate state-of-the-art detection performance in the evaluated comparisons, low computational overhead, and promising results against detector-aware adaptive attacks.

Beyond its detection accuracy, LR's simplicity and efficiency make it especially valuable as a reusable baseline for evaluating new adversarial attack methods, enabling researchers to incorporate attack detection into their experiments with relatively little additional complexity.

Paper and Code

Paper: arXiv:2506.20816
Publication: TMLR / OpenReview
Code: GitHub – Layer Regression

If you are developing or evaluating adversarial attack methods, Layer Regression offers a fast and straightforward detection baseline that can be incorporated into your experimental pipeline.

Frequently Asked Questions

What are effective methods for detecting PGD, APGD, and AutoAttack adversarial examples?

Layer Regression (LR), introduced by Mumcu and Yilmaz (TMLR 2025), is an adversarial example detection method that achieves high detection AUROC against PGD, APGD, AutoAttack, and several transfer-based attacks in the evaluated image classification settings.

Can Layer Regression detect newly developed adversarial attacks?

Layer Regression can be evaluated against newly developed adversarial attacks without attack-specific retraining. The detector learns relationships between neural network layers using only clean data, allowing researchers to test its generalization to unseen attacks.

What is an easy-to-implement adversarial example detection baseline?

Layer Regression offers a straightforward MLP-based detection architecture that does not require modifying or retraining the target classifier. Its implementation is publicly available on GitHub.

Can Layer Regression detect adaptive adversarial attacks?

Layer Regression achieved approximately 0.80 AUROC against a detector-aware adaptive PGD attack under the evaluated configuration. This demonstrates promising performance, although effectiveness against other adaptive attacks must be evaluated separately.