Logo image
Back
Exploring the Structure Resulting from Unstructured Neural Network Pruning
Journal article   Open access

Exploring the Structure Resulting from Unstructured Neural Network Pruning

Jamil Gafur, Max Milkert, Kevin Patrick Griffin, Nicholas T. Wimer, Charles Tripp and Steve Goddard
International journal of advanced computer science & applications, Vol.17(6), pp.25-34
2026
DOI: 10.14569/IJACSA.2026.0170603
url
https://doi.org/10.14569/IJACSA.2026.0170603View
Published (Version of record) Open Access

Abstract

Iterative Magnitude Pruning (IMP) is a widely used technique for compressing neural networks by progressively removing low-magnitude weights while maintaining predictive accuracy. Despite its widespread application and simplicity, the underlying reasons for its effectiveness remain underexplored. In this work, the pruning dynamics and emergent structural traits of IMP are empirically examined across three benchmark convolutional architectures (ResNet20, Vgg16, and RegNetX) and evaluated on the CIFAR10 and Tiny ImageNet datasets. A comprehensive analysis reveals that IMP preferentially removes weights from deeper layers, preserving early feature extractors until a critical network sparsity threshold of 96% is reached. Up to this 96% threshold, test accuracy remains remarkably stable across all evaluated networks before experiencing significant degradation. Quantitative evidence is provided showing that later stages are pruned more heavily, taking advantage of the fact that deeper layers contain more low-magnitude weights. Furthermore, neuron-level investigations reveal that IMP does not produce any completely pruned (100% sparse) neurons, even at extreme sparsity levels. Sensitivity analysis via neuron zeroing demonstrates that individual neurons maintain a stable range of functional importance rather than narrowing as pruning progresses. Finally, activation similarity metrics indicate that feature representations are preserved throughout the pruning process, keeping pruned and unpruned networks in close rep-resentational alignment. These findings highlight the surprising degree of structure generated by unstructured pruning, offering new insights into network compression and potential pathways for hardware-efficient sparse matrix adaptations.
 Machine Learning convolutional neural networks multi-layer perceptrons artificial intelligence

Details

Metrics

1 Record Views
Logo image