SqueezeNext: Hardware-Aware Neural Network Design

Introduction

The codes are PyTorch re-implement version for paper: SqueezeNext: Hardware-Aware Neural Network Design. (SqueezeNext)

Gholami A, Kwon K, Wu B, et al. SqueezeNext: Hardware-Aware Neural Network Design[J]. 2018. arXiv:1803.10615v1

We implement this work from amirgholami/SqueezeNext.

Structure

Here, we use a variation of the latter approach by using a two stage squeeze layer. In each SqueezeNext block, we use two bottleneck modules each reducing the channel size by a factor of 2, which is followed by two separable convolutions. We also incorporate a final 1 × 1 expansion module, which further reduces the number of output channels for the separable convolutions.

Requirements

• jupyter notebook
• Python3
• PyTorch 0.4

Results

We just test four models in three datasets: Cifar10, Cifar100 and Tiny ImageNet

Cifar-10

Models | train(Top-1) | validation(Top-1) | width | depth | ------------- | ------------ | ----------------- | ------ | ------ | SqNxt_23_1x | 98.7 | 91.9 | 1.0x | 23 | SqNxt_23_2x | 99.9 | 93.1 | 2.0x | 23 | SqNxt_23_1x_v5 | 99.4 | 91.9 | 1.0x | 23 | SqNxt_23_2x_v5 | 99.8 | 93.1 | 2.0x | 23 |

Cifar-100

Models | train(Top-1) | validation(Top-1) | width | depth | ------------- | ------------ | ----------------- | ------ | ------ | SqNxt_23_1x | 94.1 | 69.3 | 1.0x | 23 | SqNxt_23_2x | 99.7 | 73.1 | 2.0x | 23 | SqNxt_23_1x_v5 | 94.7 | 70.1 | 1.0x | 23 | SqNxt_23_2x_v5 | 99.8 | 73.2 | 2.0x | 23 |

Tiny ImageNet

Models | train(Top-1) | validation(Top-1) | width | depth | ------------- | ------------ | ----------------- | ------ | ------ | SqNxt_23_1x | 71.1 | 53.5 | 1.0x | 23 | SqNxt_23_2x | 77.2 | 56.7 | 2.0x | 23 | SqNxt_23_1x_v5 | 70.9 | 52.7 | 1.0x | 23 | SqNxt_23_2x_v5 | 72.4 | 56.7 | 2.0x | 23 |