🤵 AuthorHorizon Max

编程技巧篇各种操作小结

🎇 机器视觉篇会变魔术 OpenCV

💥 深度学习篇简单入门 PyTorch

🏆 神经网络篇经典网络模型

💻 算法篇再忙也别忘了 LeetCode

🚀 Squeeze-and-Excitation Networks

Squeeze :挤压     Excitation :激励 ;

Squeeze-and-Excitation Networks 简称 SENet ,由 Momenta 和 牛津大学 的Jie Hu等人 提出的一种新的网络结构;

目标是通过建模 卷积特征通道之间的相互依赖关系 来提高网络的表示能力;

在2017年最后一届 ImageNet 挑战赛(ILSVRC) classification 任务中获得 冠军,将错误率降低到 2.251% ;

🔗 论文地址:Squeeze-and-Excitation Networks

🚀 SENet 详解

🎨 Squeeze-and-Excitation block


Squeeze-and-Excitation block

 Squeeze-and-Excitation block

对于任意给定的变换: Ftr :X → U ,其中 X ∈ R H’xW’xC’ , U ∈ R HxWxCFtr 用作一个卷积算子 ;

🚩 Squeeze: Global Information Embedding

挤压:全局信息嵌入

(1)Squeeze :特征U通过 squeeze 压缩操作,将跨空间维度H × W的特征映射进行聚合,生成一个通道描述符,HxWxC → 1x1xC
将 全局空间信息 压缩到上述 通道描述符 中,使来这些 通道描述符 可以被 其输入的层 利用,这里采用的是 global average pooling

Squeeze

🚩 Excitation: Adaptive Recalibration

激励:自适应调整

(2)Excitation :每个通道通过一个 基于通道依赖 的自选门机制 来学习特定样本的激活,使其学会使用全局信息,有选择地强调信息特征,并抑制不太有用的特征,这里采用的是 sigmoid ,并在中间嵌入了 ReLU 函数用于限制模型的复杂性和帮助训练 ;

通过 两个全连接层(FC) 构成的瓶颈来参数化门控机制,即 W1 用于降低维度,W2 用于维度递增 ;

Excitation

(3)Reweight :将 Excitation 输出的权重通过乘法逐通道加权到输入特征上;

总的来说 SE Block 就是在 Layer 的输入和输出之间添加结构: global average pooling - FC- sigmoid - FC- ReLU

SE block 的灵活性意味着它可以直接应用于标准卷积以外的转换,通过将 SE block 集成到任何复杂模型当中来开发SENet;

🚩 在非残差网络中的应用

应用于 非残差网络 Inception network 当中,形成 SE-Inception module


非残差网络结构框图(Inception block)

SE-Inception Module

Scale : 改变(文字、图片)的尺寸大小

🚩 在残差网络中的应用

应用于 残差网络 Residual network 当中,形成 SE-ResNet module



残差网络结构框图(Residual Block)

SE-ResNet Module

论文中对 SE block 的应用用于实验对比:

SE-ResNet-50 网络的准确性优于 ResNet-50 和模型深化版的 ResNet101 网络 ;
对于224 × 224像素的输入图像,ResNet-50 需要164 ms,而 SE-ResNet-50 需要167 ms ;

🚀 SENet 复现

这里实现的是 SE-ResNet 系列网络 :

# Here is the code :

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchinfo import summary


class SE_Block(nn.Module):                         # Squeeze-and-Excitation block
    def __init__(self, in_planes):
        super(SE_Block, self).__init__()
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.conv1 = nn.Conv2d(in_planes, in_planes // 16, kernel_size=1)
        self.relu = nn.ReLU()
        self.conv2 = nn.Conv2d(in_planes // 16, in_planes, kernel_size=1)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        x = self.avgpool(x)
        x = self.conv1(x)
        x = self.relu(x)
        x = self.conv2(x)
        out = self.sigmoid(x)
        return out


class BasicBlock(nn.Module):      # 左侧的 residual block 结构(18-layer、34-layer)
    expansion = 1

    def __init__(self, in_planes, planes, stride=1):      # 两层卷积 Conv2d + Shutcuts
        super(BasicBlock, self).__init__()
        self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3,
                               stride=stride, padding=1, bias=False)
        self.bn1 = nn.BatchNorm2d(planes)
        self.conv2 = nn.Conv2d(planes, planes, kernel_size=3,
                               stride=1, padding=1, bias=False)
        self.bn2 = nn.BatchNorm2d(planes)

        self.SE = SE_Block(planes)           # Squeeze-and-Excitation block

        self.shortcut = nn.Sequential()
        if stride != 1 or in_planes != self.expansion*planes:      # Shutcuts用于构建 Conv Block 和 Identity Block
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_planes, self.expansion*planes,
                          kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(self.expansion*planes)
            )

    def forward(self, x):
        out = F.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        SE_out = self.SE(out)
        out = out * SE_out
        out += self.shortcut(x)
        out = F.relu(out)
        return out


class Bottleneck(nn.Module):      # 右侧的 residual block 结构(50-layer、101-layer、152-layer)
    expansion = 4

    def __init__(self, in_planes, planes, stride=1):      # 三层卷积 Conv2d + Shutcuts
        super(Bottleneck, self).__init__()
        self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False)
        self.bn1 = nn.BatchNorm2d(planes)
        self.conv2 = nn.Conv2d(planes, planes, kernel_size=3,
                               stride=stride, padding=1, bias=False)
        self.bn2 = nn.BatchNorm2d(planes)
        self.conv3 = nn.Conv2d(planes, self.expansion*planes,
                               kernel_size=1, bias=False)
        self.bn3 = nn.BatchNorm2d(self.expansion*planes)

        self.SE = SE_Block(self.expansion*planes)           # Squeeze-and-Excitation block

        self.shortcut = nn.Sequential()
        if stride != 1 or in_planes != self.expansion*planes:      # Shutcuts用于构建 Conv Block 和 Identity Block
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_planes, self.expansion*planes,
                          kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(self.expansion*planes)
            )

    def forward(self, x):
        out = F.relu(self.bn1(self.conv1(x)))
        out = F.relu(self.bn2(self.conv2(out)))
        out = self.bn3(self.conv3(out))
        SE_out = self.SE(out)
        out = out * SE_out
        out += self.shortcut(x)
        out = F.relu(out)
        return out


class SE_ResNet(nn.Module):
    def __init__(self, block, num_blocks, num_classes=1000):
        super(SE_ResNet, self).__init__()
        self.in_planes = 64

        self.conv1 = nn.Conv2d(3, 64, kernel_size=3,
                               stride=1, padding=1, bias=False)                  # conv1
        self.bn1 = nn.BatchNorm2d(64)
        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)       # conv2_x
        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)      # conv3_x
        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)      # conv4_x
        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)      # conv5_x
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.linear = nn.Linear(512 * block.expansion, num_classes)

    def _make_layer(self, block, planes, num_blocks, stride):
        strides = [stride] + [1]*(num_blocks-1)
        layers = []
        for stride in strides:
            layers.append(block(self.in_planes, planes, stride))
            self.in_planes = planes * block.expansion
        return nn.Sequential(*layers)

    def forward(self, x):
        x = F.relu(self.bn1(self.conv1(x)))
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        out = self.linear(x)
        return out


def SE_ResNet18():
    return SE_ResNet(BasicBlock, [2, 2, 2, 2])


def SE_ResNet34():
    return SE_ResNet(BasicBlock, [3, 4, 6, 3])


def SE_ResNet50():
    return SE_ResNet(Bottleneck, [3, 4, 6, 3])


def SE_ResNet101():
    return SE_ResNet(Bottleneck, [3, 4, 23, 3])


def SE_ResNet152():
    return SE_ResNet(Bottleneck, [3, 8, 36, 3])


def test():
    net = SE_ResNet50()
    y = net(torch.randn(1, 3, 224, 224))
    print(y.size())
    summary(net, (1, 3, 224, 224))


if __name__ == '__main__':
    test()

输出结果:

torch.Size([1, 1000])
===============================================================================================
Layer (type:depth-idx)                        Output Shape              Param #
===============================================================================================
SE_ResNet                                     --                        --
├─Conv2d: 1-1                                 [1, 64, 224, 224]         1,728
├─BatchNorm2d: 1-2                            [1, 64, 224, 224]         128
├─Sequential: 1-3                             [1, 256, 224, 224]        --
│    └─Bottleneck: 2-1                        [1, 256, 224, 224]        --
│    │    └─Conv2d: 3-1                       [1, 64, 224, 224]         4,096
│    │    └─BatchNorm2d: 3-2                  [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-3                       [1, 64, 224, 224]         36,864
│    │    └─BatchNorm2d: 3-4                  [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-5                       [1, 256, 224, 224]        16,384
│    │    └─BatchNorm2d: 3-6                  [1, 256, 224, 224]        512
│    │    └─SE_Block: 3-7                     [1, 256, 1, 1]            8,464
│    │    └─Sequential: 3-8                   [1, 256, 224, 224]        16,896
│    └─Bottleneck: 2-2                        [1, 256, 224, 224]        --
│    │    └─Conv2d: 3-9                       [1, 64, 224, 224]         16,384
│    │    └─BatchNorm2d: 3-10                 [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-11                      [1, 64, 224, 224]         36,864
│    │    └─BatchNorm2d: 3-12                 [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-13                      [1, 256, 224, 224]        16,384
│    │    └─BatchNorm2d: 3-14                 [1, 256, 224, 224]        512
│    │    └─SE_Block: 3-15                    [1, 256, 1, 1]            8,464
│    │    └─Sequential: 3-16                  [1, 256, 224, 224]        --
│    └─Bottleneck: 2-3                        [1, 256, 224, 224]        --
│    │    └─Conv2d: 3-17                      [1, 64, 224, 224]         16,384
│    │    └─BatchNorm2d: 3-18                 [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-19                      [1, 64, 224, 224]         36,864
│    │    └─BatchNorm2d: 3-20                 [1, 64, 224, 224]         128
│    │    └─Conv2d: 3-21                      [1, 256, 224, 224]        16,384
│    │    └─BatchNorm2d: 3-22                 [1, 256, 224, 224]        512
│    │    └─SE_Block: 3-23                    [1, 256, 1, 1]            8,464
│    │    └─Sequential: 3-24                  [1, 256, 224, 224]        --
├─Sequential: 1-4                             [1, 512, 112, 112]        --
│    └─Bottleneck: 2-4                        [1, 512, 112, 112]        --
│    │    └─Conv2d: 3-25                      [1, 128, 224, 224]        32,768
│    │    └─BatchNorm2d: 3-26                 [1, 128, 224, 224]        256
│    │    └─Conv2d: 3-27                      [1, 128, 112, 112]        147,456
│    │    └─BatchNorm2d: 3-28                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-29                      [1, 512, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-30                 [1, 512, 112, 112]        1,024
│    │    └─SE_Block: 3-31                    [1, 512, 1, 1]            33,312
│    │    └─Sequential: 3-32                  [1, 512, 112, 112]        132,096
│    └─Bottleneck: 2-5                        [1, 512, 112, 112]        --
│    │    └─Conv2d: 3-33                      [1, 128, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-34                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-35                      [1, 128, 112, 112]        147,456
│    │    └─BatchNorm2d: 3-36                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-37                      [1, 512, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-38                 [1, 512, 112, 112]        1,024
│    │    └─SE_Block: 3-39                    [1, 512, 1, 1]            33,312
│    │    └─Sequential: 3-40                  [1, 512, 112, 112]        --
│    └─Bottleneck: 2-6                        [1, 512, 112, 112]        --
│    │    └─Conv2d: 3-41                      [1, 128, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-42                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-43                      [1, 128, 112, 112]        147,456
│    │    └─BatchNorm2d: 3-44                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-45                      [1, 512, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-46                 [1, 512, 112, 112]        1,024
│    │    └─SE_Block: 3-47                    [1, 512, 1, 1]            33,312
│    │    └─Sequential: 3-48                  [1, 512, 112, 112]        --
│    └─Bottleneck: 2-7                        [1, 512, 112, 112]        --
│    │    └─Conv2d: 3-49                      [1, 128, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-50                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-51                      [1, 128, 112, 112]        147,456
│    │    └─BatchNorm2d: 3-52                 [1, 128, 112, 112]        256
│    │    └─Conv2d: 3-53                      [1, 512, 112, 112]        65,536
│    │    └─BatchNorm2d: 3-54                 [1, 512, 112, 112]        1,024
│    │    └─SE_Block: 3-55                    [1, 512, 1, 1]            33,312
│    │    └─Sequential: 3-56                  [1, 512, 112, 112]        --
├─Sequential: 1-5                             [1, 1024, 56, 56]         --
│    └─Bottleneck: 2-8                        [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-57                      [1, 256, 112, 112]        131,072
│    │    └─BatchNorm2d: 3-58                 [1, 256, 112, 112]        512
│    │    └─Conv2d: 3-59                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-60                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-61                      [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-62                 [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-63                    [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-64                  [1, 1024, 56, 56]         526,336
│    └─Bottleneck: 2-9                        [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-65                      [1, 256, 56, 56]          262,144
│    │    └─BatchNorm2d: 3-66                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-67                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-68                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-69                      [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-70                 [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-71                    [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-72                  [1, 1024, 56, 56]         --
│    └─Bottleneck: 2-10                       [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-73                      [1, 256, 56, 56]          262,144
│    │    └─BatchNorm2d: 3-74                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-75                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-76                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-77                      [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-78                 [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-79                    [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-80                  [1, 1024, 56, 56]         --
│    └─Bottleneck: 2-11                       [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-81                      [1, 256, 56, 56]          262,144
│    │    └─BatchNorm2d: 3-82                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-83                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-84                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-85                      [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-86                 [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-87                    [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-88                  [1, 1024, 56, 56]         --
│    └─Bottleneck: 2-12                       [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-89                      [1, 256, 56, 56]          262,144
│    │    └─BatchNorm2d: 3-90                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-91                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-92                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-93                      [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-94                 [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-95                    [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-96                  [1, 1024, 56, 56]         --
│    └─Bottleneck: 2-13                       [1, 1024, 56, 56]         --
│    │    └─Conv2d: 3-97                      [1, 256, 56, 56]          262,144
│    │    └─BatchNorm2d: 3-98                 [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-99                      [1, 256, 56, 56]          589,824
│    │    └─BatchNorm2d: 3-100                [1, 256, 56, 56]          512
│    │    └─Conv2d: 3-101                     [1, 1024, 56, 56]         262,144
│    │    └─BatchNorm2d: 3-102                [1, 1024, 56, 56]         2,048
│    │    └─SE_Block: 3-103                   [1, 1024, 1, 1]           132,160
│    │    └─Sequential: 3-104                 [1, 1024, 56, 56]         --
├─Sequential: 1-6                             [1, 2048, 28, 28]         --
│    └─Bottleneck: 2-14                       [1, 2048, 28, 28]         --
│    │    └─Conv2d: 3-105                     [1, 512, 56, 56]          524,288
│    │    └─BatchNorm2d: 3-106                [1, 512, 56, 56]          1,024
│    │    └─Conv2d: 3-107                     [1, 512, 28, 28]          2,359,296
│    │    └─BatchNorm2d: 3-108                [1, 512, 28, 28]          1,024
│    │    └─Conv2d: 3-109                     [1, 2048, 28, 28]         1,048,576
│    │    └─BatchNorm2d: 3-110                [1, 2048, 28, 28]         4,096
│    │    └─SE_Block: 3-111                   [1, 2048, 1, 1]           526,464
│    │    └─Sequential: 3-112                 [1, 2048, 28, 28]         2,101,248
│    └─Bottleneck: 2-15                       [1, 2048, 28, 28]         --
│    │    └─Conv2d: 3-113                     [1, 512, 28, 28]          1,048,576
│    │    └─BatchNorm2d: 3-114                [1, 512, 28, 28]          1,024
│    │    └─Conv2d: 3-115                     [1, 512, 28, 28]          2,359,296
│    │    └─BatchNorm2d: 3-116                [1, 512, 28, 28]          1,024
│    │    └─Conv2d: 3-117                     [1, 2048, 28, 28]         1,048,576
│    │    └─BatchNorm2d: 3-118                [1, 2048, 28, 28]         4,096
│    │    └─SE_Block: 3-119                   [1, 2048, 1, 1]           526,464
│    │    └─Sequential: 3-120                 [1, 2048, 28, 28]         --
│    └─Bottleneck: 2-16                       [1, 2048, 28, 28]         --
│    │    └─Conv2d: 3-121                     [1, 512, 28, 28]          1,048,576
│    │    └─BatchNorm2d: 3-122                [1, 512, 28, 28]          1,024
│    │    └─Conv2d: 3-123                     [1, 512, 28, 28]          2,359,296
│    │    └─BatchNorm2d: 3-124                [1, 512, 28, 28]          1,024
│    │    └─Conv2d: 3-125                     [1, 2048, 28, 28]         1,048,576
│    │    └─BatchNorm2d: 3-126                [1, 2048, 28, 28]         4,096
│    │    └─SE_Block: 3-127                   [1, 2048, 1, 1]           526,464
│    │    └─Sequential: 3-128                 [1, 2048, 28, 28]         --
├─AdaptiveAvgPool2d: 1-7                      [1, 2048, 1, 1]           --
├─Linear: 1-8                                 [1, 1000]                 2,049,000
===============================================================================================
Total params: 28,080,344
Trainable params: 28,080,344
Non-trainable params: 0
Total mult-adds (G): 63.60
===============================================================================================
Input size (MB): 0.60
Forward/backward pass size (MB): 2691.18
Params size (MB): 112.32
Estimated Total Size (MB): 2804.10
===============================================================================================