江苏龙网

  1. 首页 > 人文 > >

空白键|DropBlock正则化的介绍( 二 )


算法的简单可视化
空白键|DropBlock正则化的介绍DropBlock Mask的计算import torchfrom torch import *"""Step-1] Generate a random feature map using NCHW format Here I am considering a batch_size=1, no. of channels = 1, Height and width of the feature map to be 5 each"""x = torch.randint(1,10, (1, 1, 5, 5))print(x)class DropBlock(torch.nn.Module):"""Step-2] Parameters InitializationInitialization of block_size, keep_prob, stride and padding respectivelywhich can be altered accordingly for experimentation"""def __init__(self, block_size=3, keep_prob=0.9):self.block_size = block_sizeself.keep_prob = keep_probself.gamma = Noneself.kernel_size = (block_size, block_size)self.stride = (1, 1)self.padding = (block_size//2, block_size//2)self.training=True"""Step-3] Compute gamma using the equation mentioned aprior in the blog.Dependency can be seen on block_size and keep_prob for its computation."""def calculate_gamma(self, x):return (1 - self.keep_prob) * x.shape[-1]**2/\(self.block_size**2 * (x.shape[-1] - self.block_size + 1)**2)def forward(self, x):"""Step-4]Check on whether it is inference mode as DropBlock Algorithm like DropOut Algorithm is performed only during the time of training"""if not self.training:return xif self.gamma is None:self.gamma = self.calculate_gamma(x)"""Step-5]Use the gamma obtained to compute the sample mask using Bernoulli"""p = torch.ones_like(x) * self.gammamask = 1 - torch.nn.functional.max_pool2d(torch.bernoulli(p),self.kernel_size,self.stride,self.padding)"""Step-6]Normalization of Features"""return mask * x * (mask.numel()/mask.sum())"""Make respective function calls"""db = DropBlock()db.calculate_gamma(x)db.forward(x)总结DropBlock技术被证明可以有效的打败一些通过传统的dropout得到的最好的性能结果 ， Spatial Dropout ， DropPath ， CutOut这些也显示了更好的结果 ， 但是这些要归功于强力的数据增强的方法 。 DropBlock被证明是一种有效的目标检测正则化方法 。
【空白键|DropBlock正则化的介绍】英文原文：


推荐阅读


上一篇：江湖车侠|自动驾驶的方案之一：单车智能你听过吗？

下一篇：选择遗忘|马自达SUV销量王 新款CX-5官图发布：2.5T同级罕见