Miscellaneous#

SpikingJelly 的 杂项模块 提供了辅助层和其他实用工具。

SpikingJelly's miscellaneous module provides auxiliary layers and other utilities.

class spikingjelly.activation_based.layer.misc.SynapseFilter(tau=100.0, learnable=False, step_mode='s')[源代码]#

基类：MemoryModule

API Language: 中文 | English

  • 中文

具有滤波性质的突触。突触的输出电流满足，当没有脉冲输入时，输出电流指数衰减：

\[\tau \frac{\mathrm{d} I(t)}{\mathrm{d} t} = - I(t)\]

当有新脉冲输入时，输出电流自增1：

\[I(t) = I(t) + 1\]

记输入脉冲为 \(S(t)\)，则离散化后，统一的电流更新方程为：

\[I(t) = I(t-1) - (1 - S(t)) \frac{1}{\tau} I(t-1) + S(t)\]

输出电流不仅取决于当前时刻的输入，还取决于之前的输入，使得该突触具有了一定的记忆能力。

这种突触偶有使用，例如：

Unsupervised learning of digit recognition using spike-timing-dependent plasticity

Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network

另一种视角是将其视为一种输入为脉冲，并输出其电压的LIF神经元。并且该神经元的发放阈值为 \(+\infty\) 。

神经元最后累计的电压值一定程度上反映了该神经元在整个仿真过程中接收脉冲的数量，从而替代了传统的直接对输出脉冲计数（即发放频率）来表示神经元活跃程度的方法。因此通常用于最后一层，在以下文章中使用：

Enabling spike-based backpropagation for training deep neural network architectures

参数:

  • tau (float) -- time 突触上电流衰减的时间常数

  • learnable (bool) -- 时间常数在训练过程中是否是可学习的。若为 True，则 tau 会被设定成时间常数的初始值

  • step_mode (str) -- 步进模式，可以为 's' (单步) 或 'm' (多步)

  • English

The synapse filter that can filter input current. The output current will decay when there is no input spike:

\[\tau \frac{\mathrm{d} I(t)}{\mathrm{d} t} = - I(t)\]

The output current will increase 1 when there is a new input spike:

\[I(t) = I(t) + 1\]

Denote the input spike as \(S(t)\), then the discrete current update equation is as followed:

\[I(t) = I(t-1) - (1 - S(t)) \frac{1}{\tau} I(t-1) + S(t)\]

The output current is not only determined by the present input but also by the previous input, which makes this synapse have memory.

This synapse is sometimes used, e.g.:

Unsupervised learning of digit recognition using spike-timing-dependent plasticity

Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network

Another view is regarding this synapse as a LIF neuron with a \(+\infty\) threshold voltage.

The final output of this synapse (or the final voltage of this LIF neuron) represents the accumulation of input spikes, which substitute for traditional firing rate that indicates the excitatory level. So, it can be used in the last layer of the network, e.g.:

Enabling spike-based backpropagation for training deep neural network architectures

参数:

  • tau (float) -- time constant that determines the decay rate of current in the synapse

  • learnable (bool) -- whether time constant is learnable during training. If True, then tau will be the initial value of time constant

  • step_mode (str) -- the step mode, which can be s (single-step) or m (multi-step)

  • 代码示例 | Example

T = 50
in_spikes = (torch.rand(size=[T]) >= 0.95).float()
lp_syn = LowPassSynapse(tau=10.0)
pyplot.subplot(2, 1, 1)
pyplot.bar(torch.arange(0, T).tolist(), in_spikes, label="in spike")
pyplot.xlabel("t")
pyplot.ylabel("spike")
pyplot.legend()

out_i = []
for i in range(T):
    out_i.append(lp_syn(in_spikes[i]))
pyplot.subplot(2, 1, 2)
pyplot.plot(out_i, label="out i")
pyplot.xlabel("t")
pyplot.ylabel("i")
pyplot.legend()
pyplot.show()
../_images/SynapseFilter.png
返回:

None

返回类型:

None

static js_single_step_forward_learnable(x: Tensor, w: Tensor, out_i: Tensor)[源代码]#
static js_single_step_forward(x: Tensor, tau: float, out_i: Tensor)[源代码]#
single_step_forward(x: Tensor)[源代码]#
class spikingjelly.activation_based.layer.misc.PrintShapeModule(ext_str='PrintShapeModule')[源代码]#

基类：Module

API Language: 中文 | English

  • 中文

只打印 ext_str 和输入的 shape，不进行任何操作的网络层，可以用于debug。

参数:

ext_str (str) -- 额外打印的字符串

  • English

This layer will not do any operation but print ext_str and the shape of input, which can be used for debugging.

参数:

ext_str (str) -- extra strings for printing

返回:

None

返回类型:

None

forward(x: Tensor)[源代码]#
class spikingjelly.activation_based.layer.misc.VotingLayer(voting_size: int = 10, step_mode='s')[源代码]#

基类：Module, StepModule

API Language: 中文 | English

  • 中文

投票层，对 shape = [..., C * voting_size] 的输入在最后一维上做 kernel_size = voting_size, stride = voting_size 的平均池化

参数:

  • voting_size (int) -- 决定一个类别的投票数量

  • step_mode (str) -- 步进模式，可以为 's' (单步) 或 'm' (多步)

  • English

Applies average pooling with kernel_size = voting_size, stride = voting_size on the last dimension of the input with shape = [..., C * voting_size]

参数:

  • voting_size (int) -- the voting numbers for determine a class

  • step_mode (str) -- the step mode, which can be s (single-step) or m (multi-step)

返回:

None

返回类型:

None

single_step_forward(x: Tensor)[源代码]#
forward(x: Tensor)[源代码]#
class spikingjelly.activation_based.layer.misc.Delay(delay_steps: int, step_mode='s')[源代码]#

基类：MemoryModule

API Language: 中文 | English

  • 中文

延迟层，可以用来延迟输入，使得 y[t] = x[t - delay_steps]。缺失的数据用0填充。

参数:

  • delay_steps (int) -- 延迟的时间步数

  • step_mode (str) -- 步进模式，可以为 's' (单步) 或 'm' (多步)

  • English

A delay layer that can delay inputs and makes y[t] = x[t - delay_steps]. The nonexistent data will be regarded as 0.

参数:

  • delay_steps (int) -- the number of delayed time-steps

  • step_mode (str) -- the step mode, which can be s (single-step) or m (multi-step)

  • 代码示例 | Example

delay_layer = Delay(delay_steps=1, step_mode="m")
x = torch.rand([5, 2])
x[3:].zero_()
x.requires_grad = True
y = delay_layer(x)
print("x=")
print(x)
print("y=")
print(y)
y.sum().backward()
print("x.grad=")
print(x.grad)

The outputs are:

x=
tensor([[0.2510, 0.7246],
        [0.5303, 0.3160],
        [0.2531, 0.5961],
        [0.0000, 0.0000],
        [0.0000, 0.0000]], requires_grad=True)
y=
tensor([[0.0000, 0.0000],
        [0.2510, 0.7246],
        [0.5303, 0.3160],
        [0.2531, 0.5961],
        [0.0000, 0.0000]], grad_fn=<CatBackward0>)
x.grad=
tensor([[1., 1.],
        [1., 1.],
        [1., 1.],
        [1., 1.],
        [0., 0.]])
返回:

None

返回类型:

None

property delay_steps#
single_step_forward(x: Tensor)[源代码]#
multi_step_forward(x_seq: Tensor)[源代码]#