spikingjelly.datasets package

Submodules

spikingjelly.datasets.asl_dvs module

class spikingjelly.datasets.asl_dvs.ASLDVS(root: str, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The ASL-DVS dataset, which is proposed by Graph-based Object Classification for Neuromorphic Vision Sensing.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static load_origin_data(file_name: str) → Dict

参数:

file_name (str) – path of the events file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function defines how to read the origin binary data.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static read_mat_save_to_np(mat_file: str, np_file: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

spikingjelly.datasets.cifar10_dvs module

spikingjelly.datasets.cifar10_dvs.read_bits(arr, mask=None, shift=None)

spikingjelly.datasets.cifar10_dvs.skip_header(fp)

spikingjelly.datasets.cifar10_dvs.load_raw_events(fp, bytes_skip=0, bytes_trim=0, filter_dvs=False, times_first=False)

spikingjelly.datasets.cifar10_dvs.parse_raw_address(addr, x_mask=4190208, x_shift=12, y_mask=2143289344, y_shift=22, polarity_mask=2048, polarity_shift=11)

spikingjelly.datasets.cifar10_dvs.load_events(fp, filter_dvs=False, **kwargs)

class spikingjelly.datasets.cifar10_dvs.CIFAR10DVS(root: str, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The CIFAR10-DVS dataset, which is proposed by `CIFAR10-DVS: An Event-Stream Dataset for Object Classification

<https://internal-journal.frontiersin.org/articles/10.3389/fnins.2017.00309/full>`_.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static load_origin_data(file_name: str) → Dict

参数:

file_name (str) – path of the events file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function defines how to read the origin binary data.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static read_aedat_save_to_np(bin_file: str, np_file: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

spikingjelly.datasets.dvs128_gesture module

class spikingjelly.datasets.dvs128_gesture.DVS128Gesture(root: str, train: Optional[bool] = None, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The DVS128 Gesture dataset, which is proposed by A Low Power, Fully Event-Based Gesture Recognition System.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

Note

In SpikingJelly, there are 1176 train samples and 288 test samples. The total samples number is 1464.

from spikingjelly.datasets import dvs128_gesture

data_dir = 'D:/datasets/DVS128Gesture'
train_set = dvs128_gesture.DVS128Gesture(data_dir, train=True)
test_set = dvs128_gesture.DVS128Gesture(data_dir, train=False)
print(f'train samples = {train_set.__len__()}, test samples = {test_set.__len__()}')
print(f'total samples = {train_set.__len__() + test_set.__len__()}')

# train samples = 1176, test samples = 288
# total samples = 1464

While from the origin paper, the DvsGesture dataset comprises 1342 instances of a set of 11 hand and arm gestures. The difference may be caused by different pre-processing methods.

snnTorch have the same numbers with SpikingJelly:

from snntorch.spikevision import spikedata

train_set = spikedata.DVSGesture("D:/datasets/DVS128Gesture/temp2", train=True, num_steps=500, dt=1000)
test_set = spikedata.DVSGesture("D:/datasets/DVS128Gesture/temp2", train=False, num_steps=1800, dt=1000)
print(f'train samples = {train_set.__len__()}, test samples = {test_set.__len__()}')
print(f'total samples = {train_set.__len__() + test_set.__len__()}')

# train samples = 1176, test samples = 288
# total samples = 1464

But tonic has different numbers, which are close to 1342:

import tonic

train_set = tonic.datasets.DVSGesture(save_to='D:/datasets/DVS128Gesture/temp', train=True)
test_set = tonic.datasets.DVSGesture(save_to='D:/datasets/DVS128Gesture/temp', train=False)
print(f'train samples = {train_set.__len__()}, test samples = {test_set.__len__()}')
print(f'total samples = {train_set.__len__() + test_set.__len__()}')

# train samples = 1077, test samples = 264
# total samples = 1341

Here we show how 1176 train samples and 288 test samples are got in SpikingJelly.

The origin dataset is split to train and test set by trials_to_train.txt and trials_to_test.txt.

trials_to_train.txt:

    user01_fluorescent.aedat
    user01_fluorescent_led.aedat
    ...
    user23_lab.aedat
    user23_led.aedat

trials_to_test.txt:

    user24_fluorescent.aedat
    user24_fluorescent_led.aedat
    ...
    user29_led.aedat
    user29_natural.aedat

SpikingJelly will read the txt file and get the aedat file name like user01_fluorescent.aedat. The corresponding label file name will be regarded as user01_fluorescent_labels.csv.

user01_fluorescent_labels.csv:

    class       startTime_usec  endTime_usec
    1   80048239        85092709
    2   89431170        95231007
    3   95938861        103200075
    4   114845417       123499505
    5   124344363       131742581
    6   133660637       141880879
    7   142360393       149138239
    8   150717639       157362334
    8   157773346       164029864
    9   165057394       171518239
    10  172843790       179442817
    11  180675853       187389051

Then SpikingJelly will split the aedat to samples by the time range and class in the csv file. In this sample, the first sample user01_fluorescent_0.npz is sliced from the origin events user01_fluorescent.aedat with 80048239 <= t < 85092709 and label=0. user01_fluorescent_0.npz will be saved in root/events_np/train/0.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static load_origin_data(file_name: str) → Dict

参数:

file_name (str) – path of the events file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function defines how to read the origin binary data.

static split_aedat_files_to_np(fname: str, aedat_file: str, csv_file: str, output_dir: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

spikingjelly.datasets.es_imagenet module

spikingjelly.datasets.es_imagenet.load_events(fname: str)

class spikingjelly.datasets.es_imagenet.ESImageNet(root: str, train: Optional[bool] = None, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The ES-ImageNet dataset, which is proposed by ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static load_events_np(fname: str)

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

spikingjelly.datasets.n_caltech101 module

class spikingjelly.datasets.n_caltech101.NCaltech101(root: str, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The N-Caltech101 dataset, which is proposed by Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static load_origin_data(file_name: str) → Dict

参数:

file_name (str) – path of the events file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function defines how to read the origin binary data.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static read_bin_save_to_np(bin_file: str, np_file: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

spikingjelly.datasets.n_mnist module

class spikingjelly.datasets.n_mnist.NMNIST(root: str, train: Optional[bool] = None, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The N-MNIST dataset, which is proposed by Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static load_origin_data(file_name: str) → Dict

参数:

file_name (str) – path of the events file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function defines how to read the origin binary data.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static read_bin_save_to_np(bin_file: str, np_file: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

spikingjelly.datasets.nav_gesture module

spikingjelly.datasets.nav_gesture.peek(f, length=1)

spikingjelly.datasets.nav_gesture.readATIS_tddat(file_name, orig_at_zero=True, drop_negative_dt=True, verbose=True, events_restriction=[0, inf])

reads ATIS td events in .dat format

input: filename: string, path to the .dat file orig_at_zero: bool, if True, timestamps will start at 0 drop_negative_dt: bool, if True, events with a timestamp greater than the previous event are dismissed verbose: bool, if True, verbose mode. events_restriction: list [min ts, max ts], will return only events with ts in the defined boundaries

output: timestamps: numpy array of length (number of events), timestamps coords: numpy array of size (number of events, 2), spatial coordinates: col 0 is x, col 1 is y. polarities: numpy array of length (number of events), polarities removed_events: integer, number of removed events (negative delta-ts)

class spikingjelly.datasets.nav_gesture.NAVGestureWalk(root: str, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NeuromorphicDatasetFolder

The Nav Gesture dataset, which is proposed by Event-Based Gesture Recognition With Dynamic Background Suppression Using Smartphone Computational Capabilities.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W` is the weight of the data. For example, this function returns ``(128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static load_origin_data(file_name: str) → Dict

static read_aedat_save_to_np(bin_file: str, np_file: str)

static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

class spikingjelly.datasets.nav_gesture.NAVGestureSit(root: str, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：NAVGestureWalk

The Nav Gesture dataset, which is proposed by Event-Based Gesture Recognition With Dynamic Background Suppression Using Smartphone Computational Capabilities.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

spikingjelly.datasets.shd module

spikingjelly.datasets.shd.cal_fixed_frames_number_segment_index_shd(events_t: ndarray, split_by: str, frames_num: int) → tuple

spikingjelly.datasets.shd.integrate_events_segment_to_frame_shd(x: ndarray, W: int, j_l: int = 0, j_r: int = -1) → ndarray

spikingjelly.datasets.shd.integrate_events_by_fixed_frames_number_shd(events: Dict, split_by: str, frames_num: int, W: int) → ndarray

spikingjelly.datasets.shd.integrate_events_file_to_frames_file_by_fixed_frames_number_shd(h5_file: h5py.File, i: int, output_dir: str, split_by: str, frames_num: int, W: int, print_save: bool = False) → None

spikingjelly.datasets.shd.integrate_events_by_fixed_duration_shd(events: Dict, duration: int, W: int) → ndarray

spikingjelly.datasets.shd.integrate_events_file_to_frames_file_by_fixed_duration_shd(h5_file: h5py.File, i: int, output_dir: str, duration: int, W: int, print_save: bool = False) → None

spikingjelly.datasets.shd.custom_integrate_function_example(h5_file: h5py.File, i: int, output_dir: str, W: int)

class spikingjelly.datasets.shd.SpikingHeidelbergDigits(root: str, train: Optional[bool] = None, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：Dataset

The Spiking Heidelberg Digits (SHD) dataset, which is proposed by The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

Note

Events in this dataset are in the format of (x, t) rather than (x, y, t, p). Thus, this dataset is not inherited from spikingjelly.datasets.NeuromorphicDatasetFolder directly. But their procedures are similar.

spikingjelly.datasets.shd.custom_integrate_function_example is an example of custom_integrate_function, which is similar to the cunstom function for DVS Gesture in the Neuromorphic Datasets Processing tutorial.

classes_number()

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static get_W()

class spikingjelly.datasets.shd.SpikingSpeechCommands(root: str, split: str = 'train', data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：Dataset

The Spiking Speech Commands (SSC) dataset, which is proposed by The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks.

Refer to spikingjelly.datasets.NeuromorphicDatasetFolder for more details about params information.

Note

Events in this dataset are in the format of (x, t) rather than (x, y, t, p). Thus, this dataset is not inherited from spikingjelly.datasets.NeuromorphicDatasetFolder directly. But their procedures are similar.

spikingjelly.datasets.shd.custom_integrate_function_example is an example of custom_integrate_function, which is similar to the cunstom function for DVS Gesture in the Neuromorphic Datasets Processing tutorial.

classes_number()

static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

static get_W()

spikingjelly.datasets.speechcommands module

spikingjelly.datasets.speechcommands.load_speechcommands_item(relpath: str, path: str) → Tuple[Tensor, int, str, str, int]

class spikingjelly.datasets.speechcommands.SPEECHCOMMANDS(label_dict: Dict, root: str, silence_cnt: Optional[int] = 0, silence_size: Optional[int] = 16000, transform: Optional[Callable] = None, url: Optional[str] = 'speech_commands_v0.02', split: Optional[str] = 'train', folder_in_archive: Optional[str] = 'SpeechCommands', download: Optional[bool] = False)

基类：Dataset

参数:

• label_dict (Dict) – 标签与类别的对应字典

• root (str) – 数据集的根目录

• silence_cnt (int, optional) – Silence数据的数量

• silence_size (int, optional) – Silence数据的尺寸

• transform (Callable, optional) – A function/transform that takes in a raw audio

• url (str, optional) – 数据集版本，默认为v0.02

• split (str, optional) – 数据集划分，可以是 "train", "test", "val"，默认为 "train"

• folder_in_archive (str, optional) – 解压后的目录名称，默认为 "SpeechCommands"

• download (bool, optional) – 是否下载数据，默认为False

SpeechCommands语音数据集，出自 Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition，根据给出的测试集与验证集列表进行了划分，包含v0.01与v0.02两个版本。

数据集包含三大类单词的音频：

1. 指令单词，共10个，”Yes”, “No”, “Up”, “Down”, “Left”, “Right”, “On”, “Off”, “Stop”, “Go”. 对于v0.02，还额外增加了5个：”Forward”, “Backward”, “Follow”, “Learn”, “Visual”.

2. 0~9的数字，共10个：”One”, “Two”, “Three”, “Four”, “Five”, “Six”, “Seven”, “Eight”, “Nine”.

3. 辅助词，可以视为干扰词，共10个：”Bed”, “Bird”, “Cat”, “Dog”, “Happy”, “House”, “Marvin”, “Sheila”, “Tree”, “Wow”.

v0.01版本包含共计30类，64,727个音频片段，v0.02版本包含共计35类，105,829个音频片段。更详细的介绍参见前述论文，以及数据集的README。

代码实现基于torchaudio并扩充了功能，同时也参考了 原论文的实现。

Module contents

spikingjelly.datasets.save_every_frame_of_an_entire_DVS_dataset(dataset: str, dataset_path: str, time_steps: int, save_pic_to: str = './', number_of_threads: int = 4)

参数:

• dataset (str) – Name of the dataset to be saved. The current options available are: DVS128Gesture, CIFAR10DVS and NCaltech101.

• dataset_path (str) – Same storage path as loading dataset.

• time_steps (int) – Same T as loading the dataset.

• save_pic_to (str) – Where to store each frame’s image.

• number_of_threads (int) – How many threads are used to save images.

demo: save_every_frame_of_an_entire_DVS_dataset(dataset=’DVS128Gesture’, dataset_path=”../../datasets/DVS128Gesture”,

time_steps=16, save_pic_to=’./demo’, number_of_threads=20)

save_every_frame_of_an_entire_DVS_dataset(dataset=’CIFAR10DVS’, dataset_path=”../../datasets/cifar10dvs”,

time_steps=10, save_pic_to=’./demo’, number_of_threads=20)

save_every_frame_of_an_entire_DVS_dataset(dataset=’NCaltech101’, dataset_path=”../../datasets/NCaltech101”,

time_steps=14, save_pic_to=’./demo’, number_of_threads=20)

spikingjelly.datasets.save_as_pic(x: Union[Tensor, ndarray], save_pic_to: str = './', pic_first_name: str = 'pic') → None

参数:

• x (Union[torch.Tensor, np.ndarray]) – frames with shape=[T, 2, H, W]

• save_pic_to (str) – Where to store images.

• pic_first_name (str) – Prefix for image names before _t (stored image names are: ``pic_first_name``_t.png)

返回:

None

demo: save_as_pic(frame, ‘./demo’, ‘first_pic’)

spikingjelly.datasets.play_frame(x: Union[Tensor, ndarray], save_gif_to: Optional[str] = None) → None

参数:

• x (Union[torch.Tensor, np.ndarray]) – frames with shape=[T, 2, H, W]

• save_gif_to (str) – If None, this function will play the frames. If True, this function will not play the frames but save frames to a gif file in the directory save_gif_to

返回:

None

spikingjelly.datasets.load_aedat_v3(file_name: str) → Dict

参数:

file_name (str) – path of the aedat v3 file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function is written by referring to https://gitlab.com/inivation/dv/dv-python . It can be used for DVS128 Gesture.

spikingjelly.datasets.load_ATIS_bin(file_name: str) → Dict

参数:

file_name (str) – path of the aedat v3 file

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

返回类型:

Dict

This function is written by referring to https://github.com/jackd/events-tfds . Each ATIS binary example is a separate binary file consisting of a list of events. Each event occupies 40 bits as described below: bit 39 - 32: Xaddress (in pixels) bit 31 - 24: Yaddress (in pixels) bit 23: Polarity (0 for OFF, 1 for ON) bit 22 - 0: Timestamp (in microseconds)

spikingjelly.datasets.load_npz_frames(file_name: str) → ndarray

参数:

file_name (str) – path of the npz file that saves the frames

返回:

frames

返回类型:

np.ndarray

spikingjelly.datasets.integrate_events_segment_to_frame(x: ndarray, y: ndarray, p: ndarray, H: int, W: int, j_l: int = 0, j_r: int = -1) → ndarray

参数:

• x (numpy.ndarray) – x-coordinate of events

• y (numpy.ndarray) – y-coordinate of events

• p (numpy.ndarray) – polarity of events

• H (int) – height of the frame

• W (int) – weight of the frame

• j_l (int) – the start index of the integral interval, which is included

• j_r – the right index of the integral interval, which is not included

返回:

frames

返回类型:

np.ndarray

Denote a two channels frame as \(F\) and a pixel at \((p, x, y)\) as \(F(p, x, y)\), the pixel value is integrated from the events data whose indices are in \([j_{l}, j_{r})\):

\[F(p, x, y) = \sum_{i = j_{l}}^{j_{r} - 1} \mathcal{I}_{p, x, y}(p_{i}, x_{i}, y_{i})\]

where \(\lfloor \cdot \rfloor\) is the floor operation, \(\mathcal{I}_{p, x, y}(p_{i}, x_{i}, y_{i})\) is an indicator function and it equals 1 only when \((p, x, y) = (p_{i}, x_{i}, y_{i})\).

spikingjelly.datasets.cal_fixed_frames_number_segment_index(events_t: ndarray, split_by: str, frames_num: int) → tuple

参数:

• events_t (numpy.ndarray) – events’ t

• split_by (str) – ‘time’ or ‘number’

• frames_num (int) – the number of frames

返回:

a tuple (j_l, j_r)

返回类型:

tuple

Denote frames_num as \(M\), if split_by is 'time', then

\[\begin{split}\Delta T & = [\frac{t_{N-1} - t_{0}}{M}] \\ j_{l} & = \mathop{\arg\min}\limits_{k} \{t_{k} | t_{k} \geq t_{0} + \Delta T \cdot j\} \\ j_{r} & = \begin{cases} \mathop{\arg\max}\limits_{k} \{t_{k} | t_{k} < t_{0} + \Delta T \cdot (j + 1)\} + 1, & j < M - 1 \cr N, & j = M - 1 \end{cases}\end{split}\]

If split_by is 'number', then

\[\begin{split}j_{l} & = [\frac{N}{M}] \cdot j \\ j_{r} & = \begin{cases} [\frac{N}{M}] \cdot (j + 1), & j < M - 1 \cr N, & j = M - 1 \end{cases}\end{split}\]

spikingjelly.datasets.integrate_events_by_fixed_frames_number(events: Dict, split_by: str, frames_num: int, H: int, W: int) → ndarray

参数:

• events (Dict) – a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

• split_by (str) – ‘time’ or ‘number’

• frames_num (int) – the number of frames

• H (int) – the height of frame

• W (int) – the weight of frame

返回:

frames

返回类型:

np.ndarray

Integrate events to frames by fixed frames number. See cal_fixed_frames_number_segment_index and integrate_events_segment_to_frame for more details.

spikingjelly.datasets.integrate_events_file_to_frames_file_by_fixed_frames_number(loader: Callable, events_np_file: str, output_dir: str, split_by: str, frames_num: int, H: int, W: int, print_save: bool = False) → None

参数:

• loader (Callable) – a function that can load events from events_np_file

• events_np_file (str) – path of the events np file

• output_dir (str) – output directory for saving the frames

• split_by (str) – ‘time’ or ‘number’

• frames_num (int) – the number of frames

• H (int) – the height of frame

• W (int) – the weight of frame

• print_save (bool) – If True, this function will print saved files’ paths.

返回:

None

Integrate a events file to frames by fixed frames number and save it. See cal_fixed_frames_number_segment_index and integrate_events_segment_to_frame for more details.

spikingjelly.datasets.integrate_events_by_fixed_duration(events: Dict, duration: int, H: int, W: int) → ndarray

参数:

• events (Dict) – a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

• duration (int) – the time duration of each frame

• H (int) – the height of frame

• W (int) – the weight of frame

返回:

frames

返回类型:

np.ndarray

Integrate events to frames by fixed time duration of each frame.

spikingjelly.datasets.integrate_events_file_to_frames_file_by_fixed_duration(loader: Callable, events_np_file: str, output_dir: str, duration: int, H: int, W: int, print_save: bool = False) → None

参数:

• loader (Callable) – a function that can load events from events_np_file

• events_np_file (str) – path of the events np file

• output_dir (str) – output directory for saving the frames

• duration (int) – the time duration of each frame

• H (int) – the height of frame

• W (int) – the weight of frame

• print_save (bool) – If True, this function will print saved files’ paths.

返回:

None

Integrate events to frames by fixed time duration of each frame.

spikingjelly.datasets.save_frames_to_npz_and_print(fname: str, frames)

spikingjelly.datasets.create_same_directory_structure(source_dir: str, target_dir: str) → None

参数:

• source_dir (str) – Path of the directory that be copied from

• target_dir (str) – Path of the directory that be copied to

返回:

None

Create the same directory structure in target_dir with that of source_dir.

spikingjelly.datasets.split_to_train_test_set(train_ratio: float, origin_dataset: Dataset, num_classes: int, random_split: bool = False)

参数:

• train_ratio (float) – split the ratio of the origin dataset as the train set

• origin_dataset (torch.utils.data.Dataset) – the origin dataset

• num_classes (int) – total classes number, e.g., 10 for the MNIST dataset

• random_split (int) – If False, the front ratio of samples in each classes will be included in train set, while the reset will be included in test set. If True, this function will split samples in each classes randomly. The randomness is controlled by numpy.random.seed

返回:

a tuple (train_set, test_set)

返回类型:

tuple

spikingjelly.datasets.pad_sequence_collate(batch: list)

参数:

batch (list) – a list of samples that contains (x, y), where x is a list containing sequences with different length and y is the label

返回:

batched samples (x_p, y, x_len), where ``x_p is padded x with the same length, y` is the label, and x_len is the length of the x

返回类型:

tuple

This function can be use as the collate_fn for DataLoader to process the dataset with variable length, e.g., a NeuromorphicDatasetFolder with fixed duration to integrate events to frames. Here is an example: .. code-block:: python class VariableLengthDataset(torch.utils.data.Dataset):

def __init__(self, n=1000):

super().__init__() self.n = n

def __getitem__(self, i):

return torch.rand([i + 1, 2]), self.n - i - 1

def __len__(self):

return self.n

loader = torch.utils.data.DataLoader(VariableLengthDataset(n=32), batch_size=2, collate_fn=pad_sequence_collate,

shuffle=True)

for i, (x_p, label, x_len) in enumerate(loader):

print(f’x_p.shape={x_p.shape}, label={label}, x_len={x_len}’) if i == 2:

break

And the outputs are: .. code-block:: bash

x_p.shape=torch.Size([2, 18, 2]), label=tensor([14, 30]), x_len=tensor([18, 2]) x_p.shape=torch.Size([2, 29, 2]), label=tensor([3, 6]), x_len=tensor([29, 26]) x_p.shape=torch.Size([2, 23, 2]), label=tensor([ 9, 23]), x_len=tensor([23, 9])

spikingjelly.datasets.padded_sequence_mask(sequence_len: Tensor, T=None)

参数:

• sequence_len (torch.Tensor) – a tensor shape = [N] that contains sequences lengths of each batch element

• T (int) – The maximum length of sequences. If None, the maximum element in sequence_len will be seen as T

返回:

a bool mask with shape = [T, N], where the padded position is False

返回类型:

torch.Tensor

Here is an example: .. code-block:: python

x1 = torch.rand([2, 6]) x2 = torch.rand([3, 6]) x3 = torch.rand([4, 6]) x = torch.nn.utils.rnn.pad_sequence([x1, x2, x3]) # [T, N, *] print(‘x.shape=’, x.shape) x_len = torch.as_tensor([x1.shape[0], x2.shape[0], x3.shape[0]]) mask = padded_sequence_mask(x_len) print(‘mask.shape=’, mask.shape) print(‘mask=n’, mask)

And the outputs are: .. code-block:: bash

x.shape= torch.Size([4, 3, 6]) mask.shape= torch.Size([4, 3]) mask=

tensor([[ True, True, True],

[ True, True, True], [False, True, True], [False, False, True]])

class spikingjelly.datasets.NeuromorphicDatasetFolder(root: str, train: Optional[bool] = None, data_type: str = 'event', frames_number: Optional[int] = None, split_by: Optional[str] = None, duration: Optional[int] = None, custom_integrate_function: Optional[Callable] = None, custom_integrated_frames_dir_name: Optional[str] = None, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None)

基类：DatasetFolder

参数:

• root (str) – root path of the dataset

• train (bool) – whether use the train set. Set True or False for those datasets provide train/test division, e.g., DVS128 Gesture dataset. If the dataset does not provide train/test division, e.g., CIFAR10-DVS, please set None and use split_to_train_test_set function to get train/test set

• data_type (str) – event or frame

• frames_number (int) – the integrated frame number

• split_by (str) – time or number

• duration (int) – the time duration of each frame

• custom_integrate_function (Callable) – a user-defined function that inputs are events, H, W. events is a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray H is the height of the data and W is the weight of the data. For example, H=128 and W=128 for the DVS128 Gesture dataset. The user should define how to integrate events to frames, and return frames.

• custom_integrated_frames_dir_name (Optional[str]) – The name of directory for saving frames integrating by custom_integrate_function. If custom_integrated_frames_dir_name is None, it will be set to custom_integrate_function.__name__

• transform (callable) – a function/transform that takes in a sample and returns a transformed version. E.g, transforms.RandomCrop for images.

• target_transform (callable) – a function/transform that takes in the target and transforms it.

The base class for neuromorphic dataset. Users can define a new dataset by inheriting this class and implementing all abstract methods. Users can refer to spikingjelly.datasets.dvs128_gesture.DVS128Gesture. If data_type == 'event'

the sample in this dataset is a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray.

If data_type == 'frame' and frames_number is not None

events will be integrated to frames with fixed frames number. split_by will define how to split events. See cal_fixed_frames_number_segment_index for more details.

If data_type == 'frame', frames_number is None, and duration is not None

events will be integrated to frames with fixed time duration.

If data_type == 'frame', frames_number is None, duration is None, and custom_integrate_function is not None:

events will be integrated by the user-defined function and saved to the custom_integrated_frames_dir_name directory in root directory. Here is an example from SpikingJelly’s tutorials:

from spikingjelly.datasets.dvs128_gesture import DVS128Gesture
from typing import Dict
import numpy as np
import spikingjelly.datasets as sjds
def integrate_events_to_2_frames_randomly(events: Dict, H: int, W: int):
    index_split = np.random.randint(low=0, high=events['t'].__len__())
    frames = np.zeros([2, 2, H, W])
    t, x, y, p = (events[key] for key in ('t', 'x', 'y', 'p'))
    frames[0] = sjds.integrate_events_segment_to_frame(x, y, p, H, W, 0, index_split)
    frames[1] = sjds.integrate_events_segment_to_frame(x, y, p, H, W, index_split, events['t'].__len__())
    return frames
root_dir = 'D:/datasets/DVS128Gesture'
train_set = DVS128Gesture(root_dir, train=True, data_type='frame', custom_integrate_function=integrate_events_to_2_frames_randomly)
from spikingjelly.datasets import play_frame
frame, label = train_set[500]
play_frame(frame)

set_root_when_train_is_none(_root: str)

abstract static resource_url_md5() → list

返回:

A list url that url[i] is a tuple, which contains the i-th file’s name, download link, and MD5

返回类型:

list

abstract static downloadable() → bool

返回:

Whether the dataset can be directly downloaded by python codes. If not, the user have to download it manually

返回类型:

bool

abstract static extract_downloaded_files(download_root: str, extract_root: str)

参数:

• download_root (str) – Root directory path which saves downloaded dataset files

• extract_root (str) – Root directory path which saves extracted files from downloaded files

返回:

None

This function defines how to extract download files.

abstract static create_events_np_files(extract_root: str, events_np_root: str)

参数:

• extract_root (str) – Root directory path which saves extracted files from downloaded files

• events_np_root – Root directory path which saves events files in the npz format

返回:

None

This function defines how to convert the origin binary data in extract_root to npz format and save converted files in events_np_root.

abstract static get_H_W() → Tuple

返回:

A tuple (H, W), where H is the height of the data and W is the weight of the data. For example, this function returns (128, 128) for the DVS128 Gesture dataset.

返回类型:

tuple

static load_events_np(fname: str)

参数:

fname – file name

返回:

a dict whose keys are ['t', 'x', 'y', 'p'] and values are numpy.ndarray

This function defines how to load a sample from events_np. In most cases, this function is np.load. But for some datasets, e.g., ES-ImageNet, it can be different.

spikingjelly.datasets.random_temporal_delete(x_seq: Union[Tensor, ndarray], T_remain: int, batch_first)

参数:

• x_seq (Union[torch.Tensor, np.ndarray]) – a sequence with shape = [T, N, *], where T is the sequence length and N is the batch size

• T_remain (int) – the remained length

• batch_first (bool) – if True, x_seq will be regarded as shape = [N, T, *]

返回:

the sequence with length T_remain, which is obtained by randomly removing T - T_remain slices

返回类型:

Union[torch.Tensor, np.ndarray]

The random temporal delete data augmentation used in Deep Residual Learning in Spiking Neural Networks. Codes example:

import torch
from spikingjelly.datasets import random_temporal_delete
T = 8
T_remain = 5
N = 4
x_seq = torch.arange(0, N*T).view([N, T])
print('x_seq=\n', x_seq)
print('random_temporal_delete(x_seq)=\n', random_temporal_delete(x_seq, T_remain, batch_first=True))

Outputs:

x_seq=
 tensor([[ 0,  1,  2,  3,  4,  5,  6,  7],
        [ 8,  9, 10, 11, 12, 13, 14, 15],
        [16, 17, 18, 19, 20, 21, 22, 23],
        [24, 25, 26, 27, 28, 29, 30, 31]])
random_temporal_delete(x_seq)=
 tensor([[ 0,  1,  4,  6,  7],
        [ 8,  9, 12, 14, 15],
        [16, 17, 20, 22, 23],
        [24, 25, 28, 30, 31]])

class spikingjelly.datasets.RandomTemporalDelete(T_remain: int, batch_first: bool)

基类：Module

参数:

• T_remain (int) – the remained length

• batch_first – if True, x_seq will be regarded as shape = [N, T, *]

The random temporal delete data augmentation used in Deep Residual Learning in Spiking Neural Networks. Refer to random_temporal_delete for more details.

forward(x_seq: Union[Tensor, ndarray])

training: bool

spikingjelly.datasets.create_sub_dataset(source_dir: str, target_dir: str, ratio: float, use_soft_link=True, randomly=False)

参数:

• source_dir (str) – the directory path of the origin dataset

• target_dir (str) – the directory path of the sub dataset

• ratio (float) – the ratio of samples sub dataset will copy from the origin dataset

• use_soft_link (bool) – if True, the sub dataset will use soft link to copy; else, the sub dataset will copy files

• randomly (bool) – if True, the files copy from the origin dataset will be picked up randomly. The randomness is controlled by numpy.random.seed

Create a sub dataset with copy ratio of samples from the origin dataset.