spikingjelly.datasets.dvs128_gesture module#

class spikingjelly.datasets.dvs128_gesture.DVS128Gesture(root: str, train: bool = True, data_type: str = 'event', frames_number: int = None, split_by: str = None, duration: int = None, custom_integrate_function: Callable = None, custom_integrated_frames_dir_name: str = None, transform: Callable | None = None, target_transform: Callable | None = None)[源代码]#

基类：NeuromorphicDatasetFolder

API Language: 中文 | English

  • 中文

DVS128 Gesture 数据集，由 A Low Power, Fully Event-Based Gesture Recognition System 提出。

有关参数的更多详细信息，请参考 NeuromorphicDatasetFolder

注意

在 SpikingJelly 中，有 1176 个训练样本和 288 个测试样本。总样本数为 1464 个。

from spikingjelly.datasets import dvs128_gesture

data_dir = "D:/datasets/DVS128Gesture"
train_set = dvs128_gesture.DVS128Gesture(data_dir, train=True)
test_set = dvs128_gesture.DVS128Gesture(data_dir, train=False)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1176, test samples = 288
# total samples = 1464

虽然原始论文指出，the DvsGesture dataset comprises 1342 instances of a set of 11 hand and arm gestures。差异可能是由于不同的预处理方法导致的。

snnTorch 与 SpikingJelly 的数字相同：

from snntorch.spikevision import spikedata

train_set = spikedata.DVSGesture(
    "D:/datasets/DVS128Gesture/temp2",
    train=True,
    num_steps=500,
    dt=1000,
)
test_set = spikedata.DVSGesture(
    "D:/datasets/DVS128Gesture/temp2",
    train=False,
    num_steps=1800,
    dt=1000,
)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1176, test samples = 288
# total samples = 1464

但 tonic 的数字不同，接近 1342：

import tonic

train_set = tonic.datasets.DVSGesture(
    save_to="D:/datasets/DVS128Gesture/temp", train=True
)
test_set = tonic.datasets.DVSGesture(
    save_to="D:/datasets/DVS128Gesture/temp", train=False
)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1077, test samples = 264
# total samples = 1341

下面说明 SpikingJelly 如何获得 1176 个训练样本和 288 个测试样本。

原始数据集通过 trials_to_train.txt 和 trials_to_test.txt 划分为训练集和测试集。

trials_to_train.txt:

    user01_fluorescent.aedat
    user01_fluorescent_led.aedat
    ...
    user23_lab.aedat
    user23_led.aedat

trials_to_test.txt:

    user24_fluorescent.aedat
    user24_fluorescent_led.aedat
    ...
    user29_led.aedat
    user29_natural.aedat

SpikingJelly 将读取 txt 文件并获取 aedat 文件名，如 user01_fluorescent.aedat。对应的 标签文件名将被视为 user01_fluorescent_labels.csv。

user01_fluorescent_labels.csv:

    class       startTime_usec  endTime_usec
    1   80048239        85092709
    2   89431170        95231007
    3   95938861        103200075
    4   114845417       123499505
    5   124344363       131742581
    6   133660637       141880879
    7   142360393       149138239
    8   150717639       157362334
    8   157773346       164029864
    9   165057394       171518239
    10  172843790       179442817
    11  180675853       187389051

然后 SpikingJelly 将根据 csv 文件中的时间范围和类别将 aedat 切分为样本。在这个示例中， 第一个样本 user01_fluorescent_0.npz 是从原始事件 user01_fluorescent.aedat 中切分的， 带有 80048239 <= t < 85092709 和 label=0。user01_fluorescent_0.npz 将保存在 root/events_np/train/0。

  • English

The DVS128 Gesture dataset, which is proposed by A Low Power, Fully Event-Based Gesture Recognition System.

Refer to NeuromorphicDatasetFolder for more details about params information.

Note

In SpikingJelly, there are 1176 train samples and 288 test samples. The total samples number is 1464.

from spikingjelly.datasets import dvs128_gesture

data_dir = "D:/datasets/DVS128Gesture"
train_set = dvs128_gesture.DVS128Gesture(data_dir, train=True)
test_set = dvs128_gesture.DVS128Gesture(data_dir, train=False)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1176, test samples = 288
# total samples = 1464

While from the origin paper, the DvsGesture dataset comprises 1342 instances of a set of 11 hand and arm gestures. The difference may be caused by different pre-processing methods.

snnTorch have the same numbers with SpikingJelly:

from snntorch.spikevision import spikedata

train_set = spikedata.DVSGesture(
    "D:/datasets/DVS128Gesture/temp2",
    train=True,
    num_steps=500,
    dt=1000,
)
test_set = spikedata.DVSGesture(
    "D:/datasets/DVS128Gesture/temp2",
    train=False,
    num_steps=1800,
    dt=1000,
)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1176, test samples = 288
# total samples = 1464

But tonic has different numbers, which are close to 1342:

import tonic

train_set = tonic.datasets.DVSGesture(
    save_to="D:/datasets/DVS128Gesture/temp", train=True
)
test_set = tonic.datasets.DVSGesture(
    save_to="D:/datasets/DVS128Gesture/temp", train=False
)
print(
    f"train samples = {train_set.__len__()}, test samples = {test_set.__len__()}"
)
print(f"total samples = {train_set.__len__() + test_set.__len__()}")

# train samples = 1077, test samples = 264
# total samples = 1341

Here we show how 1176 train samples and 288 test samples are got in SpikingJelly.

The origin dataset is split to train and test set by trials_to_train.txt and trials_to_test.txt.

trials_to_train.txt:

    user01_fluorescent.aedat
    user01_fluorescent_led.aedat
    ...
    user23_lab.aedat
    user23_led.aedat

trials_to_test.txt:

    user24_fluorescent.aedat
    user24_fluorescent_led.aedat
    ...
    user29_led.aedat
    user29_natural.aedat

SpikingJelly will read the txt file and get the aedat file name like user01_fluorescent.aedat. The corresponding label file name will be regarded as user01_fluorescent_labels.csv.

user01_fluorescent_labels.csv:

    class       startTime_usec  endTime_usec
    1   80048239        85092709
    2   89431170        95231007
    3   95938861        103200075
    4   114845417       123499505
    5   124344363       131742581
    6   133660637       141880879
    7   142360393       149138239
    8   150717639       157362334
    8   157773346       164029864
    9   165057394       171518239
    10  172843790       179442817
    11  180675853       187389051

Then SpikingJelly will split the aedat to samples by the time range and class in the csv file. In this sample, the first sample user01_fluorescent_0.npz is sliced from the origin events user01_fluorescent.aedat with 80048239 <= t < 85092709 and label=0. user01_fluorescent_0.npz will be saved in root/events_np/train/0.

参数:

  • root (Union[str, Path]) -- 数据集的根路径

  • train (Optional[bool]) -- 是否使用训练集

  • data_type (str) -- "event" 或 "frame"

  • frames_number (Optional[int]) -- 积分帧的数量

  • split_by (Optional[str]) -- "time" 或 "number"

  • duration (Optional[int]) -- 每帧的时间时长

  • custom_integrate_function (Optional[Callable]) -- 用户自定义积分函数

  • custom_integrated_frames_dir_name (Optional[str]) -- 自定义积分帧目录名

  • transform (Optional[Callable]) -- 数据变换

  • target_transform (Optional[Callable]) -- 标签变换

  • root -- Root directory of the dataset

  • train -- Whether to use training set or test set

  • data_type -- "event" or "frame"

  • frames_number -- Number of frames to integrate

  • split_by -- "time" or "number"

  • duration -- Time duration per frame

  • custom_integrate_function -- User-defined integrate function

  • custom_integrated_frames_dir_name -- Custom frames directory name

  • transform -- Transform function

  • target_transform -- Target transform function

返回:

None

返回类型:

None

classmethod get_H_W() → Tuple[源代码]#

API Language: 中文 | English

  • 中文

  • 中文

返回:

(128, 128)

返回类型:

Tuple

  • English

  • English

返回:

(128, 128)

返回类型:

Tuple

classmethod resource_url_md5() → list[源代码]#

API Language: 中文 | English

  • 中文

返回:

DVS128 Gesture 数据集的下载链接与 MD5 校验值列表

返回类型:

list

  • English

返回:

List of download URLs and MD5 checksums for the DVS128 Gesture dataset

返回类型:

list

classmethod downloadable() → bool[源代码]#

API Language: 中文 | English

  • 中文

由于数据集版权限制，DVS128 Gesture 不提供自动下载，用户需手动下载。

返回:

False

返回类型:

bool

  • English

The DVS128 Gesture dataset does not provide automatic download due to copyright restrictions. Users need to download it manually.

返回:

False

返回类型:

bool

classmethod extract_downloaded_files(download_root: Path, extract_root: Path)[源代码]#

API Language: 中文 | English

  • 中文

从 download_root 中的 DvsGesture.tar.gz 提取文件到 extract_root。

参数:

  • download_root (Path) -- 下载文件所在目录

  • extract_root (Path) -- 提取目标目录

返回:

None

返回类型:

None

  • English

Extract DvsGesture.tar.gz from download_root into extract_root.

参数:

  • download_root (Path) -- Directory containing the downloaded files

  • extract_root (Path) -- Directory to extract into

返回:

None

返回类型:

None

classmethod create_raw_from_extracted(extract_root: Path, raw_root: Path)[源代码]#

API Language: 中文 | English

  • 中文

将提取后的 AEDAT 文件按标签和时间戳分割为 .npz 格式样本， 并根据 trials_to_train.txt / trials_to_test.txt 分配到训练集和测试集目录。

参数:

  • extract_root (Path) -- 包含已提取文件的目录

  • raw_root (Path) -- 保存原始数据的目录

返回:

None

返回类型:

None

  • English

Split extracted AEDAT files into .npz samples by label and timestamp, then distribute them into train/test directories according to trials_to_train.txt / trials_to_test.txt.

参数:

  • extract_root (Path) -- Directory containing the extracted files

  • raw_root (Path) -- Directory to save the raw dataset

返回:

None

返回类型:

None