基于視頻理解TSM-mobilenetv2和數(shù)據(jù)集20bn-jester-v1的27類手勢識別

基于視頻理解TSM-resnet50和數(shù)據(jù)集20bn-jester-v1的27類手勢識別

基于視頻理解TSM-resnet101和數(shù)據(jù)集20bn-jester-v1的27類手勢識別

論文下載：TSM: Temporal Shift Module for Efficient Video Understanding

# TSM: Temporal Shift Module for Efficient Video Understanding [[Website]](https://hanlab.mit.edu/projects/tsm/) [[arXiv]](https://arxiv.org/abs/1811.08383)[[Demo]](https://www.youtube.com/watch?v=0T6u7S_gq-4)
一鍵運行TSM，修改后代碼下載：下載鏈接

20bn-jester-v1數(shù)據(jù)集百度網(wǎng)盤下載：下載鏈接

識別demo:

手勢類型：Doing other things，Drumming Fingers，No gesture，Pulling Hand In，Pulling Two Fingers In，Pushing Hand Away，Pushing Two Fingers Away，Rolling Hand Backward
Rolling Hand Forward，Shaking Hand，Sliding Two Fingers Down，Sliding Two Fingers Left，Sliding Two Fingers Right，Sliding Two Fingers Up，Stop Sign，Swiping Down
Swiping Left，Swiping Right，Swiping Up，Thumb Down，Thumb Up，Turning Hand Clockwise，Turning Hand Counterclockwise，Zooming In With Full Hand，Zooming In With Two Fingers
Zooming Out With Full Hand，Zooming Out With Two Fingers。

20bn-jester-v1數(shù)據(jù)集解壓：生成可訓練的視頻幀。

cat 20bn-jester-v1-?? | tar zx

20bn-jester-v1數(shù)據(jù)集抽取：生成category.txt、train_videofolder.txt和val_videofolder.txt

import os
data_dir = "/Users/xxx/Data/20bn-jester-v1/20bn-jester-v1/"
if __name__ == '__main__':dataset_name = '/Users/xxx/Data/20bn-jester-v1/jester-v1'with open('%s-labels.csv' % dataset_name) as f:lines = f.readlines()categories = []for line in lines:line = line.rstrip()categories.append(line)categories = sorted(categories)with open('20bn-jester-v1/category.txt', 'w') as f:f.write('\n'.join(categories))dict_categories = {}for i, category in enumerate(categories):dict_categories[category] = ifiles_input = ['%s-validation.csv' % dataset_name, '%s-train.csv' % dataset_name]files_output = ['20bn-jester-v1/val_videofolder.txt', '20bn-jester-v1/train_videofolder.txt']for (filename_input, filename_output) in zip(files_input, files_output):with open(filename_input) as f:lines = f.readlines()folders = []idx_categories = []for line in lines:line = line.rstrip()items = line.split(';')folders.append(items[0])idx_categories.append(dict_categories[items[1]])output = []for i in range(len(folders)):curFolder = folders[i]curIDX = idx_categories[i]# counting the number of frames in each video foldersdir_files = os.listdir(os.path.join(data_dir, curFolder))output.append('%s %d %d' % (data_dir + curFolder, len(dir_files), curIDX))print('%d/%d' % (i, len(folders)))with open(filename_output, 'w') as f:f.write('\n'.join(output))

運行修改后的TSM源碼：修改數(shù)據(jù)集配置選項ops/dataset_config.py和基于mobilenet-v2中運行中出現(xiàn)的bug。

  python main.py jester RGB \--arch mobilenetv2 --num_segments 8 \--gd 20 --lr 0.02 --wd 1e-4 --lr_steps 20 40 --epochs 1 \--batch-size 32 -j 16 --dropout 0.5 --consensus_type=avg --eval-freq=1 \--shift --shift_div=8 --shift_place=blockres --npb

?

?

總結(jié)

以上是生活随笔為你收集整理的基于视频理解TSM和数据集20bn-jester-v1的27类手势识别的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。