代碼根據key=TransactionID來進行join操作

go.py

import datatable as dtfolder_path = './' train_identity = dt.fread(f'{folder_path}train_identity.csv') test_identity = dt.fread(f'{folder_path}test_identity.csv') train_transaction = dt.fread(f'{folder_path}train_transaction.csv') test_transaction = dt.fread(f'{folder_path}test_transaction.csv')train_identity.key = 'TransactionID' test_identity.key = 'TransactionID' train = train_transaction[:, :, dt.join(train_identity)] test = test_transaction[:, :, dt.join(test_identity)]train.to_csv("train.csv") test.to_csv("test.csv")

run method:

time python go.py

?

(env) arno@mr-dli1:~/kaggle/ieee/scripts(master)$ time python go.py 100% |██████████████████████████████████████████████████| Writing CSV 100% |██████████████████████████████████████████████████| Writing CSVreal 0m2.058s user 0m26.883s sys 0m2.221s(env) arno@mr-dli1:~/kaggle/ieee/scripts(master)$ ls -ltrah *csv -rw-r--r-- 1 arno arno 744M Jul 17 14:03 train.csv -rw-r--r-- 1 arno arno 664M Jul 17 14:03 test.csv

性能為50w數據的csv和14w數據的csv,耗時2.221s

data:

https://www.kaggle.com/c/ieee-fraud-detection/data

?

datatable的官方文檔

https://datatable.readthedocs.io/en/latest/using-datatable.html

?

?

總結

以上是生活随笔為你收集整理的datatable对两个csv的join操作的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。