ubuntu 源码编译libtorch
生活随笔
收集整理的這篇文章主要介紹了
ubuntu 源码编译libtorch
小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.
## pytorch/libtorch qq群： 1041467052
有一點兒感悟就是：
一定要去官網(wǎng)找一手資料，百度出來的都是個人根據(jù)官網(wǎng)來的，這個就隨便看看了。
官網(wǎng)：https://github.com/pytorch/pytorch
首先我跑的是pytorch 1.3版本的，conda安裝好的，現(xiàn)在需要源碼編譯。按照官網(wǎng)的流程，需要先安裝依賴包。
我是切換到pytorch1.3的conda虛擬環(huán)境進行的：這里是官網(wǎng)的教程：
其中，git clone --recursive https://github.com/pytorch/pytorch 是下載的最新版本，而我需要1.3的。
下載完成后：
會出來：
v0.1.1 v0.1.10 v0.1.11 v0.1.12 v0.1.2 v0.1.3 v0.1.4 v0.1.5 v0.1.6 v0.1.7 v0.1.8 v0.1.9 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v1.0.0 v1.0.0a0 v1.0.1 v1.0rc0 v1.0rc1 v1.1.0 v1.1.0a0 v1.2.0 v1.2.0a0 v1.3.0 v1.3.0a0 v1.3.1 v1.4.0 v1.4.0a0 v1.4.1 v1.5.0 v1.5.0-rc1 v1.5.0-rc2 v1.5.0-rc3 v1.5.0-rc4 v1.5.0-rc5然后敲：
git checkout v1.3.1然后再：
git submodule sync git submodule update --init --recursiveexport CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"} python setup.py install然后是漫長的等待
好久之后，沒有報錯，然后我不知道我需要的庫生成在哪里，然后往上翻日志：
找到
-- CMAKE_INSTALL_PREFIX : /data_2/everyday/0429/pytorch/torch
恩，安裝到了目錄torch下面了。
然后用測試代碼，cmakelist，可以加載模型。應(yīng)該是可以了。
還有注意就是cuda這些環(huán)境需要配置好。
######################################################################################
@@@@總體代碼@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
source activate snake_cuda10
?conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
conda install -c pytorch magma-cuda100
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch/
git tag
git checkout v1.1.0
git submodule sync
git submodule update --init --recursive
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
以下是20200901更新：
按照這邊的流程今天又編譯了pytorch1.1.0 cuda10的libtorch庫，把流程寫在這里：
首先是anaconda環(huán)境snake_cuda10，這個環(huán)境snake_cuda10是已經(jīng)能夠跑通一個git倉庫的工程，是基于pytorch1.0，cuda10的conda環(huán)境下的：:
命令1：
?source activate snake_cuda10
?conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
這里是按照官網(wǎng)上面的安轉(zhuǎn)依賴，會提示一些網(wǎng)絡(luò)問題：
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/mkl-2020.2-256.tar.bz2>
Elapsed:
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way
簡單的重試就可以。
命令2：
conda install -c pytorch magma-cuda100
git clone --recursive https://github.com/pytorch/pytorch
漫長的等待
cd pytorch/
git tag
###############################################################
會出來：
v0.1.1
v0.1.10
v0.1.11
v0.1.12
v0.1.2
v0.1.3
v0.1.4
v0.1.5
v0.1.6
v0.1.7
v0.1.8
v0.1.9
v0.2.0
v0.3.0
v0.3.1
v0.4.0
v0.4.1
v1.0.0
v1.0.0a0
v1.0.1
v1.0rc0
v1.0rc1
v1.1.0
v1.1.0a0
v1.2.0
v1.2.0a0
v1.3.0
v1.3.0a0
v1.3.1
v1.4.0
v1.4.0a0
v1.4.1
v1.5.0
v1.5.0-rc1
v1.5.0-rc2
v1.5.0-rc3
v1.5.0-rc4
v1.5.0-rc5
v1.5.1
v1.5.1-rc1
v1.6.0
v1.6.0-rc1
v1.6.0-rc2
v1.6.0-rc3
v1.6.0-rc4
v1.6.0-rc5
v1.6.0-rc6
v1.6.0-rc7
###############################################################
git checkout v1.1.0
git submodule sync
git submodule update --init --recursive
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install
漫長的等待+1
2021年09月01日15:59:13更新
https://oldpan.me/archives/pytorch-build-simple-instruction
只安裝libtorch庫:創(chuàng)建build文件夾，在里頭執(zhí)行python ../tools/build_libtorch.py
python的安裝方式并不是單獨利用Cmake進行構(gòu)建的，而是結(jié)合了python端的setuptools搭配cmake進行構(gòu)建，pytorch的項目還是比較龐大的，所以編譯代碼也是老長，我們主要看看編譯過程中的環(huán)境變量即可：
在setup.py里面：
```
# Environment variables you are probably interested in: # # DEBUG # build with -O0 and -g (debug symbols) # # REL_WITH_DEB_INFO # build with optimizations and -g (debug symbols) # # MAX_JOBS # maximum number of compile jobs we should use to compile your code # # NO_CUDA # disables CUDA build # # .... # .... # # Environment variables for feature toggles: # # NO_CUDNN # disables the cuDNN build # # NO_FBGEMM # disables the FBGEMM build # # NO_TEST # disables the test build # # NO_MIOPEN # disables the MIOpen build這些編譯變量根據(jù)我們的需要在執(zhí)行python setup.py install使用，如果你不想編譯CUDA，則NO_CUDA=1 python setup.py install.
執(zhí)行以上語句我們就可以進行編譯了。
```
2021年09月01日16:56:26再次編譯libtorch1.6，cuda10.2
```
git tag
git checkout v1.6.0
git submodule sync
git submodule update --init --recursive --jobs 0最新官網(wǎng)是這句話，但是有問題，還是用之前的
git submodule update --init --recursive
conda install astunparse numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
conda install -c pytorch magma-cuda102
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install
```
- Adding OpenMP CXX_FLAGS: -fopenmp -- Will link against OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/5/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so -- Found CUDA: /usr/local/cuda (found version "10.2") -- Caffe2: CUDA detected: 10.2 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 10.2 -- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so -- Found cuDNN: v? (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) CMake Error at cmake/public/cuda.cmake:170 (message):PyTorch requires cuDNN 7 and above. Call Stack (most recent call first):cmake/Dependencies.cmake:956 (include)CMakeLists.txt:411 (include)-- Configuring incomplete, errors occurred! See also "/data_1/everyday/0901/pytorch1.5_libtorch/build/CMakeFiles/CMakeOutput.log". See also "/data_1/everyday/0901/pytorch1.5_libtorch/build/CMakeFiles/CMakeError.log". Traceback (most recent call last):File "setup.py", line 744, in <module>build_deps()File "setup.py", line 316, in build_depscmake=cmake)File "/data_1/everyday/0901/pytorch1.5_libtorch/tools/build_pytorch_libs.py", line 59, in build_caffe2rerun_cmake)File "/data_1/everyday/0901/pytorch1.5_libtorch/tools/setup_helpers/cmake.py", line 323, in generateself.run(args, env=my_env)File "/data_1/everyday/0901/pytorch1.5_libtorch/tools/setup_helpers/cmake.py", line 141, in runcheck_call(command, cwd=self.build_dir, env=env)File "/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/python3.7/subprocess.py", line 363, in check_callraise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '-GNinja', '-DBUILD_PYTHON=True', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX=/data_1/everyday/0901/pytorch1.5_libtorch/torch', '-DCMAKE_PREFIX_PATH=/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2', '-DJAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/', '-DNUMPY_INCLUDE_DIR=/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/python3.7/site-packages/numpy/core/include', '-DPYTHON_EXECUTABLE=/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/bin/python', '-DPYTHON_INCLUDE_DIR=/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/include/python3.7m', '-DPYTHON_LIBRARY=/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/libpython3.7m.so.1.0', '-DTORCH_BUILD_VERSION=1.5.0a0+3c31d73', '-DUSE_NUMPY=True', '/data_1/everyday/0901/pytorch1.5_libtorch']' returned non-zero exit status 1.一堆報錯，但是從報錯中可以看到好像是cudnn問題，
-- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so ?
-- Found cuDNN: v?? (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
CMake Error at cmake/public/cuda.cmake:170 (message):
? PyTorch requires cuDNN 7 and above.
Call Stack (most recent call first):
? cmake/Dependencies.cmake:956 (include)
? CMakeLists.txt:411 (include)
我cuda10.2， cudnn8.0.
看了cmake/public/cuda.cmake:170。 里面寫法就是獲取cudnn.h剛開始的頭文件里面編寫的版本號，可是我在cudnn.h里面沒有找到，7點多版本是有的。于是是把cudnn8刪掉，下載了cudnn7.6.5可以。
但是后續(xù)又出現(xiàn)錯誤了，一大堆
-- Looking for clock_gettime in rt -- Looking for clock_gettime in rt - found -- Looking for mmap -- Looking for mmap - found -- Looking for shm_open -- Looking for shm_open - found -- Looking for shm_unlink -- Looking for shm_unlink - found -- Looking for malloc_usable_size -- Looking for malloc_usable_size - found -- Performing Test C_HAS_THREAD -- Performing Test C_HAS_THREAD - Success -- Version: 6.2.0 -- Build type: Release -- CXX_STANDARD: 14 -- Performing Test has_std_14_flag -- Performing Test has_std_14_flag - Success -- Performing Test has_std_1y_flag -- Performing Test has_std_1y_flag - Success -- Performing Test SUPPORTS_USER_DEFINED_LITERALS -- Performing Test SUPPORTS_USER_DEFINED_LITERALS - Success -- Performing Test FMT_HAS_VARIANT -- Performing Test FMT_HAS_VARIANT - Failed -- Required features: cxx_variadic_templates -- Looking for strtod_l -- Looking for strtod_l - not found -- GCC 5.4.0: Adding gcc and gcc_s libs to link line -- Performing Test HAS_WERROR_FORMAT -- Performing Test HAS_WERROR_FORMAT - Success -- Looking for backtrace -- Looking for backtrace - found -- backtrace facility detected in default set of libraries -- Found Backtrace: /usr/include -- NUMA paths: -- /usr/include -- /usr/lib/x86_64-linux-gnu/libnuma.so -- Performing Test COMPILER_SUPPORTS_NO_AVX256_SPLIT -- Performing Test COMPILER_SUPPORTS_NO_AVX256_SPLIT - Success -- Using ATen parallel backend: OMP CMake Deprecation Warning at third_party/sleef/CMakeLists.txt:20 (cmake_policy):The OLD behavior for policy CMP0066 will be removed from a future versionof CMake.The cmake-policies(7) manual explains that the OLD behaviors of allpolicies are deprecated and that a policy should be set to OLD only underspecific short-term circumstances. Projects should be ported to the NEWbehavior and not rely on setting a policy to OLD.-- Found OpenSSL: /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/libcrypto.so (found version "1.1.1k") -- Check size of long double -- Check size of long double - done -- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE -- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE - Success -- Performing Test COMPILER_SUPPORTS_FLOAT128 -- Performing Test COMPILER_SUPPORTS_FLOAT128 - Success -- Performing Test COMPILER_SUPPORTS_SSE2 -- Performing Test COMPILER_SUPPORTS_SSE2 - Success -- Performing Test COMPILER_SUPPORTS_SSE4 -- Performing Test COMPILER_SUPPORTS_SSE4 - Success -- Performing Test COMPILER_SUPPORTS_AVX -- Performing Test COMPILER_SUPPORTS_AVX - Success -- Performing Test COMPILER_SUPPORTS_FMA4 -- Performing Test COMPILER_SUPPORTS_FMA4 - Success -- Performing Test COMPILER_SUPPORTS_AVX2 -- Performing Test COMPILER_SUPPORTS_AVX2 - Success -- Performing Test COMPILER_SUPPORTS_AVX512F -- Performing Test COMPILER_SUPPORTS_AVX512F - Failed -- Performing Test COMPILER_SUPPORTS_OPENMP -- Performing Test COMPILER_SUPPORTS_OPENMP - Success -- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES -- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES - Success -- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH -- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH - Success -- Performing Test COMPILER_SUPPORTS_SYS_GETRANDOM -- Performing Test COMPILER_SUPPORTS_SYS_GETRANDOM - Success -- Configuring build for SLEEF-v3.4.0Target system: Linux-4.15.0-142-genericTarget processor: x86_64Host system: Linux-4.15.0-142-genericHost processor: x86_64Detected C compiler: GNU @ /usr/bin/cc -- Using option `-Wall -Wno-unused -Wno-attributes -Wno-unused-result -Wno-psabi -ffp-contract=off -fno-math-errno -fno-trapping-math` to compile libsleef -- Building shared libs : OFF -- MPFR : LIB_MPFR-NOTFOUND -- GMP : LIBGMP-NOTFOUND -- RT : /usr/lib/x86_64-linux-gnu/librt.so -- FFTW3 : LIBFFTW3-NOTFOUND -- OPENSSL : 1.1.1k -- SDE : SDE_COMMAND-NOTFOUND -- RUNNING_ON_TRAVIS : 0 -- COMPILER_SUPPORTS_OPENMP : 1 AT_INSTALL_INCLUDE_DIR include/ATen/core core header install: /data_1/everyday/0901/pytorch1.6_libtorch/build/aten/src/ATen/core/TensorBody.h -- Include NCCL operators -- Excluding FakeLowP operators -- Excluding ideep operators as we are not using ideep -- Excluding image processing operators due to no opencv -- Excluding video processing operators due to no opencv -- Include Observer library -- /usr/bin/c++ /data_1/everyday/0901/pytorch1.6_libtorch/caffe2/../torch/abi-check.cpp -o /data_1/everyday/0901/pytorch1.6_libtorch/build/abi-check -- Determined _GLIBCXX_USE_CXX11_ABI=1 -- MPI_INCLUDE_PATH: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent;/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include;/usr/lib/openmpi/include;/usr/lib/openmpi/include/openmpi -- MPI_LIBRARIES: /usr/lib/openmpi/lib/libmpi_cxx.so;/usr/lib/openmpi/lib/libmpi.so -- MPIEXEC: /usr/bin/mpiexec CMake Deprecation Warning at torch/lib/libshm/CMakeLists.txt:2 (cmake_minimum_required):Compatibility with CMake < 2.8.12 will be removed from a future version ofCMake.Update the VERSION argument <min> value or use a ...<max> suffix to tellCMake that the project does not need compatibility with older versions.CMake Deprecation Warning at torch/lib/libshm/CMakeLists.txt:3 (cmake_policy):Compatibility with CMake < 2.8.12 will be removed from a future version ofCMake.Update the VERSION argument <min> value or use a ...<max> suffix to tellCMake that the project does not need compatibility with older versions.-- Autodetected CUDA architecture(s): 6.1 CMake Warning (dev) at /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):The package name passed to `find_package_handle_standard_args` (OpenMP_C)does not match the name of the calling package (OpenMP). This can lead toproblems in calling code that expects `find_package` result variables(e.g., `_FOUND`) to follow a certain pattern. Call Stack (most recent call first):cmake/Modules/FindOpenMP.cmake:565 (find_package_handle_standard_args)caffe2/CMakeLists.txt:812 (find_package) This warning is for project developers. Use -Wno-dev to suppress it.CMake Warning (dev) at /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)does not match the name of the calling package (OpenMP). This can lead toproblems in calling code that expects `find_package` result variables(e.g., `_FOUND`) to follow a certain pattern. Call Stack (most recent call first):cmake/Modules/FindOpenMP.cmake:565 (find_package_handle_standard_args)caffe2/CMakeLists.txt:812 (find_package) This warning is for project developers. Use -Wno-dev to suppress it.-- pytorch is compiling with OpenMP. OpenMP CXX_FLAGS: -fopenmp. OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/5/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so. -- Caffe2 is compiling with OpenMP. OpenMP CXX_FLAGS: -fopenmp. OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/5/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so. -- Using lib/python3.7/site-packages as python relative installation path CMake Warning at CMakeLists.txt:690 (message):Generated cmake files are only fully tested if one builds with system glog,gflags, and protobuf. Other settings may generate files that are not welltested.-- -- ******** Summary ******** -- General: -- CMake version : 3.19.6 -- CMake command : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/bin/cmake -- System : Linux -- C++ compiler : /usr/bin/c++ -- C++ compiler id : GNU -- C++ compiler version : 5.4.0 -- BLAS : MKL -- CXX flags : -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -- Build type : Release -- Compile definitions : TH_BLAS_MKL;ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;MAGMA_V2;IDEEP_USE_MKL;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -- CMAKE_PREFIX_PATH : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2;/usr/local/cuda -- CMAKE_INSTALL_PREFIX : /data_1/everyday/0901/pytorch1.6_libtorch/torch -- -- TORCH_VERSION : 1.6.0 -- CAFFE2_VERSION : 1.6.0 -- BUILD_CAFFE2_MOBILE : OFF -- USE_STATIC_DISPATCH : OFF -- BUILD_BINARY : OFF -- BUILD_CUSTOM_PROTOBUF : ON -- Link local protobuf : ON -- BUILD_DOCS : OFF -- BUILD_PYTHON : True -- Python version : 3.7.7 -- Python executable : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/bin/python -- Pythonlibs version : 3.7.7 -- Python library : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/libpython3.7m.so.1.0 -- Python includes : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/include/python3.7m -- Python site-packages: lib/python3.7/site-packages -- BUILD_CAFFE2_OPS : ON -- BUILD_SHARED_LIBS : ON -- BUILD_TEST : True -- BUILD_JNI : OFF -- INTERN_BUILD_MOBILE : -- USE_ASAN : OFF -- USE_CUDA : ON -- CUDA static link : OFF -- USE_CUDNN : ON -- CUDA version : 10.2 -- cuDNN version : 7.6.5 -- CUDA root directory : /usr/local/cuda -- CUDA library : /usr/local/cuda/lib64/stubs/libcuda.so -- cudart library : /usr/local/cuda/lib64/libcudart.so -- cublas library : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/libcublas.so -- cufft library : /usr/local/cuda/lib64/libcufft.so -- curand library : /usr/local/cuda/lib64/libcurand.so -- cuDNN library : /usr/local/cuda/lib64/libcudnn.so -- nvrtc : /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/lib/libnvrtc.so -- CUDA include path : /usr/local/cuda/include -- NVCC executable : /usr/local/cuda/bin/nvcc -- NVCC flags : -DONNX_NAMESPACE=onnx_torch;-gencode;arch=compute_61,code=sm_61;-Xcudafe;--diag_suppress=cc_clobber_ignored;-Xcudafe;--diag_suppress=integer_sign_change;-Xcudafe;--diag_suppress=useless_using_declaration;-Xcudafe;--diag_suppress=set_but_not_used;-Xcudafe;--diag_suppress=field_without_dll_interface;-Xcudafe;--diag_suppress=base_class_has_different_dll_interface;-Xcudafe;--diag_suppress=dll_interface_conflict_none_assumed;-Xcudafe;--diag_suppress=dll_interface_conflict_dllexport_assumed;-Xcudafe;--diag_suppress=implicit_return_from_non_void_function;-Xcudafe;--diag_suppress=unsigned_compare_with_zero;-Xcudafe;--diag_suppress=declared_but_not_referenced;-Xcudafe;--diag_suppress=bad_friend_decl;-std=c++14;-Xcompiler;-fPIC;--expt-relaxed-constexpr;--expt-extended-lambda;-Wno-deprecated-gpu-targets;--expt-extended-lambda;-gencode;arch=compute_61,code=sm_61;-Xcompiler;-fPIC;-DCUDA_HAS_FP16=1;-D__CUDA_NO_HALF_OPERATORS__;-D__CUDA_NO_HALF_CONVERSIONS__;-D__CUDA_NO_HALF2_OPERATORS__ -- CUDA host compiler : /usr/bin/cc -- NVCC --device-c : OFF -- USE_TENSORRT : OFF -- USE_ROCM : OFF -- USE_EIGEN_FOR_BLAS : -- USE_FBGEMM : ON -- USE_FAKELOWP : OFF -- USE_FFMPEG : OFF -- USE_GFLAGS : OFF -- USE_GLOG : OFF -- USE_LEVELDB : OFF -- USE_LITE_PROTO : OFF -- USE_LMDB : OFF -- USE_METAL : OFF -- USE_MKL : ON -- USE_MKLDNN : ON -- USE_NCCL : ON -- USE_SYSTEM_NCCL : OFF -- USE_NNPACK : ON -- USE_NUMPY : ON -- USE_OBSERVERS : ON -- USE_OPENCL : OFF -- USE_OPENCV : OFF -- USE_OPENMP : ON -- USE_TBB : OFF -- USE_VULKAN : OFF -- USE_PROF : OFF -- USE_QNNPACK : ON -- USE_PYTORCH_QNNPACK : ON -- USE_REDIS : OFF -- USE_ROCKSDB : OFF -- USE_ZMQ : OFF -- USE_DISTRIBUTED : ON -- USE_MPI : ON -- USE_GLOO : ON -- USE_TENSORPIPE : ON -- Public Dependencies : Threads::Threads;caffe2::mkl -- Private Dependencies : pthreadpool;cpuinfo;qnnpack;pytorch_qnnpack;nnpack;XNNPACK;fbgemm;/usr/lib/x86_64-linux-gnu/libnuma.so;fp16;/usr/lib/openmpi/lib/libmpi_cxx.so;/usr/lib/openmpi/lib/libmpi.so;gloo;tensorpipe;aten_op_header_gen;foxi_loader;rt;fmt::fmt-header-only;gcc_s;gcc;dl -- Configuring done CMake Warning at caffe2/CMakeLists.txt:583 (add_library):Cannot generate a safe runtime search path for target torch_cpu becausefiles in some directories may conflict with libraries in implicitdirectories:runtime library [libgomp.so.1] in /usr/lib/gcc/x86_64-linux-gnu/5 may be hidden by files in:/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/libSome of these libraries may not be found correctly.CMake Warning at cmake/Modules_CUDA_fix/upstream/FindCUDA.cmake:1847 (add_library):Cannot generate a safe runtime search path for targetcaffe2_detectron_ops_gpu because files in some directories may conflictwith libraries in implicit directories:runtime library [libgomp.so.1] in /usr/lib/gcc/x86_64-linux-gnu/5 may be hidden by files in:/data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/libSome of these libraries may not be found correctly. Call Stack (most recent call first):modules/detectron/CMakeLists.txt:13 (CUDA_ADD_LIBRARY)-- Generating done CMake Warning:Manually-specified variables were not used by the project:JAVA_HOME-- Build files have been written to: /data_1/everyday/0901/pytorch1.6_libtorch/build cmake --build . --target install --config Release -- -j 8 [54/4954] Performing build step for 'nccl_external' FAILED: nccl_external-prefix/src/nccl_external-stamp/nccl_external-build nccl/lib/libnccl_static.a cd /data_1/everyday/0901/pytorch1.6_libtorch/third_party/nccl/nccl && env CCACHE_DISABLE=1 SCCACHE_DISABLE=1 make CXX=/usr/bin/c++ CUDA_HOME=/usr/local/cuda NVCC=/usr/local/cuda/bin/nvcc NVCC_GENCODE=-gencode=arch=compute_61,code=sm_61 BUILDDIR=/data_1/everyday/0901/pytorch1.6_libtorch/build/nccl VERBOSE=0 -j && /data_1/Yang/software_install/Anaconda1105/envs/DB_cuda10_2/bin/cmake -E touch /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl_external-prefix/src/nccl_external-stamp/nccl_external-build make -C src build BUILDDIR=/data_1/everyday/0901/pytorch1.6_libtorch/build/nccl make[1]: Entering directory '/data_1/everyday/0901/pytorch1.6_libtorch/third_party/nccl/nccl/src' Grabbing include/nccl_net.h > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/include/nccl_net.h Compiling channel.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/channel.o Compiling bootstrap.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/bootstrap.o Compiling init.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/init.o Compiling transport.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport.o Compiling enqueue.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/enqueue.o Compiling misc/nvmlwrap.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/nvmlwrap.o Compiling misc/group.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/group.o Compiling misc/ibvwrap.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/ibvwrap.o Generating nccl.h.in > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/include/nccl.h Compiling misc/rings.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/rings.o Compiling misc/argcheck.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/argcheck.o Compiling misc/trees.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/trees.o Compiling misc/utils.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/utils.o Compiling misc/topo.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/misc/topo.o Compiling transport/p2p.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport/p2p.o Compiling transport/shm.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport/shm.o Compiling transport/net.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport/net.o Compiling collectives/all_gather.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/collectives/all_gather.o Compiling collectives/all_reduce.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/collectives/all_reduce.o Compiling collectives/broadcast.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/collectives/broadcast.o Compiling collectives/reduce.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/collectives/reduce.o Compiling transport/net_ib.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport/net_ib.o Compiling collectives/reduce_scatter.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/collectives/reduce_scatter.o Generating nccl.pc.in > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/lib/pkgconfig/nccl.pc Compiling transport/net_socket.cc > /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/obj/transport/net_socket.o make[2]: Entering directory '/data_1/everyday/0901/pytorch1.6_libtorch/third_party/nccl/nccl/src/collectives/device' In file included from include/group.h:10:0,from misc/group.cc:7: /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/include/nccl.h:7:0: error: unterminated #ifndef#ifndef NCCL_H_^ In file included from include/group.h:10:0,from misc/group.cc:7: /data_1/everyday/0901/pytorch1.6_libtorch/build/nccl/include/nccl.h:202:5: error: ‘s’ has not been declaredsize_t recvcount, ncclDataType_t datatype, ncclRedOp_t op, ncclComm_t comm,^ In file included from /usr/include/x86_64-linux-gnu/bits/byteswap.h:27:0,from /usr/include/endian.h:60,from /usr/include/pthread.h:22,from include/core.h:10,from include/group.h:11,from misc/group.cc:7: /usr/include/x86_64-linux-gnu/bits/types.h:30:23: error: two or more data types in declaration of ‘__u_char’typedef unsigned char __u_char;^ /usr/include/x86_64-linux-gnu/bits/types.h:30:31: error: expected ‘)’ before ‘;’ tokentypedef unsigned char __u_char;^ In file included from /usr/include/c++/5/bits/stl_algobase.h:61:0,from /usr/include/c++/5/algorithm:61,from include/core.h:11,from include/group.h:11,from misc/group.cc:7: /usr/include/c++/5/bits/cpp_type_traits.h:72:3: error: template with C linkagetemplate<typename _Iterator, typename _Container>^ /usr/include/c++/5/bits/cpp_type_traits.h:85:3: error: template with C linkagetemplate<bool>^ /usr/include/c++/5/bits/cpp_type_traits.h:89:3: error: template specialization with C linkagetemplate<>^ /usr/include/c++/5/bits/cpp_type_traits.h:95:3: error: template with C linkagetemplate<class _Sp, class _Tp>^ /usr/include/c++/5/bits/cpp_type_traits.h:103:3: error: template with C linkagetemplate<typename, typename>^ /usr/include/c++/5/bits/cpp_type_traits.h:110:3: error: template with C linkagetemplate<typename _Tp>^ /usr/include/c++/5/bits/cpp_type_traits.h:118:3: error: template with C linkagetemplate<typename _Tp>^ /usr/include/c++/5/bits/cpp_type_traits.h:125:3: error: template specialization with C linkagetemplate<>^ /usr/include/c++/5/bits/cpp_type_traits.h:135:3: error: template with C linkagetemplate<typename _Tp>^ /usr/include/c++/5/bits/cpp_type_traits.h:146:3: error: template specialization with C linkagetemplate<>^ /usr/include/c++/5/bits/cpp_type_traits.h:153:3: error: template specialization with C linkagetemplate<>^ /usr/include/c++/5/bits/cpp_type_traits.h:160:3: error: template specialization with C linkagetemplate<>好像是nccl的，天啊！無解。
然后百度其他的，發(fā)現(xiàn)可以直接編譯libtorch。
只安裝libtorch庫:創(chuàng)建build文件夾，在里頭執(zhí)行python ../tools/build_libtorch.py
然后執(zhí)行這個命令沒有報錯。成功編譯出庫出來了。
總結(jié)
以上是生活随笔為你收集整理的ubuntu 源码编译libtorch的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： TreeSize工具介绍
- 下一篇： ubuntu 下耳机插上后，扬声器仍有声