CentOS7 Ambari2.7.4编译
?
?
目錄
?
?
1.相關連接
2.編譯
2.1.編譯環境安裝
2.2.下載源碼
2.3.鏡像加速
2.4.大包提前下載
2.5.各種錯誤處理
2.6.編譯成功
?
1.相關連接
編譯官方說明
https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
?
部署官方說明
https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.5
?
git地址：https://github.com/apache/ambari
?
2.編譯
2.1.編譯環境安裝
yum install -y java-1.8.0-openjdk-develyum install -y gcc-c++ rpm-build git下載maven新版3.6.3（筆者實驗時yum直接安裝的maven是3.0.x，會提示需要3.1.0以上）
http://mirror.bit.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
curl http://mirror.bit.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz -o apache-maven-3.6.3-bin.tar.gztar xzvf apache-maven-3.6.3-bin.tar.gz?
?
?
#pip安裝（官方中說安裝setuptools即可，pip比較好裝，包含了setuptools）
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.pypython get-pip.py?
nodejs環境安裝
yum?install?epel-releaseyum?install?nodejs -y升級nodejs
sudo npm install n -gsudo n stable?
vi ~/.bash_profile添加
export PATH=/usr/local/bin/:$PATH:/root/apache-maven-3.6.3/binexport JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/export JRE_HOME=${JAVA_HOME}/jreexport CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/libexport PATH=${JAVA_HOME}/bin:$PATH?
再執行
source ~/.bash_profile?
?
安裝yarn等
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.reposudo yum install yarnnpm install -g bowernpm install -g gulp?
2.2.下載源碼
到github下載
https://github.com/apache/ambari/archive/release-2.7.4.tar.gz
下載后解壓
tar xfvz apache-ambari-2.7.4-src.tar.gz cd apache-ambari-2.7.4-srcmvn versions:set -DnewVersion=2.7.4.0.0pushd ambari-metricsmvn versions:set -DnewVersion=2.7.4.0.0popdmvn -B clean install rpm:rpm -DnewVersion=2.7.4.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" apache-rat:check -Drat.numUnapprovedLicenses=600?
注
編譯過程中失敗，繼續從該項目編譯即可（mvn命令后添加參數如下，后面的ambari-metrics-host-monitoring就是提示的項目名）
mvn …… -rf :ambari-metrics-host-monitoring
?
2.3.鏡像加速
覺得下載依賴包特別慢的，可以添加阿里云鏡像
注意：后面下載storm源的時候會報找不到，此時把public的
<mirrorOf>*</mirrorOf>
改成
<mirrorOf>public</mirrorOf>
vi ${maven安裝目錄}/conf/settings.xml
<mirrors><mirror><id>ali apache snapshots</id><url>https://maven.aliyun.com/repository/apache-snapshots</url><mirrorOf>apache snapshots</mirrorOf> </mirror><mirror><id>ali central</id><url>https://maven.aliyun.com/repository/central</url><mirrorOf>central</mirrorOf> </mirror><mirror><id>ali google</id><url>https://maven.aliyun.com/repository/google</url><mirrorOf>google</mirrorOf> </mirror><mirror><id>ali gradle-plugin</id><url>https://maven.aliyun.com/repository/gradle-plugin</url><mirrorOf>gradle-plugin</mirrorOf> </mirror><mirror><id>ali jcenter</id><url>https://maven.aliyun.com/repository/jcenter</url><mirrorOf>jcenter</mirrorOf> </mirror><mirror><id>ali spring</id><url>https://maven.aliyun.com/repository/spring</url><mirrorOf>spring</mirrorOf> </mirror><mirror><id>ali spring-plugin</id><url>https://maven.aliyun.com/repository/spring-plugin</url><mirrorOf>spring-plugin</mirrorOf> </mirror><mirror><id>ali public</id><url>https://maven.aliyun.com/repository/public</url><mirrorOf>*</mirrorOf> </mirror><mirror><id>ali releases</id><url>https://maven.aliyun.com/repository/releases</url><mirrorOf>releases</mirrorOf> </mirror><mirror><id>ali snapshots</id><url>https://maven.aliyun.com/repository/snapshots</url><mirrorOf>snapshots</mirrorOf> </mirror><mirror><id>ali grails-core</id><url>https://maven.aliyun.com/repository/grails-core</url><mirrorOf>grails-core</mirrorOf> </mirror><mirror><id>ali mapr-public</id><url>https://maven.aliyun.com/repository/mapr-public</url><mirrorOf>mapr-public</mirrorOf> </mirror></mirrors>?
2.4.大包提前下載
hadoop等包
下載很慢，可以提前下載
vi ambari-metrics/pom.xml找到
<hbase.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.0-315</hbase.folder>
<hadoop.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hadoop/hadoop-3.1.1.3.1.4.0-315.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.0-315</hadoop.folder>
<grafana.folder>grafana-2.6.0</grafana.folder>
<grafana.tar>https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz</grafana.tar>
<phoenix.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/phoenix/phoenix-5.0.0.3.1.4.0-315.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.0-315</phoenix.folder>
這一段，用下載工具下載下來（PS：迅雷可以直接鏡像加速），在部署到自己本地的web服務器上，再修改這個pom.xml中的url地址，下載使用更快捷
?
2.5.各種錯誤處理
報錯
[INFO] Downloading https://nodejs.org/dist/v4.5.0/node-v4.5.0-linux-x64.tar.gz to /root/.m2/repository/com/github/eirslett/node/4.5.0/node-4.5.0-linux-x64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/4.5.0/node-4.5.0-linux-x64.tar.gz into /root/apache-ambari-2.7.5-src/ambari-web/node/tmp
……
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not extract the Node archive: Could not extract archive: '/root/.m2/repository/com/github/eirslett/node/4.5.0/node-4.5.0-linux-x64.tar.gz': EOFException -> [Help 1]
解決
手動下載
curl -o /root/.m2/repository/com/github/eirslett/node/4.5.0/node-4.5.0-linux-x64.tar.gz https://nodejs.org/dist/v4.5.0/node-v4.5.0-linux-x64.tar.gz
?
報錯
[ERROR]?warning fsevents@0.3.8: The platform "linux" is incompatible with this module.
解決
npm install pm2@latest -gpm2 install pm2-slack（執行后上面的錯誤也還存在，但是可以往下走，這里發現了需要安裝bower，安裝這個即可）
安裝bower的依賴包（個人在ambari-admin/src/main/resources/ui/admin-web/下執行）
bower install --allow-root?
報錯
Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /root/apache-ambari-2.7.5-src/target/rat.txt -> [Help 1]
?
解決（采用方案二）
一種方案是在新建的每一個類的第一行添加
?
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
第二種解決方案：
在mvn命令中添加參數
apache-rat:check -Drat.numUnapprovedLicenses=600數字600是一個大于報錯信息中的數量。
?
報錯
PhantomJS not found on PATH
?
Downloading https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] Downloading https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.7-linux-x86_64.tar.bz2
[ERROR] Saving to /tmp/phantomjs/phantomjs-1.9.7-linux-x86_64.tar.bz2
[ERROR] Receiving...
[ERROR]
[ERROR] Error making request.
[ERROR] Error: read ECONNRESET
[ERROR] at exports._errnoException (util.js:907:11)
[ERROR] at TLSWrap.onread (net.js:557:26)
應該是下載超時了，網上說用cnpm，這個需要改編譯過程，不合適
####不需要執行 npm install -g cnpm --registry=https://registry.npm.taobao.org
下載這個確實太久了，咱手動從淘寶鏡像上下載
wget https://npm.taobao.org/mirrors/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
cp phantomjs-2.1.1-linux-x86_64.tar.bz2 /tmp/phantomjs/
下載完2.1.1又會提示需要1.9.7
wget https://npm.taobao.org/mirrors/phantomjs/phantomjs-1.9.7-linux-x86_64.tar.bz2
cp phantomjs-1.9.7-linux-x86_64.tar.bz2 /tmp/phantomjs/
接下來又報錯/tmp/npm_config_tmp/phantomjs/這個目錄下也要下載phantomjs，一樣的版本，都拷貝過去……
cp phantomjs-* /tmp/npm_config_tmp/phantomjs/
wget https://npm.taobao.org/mirrors/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2 -O /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2
后面還有……
wget https://npm.taobao.org/mirrors/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2 -O /tmp/logsearch_npm_config_tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
?
?
?
報錯
No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?
解決
yum install -y java-1.8.0-openjdk-devel這個是安裝jdk，yum install -y java-1.8.0-openjdk這個是安裝jre，詳見
yum search openjdk
java-1.8.0-openjdk.x86_64 : OpenJDK Runtime Environment 8
java-1.8.0-openjdk-devel.x86_64 : OpenJDK Development Environment 8
?
?
報錯
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (Bower install) on project ambari-admin: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
往上找，發現
[INFO] Running 'npm install --unsafe-perm' in /root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web
……
[INFO] --- exec-maven-plugin:1.2.1:exec (Bower install) @ ambari-admin ---
bower error Unexpected token {
?
Stack trace:
SyntaxError: Unexpected token {
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:373:25)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/bower-registry-client/node_modules/request/lib/cookies.js:3:13)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
?
Console trace:
Trace
at StandardRenderer.error (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/lib/renderers/StandardRenderer.js:72:17)
at Logger.<anonymous> (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/bin/bower:111:22)
at emitOne (events.js:77:13)
at Logger.emit (events.js:169:7)
at Logger.emit (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/bower-logger/lib/Logger.js:29:39)
at /root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/lib/commands/index.js:40:20
at _rejected (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/q/q.js:797:24)
at /root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/q/q.js:823:30
at Promise.when (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/q/q.js:1035:31)
at Promise.promise.promiseDispatch (/root/apache-ambari-2.7.4-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/q/q.js:741:41)
處理
vi ambari-admin/pom.xml<argument>${basedir}/src/main/resources/ui/admin-web/node_modules/bower/bin/bower</argument>
改為
<argument>bower</argument>
?
?
報錯
[exec] psutil/_psutil_linux.c:12:20: 致命錯誤：Python.h：沒有那個文件或目錄
[exec] #include <Python.h>
[exec] ^
[exec] 編譯中斷。
[exec] error: command 'gcc' failed with exit status 1
?
解決
yum install -y python-devel?
報錯
各種下載超時
[INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v1.1.0/yarn-v1.1.0.tar.gz to /root/.m2/repository/com/github/eirslett/yarn/1.1.0/yarn-1.1.0.tar.gz
解決
手動下載
mkdir -p /root/.m2/repository/com/github/eirslett/yarn/1.1.0/wget https://github.com/yarnpkg/yarn/releases/download/v1.1.0/yarn-v1.1.0.tar.gz -O /root/.m2/repository/com/github/eirslett/yarn/1.1.0/yarn-1.1.0.tar.gz?
?
2.6.編譯成功
……[INFO] Ambari Server ...................................... SUCCESS [04:18 min] [INFO] Ambari Functional Tests ............................ SUCCESS [ ?3.061 s] [INFO] Ambari Agent ....................................... SUCCESS [01:05 min] [INFO] ambari-logsearch ................................... SUCCESS [ ?1.431 s] [INFO] Ambari Logsearch Appender .......................... SUCCESS [ ?0.595 s] [INFO] Ambari Logsearch Config Api ........................ SUCCESS [ ?2.027 s] [INFO] Ambari Logsearch Config JSON ....................... SUCCESS [ ?0.253 s] [INFO] Ambari Logsearch Config Solr ....................... SUCCESS [ ?0.449 s] [INFO] Ambari Logsearch Config Zookeeper .................. SUCCESS [ ?0.512 s] [INFO] Ambari Logsearch Config Local ...................... SUCCESS [ ?0.181 s] [INFO] Ambari Logsearch Log Feeder Plugin Api ............. SUCCESS [ ?0.298 s] [INFO] Ambari Logsearch Log Feeder Container Registry ..... SUCCESS [ ?0.462 s] [INFO] Ambari Logsearch Log Feeder ........................ SUCCESS [ ?7.216 s] [INFO] Ambari Logsearch Web ............................... SUCCESS [02:56 min] [INFO] Ambari Logsearch Server ............................ SUCCESS [ 10.176 s] [INFO] Ambari Logsearch Assembly .......................... SUCCESS [ ?0.185 s] [INFO] Ambari Logsearch Integration Test .................. SUCCESS [ ?2.482 s] [INFO] ambari-infra ....................................... SUCCESS [ ?0.221 s] [INFO] Ambari Infra Solr Client ........................... SUCCESS [ 22.077 s] [INFO] Ambari Infra Solr Plugin ........................... SUCCESS [ ?2.573 s] [INFO] Ambari Infra Manager ............................... SUCCESS [ ?7.893 s] [INFO] Ambari Infra Assembly .............................. SUCCESS [ ?0.267 s] [INFO] Ambari Infra Manager Integration Tests ............. SUCCESS [ ?1.871 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: ?09:25 min [INFO] Finished at: 2020-04-10T11:42:14+08:00 [INFO] ------------------------------------------------------------------------
?
成功
編譯好后的rpm包在
cd ambari-server/target/rpm/ambari-server/RPMS/x86_64/
ambari-server-2.7.4.0-0.x86_64.rpm
?
一系列漫長的折磨，斷斷續續兩三天，終于成功了！
總結教訓：
1.用阿里云鏡像，能加速很多
2.了解幾個常用的編譯工具，問題出現在這些點上的時候要想到是不是他沒安裝的問題。
?
?
?
?
?
總結
以上是生活随笔為你收集整理的CentOS7 Ambari2.7.4编译的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： bzoj 3157 bzoj 3516
- 下一篇： Euler：欧拉函数＆素数筛