代碼地址如下：
http://www.demodashi.com/demo/11986.html

前些時間做了關(guān)于Apache Nifi分布式集群的搭建分享，但很多時候要搭建分布式集群機器資源是個問題，而現(xiàn)在的單機的配置還是相當(dāng)不錯的，故現(xiàn)在就做個關(guān)于Windows上搭建個偽分布式集群的分享，同時通過另外一種方式實現(xiàn)Apache Nifi的授權(quán)認證。

系統(tǒng)環(huán)境及軟件版本

• Windows8.1

• JDK1.8.0_131

• Nifi-1.4.0

Nifi安裝目錄WEB端口

xxx\nifi-ncm	9443
xxx\nifi-cluster01	9444
xxx\nifi-cluster02	9445

(其它版本可參考此篇文章)
另在測試中發(fā)個問題，使用Apache Nifi內(nèi)嵌的Zookeeper搭建偽集群里啟動總是提示端口占用的問題，故放棄只采用了單結(jié)點啟動。

Nifi的服務(wù)證書

生成本地Nifi服務(wù)證書

解壓nifi-toolkit-1.4.0-bin.tar.gz文件后，通過CMD進入bin目錄，執(zhí)行以下的命令：

D:\DevelopTools\nifi-toolkit-1.4.0\bin>tls-toolkit.bat standalone -n "localhost( 3)" -C "CN=Admin, OU=ApacheNIFI" -o "..\target" 2017/10/26 18:21:32 INFO [main] org.apache.nifi.toolkit.tls.standalone.TlsToolki tStandaloneCommandLine: No nifiPropertiesFile specified, using embedded one. 2017/10/26 18:21:32 INFO [main] org.apache.nifi.toolkit.tls.standalone.TlsToolki tStandalone: Running standalone certificate generation with output directory ..\ target ****************************************************************************** 2017/10/26 18:21:34 INFO [main] org.apache.nifi.toolkit.tls.standalone.TlsToolki tStandalone: Successfully generated client certificate ..\target\CN=Admin_OU=Apa cheNIFI.p12 2017/10/26 18:21:34 INFO [main] org.apache.nifi.toolkit.tls.standalone.TlsToolki tStandalone: tls-toolkit standalone completed successfully

生成后的目錄結(jié)構(gòu)如下：

Folder PATH listing for volume senhui.li Volume serial number is 000000F0 FA46:A0EB D:. │ CN=Admin_OU=ApacheNIFI.p12 │ CN=Admin_OU=ApacheNIFI.password │ nifi-cert.pem │ nifi-key.key │ ├─localhost │ keystore.jks │ nifi.properties │ truststore.jks │ ├─localhost_2 │ keystore.jks │ nifi.properties │ truststore.jks │ └─localhost_3keystore.jksnifi.propertiestruststore.jks

特意注意： -C "CN=Admin, OU=ApacheNIFI" 中間的空格必須保留

拷貝Nifi服務(wù)證書

• 將localhost目錄下的文件拷貝到nifi-ncm目錄下替換所有的文件
• 將localhost_2目錄下的文件拷貝到nifi-cluster01目錄下替換所有的文件
• 將localhost_3目錄下的文件拷貝到nifi-cluster02目錄下替換所有的文件
• 將CN=Admin_OU=ApacheNIFI.p12和CN=Admin_OU=ApacheNIFI.password拷貝到桌面?zhèn)溆?#xff0c;后續(xù)登錄需要使用

配置單點Zookeeper相關(guān)

創(chuàng)建目錄及id

進入nifi-ncm的目錄，創(chuàng)建woker目錄，并把server id寫到文件中，命令如下：

D:\DevelopTools\nifi-ncm>mkdir -p state\zookeeper D:\DevelopTools\nifi-ncm>echo -n '1' > state/zookeeper/myid

更新ZK配置

進入nifi-ncm的conf目錄，打開zookeeper.properties文件，內(nèi)容更新參考如下：

clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30# 只需要配置端口服務(wù) server.1=localhost:2181

更新Nifi配置

進入nifi-ncm的conf目錄，打開nifi.properties文件，更新如下的配置屬性：

nifi.state.management.embedded.zookeeper.start=true# zookeeper properties, used for cluster management # # 另外兩個節(jié)點，只要編輯此字段即可 nifi.zookeeper.connect.string=localhost:2181

更新State配置

進入nifi-ncm的conf目錄，打開state-management.xml文件，更新zookeeper配置，如下：

<cluster-provider><id>zk-provider</id><class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class><property name="Connect String">localhost:2181</property><property name="Root Node">/nifi</property><property name="Session Timeout">10 seconds</property><property name="Access Control">Open</property> </cluster-provider>

然后把此文件拷貝到nifi-cluster01和nifi-cluster02相同的目錄下

配置Nifi Admin

添加Admin用戶

進入nifi-ncm的conf目錄，打開authorizers.xml文件，找到file-provider添加如下配置：

<authorizer><identifier>file-provider</identifier><class>org.apache.nifi.authorization.FileAuthorizer</class><property name="Authorizations File">./conf/authorizations.xml</property><property name="Users File">./conf/users.xml</property><property name="Initial Admin Identity">CN=Admin, OU=ApacheNifi</property><property name="Legacy Authorized Users File"></property><property name="Node Identity 1">CN=localhost, OU=NIFI</property><property name="Node Identity 2">CN=localhost_2, OU=NIFI</property><property name="Node Identity 3">CN=localhost_3, OU=NIFI</property> </authorizer>

然后把此文件同時拷貝到別外兩個節(jié)點目錄。

注： 在Node Identity x中的OU要寫成NIFI，嘗試過用別的名稱好像不成功，具體的原因未知，感興趣的可以自行探究一二。

安裝證書

打開谷歌瀏覽器，在設(shè)置中找到安全選項中找到管理證書，點擊Import開始導(dǎo)入上面生成的證書：CN=Admin_OU=ApacheNIFI.p12，密碼在后綴名為.password的文件中，如下圖所示：

啟動Nifi服務(wù)

進入到Nifi安裝目錄，然后在bin目錄中找到run-nifi.bat文件并雙擊運行，注意啟動的順序： nifi-ncm-->nifi-cluster01/2，等待片刻后（可能會有點久，需要一個選舉的過程）打開瀏覽器輸入https://localhost:9443/nifi，選擇剛剛導(dǎo)入的證書，如看到下面的畫面表示啟動成功：

用戶策略

剛登錄NIFI頁面時，你會發(fā)現(xiàn)圖標(biāo)都是灰色的，需要賦予相應(yīng)的權(quán)限才可以開始編輯權(quán)限才可以開始編輯。點擊頁面左側(cè)面板上的鑰匙圖標(biāo)，會彈出訪問策略的窗口，如下圖所示：

在此會看到用戶列表為空，那么就要給相應(yīng)的行為添加用戶，點擊Create鏈接即可開始添加，如下圖所示：

待所有的權(quán)限添加完成后，便可看到NIFI頁面的按鈕已經(jīng)點亮，可以開始創(chuàng)建流程。

示例演示

模板上傳

下載WordCountDemo.zip壓縮包，解壓出來有個WordCountDemo.xml文件。然后打開瀏覽器輸入NIFI訪問地址： https://localhost:9443/nifi/，點擊左側(cè)面板中的上傳按鈕上傳模板，如下圖所示：

模板代碼

Nifi所有的流程可以導(dǎo)出為模板保存，輸出的模板為XML文檔，關(guān)鍵部分都加了注釋說明，如下所示：

<?xml version="1.0" ?> <template encoding-version="1.1"><description>Local word count demo</description><groupId>326f6452-015f-1000-99be-1d670a0ae923</groupId><!-- Flow組的名稱 ---><name>WordCountDemo</name><snippet><processGroups><id>cf1fdc67-56e0-3629-0000-000000000000</id><parentGroupId>d6c1b1d9-24fa-3e40-0000-000000000000</parentGroupId><position><x>0.0</x><y>0.0</y></position><comments>Local Word Count Demo</comments><contents><connections><id>adf2a3c8-b97d-38b4-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>cf1fdc67-56e0-3629-0000-000000000000</groupId><id>be4176ae-781f-3eef-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>cf1fdc67-56e0-3629-0000-000000000000</groupId><id>5d756769-97bb-3dde-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>2870ac2c-9995-371f-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>cf1fdc67-56e0-3629-0000-000000000000</groupId><id>f257102e-8389-3fc4-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>cf1fdc67-56e0-3629-0000-000000000000</groupId><id>be4176ae-781f-3eef-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><labels><id>de134a82-8649-373d-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><position><x>808.7726989746093</x><y>39.81819076538085</y></position><height>426.0</height><label>WordCountDemo</label><style><entry><key>background-color</key><value>#36a377</value></entry><entry><key>font-size</key><value>12px</value></entry></style><width>938.0</width></labels><processors><id>be4176ae-781f-3eef-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><position><x>1165.7726989746093</x><y>294.8181945800781</y></position><bundle><artifact>nifi-scripting-nar</artifact><group>org.apache.nifi</group><version>1.4.0</version></bundle><config><bulletinLevel>INFO</bulletinLevel><comments></comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><descriptors><entry><key>Script Engine</key><value><name>Script Engine</name></value></entry><entry><key>Script File</key><value><name>Script File</name></value></entry><entry><key>Script Body</key><value><name>Script Body</name></value></entry><entry><key>Module Directory</key><value><name>Module Directory</name></value></entry></descriptors><executionNode>ALL</executionNode><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><key>Script Engine</key><value>Groovy</value></entry><entry><key>Script File</key></entry><entry><key>Script Body</key><!-- 計算WORD的代碼 --><value>import org.apache.commons.io.IOUtils import java.nio.charset.* def flowFile = session.get() if(!flowFile) return flowFile = session.write(flowFile, {inputStream, outputStream -&gt;def wordCount = [:]def tellTaleHeart = IOUtils.toString(inputStream, StandardCharsets.UTF_8)def words = tellTaleHeart.split(/(!|\?|-|\.|\"|:|;|,|\s)+/)*.toLowerCase()words.each { word -&gt;def currentWordCount = wordCount.get(word)if(!currentWordCount) {wordCount.put(word, 1)}else {wordCount.put(word, currentWordCount + 1)}}def outputMapString = wordCount.inject("", {k,v -&gt; k += "${v.key}: ${v.value}\n"})outputStream.write(outputMapString.getBytes(StandardCharsets.UTF_8)) } as StreamCallback) flowFile = session.putAttribute(flowFile, 'filename', 'telltale_heart_wordcount') session.transfer(flowFile, REL_SUCCESS)</value></entry><entry><key>Module Directory</key></entry></properties><runDurationMillis>0</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>ExecuteScript</name><relationships><autoTerminate>true</autoTerminate><name>failure</name></relationships><relationships><autoTerminate>false</autoTerminate><name>success</name></relationships><state>STOPPED</state><style></style><!-- 指定執(zhí)行器的類型 --> <type>org.apache.nifi.processors.script.ExecuteScript</type></processors><processors><id>f257102e-8389-3fc4-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><position><x>1354.7726989746093</x><y>75.81820983886718</y></position><bundle><artifact>nifi-standard-nar</artifact><group>org.apache.nifi</group><version>1.4.0</version></bundle><config><bulletinLevel>WARN</bulletinLevel><comments></comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><descriptors><entry><key>Directory</key><value><name>Directory</name></value></entry><entry><key>Conflict Resolution Strategy</key><value><name>Conflict Resolution Strategy</name></value></entry><entry><key>Create Missing Directories</key><value><name>Create Missing Directories</name></value></entry><entry><key>Maximum File Count</key><value><name>Maximum File Count</name></value></entry><entry><key>Last Modified Time</key><value><name>Last Modified Time</name></value></entry><entry><key>Permissions</key><value><name>Permissions</name></value></entry><entry><key>Owner</key><value><name>Owner</name></value></entry><entry><key>Group</key><value><name>Group</name></value></entry></descriptors><executionNode>ALL</executionNode><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><!-- 結(jié)果保存目錄 --><key>Directory</key><value>/data/tmp/</value></entry><entry><key>Conflict Resolution Strategy</key><value>fail</value></entry><entry><key>Create Missing Directories</key><value>true</value></entry><entry><key>Maximum File Count</key></entry><entry><key>Last Modified Time</key></entry><entry><key>Permissions</key></entry><entry><key>Owner</key></entry><entry><key>Group</key></entry></properties><runDurationMillis>0</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>PutFile</name><relationships><autoTerminate>true</autoTerminate><name>failure</name></relationships><relationships><autoTerminate>true</autoTerminate><name>success</name></relationships><state>STOPPED</state><style></style><type>org.apache.nifi.processors.standard.PutFile</type></processors><processors><id>5d756769-97bb-3dde-0000-000000000000</id><parentGroupId>cf1fdc67-56e0-3629-0000-000000000000</parentGroupId><position><x>834.7726989746093</x><y>81.31820983886718</y></position><bundle><artifact>nifi-standard-nar</artifact><group>org.apache.nifi</group><version>1.4.0</version></bundle><config><bulletinLevel>WARN</bulletinLevel><comments>Generate File Source</comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><descriptors><entry><key>File Size</key><value><name>File Size</name></value></entry><entry><key>Batch Size</key><value><name>Batch Size</name></value></entry><entry><key>Data Format</key><value><name>Data Format</name></value></entry><entry><key>Unique FlowFiles</key><value><name>Unique FlowFiles</name></value></entry><entry><key>generate-ff-custom-text</key><value><name>generate-ff-custom-text</name></value></entry><entry><key>character-set</key><value><name>character-set</name></value></entry></descriptors><executionNode>ALL</executionNode><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><key>File Size</key><value>0B</value></entry><entry><key>Batch Size</key><value>1</value></entry><entry><key>Data Format</key><value>Text</value></entry><entry><key>Unique FlowFiles</key><value>false</value></entry><entry><!-- 需要統(tǒng)計的文本內(nèi)容 --><key>generate-ff-custom-text</key><value>Put simply NiFi was built to automate the flow of data between systems. While the term dataflow is used in a variety of contexts, we use it here to mean the automated and managed flow of information between systems. This problem space has been around ever since enterprises had more than one system, where some of the systems created data and some of the systems consumed data. The problems and solution patterns that emerged have been discussed and articulated extensively. A comprehensive and readily consumed form is found in the Enterprise Integration Patterns [eip].Some of the high-level challenges of dataflow include:Systems fail Networks fail, disks fail, software crashes, people make mistakes.Data access exceeds capacity to consume Sometimes a given data source can outpace some part of the processing or delivery chain - it only takes one weak-link to have an issue.Boundary conditions are mere suggestions You will invariably get data that is too big, too small, too fast, too slow, corrupt, wrong, or in the wrong format.What is noise one day becomes signal the next Priorities of an organization change - rapidly. Enabling new flows and changing existing ones must be fast.</value></entry><entry><key>character-set</key><value>UTF-8</value></entry></properties><runDurationMillis>2000</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>GenerateFlowFile</name><relationships><autoTerminate>false</autoTerminate><name>success</name></relationships><state>STOPPED</state><style><entry><key>background-color</key><value>#0945eb</value></entry></style><type>org.apache.nifi.processors.standard.GenerateFlowFile</type></processors></contents><name>WordCountDemo</name></processGroups></snippet><timestamp>10/24/2017 10:46:13 CST</timestamp> </template>

創(chuàng)建流程

拖動NIFI頁面頂部的模板按鈕到畫板空白處，點擊ADD按鈕即可，然后雙擊打開WordCountDemo組找到PutFile組件，修改目錄地址為你機器的實際可訪問路徑，如下圖所示：

啟動流程

點擊NIFI頁面左下角的NiFi Flow鏈接返回到主面板，點擊WordCountDemo組，然后點擊左側(cè)面板中的開始按鈕啟動流程，如下圖所示：

如無異常那么此時你可在目錄下找到名為telltale_heart_wordcount的文件，打開便可看到如下圖的統(tǒng)計內(nèi)容：

至此在本地搭建NIFI偽集群就完成了，有問題歡迎留言。Apache Nifi在Windows環(huán)境下搭建偽群集及證書登錄

代碼地址如下：
http://www.demodashi.com/demo/11986.html

注：本文著作權(quán)歸作者，由demo大師代發(fā)，拒絕轉(zhuǎn)載，轉(zhuǎn)載需要作者授權(quán)

總結(jié)

以上是生活随笔為你收集整理的Apache Nifi在Windows环境下搭建伪群集及证书登录的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。