mlflow

Hyperparameter tuning and optimization is a powerful tool in the field of AutoML. Tuning these configurations can dramatically improve model performance. However, hyperparameter tuning can be computationally expensive and slow.

^ h yperparameter調(diào)整和優(yōu)化是AutoML領(lǐng)域的有力工具。 調(diào)整這些配置可以大大提高模型性能。 但是，超參數(shù)調(diào)整在計(jì)算上可能既昂貴又緩慢。

If you have a large network like VGG, Resnet, etc. trying out every parameter exhaustively and then choosing the best parameter is computationally intensive because in hyperparameter tuning we use various configurations. To do that, we need a principle approach to choose the best parameter.

如果您有一個大型網(wǎng)絡(luò)(如VGG，Resnet等)，則要窮盡地嘗試每個參數(shù)，然后選擇最佳參數(shù)會占用大量計(jì)算資源，因?yàn)樵诔瑓?shù)調(diào)整中，我們使用了各種配置。 為此，我們需要一種原則方法來選擇最佳參數(shù)。

Hyperparameter tuning creates complex workflows involving testing many hyperparameter settings, generating lots of models, and iterating on an ML pipeline. To simplify tracking and reproducibility for tuning workflows, we use MLflow, an open-source platform to help and manage the complete machine learning lifecycle.

超參數(shù) 調(diào)整會創(chuàng)建復(fù)雜的工作流程，其中包括測試許多超參數(shù)設(shè)置，生成大量模型以及在ML管道上進(jìn)行迭代。 為了簡化跟蹤和調(diào)整工作流程的可重復(fù)性，我們使用MLflow (一個開放源代碼平臺)來幫助和管理整個機(jī)器學(xué)習(xí)生命周期。

source資源

How can I find the best version of this model?

如何找到該模型的最佳版本？

How to track all the information and document of each model trained?

如何跟蹤每個訓(xùn)練過的模型的所有信息和文檔？

How do we do that efficiently?

我們?nèi)绾斡行У刈龅竭@一點(diǎn)？

MLflow makes this process much more efficient and convenient!!

MLflow使此過程更加高效和便捷！！

Let’s start

開始吧

什么是MLflow？ (What is MLflow?)

MLfow is a python package developed by databricks that is defined as an open-source platform for the machine learning lifecycle. There are three pillars around mlflow ().

MLfow是由databricks開發(fā)的python軟件包，被定義為機(jī)器學(xué)習(xí)生命周期的開源平臺。 mlflow()周圍有三個Struts。

Source資源

Their documentation has a nice tutorial to explain the component of mlflow. Mlflow lets you log parameters and metrics which is incredibly convenient for model comparison.

他們的文檔中有一個很好的教程來解釋mlflow的組件。 Mlflow允許您記錄參數(shù)和度量，這對于模型比較非常方便。

在本地運(yùn)行時進(jìn)行協(xié)作 (Colab on local runtime)

So before starting to MLflow we will first connect to our colab on a local runtime. You are going to need Jupyter notebooks. It is what Colab built their platform on top of and it is required to run a local notebook. If you have Jupyter, you are already ahead of the game if not, click here for information.

因此，在開始MLflow之前，我們將首先在本地運(yùn)行時連接到colab。 您將需要Jupyter筆記本。 這是Colab在其基礎(chǔ)上構(gòu)建其平臺的基礎(chǔ)，并且它是運(yùn)行本地筆記本電腦所必需的。 如果您擁有Jupyter，那么您已經(jīng)領(lǐng)先于游戲， 請單擊此處獲取信息。

Step 1: install jupyter_http_over_ws using the command ‘pip install jupyter_http_over_ws’

步驟1：使用命令'pip install jupyter_http_over_ws'安裝jupyter_http_over_ws

Step 2: Enable the jupyter_http_over_ws jupyter extension using command jupyter serverextension enable — py jupyter_http_over_ws

步驟2：使用命令jupyter serverextension enable啟用jupyter_http_over_ws jupyter擴(kuò)展— py jupyter_http_over_ws

Step 3 : Start server and authenticate using command jupyter notebook \ — NotebookApp.allow_origin=’https://colab.research.google.com' \ — port=8888 \ — NotebookApp.port_retries=0

步驟3：使用命令jupyter notebook啟動服務(wù)器并進(jìn)行身份驗(yàn)證\-NotebookApp.allow_origin ='https：//colab.research.google.com'\-端口= 8888 \-NotebookApp.port_retries = 0

Once the server has started, it will print a message with the initial backend URL used for authentication. Make a copy of this URL as you’ll need to provide this in the next step.

服務(wù)器啟動后，它將打印一條消息，其中包含用于身份驗(yàn)證的初始后端URL。 復(fù)制此URL，因?yàn)槟枰谙乱徊街刑峁┧?

Jupyter notebook will automatically pop-up in your browser, but if it does not you can look at the highlighted link, that is the local http:// you will need to copy and paste in your browser.

Jupyter Notebook將自動在您的瀏覽器中彈出，但是如果沒有，您可以查看突出顯示的鏈接，即本地http：//，您需要在瀏覽器中進(jìn)行復(fù)制和粘貼。

now the notebook is generated, but the notebook still needs to be connected to computing power. You will be able to do this by going to your colab page. Make sure your colab notebook settings are set to the GPU. Go to Edit > Notebook Settings and make sure you are running the right environment and that the hardware accelerator is set to the GPU.

現(xiàn)在已生成筆記本，但筆記本仍需要連接到計(jì)算能力。 您可以通過轉(zhuǎn)到colab頁面來執(zhí)行此操作。 確保將您的colab筆記本設(shè)置設(shè)置為GPU。 轉(zhuǎn)到“編輯”>“筆記本設(shè)置”，并確保您運(yùn)行的環(huán)境正確，并且硬件加速器已設(shè)置為GPU。

Click the “Connect” button and select “Connect to local runtime…”. The local connection setting window will pop up as shown below.

單擊“連接”按鈕，然后選擇“連接到本地運(yùn)行時…”。 如下所示，將彈出本地連接設(shè)置窗口。

Enter the URL from the previous step in the dialog that appears and click the “Connect” button. After this, you should now be connected to your local runtime.

在出現(xiàn)的對話框中輸入上一步的URL，然后單擊“連接”按鈕。 之后，您現(xiàn)在應(yīng)該連接到本地運(yùn)行時。

Now you have GPU power through your local google colab notebook.

現(xiàn)在，您可以通過本地Google colab筆記本使用GPU了。

MLFlow的安裝和基本用法： (MLFlow installation and basic usage:)

Step 1: pip install mlflow

步驟1：pip安裝mlflow

Step 2: MLflow Python API logs run locally, in a mlruns directory wherever you ran your program. You can then run mlflow ui to see the logged runs.

步驟2：無論您在哪里運(yùn)行程序，MLflow Python API日志都在mlruns目錄中本地運(yùn)行。 然后，您可以運(yùn)行mlflow ui以查看記錄的運(yùn)行。

Enter the URL shown in the above image which provides a simple interface to various functionality in MLflow.

輸入上圖所示的URL，該URL提供了MLflow中各種功能的簡單接口。

超參數(shù)調(diào)整 (Hyperparameter Tuning)

Now we have connected google colab notebook to runtime and also installed mlflow.

現(xiàn)在，我們已將google colab筆記本連接到運(yùn)行時，還安裝了mlflow。

Let’s solve one practical example, Digit recognition where mlflow will help you to find the right set of optimal hyperparameters for a learning algorithm.

讓我們解決一個實(shí)際的例子， 數(shù)字識別 ，其中mlflow將幫助您為學(xué)習(xí)算法找到正確的最佳超參數(shù)集。

We are using the MNIST dataset of 60,000 small square 28×28 pixel grayscale images of handwritten single digits between 0 and 9. The task is to classify a given image of a handwritten digit into one of 10 classes representing integer values from 0 to 9, inclusively. It has three tunable hyperparameters that we try to optimize: learning-rate, momentum, and number of hidden nodes.

我們正在使用MNIST數(shù)據(jù)集，其中包含60,000個手寫數(shù)字在0到9之間的小正方形28×28像素灰度圖像。任務(wù)是將給定的手寫數(shù)字圖像分類為10個類之一，這些類表示從0到9的整數(shù)值。包括在內(nèi)。 它具有我們嘗試優(yōu)化的三個可調(diào)超參數(shù)：學(xué)習(xí)速率，動量和隱藏節(jié)點(diǎn)的數(shù)量。

We have loaded the Data set and transformed it as required for now we have everything in place.

我們已經(jīng)加載了數(shù)據(jù)集，并根據(jù)需要對其進(jìn)行了轉(zhuǎn)換，現(xiàn)在我們已經(jīng)準(zhǔn)備就緒。

MLflow Tracking is organized around the concept of runs (mlruns), Each run records some information like which is explained as below.

MLflow跟蹤是圍繞運(yùn)行(mlruns)的概念組織的，每次運(yùn)行都會記錄一些信息，如下所述。

Create an experiment id using mlflow.create_experiment() creates a new experiment and returns its ID. Runs can be launched under the experiment bypassing the experiment ID to mlflow.start_run() which returns current active run if you have any else it will create new active run and return its object.

使用mlflow創(chuàng)建實(shí)驗(yàn)ID。create_experiment()創(chuàng)建一個新實(shí)驗(yàn)并返回其ID。 可以在實(shí)驗(yàn)下啟動運(yùn)行，而將實(shí)驗(yàn)ID繞過mlflow.start_run() ，如果您有其他運(yùn)行，則返回當(dāng)前的運(yùn)行，它將創(chuàng)建新的運(yùn)行并返回其對象。

We have logged the value of the metric train loss, test loss, and test accuracy using mlflow.log_metric() which logs into a single key-value metric.

我們已經(jīng)使用mlflow.log_metric()記錄了度量標(biāo)準(zhǔn)火車損失，測試損失和測試準(zhǔn)確性的值，該值登錄到單個鍵值度量標(biāo)準(zhǔn)中。

At each run, we have logged parameter like start time, batch size, epochs, Learning rate, momentum, hidden nodes, and test loss along with its source code using mlflow.log_param() which logs into a single key-value param in the currently active run.

在每次運(yùn)行中，我們都使用mlflow.log_param()記錄了諸如開始時間，批處理大小，時期，學(xué)習(xí)率，動量，隱藏節(jié)點(diǎn)和測試損失等參數(shù)及其源代碼，該參數(shù)登錄到了單個鍵值參數(shù)中。當(dāng)前正在運(yùn)行。

select the different models for comparison and click on the compare button.

選擇不同的模型進(jìn)行比較，然后單擊“比較”按鈕。

Select the x-axis and y-axis parameters to compare the selected model and check the performance of the model.

選擇x軸和y軸參數(shù)以比較所選模型并檢查模型的性能。

you can also check the performance of the individual model and check different metrics plots(for example, to track how your model’s loss function is converging).

您還可以檢查單個模型的性能并檢查不同的指標(biāo)圖(例如，跟蹤模型的損失函數(shù)如何收斂)。

you can record images (for example, confusing metric PNGs for each run), models (for example, a pth PyTorch model), or even different data files as artifacts using mlflow.log_artifact().

您可以使用mlflow.log_artifact()將圖像(例如，每次運(yùn)行時混淆的度量PNG)，模型(例如，pth PyTorch模型)或不同的數(shù)據(jù)文件記錄為工件。

So, we logged all run for each hyperparameter setting, and each of those runs includes the hyperparameter setting and the evaluation metric. Comparing these runs in the MLflow UI helps with visualizing the effect of tuning each hyperparameter.

因此，我們記錄了每個超參數(shù)設(shè)置的所有運(yùn)行，并且每個運(yùn)行都包括超參數(shù)設(shè)置和評估指標(biāo)。 在MLflow UI中比較這些運(yùn)行有助于可視化調(diào)整每個超參數(shù)的效果。

Once you come up with a better configuration of hyperparameter you can load that model specifying the run id and use for inference.

一旦提出了更好的超參數(shù)配置，就可以加載指定運(yùn)行ID的模型并用于推理。

To know more about mlflow functions you can visit the official page of mlflow.

要了解有關(guān)mlflow功能的更多信息，請?jiān)L問mlflow的官方頁面 。

So this is how we can do hyperparameter tuning using Mlflow tracking.

因此，這就是我們可以使用Mlflow跟蹤進(jìn)行超參數(shù)調(diào)整的方法。

結(jié)論 (Conclusion)

Using Mlflow we can easily track and manage the different configured trained models and compare them easily to find the best set of hyperparameters.

使用Mlflow，我們可以輕松地跟蹤和管理不同的已配置訓(xùn)練模型，并輕松進(jìn)行比較以找到最佳的超參數(shù)集。

Feel free to comment if you have any feedback for me to improve on, or if you want to share any thoughts or experience on the same.

如果您有任何需要改進(jìn)的反饋意見，或者想分享任何想法或經(jīng)驗(yàn)，請隨時發(fā)表評論。

Do you want more? Follow me on LinkedIn, and GitHub.

你還要嗎？ 在LinkedIn和GitHub上關(guān)注我。

翻譯自: https://medium.com/swlh/hyperparameter-tuning-with-mlflow-tracking-b67ec4de18c9

mlflow

總結(jié)

以上是生活随笔為你收集整理的mlflow_使用MLflow跟踪进行超参数调整的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。