mlflow

Hyperparameter tuning and optimization is a powerful tool in the field of AutoML. Tuning these configurations can dramatically improve model performance. However, hyperparameter tuning can be computationally expensive and slow.

^ h yperparameter調整和優化是AutoML領域的有力工具。 調整這些配置可以大大提高模型性能。 但是，超參數調整在計算上可能既昂貴又緩慢。

If you have a large network like VGG, Resnet, etc. trying out every parameter exhaustively and then choosing the best parameter is computationally intensive because in hyperparameter tuning we use various configurations. To do that, we need a principle approach to choose the best parameter.

如果您有一個大型網絡(如VGG，Resnet等)，則要窮盡地嘗試每個參數，然后選擇最佳參數會占用大量計算資源，因為在超參數調整中，我們使用了各種配置。 為此，我們需要一種原則方法來選擇最佳參數。

Hyperparameter tuning creates complex workflows involving testing many hyperparameter settings, generating lots of models, and iterating on an ML pipeline. To simplify tracking and reproducibility for tuning workflows, we use MLflow, an open-source platform to help and manage the complete machine learning lifecycle.

超參數 調整會創建復雜的工作流程，其中包括測試許多超參數設置，生成大量模型以及在ML管道上進行迭代。 為了簡化跟蹤和調整工作流程的可重復性，我們使用MLflow (一個開放源代碼平臺)來幫助和管理整個機器學習生命周期。

source資源

How can I find the best version of this model?

如何找到該模型的最佳版本？

How to track all the information and document of each model trained?

如何跟蹤每個訓練過的模型的所有信息和文檔？

How do we do that efficiently?

我們如何有效地做到這一點？

MLflow makes this process much more efficient and convenient!!

MLflow使此過程更加高效和便捷！！

Let’s start

開始吧

什么是MLflow？ (What is MLflow?)

MLfow is a python package developed by databricks that is defined as an open-source platform for the machine learning lifecycle. There are three pillars around mlflow ().

MLfow是由databricks開發的python軟件包，被定義為機器學習生命周期的開源平臺。 mlflow()周圍有三個Struts。

Source資源

Their documentation has a nice tutorial to explain the component of mlflow. Mlflow lets you log parameters and metrics which is incredibly convenient for model comparison.

他們的文檔中有一個很好的教程來解釋mlflow的組件。 Mlflow允許您記錄參數和度量，這對于模型比較非常方便。

在本地運行時進行協作 (Colab on local runtime)

So before starting to MLflow we will first connect to our colab on a local runtime. You are going to need Jupyter notebooks. It is what Colab built their platform on top of and it is required to run a local notebook. If you have Jupyter, you are already ahead of the game if not, click here for information.

因此，在開始MLflow之前，我們將首先在本地運行時連接到colab。 您將需要Jupyter筆記本。 這是Colab在其基礎上構建其平臺的基礎，并且它是運行本地筆記本電腦所必需的。 如果您擁有Jupyter，那么您已經領先于游戲， 請單擊此處獲取信息。

Step 1: install jupyter_http_over_ws using the command ‘pip install jupyter_http_over_ws’

步驟1：使用命令'pip install jupyter_http_over_ws'安裝jupyter_http_over_ws

Step 2: Enable the jupyter_http_over_ws jupyter extension using command jupyter serverextension enable — py jupyter_http_over_ws

步驟2：使用命令jupyter serverextension enable啟用jupyter_http_over_ws jupyter擴展— py jupyter_http_over_ws

Step 3 : Start server and authenticate using command jupyter notebook \ — NotebookApp.allow_origin=’https://colab.research.google.com' \ — port=8888 \ — NotebookApp.port_retries=0

步驟3：使用命令jupyter notebook啟動服務器并進行身份驗證\-NotebookApp.allow_origin ='https：//colab.research.google.com'\-端口= 8888 \-NotebookApp.port_retries = 0

Once the server has started, it will print a message with the initial backend URL used for authentication. Make a copy of this URL as you’ll need to provide this in the next step.

服務器啟動后，它將打印一條消息，其中包含用于身份驗證的初始后端URL。 復制此URL，因為您需要在下一步中提供它。

Jupyter notebook will automatically pop-up in your browser, but if it does not you can look at the highlighted link, that is the local http:// you will need to copy and paste in your browser.

Jupyter Notebook將自動在您的瀏覽器中彈出，但是如果沒有，您可以查看突出顯示的鏈接，即本地http：//，您需要在瀏覽器中進行復制和粘貼。

now the notebook is generated, but the notebook still needs to be connected to computing power. You will be able to do this by going to your colab page. Make sure your colab notebook settings are set to the GPU. Go to Edit > Notebook Settings and make sure you are running the right environment and that the hardware accelerator is set to the GPU.

現在已生成筆記本，但筆記本仍需要連接到計算能力。 您可以通過轉到colab頁面來執行此操作。 確保將您的colab筆記本設置設置為GPU。 轉到“編輯”>“筆記本設置”，并確保您運行的環境正確，并且硬件加速器已設置為GPU。

Click the “Connect” button and select “Connect to local runtime…”. The local connection setting window will pop up as shown below.

單擊“連接”按鈕，然后選擇“連接到本地運行時…”。 如下所示，將彈出本地連接設置窗口。

Enter the URL from the previous step in the dialog that appears and click the “Connect” button. After this, you should now be connected to your local runtime.

在出現的對話框中輸入上一步的URL，然后單擊“連接”按鈕。 之后，您現在應該連接到本地運行時。

Now you have GPU power through your local google colab notebook.

現在，您可以通過本地Google colab筆記本使用GPU了。

MLFlow的安裝和基本用法： (MLFlow installation and basic usage:)

Step 1: pip install mlflow

步驟1：pip安裝mlflow

Step 2: MLflow Python API logs run locally, in a mlruns directory wherever you ran your program. You can then run mlflow ui to see the logged runs.

步驟2：無論您在哪里運行程序，MLflow Python API日志都在mlruns目錄中本地運行。 然后，您可以運行mlflow ui以查看記錄的運行。

Enter the URL shown in the above image which provides a simple interface to various functionality in MLflow.

輸入上圖所示的URL，該URL提供了MLflow中各種功能的簡單接口。

超參數調整 (Hyperparameter Tuning)

Now we have connected google colab notebook to runtime and also installed mlflow.

現在，我們已將google colab筆記本連接到運行時，還安裝了mlflow。

Let’s solve one practical example, Digit recognition where mlflow will help you to find the right set of optimal hyperparameters for a learning algorithm.

讓我們解決一個實際的例子， 數字識別 ，其中mlflow將幫助您為學習算法找到正確的最佳超參數集。

We are using the MNIST dataset of 60,000 small square 28×28 pixel grayscale images of handwritten single digits between 0 and 9. The task is to classify a given image of a handwritten digit into one of 10 classes representing integer values from 0 to 9, inclusively. It has three tunable hyperparameters that we try to optimize: learning-rate, momentum, and number of hidden nodes.

我們正在使用MNIST數據集，其中包含60,000個手寫數字在0到9之間的小正方形28×28像素灰度圖像。任務是將給定的手寫數字圖像分類為10個類之一，這些類表示從0到9的整數值。包括在內。 它具有我們嘗試優化的三個可調超參數：學習速率，動量和隱藏節點的數量。

We have loaded the Data set and transformed it as required for now we have everything in place.

我們已經加載了數據集，并根據需要對其進行了轉換，現在我們已經準備就緒。

MLflow Tracking is organized around the concept of runs (mlruns), Each run records some information like which is explained as below.

MLflow跟蹤是圍繞運行(mlruns)的概念組織的，每次運行都會記錄一些信息，如下所述。

Create an experiment id using mlflow.create_experiment() creates a new experiment and returns its ID. Runs can be launched under the experiment bypassing the experiment ID to mlflow.start_run() which returns current active run if you have any else it will create new active run and return its object.

使用mlflow創建實驗ID。create_experiment()創建一個新實驗并返回其ID。 可以在實驗下啟動運行，而將實驗ID繞過mlflow.start_run() ，如果您有其他運行，則返回當前的運行，它將創建新的運行并返回其對象。

We have logged the value of the metric train loss, test loss, and test accuracy using mlflow.log_metric() which logs into a single key-value metric.

我們已經使用mlflow.log_metric()記錄了度量標準火車損失，測試損失和測試準確性的值，該值登錄到單個鍵值度量標準中。

At each run, we have logged parameter like start time, batch size, epochs, Learning rate, momentum, hidden nodes, and test loss along with its source code using mlflow.log_param() which logs into a single key-value param in the currently active run.

在每次運行中，我們都使用mlflow.log_param()記錄了諸如開始時間，批處理大小，時期，學習率，動量，隱藏節點和測試損失等參數及其源代碼，該參數登錄到了單個鍵值參數中。當前正在運行。

select the different models for comparison and click on the compare button.

選擇不同的模型進行比較，然后單擊“比較”按鈕。

Select the x-axis and y-axis parameters to compare the selected model and check the performance of the model.

選擇x軸和y軸參數以比較所選模型并檢查模型的性能。

you can also check the performance of the individual model and check different metrics plots(for example, to track how your model’s loss function is converging).

您還可以檢查單個模型的性能并檢查不同的指標圖(例如，跟蹤模型的損失函數如何收斂)。

you can record images (for example, confusing metric PNGs for each run), models (for example, a pth PyTorch model), or even different data files as artifacts using mlflow.log_artifact().

您可以使用mlflow.log_artifact()將圖像(例如，每次運行時混淆的度量PNG)，模型(例如，pth PyTorch模型)或不同的數據文件記錄為工件。

So, we logged all run for each hyperparameter setting, and each of those runs includes the hyperparameter setting and the evaluation metric. Comparing these runs in the MLflow UI helps with visualizing the effect of tuning each hyperparameter.

因此，我們記錄了每個超參數設置的所有運行，并且每個運行都包括超參數設置和評估指標。 在MLflow UI中比較這些運行有助于可視化調整每個超參數的效果。

Once you come up with a better configuration of hyperparameter you can load that model specifying the run id and use for inference.

一旦提出了更好的超參數配置，就可以加載指定運行ID的模型并用于推理。

To know more about mlflow functions you can visit the official page of mlflow.

要了解有關mlflow功能的更多信息，請訪問mlflow的官方頁面 。

So this is how we can do hyperparameter tuning using Mlflow tracking.

因此，這就是我們可以使用Mlflow跟蹤進行超參數調整的方法。

結論 (Conclusion)

Using Mlflow we can easily track and manage the different configured trained models and compare them easily to find the best set of hyperparameters.

使用Mlflow，我們可以輕松地跟蹤和管理不同的已配置訓練模型，并輕松進行比較以找到最佳的超參數集。

Feel free to comment if you have any feedback for me to improve on, or if you want to share any thoughts or experience on the same.

如果您有任何需要改進的反饋意見，或者想分享任何想法或經驗，請隨時發表評論。

Do you want more? Follow me on LinkedIn, and GitHub.

你還要嗎？ 在LinkedIn和GitHub上關注我。

翻譯自: https://medium.com/swlh/hyperparameter-tuning-with-mlflow-tracking-b67ec4de18c9

mlflow

總結

以上是生活随笔為你收集整理的mlflow_使用MLflow跟踪进行超参数调整的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。