當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

netflix_Netflix的Polynote

發(fā)布時間：2023/11/29 编程问答 33 豆豆

生活随笔收集整理的這篇文章主要介紹了 netflix_Netflix的Polynote 小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.

netflix

Netflix open source Polynote is a new notebook environment and was born out of the necessity to accelerate data science experimentation at Netflix.

Netflix開源Polynote是一種新的筆記本環(huán)境，其誕生是出于加速Netflix數(shù)據(jù)科學實驗的需要。

Over the last few years, Netflix has transformed its use of data science notebooks from an experimentation artifact to a key component of the lifecycle of machine learning solutions. Initially, Netflix adopted Jupyter Notebooks like a data exploration and analysis tools.

在過去的幾年中，Netflix已將其對數(shù)據(jù)科學筆記本的使用從實驗工件轉變?yōu)闄C器學習解決方案生命周期的關鍵組成部分。最初，Netflix將Jupyter Notebooks用作數(shù)據(jù)探索和分析工具。

Polynote (Polynote)

Polynote is a multi-language notebook experimentation environment. In addition to Python, the current release supports languages such as SQL, Vega(visualizations) and, of course, Scala. The platform is also integrated with data science infrastructures such as Apache Spark. At its core, Polynote includes the following capabilities:

Polynote是一種多語言筆記本實驗環(huán)境。除Python外 ，當前版本還支持SQL ， Vega (可視化)，當然還有Scala等語言 。該平臺還與數(shù)據(jù)科學基礎架構(例如Apache Spark)集成在一起。 Polynote的核心包括以下功能：

a) Improved Editing Experience: Polynote tries to enable an editing experience closer to modern IDEs. b) Multi-Language Support: Polynote introduces first-class support for Scala and other languages used in data science environmenhts. c) Data Visualization Improvements: Polynote integrates native data visualizations into notebooks’ dataset without the need of adding a lot of code. d) Configuration and Dependency Management: Languages like Scala require complex package dependencies in its programs. Polynote saves the package dependency configuration within the notebook itself addressing some of the common challenges in this area experienced by JVM developers. e) Reproducibility: The combination of code, data and execution results into a single document makes notebooks powerful, but also difficult to reproduce. Polynote includes reproducibility as a first-class capability of the framework.

a) 改進的編輯體驗： Polynote試圖使編輯體驗更接近現(xiàn)代IDE。 b) 多語言支持： Polynote引入了對Scala和數(shù)據(jù)科學環(huán)境中使用的其他語言的一流支持。 c) 數(shù)據(jù)可視化方面的改進： Polynote將原生數(shù)據(jù)可視化集成到筆記本的數(shù)據(jù)集中，而無需添加大量代碼。 d) 配置和依賴性管理：諸如Scala之類的語言在其程序中需要復雜的軟件包依賴性。 Polynote將包依賴項配置保存在筆記本自身中，以解決JVM開發(fā)人員在該領域遇到的一些常見挑戰(zhàn)。 e)可復制性：將代碼，數(shù)據(jù)和執(zhí)行結果組合到一個文檔中，使筆記本功能強大，但也難以復制。 Polynote將可再現(xiàn)性作為框架的一流功能。

改進的編輯體驗 (Improved Editing Experience)

Polynote includes common features in IDEs such as code auto-completion or syntax error highlighting which improves the experience for data scientists and researchers building Notebooks. More of the editing capabilities are powered by the Monaco editor which powers the experience of Visual Studio Code.

Polynote包含IDE中的常見功能，例如代碼自動完成或語法錯誤突出顯示，從而改善了構建筆記本電腦的數(shù)據(jù)科學家和研究人員的體驗。摩納哥編輯器提供了更多的編輯功能，該編輯器為Visual Studio Code的體驗提供了支持。

多國語言支持 (Multi-Language Support)

Polynote does not only provide support for multiple languages but it also allows those languages to be combined in a single program. In Polynote, every cell can be based on a different language. When a cell is run, the kernel provides the available typed input values to the cell’s language interpreter. In turn, the interpreter provides the resulting typed output values back to the kernel. This allows cells in Polynote notebooks to operate within the same context. The example below shows a Python library, to compute an isotonic regression of a dataset generated with Scala.

Polynote不僅提供對多種語言的支持，而且還允許將這些語言組合在一個程序中。在Polynote中，每個單元格可以基于不同的語言。當單元運行時，內核將可用的類型化輸入值提供給單元的語言解釋器。反過來，解釋器將結果輸入的輸出值提供回內核。這使Polynote筆記本中的單元格可以在相同的上下文中運行。下面的示例顯示了一個Python庫，用于計算使用Scala生成的數(shù)據(jù)集的等滲回歸。

Source: https://polynote.org/資料來源： https : //polynote.org/

數(shù)據(jù)可視化 (Data Visualization)

Data visualizations are a common component of most notebook environment. However, Polynote takes the visualization value proposition to another level by including it as a native component of the platform which does not require developers to write any code in order to visually explore a dataset.

數(shù)據(jù)可視化是大多數(shù)筆記本環(huán)境的常見組件。但是，Polynote通過將可視化價值主張作為平臺的本機組件，將可視化價值主張?zhí)嵘搅肆硪粋€層次，不需要開發(fā)人員編寫任何代碼即可直觀地瀏覽數(shù)據(jù)集。

組態(tài) (Configuration)

Most of the time, data scientists working on notebooks can enjoy the efficiency of Python’s package management model to handle the dependencies of a program. However, in JVM-languages like Scala dependency management can become a total night mare. Polynote addresses that challenge by storing the configuration and dependency information directly in the notebook itself, rather than relying on external files. Additionally, Polynote provides a user-friendly Configuration section where users can set dependencies for each notebook.

大多數(shù)時候，從事筆記本工作的數(shù)據(jù)科學家可以享受Python的包管理模型處理程序依賴關系的效率。但是，在諸如Scala依賴關系管理之類的JVM語言中，它們可能會變成一頭噩夢。 Polynote通過將配置和相關性信息直接存儲在筆記本本身中而不是依賴于外部文件來解決這一挑戰(zhàn)。此外，Polynote還提供了一個用戶友好的“配置”部分，用戶可以在其中為每個筆記本設置依賴性。

Source: https://polynote.org/資料來源： https : //polynote.org/

重現(xiàn)性 (Reproducibility)

With Polynote, Netflix a new code interpretation block instead of relying on a REPL model like a traditional notebook. One of the key capabilities of the new interpretation model is that it removes hidden states which allows data scientists to copy cells within a notebook without introducing any state from the previous position.

借助Polynote，Netflix有了新的代碼解釋模塊，而不再像傳統(tǒng)筆記本那樣依賴REPL模型。新解釋模型的關鍵功能之一是，它消除了隱藏狀態(tài)，這使數(shù)據(jù)科學家可以在筆記本中復制單元而無需從先前位置引入任何狀態(tài)。

Polynote is a new release in the ambitious competitive of data science notebooks but one that stands in its own merits. The support for JVM-based languages could make Polynote a favorite of developers working on Spark infrastructures. Also the editing and reproducatility capabilities are definitely welcomed enhancements to traditional notebook environments. Polynote is available in Github and you can also follow the project’s website.

Polynote是在雄心勃勃的數(shù)據(jù)科學筆記本電腦競爭中推出的新版本，但它有自己的優(yōu)點。對基于JVM的語言的支持可能使Polynote成為使用Spark基礎結構的開發(fā)人員的最愛。同樣，編輯和再現(xiàn)性功能無疑是對傳統(tǒng)筆記本環(huán)境的增強。 Polynote 在Github中可用，您也可以訪問該項目的網站。

Source: https://polynote.org/資料來源： https : //polynote.org/

生活随笔

生活随笔

编程问答

netflix_Netflix的Polynote

Polynote (Polynote)

改進的編輯體驗 (Improved Editing Experience)

多國語言支持 (Multi-Language Support)

數(shù)據(jù)可視化 (Data Visualization)

組態(tài) (Configuration)

重現(xiàn)性 (Reproducibility)

推薦文章 (Recommended Articles)

總結