This is a memo to share what I have learnt in Apache Airflow, capturing the learning objectives as well as my personal notes. The course is taught by Mike Metzger from DataCamp.

這是一份備忘錄，旨在分享我在Apache Airflow中學到的知識，記錄學習目標以及我的個人筆記。 該課程由DataCamp的Mike Metzger教授。

A data engineer’s job includes writing scripts, adding complex CRON tasks, and trying various ways to meet an ever-changing set of requirements to deliver data on schedule. Airflow can do all these while adding scheduling, error handling, and reporting.

數據工程師的工作包括編寫腳本，添加復雜的CRON任務以及嘗試各種方法來滿足日新月異的要求，以按計劃交付數據。 Airflow可以在添加計劃，錯誤處理和報告的同時完成所有這些工作。

I have learnt the following topics:

我已經學習了以下主題：

• Workflows / DAGs / Tasks
  工作流程/ DAG /任務
• Operators (BashOperator, PythonOperator, BranchPythonOperator, EmailOperator)
  運算子(BashOperator，PythonOperator，BranchPythonOperator，EmailOperator)
• Dependencies between tasks / Bitshift operators
  任務之間的依賴關系/移位運算符
• Sensors (to react to workflow conditions and state)
  傳感器(對工作流程條件和狀態做出React)
• Scheduling DAGs
  安排DAG
• SLAs / Alerting to maintain visibility on workflows
  SLA /警報以保持工作流程的可見性
• Templates for maximum flexibility when defining tasks
  定義任務時具有最大靈活性的模板
• Branching, to add conditional logic to DAGs
  分支，為DAG添加條件邏輯
• Airflow interfaces: command line / UI
  氣流接口：命令行/ UI
• Airflow executors
  氣流執行器
• Debugging / Troubleshooting
  調試/故障排除

My next steps would be:

我的下一步將是：

• Set up my own environment for practice
  建立自己的練習環境
• Explore other operators (eg. Amazon’s S3, Postgresql) and sensors (eg. HDFS)
  探索其他運營商(例如Amazon的S3，Postgresql)和傳感器(例如HDFS)
• Experiment with dependencies with a large number of tasks
  試驗具有大量任務的依賴項
• Look into parts of Airflow: XCom, Connections, etc
  查看氣流的各個部分：XCom，連接等
• Refer to Airflow documentations
  請參閱氣流文檔
• Keep building workflows
  繼續構建工作流程

More notes and codes can be found on my GitHub.

在我的GitHub上可以找到更多注釋和代碼。

Overall, I have enjoyed learning this course and would highly recommend it!

總的來說，我很喜歡學習這門課程，并強烈推薦它！

翻譯自: https://medium.com/swlh/introduction-to-airflow-in-python-67b554f06f0b

總結

以上是生活随笔為你收集整理的Python气流介绍的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。