I have a 'user' table in MySQL. I want to read it to my Spark SQL program. How can I read the table from MySQL to the Apache Spark's SparkSQL module using Python? Is there a connector I can use for this task?

Thanks.

解決方案

There is a similar question answered. Start pyspark like this

./bin/pyspark --packages mysql:mysql-connector-java:5.1.38

Then just run

sqlContext.read.format("jdbc").options(

url ="jdbc:mysql://localhost/mysql",

driver="com.mysql.jdbc.Driver",

dbtable="user",

user="root",

password=""

).load().take(10)

This would most likely just work. But this depends on your mysql set-up, so if it doesn't try changing password, username, db-url and other settings.

總結

以上是生活随笔為你收集整理的python调用sparksql,使用Python从MySQL数据库表读取SparkSQL的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。