當前位置：首頁 > 编程语言 > python >内容正文

python

python分析犯罪数据_使用 SparkSQL 分析纽约市犯罪数据

發布時間：2025/3/12 python 25 豆豆

生活随笔收集整理的這篇文章主要介紹了 python分析犯罪数据_使用 SparkSQL 分析纽约市犯罪数据小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

在實驗《使用 SparkSQL 分析紐約市犯罪數據》時候

創建 DataFrame 并注冊成表，報錯！

sqlContext.createDataFrame(Crimes).registerTempTable("Crimes")

17/11/10 07:55:17 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)

org.apache.spark.api.python.PythonException: Traceback (most recent call last):

File "/opt/spark-1.6.1-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/worker.py", line 111, in main

process()

File "/opt/spark-1.6.1-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/worker.py", line 106, in process

serializer.dump_stream(func(split_index, iterator), outfile)

File "/opt/spark-1.6.1-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream

vs = list(itertools.islice(iterator, batch))

File "/opt/spark-1.6.1-bin-hadoop2.6/python/pyspark/rdd.py", line 1293, in takeUpToNumLeft

yield next(iterator)

File "", line 1, in

NameError: global name 'P' is not defined

at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRDD.scala:166)

at org.apache.spark.api.python.PythonRunner$$anon$1.(PythonRDD.scala:207)

at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:125)

at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:70)

at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)

at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)

at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)

at org.apache.spark.scheduler.Task.run(Task.scala:89)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

17/11/10 07:55:17 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last):