Skewcheck — audit a PySpark job before it runs at scale, then fix it

Paste or drop one PySpark job with its spark-submit script and requirements.txt. Free in your browser: 20 checks - collect()/toPandas() of data nothing bounds, Python UDFs, count() for emptiness, actions and withColumn in Python loops, windows with no partitionBy, keyless joins, adaptive execution or broadcast switched off, repartition(1) before a write, partitionBy on an id, inferSchema, one-connection JDBC reads, credential literals, overwrites that delete other partitions. Then an audit with exact edits per finding, or the whole fixed job, re-checked. Derived from @wshobson/spark-optimization (Seth Hobson, MIT). Not affiliated with or endorsed by the Apache Software Foundation; Apache Spark and PySpark are its trademarks.