Release builds for newer versions of Scala and Spark

[colindean]

Is your feature request related to a problem? Please describe.

We currently only ship for Scala 2.11 and Spark 2.3.x.

Describe the solution you'd like

We should ship for newer versions in whatever pairs are appropriate.

Additional context

https://github.com/sbt/sbt-projectmatrix is probably the right tool for the job.

[colindean]

Minimal:

groups:
  - spark:
    - 2.3.4
    scala:
    - 2.11.12

Max:

groups:
  - spark:
    - 2.3.4
    scala:
    - 2.11.12
  - spark:
    - 2.4.8
    - 3.0.3
    - 3.1.3
    scala:
    - 2.12.15
  - spark:
    - 3.2.1
    scala:
    - 2.13.8

Spark versions: https://spark.apache.org/downloads.html

[colindean]

We should continue to build on JDK 1.8. Spark still supports that. However, Spark 3.0 allows JDK 1.11, so that could be another variable, but it'd have to be at the GHA level instead of sbt-projectmatrix.

[colindean]

I've got a start on this in branch gh-84-crossbuild. I've got builds for [Scala 2.11 + Spark 2.3.4] and [Scala 2.12 + Spark 2.4.8] working pretty easily but not Scala 2.13 yet. It'll need some refactoring and I've yet to address a spurious extra build, too.

[colindean]

sbt 2.x builds in the build matrix: https://www.scala-sbt.org/2.x/docs/en/reference/cross-building-setup.html