# Designing libraries for source and binary compatibility Sébastien Doeraene @sjrdoeraene

April 17, 2018 -- Scala.sphere.it

LAMP, lamp.epfl.ch
École polytechnique fédérale de Lausanne  ### Who am I ...

to lecture you about source and binary compatibility?

Answer: the author of Scala.js

## Part Ⅰ

What is:

• Source compatibility?
• Binary compatibility?
• Forward/backward compatibility?
• Compatibility?

What is compatibility? What is compatibility? What is library compatibility? (backward)

What is library compatibility? (forward)

Source v binary compatibility

A v1

``````
class Foo {
def bar(x: Int): Int = x + 1
}
``````

A v2

``````
class Foo {
def bar(x: Double): Double = x + 1
}
``````

Source compat -- Binary compat

A v1

``````
class Foo {
def bar(x: Int): Int = x + 1
}
``````

A v2

``````
class Foo {
private val one: Int = 1

def bar(x: Int): Int = x + one
}
``````

Source compat -- Binary compat

A v1

``````
trait Foo {
def bar(x: Int): Int = x + 1
}
``````

A v2

``````
trait Foo {
private val one: Int = 1

def bar(x: Int): Int = x + one
}
``````

Source compat -- Binary compat

A v1

``````
class Foo {
def bar(x: Int): Int = x + 1
}
``````

A v2

``````
class Foo {
def bar(x: Int): Int = x + 1

def foobar(y: Int): Int = y * 2
}
``````

Source compat -- Binary compat

``````
class Foo {
def bar(x: Int): Int = x + 1

def foobar(y: Int): Int = y * 2
}
``````
``````
implicit class FooOps(val self: Foo) {
def foobar(y: Int): Int = y / 3
}
``````

Not a theoretical issue

### Writing an application X Neither source nor binary compat matter

### Writing a library A used by an app X Only source compatibility matters

### Writing a library A used by other libraries ### Writing a library A used by other libraries Source compat matters for C; binary compat matters for X    Binary incompatibilities lock down the ecosystem

Source incompatibilities are only a one-time inconvenience to directly dependent projects

### Summary so far

• Binary compatibility:
• hard to get right
• when broken: disastrous for the ecosystem
• Source compatibility:
• virtually impossible to achieve
• when broken: inconvenient for direct dependents

### Versioning scheme

When the new version is:

• Binary incompatible: major version bump
• Source incompatible: minor version bump
• Compatible: patch version bump

## Part Ⅱ

### How to avoid binary incompatibilities

Sometimes needs filters
requires good knowledge about the compile scheme
(even scalac people get it wrong sometimes)

### How to avoid source incompatibilities

Do not change your public/protected API

Use MiMa both ways (forward+backward) as an approximation

### When MiMa is not enough

Learn the compilation scheme with `-Xprint:mixin`

### Default parameters

``````
class Test {
def foo(x: Int, y: Int = 1, z: String = "hello"): Int = ???
}

test.foo(3, 4)
``````
``````
def foo(x: Int, y: Int, z: String): Int = scala.Predef.???();
<synthetic> def foo\$default\$2(): Int = 1;
<synthetic> def foo\$default\$3(): String = "hello";

test.foo(3, 4, test.foo\$default\$3());
``````

### How to recover?

``````
class Test {
@deprecated("...", "...")
def foo(x: Int, y: Int = 1, z: String = "hello"): Int = ???

def foo(x: Int, y: Int, z: String, w: Boolean): Int = ???
}

test.foo(3, 4) // deprecated
test.foo(3, 4, "hello", false)
``````

### Option bags

``````
case class FooOptions(x: Int, y: Int = 1)
``````
``````
case class FooOptions extends Object with Product with Serializable {
private[this] val x: Int = _;
def x(): Int = ...;
private[this] val y: Int = _;
def y(): Int = ...;
def copy(x: Int, y: Int): bincompat.FooOptions = ...;
def copy\$default\$1(): Int = ...;
def copy\$default\$2(): Int = ...;
override  def productPrefix(): String = ...;
def productArity(): Int = 2;
def productElement(x\$1: Int): Object = ...;
override  def productIterator(): Iterator = ...;
def canEqual(x\$1: Object): Boolean = ...;
override  def hashCode(): Int = ...;
override  def toString(): String = ...;
override  def equals(x\$1: Object): Boolean = ...;
def (x: Int, y: Int): bincompat.FooOptions = ...
};
object FooOptions extends AbstractFunction2 with Serializable {
def \$default\$2(): Int = ...;
final override  def toString(): String = ...;
case  def apply(x: Int, y: Int): bincompat.FooOptions = ...;
def apply\$default\$2(): Int = ...;
case  def unapply(x\$0: bincompat.FooOptions): Option = if (x\$0.==(null))
scala.None
else
new Some(new Tuple2\$mcII\$sp(x\$0.x(), x\$0.y()));
private def readResolve(): Object = ...;
case    def apply(v1: Object, v2: Object): Object = ...;
def (): bincompat.FooOptions.type = ...
};
``````

### How to recover?

Tough! Desugar it by hand.

``````
final class FooOptions private (val x: Int, val y: Int) {
private def this() = this(x = 0, y = 1)

private def withX(x: Int): FooOptions = copy(x = x)
private def withY(y: Int): FooOptions = copy(y = y)

private def copy(x: Int = x, y: Int = y): FooOptions =
new FooOptions(x, y)

override def toString(): String = s"FooOptions(\$x, \$y)"

override def equals(that: Any): Boolean = that match {
case that: FooOptions => this.x == that.y && this.y == y
case _                => false
}

override def hashCode(): Int = {
import scala.util.hashing.MurmurHash3._
var acc = classOf[FooOptions].getName.##
acc = mix(acc, x.##)
acc = mixLast(acc, y.##)
finalizeHash(acc, 2)
}
}

object FooOptions {
def apply(): FooOptions = new FooOptions()
}
``````

### What are case classes good for, then?

For pattern matching, in order words, if you intend to use them in case clauses!

Oh and ... make them final!

### Open traits

``````
trait Test {
private var bar: Int = 5
def foo(x: Int): Int = x + bar
}
class TestImpl extends Test
``````
``````
abstract trait Test extends Object {
<accessor> <sub_synth> def bar(): Int;
<accessor> <sub_synth> def bar_=(x\$1: Int): Unit;
def foo(x: Int): Int = x.+(Test.this.bar());
def /*Test*/\$init\$(): Unit = {
Test.this.bar_=((5: Int));
()
}
};
class TestImpl extends Object with bincompat.Test {
def foo(x: Int): Int = TestImpl.super.foo(x);
override <accessor> def bar(): Int = this.bar;
private[this] var bar: Int = _;
override <accessor> def bar_=(x\$1: Int): Unit =
TestImpl.this.bar = (x\$1: Int);
def <init>(): bincompat.TestImpl = {
TestImpl.super.<init>();
TestImpl.super./*Test*/\$init\$();
()
}
};
``````

### Open traits

• Seal your traits
• Or make sure they are completely abstract interfaces (Java-like) -- and never change them

### Write-only ADTs

``````
abstract sealed class Foo private ()

object Foo {
private[lib] case class Bar(x: Int) extends Foo
...

def bar(x: Int): Foo = Bar(x)
}
``````

Construct in user-space; extract in library

### Disappearing methods

``````
final class Test {
@deprecated("don't use this", "1.1.0")
def foo(x: Int): Int = x + 1
}
``````
``````
final class Test {
protected[Test] def foo(x: Int): Int = x + 1
}
``````

### I have stopped using ...

• Result type inference
• Default parameters
• Case classes (unless write-only)
• Non-sealed classes and traits
• Concrete members in open traits
• Public APIs

### How long can we sustain this?

Outlast each Scala major version

### Will TASTY solve this?

A bit, but mostly no
(it also makes some things worse)

### Summary

• Binary compat matters more than source compat
-> reflect in version numbers
• MiMa prevents you from creating an incompatible version
• It won't help you if you've locked yourself down in a previous version
• Learn the compilation scheme
• Same reasoning and tools apply to Scala.js and Scala Native libraries scala-js.org