Scala Puzzlers

What's Scala?

Object-oriented and functional

“A drunken Martin Odersky sees a Reese's Peanut Butter Cup ad featuring somebody's peanut butter getting on somebody else's chocolate and has an idea.
He creates Scala, a language that unifies constructs from both object oriented and functional languages. This pisses off both groups and each promptly declares jihad.” ¹

¹ A Brief, Incomplete, and Mostly Wrong History of Programming Languages

What's Scala?

Statically typed with local type inference

val hi: String = "Hi KaFP!"

What's Scala?

Class/trait-based inheritance and subtyping

trait HasName { def name: String } class Person(val name: String) extends HasName

What's Scala?

First-class functions/lambdas/methods

def isEven(i: Int) = i % 2 == 0 (1 to 16) .filter(isEven) .map(_.toBinaryString)

What's Scala?

Compiles to Java bytecode, runs on the JVM
A Java class is a Scala class, too

The Plan for Today

No introduction to the language …
… but all code snippets will be explained
Examples of small & huge simplifications …
- mostly in the language,
- but also in the standard library
Lessons learned?

The Three Laws of
Programming Language Design

Joe Armstrong (creator of Erlang):

What you get right nobody mentions
What you get wrong, people bitch about
What is difficult to understand you have
to explain to people over and over again

We'll focus on 2. and 3. today.

Why isn't everyting dead-simple?

Expressivity and power come with a price
Unintended feature interactions
Feature & scope creep
Legacy issues and compatibility bloat

Simplicity

Simple to specify?
Simple to implement?
Simple to use?

Simplest solution which works consistently
(and still takes all known issues into account)?

Example I: Array Syntax

// Java/C#/…: special syntax rules int[] nums = new int[5]; nums[0] = 42; int answer = nums[0];

// Scala: apply and update methods val nums = new Array[Int](5) nums(0) = 42 val answer = nums(0)

Scala's Approach

val nums = new Array[Int](5) nums(0) = 42 // nums.update(0, 42) val answer = nums(0) // val answer = nums.apply(0)

Works for all types which define apply or update!

Example II: Operators …

… because readability is not optional!

// Compare ... 1 + 2 * 3 / 4 - 5 // ... to: minus(add(1, divide(multiply(2, 3), 4)) - 5)

Scala's Solution in Theory

No operators, just methods
No restriction on method names
Binary calls can leave out . and ()
Precedence is determined by first char

Scala's Solution in Practice

val hello = "Hello KaFP".take(5)

val hello = "Hello KaFP" take 5

val three: Int = 1 + 2

val three: Int = 1.+(2)

The Past

Floating-Point Literals

Floating-point-literal-without-digit-after-dot val num = 42. val sum = 23.+(17)

Floating-Point Literals

Floating-point-literal-without-digit-after-dot
(deprecated in 2.10) val num = 42. val sum = 23.+(17)

Octal Literals

Octal numeric literal val bond = 007

Octal escape literal val abc = "\141\142\143"

Octal Literals

Octal numeric literal (deprecated in 2.10) val bond = 007

Octal escape literal (deprecated in 2.11) val abc = "\141\142\143"

Primitive Type Aliases

def getRandomNumber: Int = 4 val myRandomNumber: int = getRandomNumber

Pointless choices to achieve the same result

Type Hierarchy (2.9/2.10)

Type Hierarchy (2.10/2.11)

Library Clean-up I (2.10)

Library Clean-up I (2.11)

Library Clean-up II (2.9)

scala.mobile /** The class Location provides a create * method to instantiate objects from a * network location by specifying the * URL address of the jar/class file. * ... * @version 1.0, 04/05/2004 */ class Location { ... }

Library Clean-up III (2.10)

scala.text /** A basic pretty-printing library, based * on Lindig's strict version of Wadler's * adaptation of Hughes' pretty-printer. */ abstract class Document

Library Clean-up IV (2.10)

scala.util.regex.automata /** This class turns a regular expression * over `A` into a `NondetWordAutom` over * `A` using the celebrated position * automata construction (also called * ''Berry-Sethi'' or ''Glushkov''). */ abstract class BaseBerrySethi

The Present

Procedures I

Special syntax for methods without return value
Return type of Unit (~ void)

def foo1() { 1 + 2 } def foo2() = 1 + 2 def foo3(): Unit = { 1 + 2 }

Extremely error-prone and silent error path

Procedures I

// multiply a and b: def multiply(a: Int, b: Int) { a * b } // add a and b: def add(a: Int, b: Int) = { a + b } // check result of (5 + 2) * 3: val result = (5 + 2) * 3 == multiply(add(5, 2), 3)

Procedures II

trait ExampleInterface { def add(a: Int, b: Int): Int def foo // Isn't something missing here? }

Is foo …
- dynamically typed?
- typed with global type inference?
- using Top/Bottom types?

Procedures

Unit `()` Insertion

class Cell[T](value: T) new Cell(42) new Cell("Hi") new Cell // ???

val set1 = List(1,2,3).toSet val set2 = List(1,2,3).toSet()

XML Literals

val john = "John Doe" val xml = <person> <name>{john}</name> <age>23</age> </person> val age = xml \ "age"

XML Literals

Unsolved implementation issues
Modularized in 2.11
Removed from "core Scala"
Hooks for third-party XML libraries

String Interpolation

New in 2.10

val john = "John Doe" val age = 23 val text = s"$john is $age years old."

You know you need string interpolation,
when people start using XML literals as a replacement!

String Interpolation

Opportunity to remove dreaded
"from everything to String"-conversion

val sum = "1 + 2" + 3 + 4

String Interpolation

Extensible by design
Example: formatting interpolator

val height = 1.9 val name = "John" val text = f"$name%s is $height%2.2f meters tall."

String Interpolation

Example: SQL interpolator

val name = "<?!''&|*" sql"select * from coffees where name = $name".as[Coffee]

View Bounds

// Only accept types which // are convertible to an Int def foo[A <% Int](value: A) = value.toDouble /* Example 1 */ // foo(1)

View Bounds

// Only accept types which // are convertible to an Int def foo[A <% Int](value: A) = value.toDouble /* Example 2 */ // implicit def strToInt(str: String) = // str.toInt // foo("42")

View Bounds

Limited usefulness and usage compared
to Context Bounds (typeclasses)
<% syntax deprecated in 2.11

// def foo[A <% Int](value: A) = // value.toDouble def foo[A](value: A) (implicit conv: A => Int) = value.toDouble

The Future

Constant Inlining

// File A: final val foo = 42 //File B: def getFoo = foo

Swaps run-time for compile-time dependency
No performance benefit
Increased compile time

Default Imports

Types, members and packages from these
scopes are available without import:

scala.Predef
scala._
java.lang._

Default Imports: `java.lang`

Useless types: Compiler, Enum, Number, Object, Void, ...

Shadowed types: Boolean, Byte, Double, Float, Long, ...

Default Imports: `java.lang`

Not importing java.lang._ by default:

Easier portability (think JS)
Less noise in the default namespace
Better average code quality

Implicit Widening Conversions

val nums = List(123456789, 1.0f)

val nums = List(List(123456789), List(1.0f))

Implicit Widening Conversions

Complete change of semantics regarding
- Precision
- Division-By-Zero, error propagation
- Overflow handling, infinity
Silent conversion inherently dangerous,
not just for conversions which lose precision
(e. g. Int to Float, Long to Float, Long to Double)

Enums

Current options in Scala:

scala.Enumeration
sealed trait + objects pattern
Use Java

Enums: `scala.Enumeration`

object WeekDay extends Enumeration { type WeekDay = Value val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value }

Enums: `scala.Enumeration`

Unintuitive to use
Not compatible with Java's enums
Terrible in more ways than you can imagine

Enums: `trait`/`object` pattern

sealed trait WeekDay object Mon extends WeekDay object Tue extends WeekDay object Wed extends WeekDay object Thu extends WeekDay object Fri extends WeekDay object Sat extends WeekDay object Sun extends WeekDay

Enums: `trait`/`object` pattern

Extremely verbose
Extremely powerful
Not compatible with Java's enums

Enums: Use Java

enum WeekDay { Mon, Tue, Wed, Thu, Fri, Sat, Sun }

Enums: Use Java

Seriously?
That's embarrising!

Enums: `@enum` macro

@enum class WeekDay { Mon; Tue; Wed; Thu; Fri; Sat; Sun }

Enums: `@enum` macro

Easy to use
Java compatible
Good replacement for scala.Enumeration

Enums

Future options in Scala:

~~scala.Enumeration~~
GADTs: sealed trait + objects
~~Use Java~~
Scala @enum

Union and Intersection Types

Bug ticket SI-7402:

val result = if (true) List(1) else Vector(1)

Computed type is correct, but … useful?

Union and Intersection Types

Native support for "Set" operations on types
Concise, precise and readable types

val result: List[Int] | Vector[Int] = if (true) List(1) else Vector(1)

val result: List[Int|String] = List(1, "")

Union and Intersection Types

Scala already has some kind of union and intersection types …

class Foo extends Bar with Baz

def foo(a: Any) = a match { case _: String | _: Int => case _ => }

… but lacks first-class support to reason about them in the type system

Union and Intersection Types

Native support for unions and intersection types in DOT, the formalization of Scala's next typesystem
Stay tuned!

Better VM Support

Generics
Value types
Proper tail calls
Delimited continuations

Lessons learned?

Basics

Distributed version control
Bug tracking
Test suites
Continuous Integration
Reviews and coding standards
Specification

Language/Code Evolution

Code/Features/Libraries need time to mature!

Mark as experimental to manage user expectations
Let different parts evolve at their own speed
When in doubt, keep things out

Language/Code Evolution

Example: Scala Macros

Experimental
Macro styles are promoted from incubator to mainline when they are ready
Not everything survives the incubator

Language/Code Evolution

Example: Modularization

2.11 will ship modularized library
Deprecated parts moved to own JAR files
Libraries get own version number

Bug Fixes

Compatibility

Source or binary compatibility? Both?

Java: “Never change anything, ever!”
C#: “It depends”
Python: “Where code goes to die”
PHP: — No comment —

Compatibility

Scala:

Minor and update versions are source and binary compatible
Major Verions are not binary compatible
How to guarantee these rules?

Tooling

Deprecation Policy

Code/feature must be marked as "deprecated" for at least one major version
Example:
- Issue is found in 2.9
- Marked as deprecated in 2.10
- Fixed/removed in 2.11

Evaluation Criteria

Quality of code/feature
Impact of change
Internal dependencies
Available alternatives

Language Deprecation

Requires modification of:

Compiler
Libraries
IDE
Specification
Style guide
Test suite

Code Deprecation

Versioning @deprecated("Use `Foo` instead.", "2.10") class Bar

Code Deprecation

Versioning
Precision
- Inheritance @deprecatedInheritance( "`Bar` will be made final.", "2.10") class Bar

Code Deprecation

Versioning
Precision
- Inheritance
- Parameter naming def foo(bar: Int)

Code Deprecation

Versioning
Precision
- Inheritance
- Parameter naming def foo(@deprecatedName('bar) baz: Int)

Code Deprecation

Versioning
Precision
- Inheritance
- Parameter naming
- Overriding class Bar { def foo = ??? }

Code Deprecation

Versioning
Precision
- Inheritance
- Parameter naming
- Overriding class Bar { @deprecatedOverriding( "`foo` will become final.", "2.10") def foo = ??? }

Results

Steady progress towards a smaller, more consistent Scala
Requirements for changes are growing (migration, tooling, ...)
Managing pressure to add new features (macros!)

Conclusion

Manage expectations!
Remove more things than you add!
Keep things small, simple and consistent!
Look out for UI/UX issues!
Listen to/watch new users!
Recognize common error patterns …
… and emit warnings/compiler errors!

Thanks!

Questions?

Bonus Slides

Optimistic APIs

What happens if the behavior in a case of error is left unspecified?
Example: Method remove takes two arguments
- Position at which elements are removed
- Amount of elements to remove

import scala.collection.mutable.ListBuffer val buf = ListBuffer(1, 2, 3) buf remove (4, 4) // <-- Datenstruktur korrumpiert!

Parallel Nightmares

Data structures implement the collection API
The collection API supports sequential and parallel operations

def twice(i: Int) = i * 2 (1 to 1000) map twice

Parallel Nightmares

Data structures implement the collection API
The collection API supports sequential and parallel operations

def twice(i: Int) = i * 2 (1 to 1000) map twice

def brokenTwice(i: Int) = if (i%42 == 0) throw new RuntimeException(s"Failed: $i") else i * 2 (1 to 1000) map brokenTwice

Parallel Nightmares

def brokenTwice(i: Int) = if (i%42 == 0) throw new RuntimeException(s"Failed: $i") else i * 2 (1 to 1000).par map brokenTwice

Parallel operations with multiple failures
- Which and how many exceptions will be reported?

class CompositeThrowable(exceptions: Set[Exception])

Parallel Nightmares

Parallel operations with multiple failures
- Which and how many exceptions will be reported?

class CompositeThrowable(exceptions: Set[Exception])

If more than one exception is encountered,
exceptions will be wrapped in CompositeThrowable

Parallel Nightmares

Parallel operations with multiple failures
- Which and how many exceptions will be reported?

class CompositeThrowable(exceptions: Set[Exception])

If more than one exception is encountered,
exceptions will be wrapped in CompositeThrowable
Sounds reasonable, but ...
completely non-deterministic exception handling!

Parallele Albträume

Sounds reasonable, but ...
completely non-deterministic exception handling!
Number of exceptions?
Type of exception?
Broken tests!
Non-reproducible behavior!

The thing with the double colon …

A nethod naming ending with :
- is right-associative …
- but only in operator syntax!

val list1 = 1 :: 2 :: 3 :: Nil val list2 = Nil.::(3).::(2).::(1) assert(list1 == list2)

The thing with the double colon …

← → #

Scala Puzzlers

What's Scala?

What's Scala?

What's Scala?

What's Scala?

What's Scala?

The Plan for Today

The Three Laws of Programming Language Design

Why isn't everyting dead-simple?

Simplicity

Example I: Array Syntax

Scala's Approach

Example II: Operators …

Scala's Solution in Theory

Scala's Solution in Practice

The Past

Floating-Point Literals

Floating-Point Literals

Octal Literals

Octal Literals

Primitive Type Aliases

Type Hierarchy (2.9/2.10)

Type Hierarchy (2.10/2.11)

Library Clean-up I (2.10)

Library Clean-up I (2.11)

Library Clean-up II (2.9)

Library Clean-up III (2.10)

Library Clean-up IV (2.10)

The Present

Procedures I

Procedures I

Procedures II

Procedures

Unit () Insertion

XML Literals

XML Literals

String Interpolation

String Interpolation

String Interpolation

String Interpolation

View Bounds

View Bounds

View Bounds

The Future

Constant Inlining

Default Imports

Default Imports: java.lang

Default Imports: java.lang

Default Imports: java.lang

Implicit Widening Conversions

Implicit Widening Conversions

Enums

Enums: scala.Enumeration

Enums: scala.Enumeration

Enums: trait/object pattern

Enums: trait/object pattern

Enums: Use Java

Enums: Use Java

Enums: @enum macro

Enums: @enum macro

Enums

Union and Intersection Types

Union and Intersection Types

Union and Intersection Types

Union and Intersection Types

Better VM Support

Lessons learned?

Basics

Language/Code Evolution

Language/Code Evolution

Language/Code Evolution

Bug Fixes

Compatibility

Compatibility

Tooling

Deprecation Policy

Evaluation Criteria

Language Deprecation

Code Deprecation

Code Deprecation

The Three Laws of
Programming Language Design

Unit `()` Insertion

Default Imports: `java.lang`

Default Imports: `java.lang`

Default Imports: `java.lang`

Enums: `scala.Enumeration`

Enums: `scala.Enumeration`

Enums: `trait`/`object` pattern

Enums: `trait`/`object` pattern

Enums: `@enum` macro

Enums: `@enum` macro