Over the last two years, I had the opportunity to gain knowledge on Scala and also re-learn Java (after about 3 years without using it). I built network analysis tools in Scala and used it to entirely write the backend of Code Inspector with it. In the meantime, when starting to recently form a new engineering team, I had to use (and re-learn) Java again. I wanted to reflect on this experience to share some thoughts as to where these languages stand, the directions they are taking and answer the question too-often asked: which one to use?

Java has evolved a lot, and for the better

Java evolved a lot over its about 20 years of existence. From a simple object-oriented language, it brought fundamental changes. One example is the change in the for-loop syntax in Java 5 or the introduction of lambda in Java 8. As the language became very popular, developers added a ton of libraries for it and the language became very popular with infamous application (such as Hadoop, Spring or Play).

Also, the community around the language built libraries to overcome some of its limitations. One good example is Guava, the library developed by Google that provides a lot of helper methods and class to overcome some clear limitations of the language (e.g. the lack of support of Optional - more on this later).

Functional Programming has proven to be clean and effective

In the meantime, it is becoming clear that Object-Oriented programming (OO) is not the silver bullet we thought it will be. Many people believed it was a revolution in the programming world (e.g. Steve Jobs at was pitching the use of OO for NeXT long time ago, seeing it as revolutionary). I belong to the group of engineers that were taught OO was the right way to build programs (and even learned Merise or UML). It is then no surprise that at first, I was very skeptical about the benefits of FP, especially after having used Java and C++ for the last 5+ years. My interest in FP really started to increase after watching this video of John Carmack on the topic where he explains the real issues our industry is facing and how FP is helping to write simpler programs.

After few months, it became obvious that Functional Programming (FP) helps to write programs that are easier to maintain and reason about it. The use of immutability avoids side-effects and basic errors such as NullPointerException that have been plaguing programs for decades. As human, we fail to reason about programs structure and do not build systems with the idea that people will maintain it for the years to come. And these problems increase as the size of the programs (and the number of engineers working on it) increases. We can argue that these problems can be solved using more processes and tools (static analyzer, automated checks during reviews, etc.), but at the end of the day, what matters is to write something simple from the start to avoid problems and additional work later in the development process. The key idea is to keep thing simple from the start. FP forces you to separate concerns (data vs. functions) and write understandable code using pure functions where outputs only depends on the inputs.

From a mental model and design perspective, the clear separation between data (case class in Scala) and the functions that process it helps you to reasons about your data flow. Finally, the use of pure functions facilitates testing and is probably the biggest take-away from FP.

There are many reasons that make FP superior to OO and so far, I believe that the best summary has been written by Joe Amstrong, creator of Erlang, who summarizes most of them in a great essay.

OO was probably a good intention but good intentions don’t work (probably the best thing I learned from my time at Amazon).

Java evolves in the direction of Scala

It is now clear that Java is now taking the direction of Scala since several years One of the obvious sign is the introduction of lambda in Java 8, a first-class citizen concept of the Scala language.

Another example is the push from the community to support Scala-like constructions such as the Optional in guava (or even the helper methods to improve the default implementation of toString, equals or hashCode - there are plenty of examples).

The last example in date is the project [Amber from Java|https://openjdk.java.net/projects/amber/) that is copying core Scala features such as [pattern matching|https://openjdk.java.net/jeps/305], case class (renamed data classes) or type inference. Once project Amber is done, you could write code like this:

if (obj instanceof String s || s.length() > 5) {s.contains("foo")}

The exact same functionality exists in Scala since its inception and the same code would look like

obj match {
  case s: String if s.length() > 5 => s.contains("foo")
  case _ => false
}

Is that a bad thing that Java is copying Scala? In fact, this is really healthy. And definitively shows the value of the concepts behind the language. However, as these features are added on top of a already complex language, it brings a new set of problems.

You cannot get rid of the past

The biggest issue by introducing this new FP-compliant features on top of an old language is the new level of complexity and the related ambiguity it might bring. As the language needs to stay backward-compatible with the existing feature, the new one are added without deprecating the old one, which let programmers do whatever they wants since no rule will be enforced by the compiler. This brings consistency issues in the codebase (use of multiple coding patterns in the language), which confuses developers and make program maintenance even more complex. One good example of such problem is large C++ codebase that tried to evolve to later versions (for example, when mixing raw and smart pointers in C++).

These additional features do not address the main shortcoming of the language (OO, mutable variables, separation of concerns, etc.) and do not force the developer to use FP. It offers the possibility to use it but does not mandate it, which means that using FP must be enforced through a human process that is fallible by definition. Not good.

So what language to choose?

It is now clear that FP languages are superior in terms of maintenance, which is the biggest cost in software today.

One argument that is often made against Scala is the lack of candidates that know Scala. That might be true if you are looking for a temporary position but otherwise, if you hire a developer because of its familiarity with a particular language, this is not good for the long term. It is better to focus on the fundamentals (data structure, algorithms, etc.) than on a particular technology. Learning Scala does not take much time. My experience after one year of teaching Scala showed me that engineers understands the fundamentals within a month).

If you are starting a new project from scratch, Scala is a good choice and its compatibility with Java lets you use any existing Java library (and there are tons of it). That should open the best of both worlds by having a great language a a ton of existing and reliable libraries.

But what if you already have an existing codebase, should you dump Java and start to rewrite everything in Scala? Probably not. Or at least, not that fast. This would be a non-sense to dump a large codebase built, matured and stabilized for years to only switch to a new language. And it would also be an immediate, visible loss of productivity as all engineers are starting to learn Scala. The best way forward is to start using new features of the language for the existing code, and put processes (such as automatic checks) in place to make sure the codebase stays consistent. If you start re-writing some services or decompose them, it can be a good idea to start using Scala (and its compatibility with Java helps in that direction) and let few engineers explore and experiment the language (reducing the potential loss of productivity). You can then judge for yourself how comfortable your engineers are with this transition but also evaluates the potential benefits gradually.

Final Words

While this small article is in favor of Scala, the most important idea is that Functional Programming principles help you writing re-usable and maintainable code and that Object-Oriented principles were probably a mistake the industry followed blindly. FP rules and principles can be implemented by developers in almost any language (see Carmack talk mentioned above) And they help you write better programs.