Scala Center Roadmap for 2023 and Beyond

In this post, we summarize the achievements of the Scala Center in 2022, and share our plans going forward to ensure the longevity of Scala.

Role of the Scala Center in the Scala Ecosystem

Overall, the mission of the Scala Center is to improve the experience of becoming and being a Scala developer, and to help the community to build a rich ecosystem of libraries.

The Scala ecosystem is made of the following pillars:

• Language, compiler, standard library: they are the core tools that all programmers interact with when they work in Scala. The role of the Scala Center is to reduce the number of bugs in the compiler implementation, to help the community to contribute to these tools, and to make sure they evolve in a way that takes into account the needs of the community.
• Documentation and MOOCs: this website is the entry point to the ecosystem. It showcases the strengths of the language and its use-cases, and it hosts all its documentation. The role of the Scala Center is to simplify the structure and the content of the website, to create and maintain high-quality online educational content (including online courses), and to help the community to contribute to the website.
• Developer experience: Scala programmers often don’t interact directly with the compiler, but they use a tool (build tool, compile server) that does that for them. They also use tools to edit, analyze, navigate through, transform, compile, run, and debug Scala programs. The role of the Scala Center is to make sure these tools are as easy to use as possible, that they work reliably for everyone, and deliver a great developer experience.
• Community and contributor experience: the last pillar is the result of the work done outside the Scala Center. The community has created thousands of projects that bring simple solutions to complex problems. The role of the Scala Center is to create the best environment for the emergence of such libraries.

The remainder of this article summarizes our achievements in those four areas in 2022, and then presents our goals going forward.

Highlights of 2022

In this section, we highlight our main achievements in 2022. You can find our complete activity reports on our website.

Language, Compiler, Standard Library

We overhauled and restarted the Scala Improvement Process. Since last summer, the SIP Committee members meet monthly to decide the evolution of the language. As we explained in the announcement, the main changes of the new process are the following. Firstly, the proposals are thoroughly discussed by a team of three reviewers, publicly on GitHub before the whole Committee votes on them during their monthly meetings. Second, there is a new stage where proposals are accepted as experimental features before the Committee votes again to promote them to stable features. The new Scala Improvement Process is the first result of our governance strategy whose goal is to establish all the necessary decision-making processes for the community, by the community.

We improved meta-programming, especially generic Mirror synthesis. Mirrors are critical to metaprogramming in Scala 3, they enable implementation of type class derivation without advanced metaprogramming techniques such as macros. Mirror synthesis now supports more cases (generic tuples, local and inner classes), is more reliable (dotty#15279, dotty#15404, dotty#15814), and provides more detailed explanations in case of failure.

We generalized the definition of methods in Scala by allowing type parameters to be interleaved with regular parameters. This change notably allows methods to take type parameters whose bounds depend on value parameters. You can learn more about the motivation behind this change in the proposal document: SIP-47 - Clause Interleaving. Lastly, we lifted a current language restriction that prevents eta-expansion to be applied to polymorphic methods. You can learn more about this change in the proposal document: SIP-49 - Polymorphic Eta-Expansion. Both proposals have been accepted by the SIP Committee and are under development in the compiler.

Documentation and MOOCs

We improved the usability of the Scala website (this website). The website had not seen a major rework since 2013, and it does not represent the full picture that we would like Scala newcomers to see. To this effect, we have removed the split of Scala 2 vs Scala 3 on the landing page and in the documentation, consolidated the “Getting Started” instructions (scala-lang#1344, scala-lang#1348, scala-lang#1352, scala-lang#1373, docs.scala-lang#2388, docs.scala-lang#2520), improved the contributing guide, and created an infrastructure to show both Scala 2 and Scala 3 versions of code examples in the documentation (docs.scala-lang#2450). We created an issue to coordinate the update of the whole documentation to always show both Scala 2 and Scala 3 code examples when possible.

We offered individualized support to our MOOCs’ learners. We published the course “Effective Programming in Scala” to the Extension School platform. This platform allows the learners to have regular 1-on-1 meetings with our instructors along their learning journey. If you need to learn Scala, or if your company needs to train developers, the Extension School provides a cost-effective way to be trained by the Scala Center team. Learn more about the motivation in the previous announcement.

Developer Experience

We improved the debugger for Scala 3. The Metals’ debugger is now able to evaluate arbitrary Scala 3 expressions while the debugger is paused in the middle of a program. Step-by-step execution now automatically skips the methods generated by the compiler (e.g. mix-in forwarders, getters, setters, bridge methods, and synthetic methods of case classes). Learn more in this video.

We fixed the apps installed by the setup instructions. The command-line apps sbt and scala now behave consistently regardless of the installation process (cs install vs manual download of the release artifacts).

Community and Contributor Experience

We co-organized the ScalaCon online conference and participated in several in-person community events. In April 2022, for the first time in 2.5 years, events could finally take place in person. Our team encouraged many Scala User Groups to restart their regular Scala Meetups. We gave talks and organized Open-Source Sprees at local events in Europe. We were also involved in the online conference ScalaCon: we were in charge of the program and the program committee, and we presented several talks including a keynote Towards a Healthy & Resilient Scala Community where we presented our strategy for the governance of Scala. The talk videos of the whole conference are available online here.

We published videos about Scala 3. Let’s Talk About Scala 3 is a series of videos where we share interesting, useful, and cool things related to Scala 3. We published 4 new videos in 2022.

We improved the code editing experience in Scastie. Scastie is the online Scala code editor that is used every day by one thousand developers to share code snippets in the community. We have implemented “IDE features” such as autocompletion and showing type information under the mouse pointer. Under the hood, we reused some components of Metals.

We mentored new contributors to the Scala 3 compiler. The Compiler Academy organizes online pair-programming events to mentor new contributors on the compiler codebase. You can learn more about the Compiler Academy in this blog article.

We implemented the building blocks of a new tool to detect incompatibilities between programs. Guaranteeing the absence of incompatibilities between library dependencies is the cornerstone of a seamless Scala ecosystem. We have implemented a prototype of a new tool to detect incompatibilities between two versions of a program (like MiMa, but based on the TASTy representation of programs instead of JVM bytecode). We explained here why we need this tool in addition to the classic MiMa. Our work is based on TASTy Query, which will be the basis of tools to perform static analysis of Scala programs (it is already used by the Metals debugger).

Roadmap for 2023 (and Beyond)

It is now time to look forward and present our goals for 2023. Our mission remains the same: improve the experience of becoming and being a Scala developer, and help the community to build a rich ecosystem of libraries. Ultimately, our work will ensure the longevity of Scala by creating a reliable and resilient ecosystem.

We have identified the priorities for 2023 through our discussions with the community (online or at conferences), with our Advisory Board members, and with the main organizations that are behind Scala (LAMP, Lightbend, and VirtusLab). We are grateful to all of them.

In the following subsections we remind you of our ongoing and recurring projects, and we present our most important goals as well as some additional stretch goals that would need more resources.

The roadmap we present here is of course subject to adjustments throughout the year.

Language, Compiler, Standard Library

In addition to keeping the Scala Improvement Process ongoing, and reducing the number of bugs in the compiler and standard library, we will:

• Make the compiler error messages clearer and more actionable. The compiler should help developers write correct code instead of “just complaining” about incorrect code. The compiler should also guide you to write maintainable code (ie, it should provide linting features). You can find a list of related issues here.
• Bring the compiler’s suggestions into the IDEs. Instead of emitting only text output, the compiler will produce structured output that IDEs (Metals and IntelliJ) will be able to read to automatically provide “quick fixes” to the users. This should reduce the implementation (and maintenance) costs on the IDE side.
• Unfreeze the Scala standard library. The standard library has not changed (except for bug fixes or some performance improvements) since Scala 2.13, which was released almost four years ago. Dropping the forwards binary compatibility requirement would be a first step to allow the introduction of new classes or the addition of new members to the existing classes. Then, we will create a process to validate what should be in the standard library and what should stay outside of it.
• Clarify and communicate the Scala 3 roadmap. We will coordinate with the compiler team to clarify what will go into the Long-Term Support version of Scala 3, and make sure that roadmap is well communicated. We will focus primarily on the items that will enable more users to migrate from Scala 2 to Scala 3 (e.g., missing compiler options).

Our stretch goals include: publish a formal specification for Scala 3, investigate opportunities for reducing compilation times, implement multi-threading in Scala Native, and remove the dependency to the Google Closure Compiler in Scala.js.

Documentation and MOOCs

In addition to maintaining the Scala website and managing our online course learners, we will:

• Add “task-oriented” content to the documentation. Currently, the documentation is rather “language features oriented”, but it does not really help solve concrete problems. We will select a set of libraries (the “Scala Toolkit”) and we will write tutorials showing how to perform common programming tasks (working with JSON, writing tests, etc.). We will use libraries created by the community and that do not require advanced knowledge of Scala. Lastly, we will help the authors of those libraries to perform the usual maintenance tasks.
• Improve the content of the course Effective Programming in Scala. We will address the feedback we received from the learners of the course to smooth the learning curve.

Our stretch goals include: update our Spark course, teach Scala to first year Bachelor students at EPFL, and create a new Scala course for beginners (with no experience at all in programming).

Developer Experience

In addition to maintaining some core tools of the ecosystem (Bloop, Coursier, sbt, scala-debug-adapter), we will:

• Streamline the distribution of cs (Coursier). The current setup process is not well integrated with the various operating systems. There are no packages for Linux distributions, no MSI installer for Windows, and no PKG or DMG installer for macOS.
• Reduce the number of ways to import Scala projects in IDEs. Currently, IDEs support several ways for importing Scala projects (via sbt or via a build server, which itself can be provided by bloop or the underlying build tool). This is a source of confusion for the users, and it increases the maintenance costs.
• Make Metals work with more versions of Scala. By creating a stable interface for the presentation compiler, Metals will be less tied to the release cycles of the compiler, and it will easier to use it to develop the compiler itself.
• Improve the Scala 3 support on Metals and IntelliJ. We heard many developers stating that the Scala 3 support in IDEs was not at the same level as Scala 2, and that is one of the blockers to migrate to Scala 3. We will identify and put weigh on the main issues that currently degrade the developer experience on Scala 3 projects.

Our stretch goals include: support standalone .scala files out of the box in Metals and IntelliJ (including support for Scala CLI configuration directives), investigate opportunities to reduce build times, reduce the maintenance costs of implementing the debugSession/start endpoint in all the build servers by running the debug server in Metals, update scalameta to Scala 3, publishing sbt 2, implement a Scala 2 expression evaluator in scala-debug-adapter, be able to choose a specific build target in Metals, make sure Scala programs are easy to deploy especially in IaaS, and provide TASTy trees to semantic rules in Scalafix.

Community and Contributor Experience

In addition to participating to events, encouraging and supporting local Scala user groups and Scala conferences around the world, publishing positive content about Scala, leading the Compiler Academy, coordinating the Google Summer of Code, and maintaining tools for contributors (Scastie, Scaladex, scalafix), we will:

• Co-organize Scala Days. We will be responsible for the program and co-located events, such as ScalaBridge, Scala Spree, and other community summits. The conference will take place June 4th-7th in Seattle, USA, and September 11th-14th in Madrid, Spain.
• Organize a Tooling Summit. We will bring together the main actors of the tooling ecosystem (IntelliJ team, Metals team, compiler team, build tools maintainers), and agree on the next steps for a reliable and seamless integration of the pieces of the tooling ecosystem.
• Add support for standalone Scala programs in Scastie. Currently, Scastie creates an sbt project for every snippet. This adds a performance overhead, and makes it impossible to just copy-paste a snippet to run it locally (because you also need to re-create the sbt configuration locally). We will fix this issue by supporting a “standalone” mode where the configuration will be defined via Scala CLI directives.
• Publish TASTy-MiMa. This tool will be able to detect incompatibilities between two versions of a program, based on their TASTy representation. We will complete its implementation and make it available to the community.

Our stretch goals include: create a Scala Center online shop, finalize the training materials for Scala moderators, revise and update the Scala Center bylaws, stabilize and document the HTTP API of Scaladex, make it easy to find projects to contribute to in Scaladex, streamline the way to publish the documentation of Scala libraries, implement GitHub precise code navigation for Scala, and simplify the usage of remote caching techniques in continuous integration systems.

Conclusion

In this article, we have looked at the pillars of the Scala ecosystem, and for each of them we have listed the main outcome of our work in 2022, and our main goals for 2023.

Thanks to your support, and with the help of all the people behind Scala, we came this far! Help us go even further by supporting the Scala Center.

You can find our detailed roadmap for the current quarter here, and track our progress by looking at our quarterly reports, or by browsing the Scala Center Updates category of the Scala Contributors forum.