Proguard shrinking and Scala collection library [was 'a kill to a view']

On Sat, Dec 17, 2011 at 9:06 AM, Johannes Rudolph <johannes.rudolph@googlemail.com> wrote:

In general, it seems to me that shrinking is really hard (probably
even undecidable?) for a language with virtual calls. Especially,
Scala's collections class hierarchy appears to be a particular hard
case because it really needs and uses virtual calls all over the
place. From what I've seen proguard's shrinker isn't sophisticated
enough to get even near an optimal solution in those cases.

One thing which has been pointed out is that every call to every method in a module is potentially side-effecting, because even if the bodies are pure, the objects are lazily loaded - they are implemented as instance methods of the MODULE$ instance instead of at the static level.  So at the level of sophistication applied by proguard, none of them can ever be eliminated, and then this probably has a bunch of additional spinout in lost cumulative optimizations.

Also note that any code which uses structural typing will be silently broken by ProGuard.

For example, innocently looking "times" sugar recently adviced in scala-user:
implicit def liftToTimes(a: Int) = new {
  def times[A](f: => A) = 1 to a foreach { _ => f }
}
10 times println("hello world!") // Oops! reflection call to "times" there
Proguard may freely rename "times" method in synthetic class returned by "liftToTimes" here (and it actually does it).

I fear that with current structural types implementation using Proguard for Scala is just a way to inject subtle bugs into the code.
For now, it would be good at least to have a compiler option to produce the list of such classes/methods to help feeding them manually into proguard projects.

Quick grep locates one such place in scala-library.jar nightly build:

class ParHashMapCombiner, method result:
    val table = new HashTable[K, DefaultEntry[K, V]] {
      def insertEntry(e: DefaultEntry[K, V]) = ...
    }
    ...
    for (elem <- buckets(i)) table.insertEntry(elem)

So, mutable.ParHashMap will not work after being processed by ProGuard.

In 2.9.1 library there is one more such place in class scala/reflect/generic/UnPickler.

On Tue, Dec 27, 2011 at 12:49 AM, Pavel Pavlov <pavel.e.pavlov@gmail.com> wrote:

class ParHashMapCombiner, method result:
    val table = new HashTable[K, DefaultEntry[K, V]] {
      def insertEntry(e: DefaultEntry[K, V]) = ...
    }
    ...
    for (elem <- buckets(i)) table.insertEntry(elem)

% pscalac -Ywarn-reflective-call src/library/scala/collection/parallel/mutable/ParHashMap.scalasrc/library/scala/collection/parallel/mutable/ParHashMap.scala:200: warning: method invocation uses reflection         for (elem <- buckets(i)) table.insertEntry(elem)                                                  ^one warning found
Looks like that's the only one in the library outside of a few in swing.  But hmm, here's something interesting/bad.  This is the buzzkill inflicted by
    type AbstractFileType = AbstractFile
(The irony is that AbstractFile is the concrete file type... concretely abstract.)
% pscalac -Ywarn-reflective-call src/compiler/scala/reflect/internal/Symbols.scala  src/compiler/scala/reflect/internal/Symbols.scala:1505: warning: method invocation uses reflection          if (this.sourceFile.path != that.sourceFile.path) {                              ^ src/compiler/scala/reflect/internal/Symbols.scala:1505: warning: method invocation uses reflection          if (this.sourceFile.path != that.sourceFile.path) {                                                      ^ src/compiler/scala/reflect/internal/Symbols.scala:1508: warning: method invocation uses reflection            if (this.sourceFile.canonicalPath != that.sourceFile.canonicalPath)                                ^ src/compiler/scala/reflect/internal/Symbols.scala:1508: warning: method invocation uses reflection            if (this.sourceFile.canonicalPath != that.sourceFile.canonicalPath)                                                                 ^ src/compiler/scala/reflect/internal/Symbols.scala:2543: warning: method invocation uses reflection    "  Found in " + sym1.sourceFile.canonicalPath + " and " + sym2.sourceFile.canonicalPath                                     ^src/compiler/scala/reflect/internal/Symbols.scala:2543: warning: method invocation uses reflection    "  Found in " + sym1.sourceFile.canonicalPath + " and " + sym2.sourceFile.canonicalPath                                                                               ^6 warnings found