@ScalaSignature. Is it just wasting a significant chunk of jar size or is it useful?

@ScalaSIgnature is Scala's new mechanism of storing the scala-related important bits of method signatures inside your class file.   By storing it in an annotation, it opens the possibility for Scala-aware reflection libraries.    This change happened between Beta + RC.

I'm pretty sure you can't remove it *yet*, as that would mean no one could compile against your library intelligently.   Perhaps some work can be done for tools like proguard to be able to remove this for completed applications where it's no longer needed.

- Josh

On Wed, Jun 9, 2010 at 9:31 PM, Lex <lexn82@gmail.com> wrote:

The size of my library went up from 400k-500k to 600k+ after switching
from beta1 to RC3. It seem that every single class is a little bit
bigger, and the main difference I see in decompiled code is a presense
of @ScalaSignature. The big problem is that @ScalaSignature contains
seemingly random bits that do not compress at all. I have spend many
hours reducing the size of my library and this sudden increase in size
is a big slap in the face.

My question is: do I absolutely need @ScalaSignature? And if no, is
there a compiler switch to remove it?

Thanx,
Lex

Hello Lex,
Hello List,

As Josh rightly points out, ScalaSignatures store Scala public
interfaces (pickled symbols). You can find a document explaining their
use, and why they changed, here:

http://www.scala-lang.org/sid/10

You need a ScalaSignatures in a class to
1. compile against it;
2. in the future, use Scala reflection (Java reflection obviously
doesn't need it).

In other words, you can get rid of it in many cases.

Question for the list: Would a compiler flag that inhibits generation
of ScalaSignature annotations (thereby preventing the two uses above)
be useful? The advantage of such a flag to Lex and other people is
obvious, but keep in mind the drawbacks too:

1. It is non-trivial to test if a library has ScalaSignatures or not
(it requires decompiling the classes), and therefore if it can be
compiled against. For the time being, any library can be compiled
against without thinking about it. This creates an annoying new class
of errors.

2. In situations were library size is critical (Android applications
for example), most developers would use something similar to ProGuard
or yGuard to reduce library sizes. If this is run in any case, it may
be simpler and safer to strip ScalaSignatures at that point.

Overall, I am not much in favour of a flag that inhibits generation of
ScalaSignature, but I'd be happy to hear your input.

Cheers,
Gilles.

> @ScalaSIgnature is Scala's new mechanism of storing the scala-related
> important bits of method signatures inside your class file.   By
> storing it in an annotation, it opens the possibility for Scala-aware
> reflection libraries.    This change happened between Beta + RC.
>
> I'm pretty sure you can't remove it *yet*, as that would mean no one
> could compile against your library intelligently.   Perhaps some work
> can be done for tools like proguard to be able to remove this for
> completed applications where it's no longer needed.
>
>> The size of my library went up from 400k-500k to 600k+ after switching
>> from beta1 to RC3. It seem that every single class is a little bit
>> bigger, and the main difference I see in decompiled code is a presense
>> of @ScalaSignature. The big problem is that @ScalaSignature contains
>> seemingly random bits that do not compress at all. I have spend many
>> hours reducing the size of my library and this sudden increase in size
>> is a big slap in the face.
>>
>> My question is: do I absolutely need @ScalaSignature? And if no, is
>> there a compiler switch to remove it?

Thank your for an informative response.

Since @ScalaSignature is required to compile against the library, the
compiler should not be stripping it. Perhaps a separate dedicated
commnad line tool could strip jars en-mass. It should be trivial to
write a tool that strips @ScalaSignature using asm or another bytecode
manipulation library.

Using @ScalaSignature for scala reflection on the other hand is a
cause for concern. How is this going to affect structural types? Will
the scala reflection honour the custom scala visibility rules, and if
so, how is it going to change the performance? These questions will
probably be answered when the implementation is near completion.

The most important question would be: Is the reflection call going to
fail when @ScalaSignature is not present? If so, then @ScalaSignature
cannot be safely removed.

Sincerely,
Lex

On Thu, Jun 10, 2010 at 4:09 AM, Gilles Dubochet wrote:
> Hello Lex,
> Hello List,
>
>
> As Josh rightly points out, ScalaSignatures store Scala public interfaces
> (pickled symbols). You can find a document explaining their use, and why
> they changed, here:
>
> http://www.scala-lang.org/sid/10
>
> You need a ScalaSignatures in a class to
>  1. compile against it;
>  2. in the future, use Scala reflection (Java reflection obviously doesn't
> need it).
>
> In other words, you can get rid of it in many cases.
>
>
> Question for the list: Would a compiler flag that inhibits generation of
> ScalaSignature annotations (thereby preventing the two uses above) be
> useful? The advantage of such a flag to Lex and other people is obvious, but
> keep in mind the drawbacks too:
>
>  1. It is non-trivial to test if a library has ScalaSignatures or not (it
> requires decompiling the classes), and therefore if it can be compiled
> against. For the time being, any library can be compiled against without
> thinking about it. This creates an annoying new class of errors.
>
>  2. In situations were library size is critical (Android applications for
> example), most developers would use something similar to ProGuard or yGuard
> to reduce library sizes. If this is run in any case, it may be simpler and
> safer to strip ScalaSignatures at that point.
>
> Overall, I am not much in favour of a flag that inhibits generation of
> ScalaSignature, but I'd be happy to hear your input.
>
>
> Cheers,
> Gilles.
>
>> @ScalaSIgnature is Scala's new mechanism of storing the scala-related
>> important bits of method signatures inside your class file.   By storing it
>> in an annotation, it opens the possibility for Scala-aware reflection
>> libraries.    This change happened between Beta + RC.
>>
>> I'm pretty sure you can't remove it *yet*, as that would mean no one could
>> compile against your library intelligently.   Perhaps some work can be done
>> for tools like proguard to be able to remove this for completed applications
>> where it's no longer needed.
>>
>>> The size of my library went up from 400k-500k to 600k+ after switching
>>> from beta1 to RC3. It seem that every single class is a little bit
>>> bigger, and the main difference I see in decompiled code is a presense
>>> of @ScalaSignature. The big problem is that @ScalaSignature contains
>>> seemingly random bits that do not compress at all. I have spend many
>>> hours reducing the size of my library and this sudden increase in size
>>> is a big slap in the face.
>>>
>>> My question is: do I absolutely need @ScalaSignature? And if no, is
>>> there a compiler switch to remove it?
>
>
>

Hello,

> Since @ScalaSignature is required to compile against the library, the
> compiler should not be stripping it. Perhaps a separate dedicated
> commnad line tool could strip jars en-mass. It should be trivial to
> write a tool that strips @ScalaSignature using asm or another bytecode
> manipulation library.

I agree.

> Using @ScalaSignature for scala reflection on the other hand is a
> cause for concern. How is this going to affect structural types?

It won't. Structural method calls are implemented internally using Java
reflection, but that is happening in the Scala compiler backend anyway
and relies on the standard Java reflection library, which will not
disapear.

Maybe this is not clear to many people, but the fact that structural
types are implemented using Java reflection is an implementation
detail. There is usually no need to know, far less to worry about it.
Thanks to agressive caching, performance is not really an issue. And
whilst this implementation creates limitations in structural types, it
is easy to see these limitations as intrisinc to structural types,
without understanding their link to the implementation (which is
tricky).

> Will the scala reflection honour the custom scala visibility rules, and if
> so, how is it going to change the performance?

Scala reflection will give a "correct" reflective view of Scala
programs (i.e. corresponding to the Scala code that is being
reflected), including visibility rules.

Yes, there will be a performance penalty for using the Scala reflection
library. Users who don't want to pay this penalty will continue to use
Java reflection. Scala reflection is a layer on top of Java reflection
(and of the Scala compiler), but does not replace Java reflection. Note
also that, since Java reflection will still be available to bypass
Scala visibilty rules, you will not be able to use such rules to
enforce strong access protection.

> These questions will probably be answered when the implementation is
> near completion.
>
> The most important question would be: Is the reflection call going to
> fail when @ScalaSignature is not present? If so, then @ScalaSignature
> cannot be safely removed.

The call will not fail, but the class will be treated as if it were a
Java class (erased types). Basically, you can still do Scala
reflection, but roughly with the same level of ease as if you were
using Java reflection on Scala programs.

As I explained in the SID that I referenced in my previous answer, the
reason to change the signature storage format now is to make 2.8
binaries compatible with the yet-to-be-released Scala reflection
library. It is only if you want binaries compiled today to be
compatible with the unreleased Scala reflection library that you must
not remove ScalaSignatures. In most cases, it is easy to recompile your
code if you find that you want to use Scala reflection in the future.

Cheers,
Gilles.

>> As Josh rightly points out, ScalaSignatures store Scala public interfaces
>> (pickled symbols). You can find a document explaining their use, and why
>> they changed, here:
>>
>> http://www.scala-lang.org/sid/10
>>
>> You need a ScalaSignatures in a class to
>>  1. compile against it;
>>  2. in the future, use Scala reflection (Java reflection obviously doesn't
>> need it).
>>
>> In other words, you can get rid of it in many cases.
>>
>>
>> Question for the list: Would a compiler flag that inhibits generation of
>> ScalaSignature annotations (thereby preventing the two uses above) be
>> useful? The advantage of such a flag to Lex and other people is obvious,but
>> keep in mind the drawbacks too:
>>
>>  1. It is non-trivial to test if a library has ScalaSignatures or not (it
>> requires decompiling the classes), and therefore if it can be compiled
>> against. For the time being, any library can be compiled against without
>> thinking about it. This creates an annoying new class of errors.
>>
>>  2. In situations were library size is critical (Android applications for
>> example), most developers would use something similar to ProGuard or yGuard
>> to reduce library sizes. If this is run in any case, it may be simpler and
>> safer to strip ScalaSignatures at that point.
>>
>> Overall, I am not much in favour of a flag that inhibits generation of
>> ScalaSignature, but I'd be happy to hear your input.
>>
>>
>> Cheers,
>> Gilles.
>>
>>> @ScalaSIgnature is Scala's new mechanism of storing the scala-related
>>> important bits of method signatures inside your class file.   By storing it
>>> in an annotation, it opens the possibility for Scala-aware reflection
>>> libraries.    This change happened between Beta + RC.
>>>
>>> I'm pretty sure you can't remove it *yet*, as that would mean no one could
>>> compile against your library intelligently.   Perhaps some work canbe done
>>> for tools like proguard to be able to remove this for completed applications
>>> where it's no longer needed.
>>>
>>>> The size of my library went up from 400k-500k to 600k+ after switching
>>>> from beta1 to RC3. It seem that every single class is a little bit
>>>> bigger, and the main difference I see in decompiled code is a presense
>>>> of @ScalaSignature. The big problem is that @ScalaSignature contains
>>>> seemingly random bits that do not compress at all. I have spend many
>>>> hours reducing the size of my library and this sudden increase in size
>>>> is a big slap in the face.
>>>>
>>>> My question is: do I absolutely need @ScalaSignature? And if no, is
>>>> there a compiler switch to remove it?