This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Compile to DalvikVM

27 replies
chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.

Does anybody know if there is any effort to get Scala to compile
natively to the DalvikVM? I was thinking of trying to write such a
backend for the Scala compiler. However, I wanted to make sure this
isn't something that is already in progress.

paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 29 Nov 2011, at 05:29, Chris Dow wrote:
> Does anybody know if there is any effort to get Scala to compile
> natively to the DalvikVM? I was thinking of trying to write such a
> backend for the Scala compiler. However, I wanted to make sure this
> isn't something that is already in progress.

Chris,

This sounds like it would be an interesting project :-)

I wonder what benefit you see from such a compiler? Right now, the primary problem (and it's a *big* problem) with Scala on Android is that of fitting the Scala libraries into the limits imposed by the Dalvik VM as discussed in this thread:

https://groups.google.com/forum/#!topic/scala-debate/El3QZEbIwHg

Do you think that a native code generator will help with this?

We make heavy use of Scala on Android for testing and would love to start doing so for production code, but really don't feel able to do so with these problems still extant :-(

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Miguel Garcia 2
Joined: 2011-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM


I don't have a lot of experience with the DalvikVM, but I think that "fitting the Scala libraries into it" is an orthogonal problem to that mentioned by Chris. The Scala+GWT project faced a similar problem, and settled down on an automated source-to-source conversion of the libraries, inspired by another source-to-source converter [1]. Details about this on the Scala+GWT google group.

Regarding compiling for the DalvikVM, there's related work on transforming one of the backend-ish internal representations (IRs) of the Scala compiler into the instruction set of a new platform ([2], among others). In some cases, rather than taking the ICode IR as starting point, a C-like IR can prove more useful (probably when targeting Dalvik). Two prototypes for this are described in [3] and [4].

regards,

Miguel


[1] A source-level conversion tool to migrate Scala programs from JDK to .NET,
    http://lamp.epfl.ch/~magarcia/jdk2ikvm/

[2] http://greedy.github.com/scala-llvm/

[3] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/2011Q2/Moving3A.pdf

[4] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/2011Q4/PartialEval3A.pdf


paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 29 Nov 2011, at 10:14, Miguel Garcia wrote:
> I don't have a lot of experience with the DalvikVM, but I think that "fitting the Scala libraries into it" is an orthogonal problem to that mentioned by Chris.

That would be my expectation too.

I'd hate to see Chris invest a lot of effort in creating a Dalvik back-end for the compiler and end up with a chocolate teapot. Dalvik binaries are of little use if one can only run toy examples before exceeding the limits of the platform...

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

I appreciate the concern. I know there are some definite size issues
with the Scala standard libraries on the DalvikVM. However, there are
potential workarounds (like using a subset of the Scala libraries).

My motivation isn't entirely about being able to run the Scala
compiler on Android (although that is a nice side effect). I'm
largely looking in to this as part of an effort to improve the
development experience using Scala on Android. The build time for
Scala projects on Android can be rather long since code has to pass
though the compiler (which is already slower than Java) and then it
has to be dexed. By cutting out a step we could get a much nicer
experience.

On Nov 29, 12:42 am, Paul Butcher

wrote:
> On 29 Nov 2011, at 05:29, Chris Dow wrote:
>
> > Does anybody know if there is any effort to get Scala to compile
> > natively to the DalvikVM?  I was thinking of trying to write such a
> > backend for the Scala  compiler.  However, I wanted to make sure this
> > isn't something that is already in progress.
>
> Chris,
>
> This sounds like it would be an interesting project :-)
>
> I wonder what benefit you see from such a compiler? Right now, the primary problem (and it's a *big* problem) with Scala on Android is that of fitting the Scala libraries into the limits imposed by the Dalvik VM as discussed in this thread:
>
> https://groups.google.com/forum/#!topic/scala-debate/El3QZEbIwHg
>
> Do you think that a native code generator will help with this?
>
> We make heavy use of Scala on Android for testing and would love to start doing so for production code, but really don't feel able to do so with these problems still extant :-(
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn:http://www.linkedin.com/in/paulbutcher
> MSN: p...@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

I'll have to check out those links. Thanks!

On Nov 29, 2:14 am, Miguel Garcia wrote:
> I don't have a lot of experience with the DalvikVM, but I think that
> "fitting the Scala libraries into it" is an orthogonal problem to that
> mentioned by Chris. The Scala+GWT project faced a similar problem, and
> settled down on an automated source-to-source conversion of the libraries,
> inspired by another source-to-source converter [1]. Details about this on
> the Scala+GWT google group.
>
> Regarding compiling for the DalvikVM, there's related work on transforming
> one of the backend-ish internal representations (IRs) of the Scala compiler
> into the instruction set of a new platform ([2], among others). In some
> cases, rather than taking the ICode IR as starting point, a C-like IR can
> prove more useful (probably when targeting Dalvik). Two prototypes for this
> are described in [3] and [4].
>
> regards,
>
> Miguel
>
> [1] A source-level conversion tool to migrate Scala programs from JDK to
> .NET,
>    http://lamp.epfl.ch/~magarcia/jdk2ikvm/
>
> [2]http://greedy.github.com/scala-llvm/
>
> [3]http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/2011Q2/Movi...
>
> [4]http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/2011Q4/Part...

paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 30 Nov 2011, at 07:02, Chris Dow wrote:
> I appreciate the concern. I know there are some definite size issues
> with the Scala standard libraries on the DalvikVM. However, there are
> potential workarounds (like using a subset of the Scala libraries).

Are there? Right now, I'm not aware of any way of getting a non-trivial Scala app which uses the 2.9.x libraries working on Dalvik :-(

> My motivation isn't entirely about being able to run the Scala
> compiler on Android (although that is a nice side effect). I'm
> largely looking in to this as part of an effort to improve the
> development experience using Scala on Android. The build time for
> Scala projects on Android can be rather long since code has to pass
> though the compiler (which is already slower than Java) and then it
> has to be dexed. By cutting out a step we could get a much nicer
> experience.

I completely agree that these are problems that it would be beneficial to solve. But until the library size issue is solved, everything else is moot, isn't it?

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Simon Ochsenreither
Joined: 2011-07-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM
A patch shrinking the library by 1.5 MB landed in trunk recently. Did that improve things?
paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 30 Nov 2011, at 14:38, Simon Ochsenreither wrote:
> A patch shrinking the library by 1.5 MB landed in trunk recently. Did that improve things?

Ooh - I was entirely unaware of that. Can you point me at any additional details?

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Johannes Rudolph 2
Joined: 2010-02-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM

I think that referred to that thread (and the resulting changes):

https://groups.google.com/d/topic/scala-language/YJjA7BV3NWs/discussion

On Wed, Nov 30, 2011 at 4:08 PM, Paul Butcher wrote:
> On 30 Nov 2011, at 14:38, Simon Ochsenreither wrote:
>> A patch shrinking the library by 1.5 MB landed in trunk recently. Did that improve things?
>
>
> Ooh - I was entirely unaware of that. Can you point me at any additional details?
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn: http://www.linkedin.com/in/paulbutcher
> MSN: paul@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher
>

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

Can't you just use Proguard or some similar tool to cut down the size
of the 2.9 libraries? I admit I have only written small Android apps
using Scala, but I would think you could remove a large portion of the
Scala libraries and still be able leverage the language. Am I missing
something?

On Nov 30, 5:47 am, Paul Butcher

wrote:
> On 30 Nov 2011, at 07:02, Chris Dow wrote:
>
> > I appreciate the concern.  I know there are some definite size issues
> > with the Scala standard libraries on the DalvikVM.  However, there are
> > potential workarounds (like using a subset of the Scala libraries).
>
> Are there? Right now, I'm not aware of any way of getting a non-trivial Scala app which uses the 2.9.x libraries working on Dalvik :-(
>
> > My motivation isn't entirely about being able to run the Scala
> > compiler on Android (although that is a nice side effect).  I'm
> > largely looking in to this as part of an effort to improve the
> > development experience using Scala on Android.  The build time for
> > Scala projects on Android can be rather long since code has to pass
> > though the compiler (which is already slower than Java) and then it
> > has to be dexed.  By cutting out a step we could get a much nicer
> > experience.
>
> I completely agree that these are problems that it would be beneficial to solve. But until the library size issue is solved, everything else is moot, isn't it?
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn:http://www.linkedin.com/in/paulbutcher
> MSN: p...@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher

Adam Jorgensen
Joined: 2011-04-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Compile to DalvikVM
Indeed, as I understand it, Proguard is somewhat of a necessity for Scala on Android...

On 1 December 2011 09:05, Chris Dow <chris.dow8@gmail.com> wrote:
Can't you just use Proguard or some similar tool to cut down the size
of the 2.9 libraries?  I admit I have only written small Android apps
using Scala, but I would think you could remove a large portion of the
Scala libraries and still be able leverage the language.  Am I missing
something?

On Nov 30, 5:47 am, Paul Butcher <p...@paulbutcher.com> wrote:
> On 30 Nov 2011, at 07:02, Chris Dow wrote:
>
> > I appreciate the concern.  I know there are some definite size issues
> > with the Scala standard libraries on the DalvikVM.  However, there are
> > potential workarounds (like using a subset of the Scala libraries).
>
> Are there? Right now, I'm not aware of any way of getting a non-trivial Scala app which uses the 2.9.x libraries working on Dalvik :-(
>
> > My motivation isn't entirely about being able to run the Scala
> > compiler on Android (although that is a nice side effect).  I'm
> > largely looking in to this as part of an effort to improve the
> > development experience using Scala on Android.  The build time for
> > Scala projects on Android can be rather long since code has to pass
> > though the compiler (which is already slower than Java) and then it
> > has to be dexed.  By cutting out a step we could get a much nicer
> > experience.
>
> I completely agree that these are problems that it would be beneficial to solve. But until the library size issue is solved, everything else is moot, isn't it?
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn:http://www.linkedin.com/in/paulbutcher
> MSN: p...@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher

paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 1 Dec 2011, at 07:05, Chris Dow wrote:
> Can't you just use Proguard or some similar tool to cut down the size
> of the 2.9 libraries? I admit I have only written small Android apps
> using Scala, but I would think you could remove a large portion of the
> Scala libraries and still be able leverage the language. Am I missing
> something?

Proguard is absolutely essential to running Scala code on Android, yes. And when using Scala 2.8.x it works pretty well.

The problem (described in detail in the e-mail thread I referenced earlier) is that Proguard *fails* to reduce the libraries to a small enough size to fit within Dalvik's limits when using the 2.9.x libraries. I don't think that we've yet identified the reason for this (there have been various suggestions - such as specialisation - but nothing definitive yet as far as I know).

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

I think I understand now. Thanks for clearing that up. Apparently I
did not read that thread close enough.

It seems like the current issue that is occurring with the 2.9
libraries is a symptom of a larger issue. Whenever we try to run
Scala on Android we have to run it through 2 phases. First we compile
it to bytecode that is optimised for what the JVM can handle. Then we
dex it using a tool that is optimised for generating Dalvik bytecodes
from JVM bytecodes that are using Java idioms. We are running the
Scala code thorough 2 tools that are optimised for things other than
what we are trying to accomplish. This seems to give slightly
unpredictable results.

I would think that the best way to improve this situation by creating
Dalvik backend for the Scala compiler. This would give us greater
control over the how we create Dalvik binaries. Otherwise we are left
at the mercy of how the Scala compiler and and the dexer interact.

On Dec 1, 2:47 am, Paul Butcher

wrote:
> On 1 Dec 2011, at 07:05, Chris Dow wrote:
>
> > Can't you just use Proguard or some similar tool to cut down the size
> > of the 2.9 libraries?  I admit I have only written small Android apps
> > using Scala, but I would think you could remove a large portion of the
> > Scala libraries and still be able leverage the language.  Am I missing
> > something?
>
> Proguard is absolutely essential to running Scala code on Android, yes. And when using Scala 2.8.x it works pretty well.
>
> The problem (described in detail in the e-mail thread I referenced earlier) is that Proguard *fails* to reduce the libraries to a small enough size to fit within Dalvik's limits when using the 2.9.x libraries. I don't think that we've yet identified the reason for this (there have been various suggestions - such as specialisation - but nothing definitive yet as far as I know).
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn:http://www.linkedin.com/in/paulbutcher
> MSN: p...@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher

paulbutcher
Joined: 2010-03-08,
User offline. Last seen 10 weeks 5 days ago.
Re: Compile to DalvikVM

On 2 Dec 2011, at 07:02, Chris Dow wrote:
> I would think that the best way to improve this situation by creating
> Dalvik backend for the Scala compiler. This would give us greater
> control over the how we create Dalvik binaries. Otherwise we are left
> at the mercy of how the Scala compiler and and the dexer interact.

OK - I can see how that might work :-)

I'm not sure that I'd like to bet on it until we've found a definitive root cause for what stops Proguard from successfully shrinking the 2.9.x libraries though?

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: paul@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Johannes Rudolph 2
Joined: 2010-02-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM

The main problem with Android is, that the leaf classes of the scala
collection have to many methods and a too deep inheritance hierarchy,
so that the fixed size "PermGen space" of the dalvikvm, the
"LinearAlloc" is overflodded with class definitions. It seems that
alone the size of the generated vtables for some classes of the
collections are too big for Android.

I don't see how a dalvik backend would help there. Dexing is basically
a somewhat direct translation from one bytecode form into another and
I know of no obvious optimization that would help remove the
"polymorphism induced burden" that Scala puts onto the VM it runs on.

BTW: The situation doesn't even change with the newly reduced size of
the collection class files mentioned earlier in this thread. If I find
some time on the weekend I can put together what I found out after I
wrote that post:
https://groups.google.com/forum/#!topic/scala-debate/El3QZEbIwHg

Johannes

On Fri, Dec 2, 2011 at 9:44 AM, Paul Butcher wrote:
> On 2 Dec 2011, at 07:02, Chris Dow wrote:
>> I would think that the best way to improve this situation by creating
>> Dalvik backend for the Scala compiler.  This would give us greater
>> control over the how we create Dalvik binaries.  Otherwise we are left
>> at the mercy of how the Scala compiler and and the dexer interact.
>
>
> OK - I can see how that might work :-)
>
> I'm not sure that I'd like to bet on it until we've found a definitive root cause for what stops Proguard from successfully shrinking the 2.9.x libraries though?
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn: http://www.linkedin.com/in/paulbutcher
> MSN: paul@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher
>

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

I admit that it is all just conjecture at this point. I think you're
right that I should probably wait until the cause of the issue with
the 2.9 libraries is solved.

On Dec 2, 12:44 am, Paul Butcher

wrote:
> On 2 Dec 2011, at 07:02, Chris Dow wrote:
>
> > I would think that the best way to improve this situation by creating
> > Dalvik backend for the Scala compiler.  This would give us greater
> > control over the how we create Dalvik binaries.  Otherwise we are left
> > at the mercy of how the Scala compiler and and the dexer interact.
>
> OK - I can see how that might work :-)
>
> I'm not sure that I'd like to bet on it until we've found a definitive root cause for what stops Proguard from successfully shrinking the 2.9.x libraries though?
>
> --
> paul.butcher->msgCount++
>
> Snetterton, Castle Combe, Cadwell Park...
> Who says I have a one track mind?
>
> http://www.paulbutcher.com/
> LinkedIn:http://www.linkedin.com/in/paulbutcher
> MSN: p...@paulbutcher.com
> AIM: paulrabutcher
> Skype: paulrabutcher

Miguel Garcia 2
Joined: 2011-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM

It would be useful to have a listing with all the different "max size limits" for Dalvik.
Things like, "max method references per dex file" [1] etc.

Miguel
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/

References:

[1] http://code.google.com/p/android/issues/detail?id=7147


chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

I would think that that the conversion from JVM byte code to Dalvik
wouldn't always be straight forward. Especially since the JVM is a
stack based VM and Dalvik is register based. This makes me think that
there could be opportunities to encode Scala features in a way that
wouldn't run in to as my Dalvik limitations. Most likely for features
that are aren't natively supported by the JVM (e.g. traits). All of
this largely depends on what primitives Dalvik provides and also if
there are alternative ways to encode Scala features.

If nothing else, this should give us more consistent Dalvik binaries
as a base to build from so we don't end up with increased issues when
we move from one release to another (like we appear to have with 2.9).

On Dec 2, 1:30 am, Johannes Rudolph
wrote:
> The main problem with Android is, that the leaf classes of the scala
> collection have to many methods and a too deep inheritance hierarchy,
> so that the fixed size "PermGen space" of the dalvikvm, the
> "LinearAlloc" is overflodded with class definitions. It seems that
> alone the size of the generated vtables for some classes of the
> collections are too big for Android.
>
> I don't see how a dalvik backend would help there. Dexing is basically
> a somewhat direct translation from one bytecode form into another and
> I know of no obvious optimization that would help remove the
> "polymorphism induced burden" that Scala puts onto the VM it runs on.
>
> BTW: The situation doesn't even change with the newly reduced size of
> the collection class files mentioned earlier in this thread. If I find
> some time on the weekend I can put together what I found out after I
> wrote that post:https://groups.google.com/forum/#!topic/scala-debate/El3QZEbIwHg
>
> Johannes
>
>
>
>
>
>
>
>
>
> On Fri, Dec 2, 2011 at 9:44 AM, Paul Butcher

wrote:
> > On 2 Dec 2011, at 07:02, Chris Dow wrote:
> >> I would think that the best way to improve this situation by creating
> >> Dalvik backend for the Scala compiler.  This would give us greater
> >> control over the how we create Dalvik binaries.  Otherwise we are left
> >> at the mercy of how the Scala compiler and and the dexer interact.
>
> > OK - I can see how that might work :-)
>
> > I'm not sure that I'd like to bet on it until we've found a definitive root cause for what stops Proguard from successfully shrinking the 2.9.x libraries though?
>
> > --
> > paul.butcher->msgCount++
>
> > Snetterton, Castle Combe, Cadwell Park...
> > Who says I have a one track mind?
>
> >http://www.paulbutcher.com/
> > LinkedIn:http://www.linkedin.com/in/paulbutcher
> > MSN: p...@paulbutcher.com
> > AIM: paulrabutcher
> > Skype: paulrabutcher
>
> --
> Johannes
>
> -----------------------------------------------
> Johannes Rudolphhttp://virtual-void.net

chris.dow8
Joined: 2011-11-24,
User offline. Last seen 49 weeks 4 days ago.
Re: Compile to DalvikVM

That would be a very good thing to have.

That link is very interesting. It seems to suggest that it is
possible to split code into multiple dex files for a single
application. The ability to do that could make things much easier.

On Dec 2, 8:26 am, Miguel Garcia wrote:
> It would be useful to have a listing with all the different "max size
> limits" for Dalvik.
> Things like, "max method references per dex file" [1] etc.
>
> Miguelhttp://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/
>
> References:
>
> [1]http://code.google.com/p/android/issues/detail?id=7147

Miguel Garcia 2
Joined: 2011-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM


< I've cross-posted this to another thread in scala-debate discussing this very same topic. Hmm perhaps we should start a discussion at scala-on-android >

Just in case this helps, compiling the scala lib sources with -no-specialization and -sourcepath results in jar size close to 6MB instead of the usual 8MB resulting from using specialization (the default) and -optimize (which inlines methods and thus in general increases size).

A further reduction should be possible by leaving out all parallel collection classes, which don't seem necessary on Android (if I remember correctly, Scala+GWT has a source-level conversion utility based on jdk2ikvm [1] to perform just that). Details in the Scala+GWT google group (I guess).

Another round of size reduction could result from "modularizing the Scala library".
  (a) In the Java world, a big jar file can be broken up into several smaller ones using tools like [2].
  (b) Two modularization efforts:
         https://github.com/scala-incubator/modularisation
         and
         https://github.com/paulp/scala-modular

Miguel
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/


References

[1] http://lamp.epfl.ch/~magarcia/jdk2ikvm/

[2] http://www.kirkk.com/main/Main/JarAnalyzer


Miguel Garcia 2
Joined: 2011-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Compile to DalvikVM

Chris,

Remember your question about a Dalvik backend? We just got to know [1] that some interfaces are redundant, increasing jar size.

Maybe you want to do sthg about it, e.g. in GenJVM:

      sortedClasses foreach { c =>
        try codeGenerator.genClass(c)
        catch {
          case e: JCode.CodeSizeTooBigException =>
            log("Skipped class %s because it has methods that are too long.".format(c))
        }
      }

more specifically:

      val ifaces = superInterfaces match {
        case Nil => JClass.NO_INTERFACES
        case _   => mkArray(superInterfaces map (x => javaName(x.typeSymbol)))
      }

Contributions are welcome :-) 

Miguel
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/


[1] https://groups.google.com/forum/#!msg/scala-debate/El3QZEbIwHg/duj-SA0iUPUJ


Johannes Rudolph 2
Joined: 2010-02-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Compile to DalvikVM

On Mon, Dec 5, 2011 at 4:11 PM, Miguel Garcia wrote:
> Remember your question about a Dalvik backend? We just got to know [1] that
> some interfaces are redundant, increasing jar size.
>
> Maybe you want to do sthg about it, e.g. in GenJVM:

Thanks Miguel for getting this started.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: Compile to DalvikVM

On Mon, Dec 5, 2011 at 7:11 AM, Miguel Garcia wrote:
> Contributions are welcome :-)

Also for anyone who hasn't noticed, I'm actually able to keep up with
pull requests now that we're on git. This is partly because the
evaluation of the patches thus far hasn't been overly taxing, but
still: if generic-you has in the past hesitated to submit patches on
the basis that we have in the past been less than speedy at
integrating external contributions, I can tell you that both theory
and practice say that it's a completely different kettle of wax now.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: Compile to DalvikVM

I missed this the first time around:

> But there is something simple which could be done: There are a lot of
> useless interface declarations which are already covered by super
> interfaces. Most obvious example is ScalaObject which is a super
> interface of all those interfaces. Removing all those interfaces would
> mean a reduction to approx. 2000 interfaces for this example (but less
> for other less spectacular cases).

This was easy to do, I'll check it in today. (Did you open a ticket?
paulp can't be held responsible for issues which are only raised on
mailing lists.)

Ismael Juma 2
Joined: 2011-01-22,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Compile to DalvikVM
On Mon, Dec 5, 2011 at 3:27 PM, Paul Phillips <paulp@improving.org> wrote:
Also for anyone who hasn't noticed, I'm actually able to keep up with
pull requests now that we're on git.

I can confirm this as Paul integrated pretty quickly a minor performance improvement I submitted as a pull request to test the process. Thanks! :)
Best,Ismael 
Johannes Rudolph 2
Joined: 2010-02-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Compile to DalvikVM

No, not yet, I focussed on putting out that mail.

For reference I created one here: https://issues.scala-lang.org/browse/SI-5278

On Mon, Dec 5, 2011 at 4:37 PM, Paul Phillips wrote:
> I missed this the first time around:
>
>> But there is something simple which could be done: There are a lot of
>> useless interface declarations which are already covered by super
>> interfaces. Most obvious example is ScalaObject which is a super
>> interface of all those interfaces. Removing all those interfaces would
>> mean a reduction to approx. 2000 interfaces for this example (but less
>> for other less spectacular cases).
>
> This was easy to do, I'll check it in today.  (Did you open a ticket?
> paulp can't be held responsible for issues which are only raised on
> mailing lists.)

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: Compile to DalvikVM

On Mon, Dec 5, 2011 at 7:53 AM, Johannes Rudolph
wrote:
> No, not yet, I focussed on putting out that mail.

Sorry, in my haste I thought that was a link to an older thread, not
to an email you'd barely finished writing.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland