FOSDEM talk and interview

martin odersky wrote:
> I'll be giving an introductory talk on Scala at FOSDEM 2009, Feb 7 to
> 8. Here's an interview which gives some background:
>
> http://fosdem.org/2009/interview/martin+odersky

Nice. I'm really glad to read that you are continually improving Scala
with additions like an effects system, specialization of primitive
instances (and maybe even not-null types) etc. Even if the proposals are
not complete, it would be interesting to hear some details about this in
your talk or read about it online.

/Jesper Nordenberg

Informative interview.

Re 'the biggest disadvantages of Scala at the moment', this made me think of the following:

* applets/Web Start applications require scala-lang.jar (which is rather large)
* Java's blank finals aren't allowed, which makes the Scala equivalent less concise (and less gratifying to write)

Rob

2009/1/7 martin odersky <martin.odersky@epfl.ch>

I'll be giving an introductory talk on Scala at FOSDEM 2009, Feb 7 to
8. Here's an interview which gives some background:

http://fosdem.org/2009/interview/martin+odersky

 -- Martin

On Wed, Jan 7, 2009 at 1:11 PM, Rob Dickens wrote:
> * Java's blank finals aren't allowed, which makes the Scala equivalent less
> concise (and less gratifying to write)

I don't see that ... example?

Cheers,

Miles

Okay. Here's an example, which first obtains some strings, then prints them out.
First the Java version:

final String pkgName, vSpec, vImp; {
  final java.util.Properties props; {
    final Class _class = Bla.class;
    pkgName = _class.getPackage().getName();
    props = Util.getProperties(_class);
  }
  if (props != null) {
    vSpec = props.getProperty(pkgName +".specification.version");
    vImp = props.getProperty(pkgName +".version");
  }
  else {
    vSpec = null;
    vImp = null;
  }
}
System.out.println(pkgName +" "+ vSpec +"r"+ vImp);

Now the Scala one:

val (pkgName, vSpec, vImp) = {
  val (_pkgName, props) = {
    val _class = classOf[Bla]
    (_class.getPackage.getName, Util.getProperties(_class))
  }
  if (props != null)
    (_pkgName,
     props.getProperty(pkgName +".specification.version"),
     props.getProperty(pkgName +".version"))
  else
    (_pkgName, null, null)
}
println(pkgName +" "+ vSpec +"r"+ vImp)

Okay, so maybe the Scala IS more concise, but the point is, I have to resort to using an intermediate tuple containing a duplicate value (_pkgName) if I want to insist on limiting the scope of vals to where they're actually used.

Rob

2009/1/7 Miles Sabin <miles@milessabin.com>

On Wed, Jan 7, 2009 at 1:11 PM, Rob Dickens <arctic.bob@googlemail.com> wrote:
> * Java's blank finals aren't allowed, which makes the Scala equivalent less
> concise (and less gratifying to write)

I don't see that ... example?

Cheers,


Miles

>>* applets/Web Start applications require scala-lang.jar (which is rather large)

Has anyone investigated the possibility of using proguard or equivalent to discard unused classes from scala-lang.jar for a given application?  Does proguard work with Scala at all?  Also, is there a lot of reflection in scala-lang.jar that will be difficult to catch?

Sam Reid

Rob Dickens wrote:

c84aee200901070511w7446f7c6xc49c56b59b8c7bfd [at] mail [dot] gmail [dot] com" type="cite">Informative interview.

Re 'the biggest disadvantages of Scala at the moment', this made me think of the following:

* applets/Web Start applications require scala-lang.jar (which is rather large)
* Java's blank finals aren't allowed, which makes the Scala equivalent less concise (and less gratifying to write)

Rob

2009/1/7 martin odersky

I'll be giving an introductory talk on Scala at FOSDEM 2009, Feb 7 to
8. Here's an interview which gives some background:

http://fosdem.org/2009/interview/martin+odersky

 -- Martin

On Wed, Jan 7, 2009 at 2:11 PM, Rob Dickens wrote:
> Informative interview.
>
> Re 'the biggest disadvantages of Scala at the moment', this made me think of
> the following:
>
> * applets/Web Start applications require scala-lang.jar (which is rather
> large)

I read some good things about ProGuard. Check out
http://technically.us/code/x/flocking-with-spde

Cheers

Yes, I did exactly that for a Scala web applet I wrote. I used
proguard to reduce the scala libary jar file from 3.3M to 290K.
Given that my app was otherwise only 200K, this meant the total
download was 500K rather than 3.5M.

I did not use any reflection in my applet, so that was not an issue.

A few more details, including the ant fragment I used to run proguard,
are in my blog entry about the applet:

--
Jim

On Wed, Jan 07, 2009 at 09:06:05AM -0700, Samuel Robert Reid wrote:
> Date: Wed, 07 Jan 2009 09:06:05 -0700
> From: Samuel Robert Reid
> To: Rob Dickens
> CC: martin odersky ,
> Scala list
> Subject: Re: [scala] FOSDEM talk and interview
>
> >>* applets/Web Start applications require scala-lang.jar (which is
> rather large)
> Has anyone investigated the possibility of using proguard or equivalent
> to discard unused classes from scala-lang.jar for a given application?
> Does proguard work with Scala at all? Also, is there a lot of
> reflection in scala-lang.jar that will be difficult to catch?
> Sam Reid
> Rob Dickens wrote:
>
> Informative interview.
> Re 'the biggest disadvantages of Scala at the moment', this made me
> think of the following:
> * applets/Web Start applications require scala-lang.jar (which is
> rather large)
> * Java's blank finals aren't allowed, which makes the Scala
> equivalent less concise (and less gratifying to write)
> Rob
>
> 2009/1/7 martin odersky
>
> I'll be giving an introductory talk on Scala at FOSDEM 2009, Feb 7
> to
> 8. Here's an interview which gives some background:
> http://fosdem.org/2009/interview/martin+odersky
> -- Martin

On Wed, Jan 07, 2009 at 09:06:05AM -0700, Samuel Robert Reid wrote:
> Has anyone investigated the possibility of using proguard or equivalent to
> discard unused classes from scala-lang.jar for a given application? Does
> proguard work with Scala at all? Also, is there a lot of reflection in
> scala-lang.jar that will be difficult to catch?

I tried proguard on the scala distribution, as well as a couple proguard-like alternatives whose names
escape me now. I ran into the difficulties described in this ticket:

https://lampsvn.epfl.ch/trac/scala/ticket/1572

With some effort one could probably work around that specific issue, but as I was only experimenting
to get a sense of how much redundancy there is in the scala classfiles, I didn't.

Excellent - will give ProGuard a try. (Am relieved to learn this!)

2009/1/7 Jim McBeath <scala@j.jimmc.org>

Yes, I did exactly that for a Scala web applet I wrote.  I used
proguard to reduce the scala libary jar file from 3.3M to 290K.
Given that my app was otherwise only 200K, this meant the total
download was 500K rather than 3.5M.

I did not use any reflection in my applet, so that was not an issue.

A few more details, including the ant fragment I used to run proguard,
are in my blog entry about the applet:

 <http://jim-mcbeath.blogspot.com/2008/08/stringart-scala-applet.html#scalalib>

--
Jim

On Wed, Jan 07, 2009 at 09:06:05AM -0700, Samuel Robert Reid wrote:
> Date: Wed, 07 Jan 2009 09:06:05 -0700
> From: Samuel Robert Reid <Reids@Colorado.EDU>
> To: Rob Dickens <arctic.bob@googlemail.com>
> CC: martin odersky <martin.odersky@epfl.ch>,
>       Scala list <scala@listes.epfl.ch>
> Subject: Re: [scala] FOSDEM talk and interview
>
>    >>* applets/Web Start applications require scala-lang.jar (which is
>    rather large)
>    Has anyone investigated the possibility of using proguard or equivalent
>    to discard unused classes from scala-lang.jar for a given application?
>    Does proguard work with Scala at all?  Also, is there a lot of
>    reflection in scala-lang.jar that will be difficult to catch?
>    Sam Reid
>    Rob Dickens wrote:
>
>      Informative interview.
>      Re 'the biggest disadvantages of Scala at the moment', this made me
>      think of the following:
>      * applets/Web Start applications require scala-lang.jar (which is
>      rather large)
>      * Java's blank finals aren't allowed, which makes the Scala
>      equivalent less concise (and less gratifying to write)
>      Rob
>
>    2009/1/7 martin odersky <martin.odersky@epfl.ch>
>
>      I'll be giving an introductory talk on Scala at FOSDEM 2009, Feb 7
>      to
>      8. Here's an interview which gives some background:
>      http://fosdem.org/2009/interview/martin+odersky
>       -- Martin

Webstart (and thus, the new Java plugin from Java 6u10 and later) caches
downloaded jars, don't they? Wouldn't a central repository for Scala
library jars make more sense then? If everybody includes their own
stripped Scala library, a user running multiple non-trivial
applets/webstart apps might end up downloading more and polluting his
webstart cache.

My 2 cents

Ingo

Paul Phillips wrote:
> On Wed, Jan 07, 2009 at 09:06:05AM -0700, Samuel Robert Reid wrote:
>> Has anyone investigated the possibility of using proguard or equivalent to
>> discard unused classes from scala-lang.jar for a given application? Does
>> proguard work with Scala at all? Also, is there a lot of reflection in
>> scala-lang.jar that will be difficult to catch?
>
> I tried proguard on the scala distribution, as well as a couple proguard-like alternatives whose names
> escape me now. I ran into the difficulties described in this ticket:
>
> https://lampsvn.epfl.ch/trac/scala/ticket/1572
>
> With some effort one could probably work around that specific issue, but as I was only experimenting
> to get a sense of how much redundancy there is in the scala classfiles, I didn't.
>

On Wed, Jan 7, 2009 at 5:16 PM, Ingo Maier wrote:
> Webstart (and thus, the new Java plugin from Java 6u10 and later) caches
> downloaded jars, don't they? Wouldn't a central repository for Scala library
> jars make more sense then?

For what is worth, this is the approach taken by Sun with JavaFX.

Ismael

Using a central repository for applet/webstart depencencies is not a good idea :
* applet provider don't want to depend of jar hosted on an third-party server :
** can't manage the load of the third-party server
** can't manage the immutability of the third-party (what appends if
third-party change the url layout, the content of the jar (remember
the double release of 2.7.1)

and third-party (like EPFL) can't/must'nt support heavy load for a
popular applet/webstart.

/davidB

On Wed, Jan 7, 2009 at 18:24, Ismael Juma wrote:
> On Wed, Jan 7, 2009 at 5:16 PM, Ingo Maier wrote:
>> Webstart (and thus, the new Java plugin from Java 6u10 and later) caches
>> downloaded jars, don't they? Wouldn't a central repository for Scala library
>> jars make more sense then?
>
> For what is worth, this is the approach taken by Sun with JavaFX.
>
> Ismael
>

David Bernard wrote:
> Using a central repository for applet/webstart depencencies is not a good idea :
> * applet provider don't want to depend of jar hosted on an third-party server :
> ** can't manage the load of the third-party server
> ** can't manage the immutability of the third-party (what appends if
> third-party change the url layout, the content of the jar (remember
> the double release of 2.7.1)

URL layout must not change and the 2.7.1 was a mistake that should not
happen again.

>
> and third-party (like EPFL) can't/must'nt support heavy load for a
> popular applet/webstart.

How do project hosting services handle this? Okay, sourceforge is slow
and uses mirros but what about google code? How does Sun do it? If Sun
really wants to push applets with JavaFX again, I think everybody agrees
that they have to do something about the startup time. They use this new
kernel architecture that splits the rt.jar into smaller pieces and loads
them on demand from a central rep. They do similar things with JavaFX as
Ismael pointed out. Furthermore, what about native bindings like JOGL?
You can't shrink the .so/.dlls with a byte code tree shaker.

Ingo

>
> /davidB
>
> On Wed, Jan 7, 2009 at 18:24, Ismael Juma wrote:
>> On Wed, Jan 7, 2009 at 5:16 PM, Ingo Maier wrote:
>>> Webstart (and thus, the new Java plugin from Java 6u10 and later) caches
>>> downloaded jars, don't they? Wouldn't a central repository for Scala library
>>> jars make more sense then?
>> For what is worth, this is the approach taken by Sun with JavaFX.
>>
>> Ismael
>>

On Thu, Jan 8, 2009 at 8:45 AM, David Bernard
wrote:
> Using a central repository for applet/webstart depencencies is not a good idea :

I don't think it's as black and white as you say. I think many would
be happy to rely on a central server (there are examples of this with
maven dependencies and the Sun JDK itself), while others would like
better control. Usually the ones who like better control have a
controlled environment where library size is a lot less important.

> * applet provider don't want to depend of jar hosted on an third-party server :
> ** can't manage the load of the third-party server

Current practice seems to imply that this is not a problem for many.

> ** can't manage the immutability of the third-party (what appends if
> third-party change the url layout, the content of the jar (remember
> the double release of 2.7.1)

As Ingo said, this was a mistake and should not be repeated.

>
> and third-party (like EPFL) can't/must'nt support heavy load for a
> popular applet/webstart.

This is a better question. There are some blog entries about the
difficulties they face when it comes to this in their blog. One
example is:

http://blogs.sonatype.com/people/2008/12/central-maven-repository-taffic...

Ismael

An other point I forgot. signing of jar :
* If application A depends of B and C (each one provide signed jar)
* If application A need permissions to exceute
=> user need to agree with 3 signatures from 3 providers ?

> URL layout must not change

except maven repository where the layout, naming of version,... are
specify. I didn't know admin that didn't change the layout of URL (ex
scala-lang change the layout of dist files when they change the site,
the nightly build change (I remember to update 3 times the script I
used to download snapshots).

On Thu, Jan 8, 2009 at 11:00, Ismael Juma wrote:
> On Thu, Jan 8, 2009 at 8:45 AM, David Bernard
> wrote:
>> Using a central repository for applet/webstart depencencies is not a good idea :
>
> I don't think it's as black and white as you say. I think many would
> be happy to rely on a central server (there are examples of this with
> maven dependencies and the Sun JDK itself), while others would like
> better control. Usually the ones who like better control have a
> controlled environment where library size is a lot less important.

I agree it's not black and white.

>> * applet provider don't want to depend of jar hosted on an third-party server :
>> ** can't manage the load of the third-party server
>
> Current practice seems to imply that this is not a problem for many.

I agree, but what's happen if third-party is offline => my applet no more work
how (professional) could retreive info about load/availability of third-party

>
>> ** can't manage the immutability of the third-party (what appends if
>> third-party change the url layout, the content of the jar (remember
>> the double release of 2.7.1)
>
> As Ingo said, this was a mistake and should not be repeated.

at epfl, but what for other lib provider.
what happen if there is no version number, and the content change,...

It's too risky for "professionnal" to depends of third-party.

>
>>
>> and third-party (like EPFL) can't/must'nt support heavy load for a
>> popular applet/webstart.
>
> This is a better question. There are some blog entries about the
> difficulties they face when it comes to this in their blog. One
> example is:
>
> http://blogs.sonatype.com/people/2008/12/central-maven-repository-taffic...

> How do project hosting services handle this? Okay, sourceforge is slow and uses mirros but what about google code? How does Sun do it? If Sun really wants to push applets with JavaFX again, I think > everybody agrees that they have to do something about the startup time. They use this new kernel architecture that splits the rt.jar into smaller pieces and loads them on demand from a central rep. They do similar things with JavaFX as Ismael pointed out.

=> jogl/javafx are hosted by the same company SUN
SUN and Google are internet company
SUN sell servers

scala-tools.org nor epfl have the same budget as those company.

> Furthermore, what about native bindings like JOGL? You can't shrink the .so/.dlls with a byte code tree shaker.

we don't shrink dll/so but gain of 2 Mo isn't nothing

On Thu, Jan 8, 2009 at 10:25 AM, David Bernard
wrote:
> at epfl, but what for other lib provider.
> what happen if there is no version number, and the content change,...
>
> It's too risky for "professionnal" to depends of third-party.

I think you're misunderstanding. It's not about having _all_ scala
libraries in a central repository for webstart (that is a separate
issue). This is about having the standard library in a central
repository (like JavaFX). The idea would be to provide a service with
similar quality of service as Sun or Google would do. One simple
approach is to just host the libraries in Google code.

Ismael

David Bernard wrote:

> except maven repository where the layout, naming of version,... are
> specify. I didn't know admin that didn't change the layout of URL (ex
> scala-lang change the layout of dist files when they change the site,
> the nightly build change (I remember to update 3 times the script I
> used to download snapshots).

It wasn't advertised as a site with stable links. A rep would be.

>
> It's too risky for "professionnal" to depends of third-party.

I was pretty much only talking about a rep for the Scala standard libs.

> => jogl/javafx are hosted by the same company SUN
> SUN and Google are internet company
> SUN sell servers

Google code is open for open source projects so we could host things
there for example.

>
> scala-tools.org nor epfl have the same budget as those company.
>
>> Furthermore, what about native bindings like JOGL? You can't shrink the .so/.dlls with a byte code tree shaker.
>
> we don't shrink dll/so but gain of 2 Mo isn't nothing

But the problem would go away with a central rep, wouldn't it? I am not
advertising this as the ultimate solution that is straightforward to
implement, but it has some advantages one cannot deny.

Ingo

On Thu, Jan 8, 2009 at 11:39, Ismael Juma wrote:
> On Thu, Jan 8, 2009 at 10:25 AM, David Bernard
> wrote:
>> at epfl, but what for other lib provider.
>> what happen if there is no version number, and the content change,...
>>
>> It's too risky for "professionnal" to depends of third-party.
>
> I think you're misunderstanding. It's not about having _all_ scala
> libraries in a central repository for webstart (that is a separate
> issue). This is about having the standard library in a central
> repository (like JavaFX).

scala-library.jar is available from scala-tools.org in a "normalized"
URL layout but not signed (required by webstart)

> The idea would be to provide a service with
> similar quality of service as Sun or Google would do. One simple
> approach is to just host the libraries in Google code.

So not a solution ready to use
Google code is not provide as a "free" download server (if it is the
case , why paying for S3 ;) )

So until the infra-structure and budget will be here then shrinker
could be a good idea.

>
> Ismael
>

On Thu, Jan 8, 2009 at 11:51, Ingo Maier wrote:
> David Bernard wrote:
>
>> except maven repository where the layout, naming of version,... are
>> specify. I didn't know admin that didn't change the layout of URL (ex
>> scala-lang change the layout of dist files when they change the site,
>> the nightly build change (I remember to update 3 times the script I
>> used to download snapshots).
>
> It wasn't advertised as a site with stable links. A rep would be.

life isn't immutable ;)

>>
>> It's too risky for "professionnal" to depends of third-party.
>
> I was pretty much only talking about a rep for the Scala standard libs.

For our project -and any end-user applet, scala-library (epfl) is a third-party

>> => jogl/javafx are hosted by the same company SUN
>> SUN and Google are internet company
>> SUN sell servers
>
> Google code is open for open source projects so we could host things there
> for example.
>
>>
>> scala-tools.org nor epfl have the same budget as those company.
>>
>>> Furthermore, what about native bindings like JOGL? You can't shrink the
>>> .so/.dlls with a byte code tree shaker.
>>
>> we don't shrink dll/so but gain of 2 Mo isn't nothing
>
> But the problem would go away with a central rep, wouldn't it? I am not
> advertising this as the ultimate solution that is straightforward to
> implement, but it has some advantages one cannot deny.

I'm pro central repository + cache (as scala-tools.org maintenaires)
for dev and aware users, but not for end-users (due to bad experiences
with the maven's repositories.

> Ingo
>

Both solution could exists, but currently only one is available.
you're free to try to use google to host you jar (but IMO it's a
violation of usage terms)

/davidB

On Thu, Jan 8, 2009 at 10:52 AM, David Bernard
wrote:
> So not a solution ready to use
> Google code is not provide as a "free" download server (if it is the
> case , why paying for S3 ;) )

There are many reasons for using S3 instead of Google Code. For one,
Google code is only available for open-source projects and it has
limits in terms of size. They are flexible about increasing the limit
if the project has a need for more size though.

In the other message, you claim that this would be a violation of
Google Code's Terms of Service, can you please give a quote with the
respective reference that confirms this?

Ismael

I have been using Proguard for the past couple of months to generate pretty
small (25-30Kb) applet jar files - for applets written in Scala. Here's an
example:
http://apps.kogics.net/multiply

There was a bump in the road when 2.7.2 rolled around, as described here:

http://sourceforge.net/tracker/?func=detail&atid=474704&aid=2197633&grou...
This issue has been fixed in Proguard 4.3.

The problem described here:
https://lampsvn.epfl.ch/trac/scala/ticket/1572
still remains. I went ahead and added some more detail to the ticket today.
Hopefully this will give someone from the Scala team additional information
to help with fixing the issue.

But despite this problem, things actually work out fine when you process
your (applet) jar files with Proguard. Proguard correctly issues warnings
about missing referenced classes, but if you ask it to ignore these
warnings, Proguard happily generates an output jar file for you (which works
fine at runtime because the problem (in one case definitely and in another
case most-likely) relates to compile-time artifacts).

Cheers,
- Lalit

Paul Phillips wrote:
>
> On Wed, Jan 07, 2009 at 09:06:05AM -0700, Samuel Robert Reid wrote:
>> Has anyone investigated the possibility of using proguard or equivalent
>> to
>> discard unused classes from scala-lang.jar for a given application? Does
>> proguard work with Scala at all? Also, is there a lot of reflection in
>> scala-lang.jar that will be difficult to catch?
>
> I tried proguard on the scala distribution, as well as a couple
> proguard-like alternatives whose names
> escape me now. I ran into the difficulties described in this ticket:
>
> https://lampsvn.epfl.ch/trac/scala/ticket/1572
>
> With some effort one could probably work around that specific issue, but
> as I was only experimenting
> to get a sense of how much redundancy there is in the scala classfiles, I
> didn't.
>