Why can't Scala auto-build like Python?

I've tried them all. Haskell is by far the most practical build tool for
Scala. I like this magic idea though; perhaps I am missing out.

Russ Paielli wrote:
> I'm trying to migrate over from Python to Scala. I recently looked
> into build tools for Scala. It appears that buildr is the way to go,
> but as I started reading up on it, I started wondering why I should
> even need to bother with a build tool. I'm an aerospace engineer, so I
> may be naive about this, but one of the great features about Python is
> that you never need to explicitly build anything. Everything just
> happens automagically (including the generation of .pyc files when
> they need to be created or updated). Why can't Scala work that way too?
>
> Yes, I realize that you may want to compile explicitly, so that
> capability obviously should not be eliminated, but I personally never
> need or want to do that. I prefer to just let things get built
> automagically as needed.
>
> Russ P.
>

Use either maven or sbt. Maven has a mvn scala::cc, which will keep compiling everything. SBT has something similar.

On Sat, Dec 19, 2009 at 9:47 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

I'm trying to migrate over from Python to Scala. I recently looked into build tools for Scala. It appears that buildr is the way to go, but as I started reading up on it, I started wondering why I should even need to bother with a build tool. I'm an aerospace engineer, so I may be naive about this, but one of the great features about Python is that you never need to explicitly build anything. Everything just happens automagically (including the generation of .pyc files when they need to be created or updated). Why can't Scala work that way too?

Yes, I realize that you may want to compile explicitly, so that capability obviously should not be eliminated, but I personally never need or want to do that. I prefer to just let things get built automagically as needed.

Russ P.

--
http://RussP.us

--
Daniel C. Sobral

I travel to the future all the time.

Neither of the previous responses addressed the OP.

Scala has a different execution model from Python. It can be run
interpreted but normally it is first compiled to Java bytecodes. Scala
was designed from the beginning to have maximal Java interop, so it
shares the same compilation model. This has certain benefits: without
special libraries like numpy, Scala will almost always be much faster
than Python.

Scala and Python aren't better or worse than one another, just
different. Use what works for you.

Warren

On Sat, Dec 19, 2009 at 3:47 PM, Russ Paielli wrote:
> I'm trying to migrate over from Python to Scala. I recently looked into
> build tools for Scala. It appears that buildr is the way to go, but as I
> started reading up on it, I started wondering why I should even need to
> bother with a build tool. I'm an aerospace engineer, so I may be naive about
> this, but one of the great features about Python is that you never need to
> explicitly build anything. Everything just happens automagically (including
> the generation of .pyc files when they need to be created or updated). Why
> can't Scala work that way too?
>
> Yes, I realize that you may want to compile explicitly, so that capability
> obviously should not be eliminated, but I personally never need or want to
> do that. I prefer to just let things get built automagically as needed.
>
> Russ P.
>
> --
> http://RussP.us
>
>

On Saturday December 19 2009, Russ Paielli wrote:
> I'm trying to migrate over from Python to Scala. I recently looked
> into build tools for Scala. It appears that buildr is the way to go,
> ...

It is no doubt a matter of opinion, but I'm very pleased with Simple
Build Tool (SBT).

> Russ P.

Randall Schulz

have you looked at SBT?  it's fantastic.

% xsbt
Project does not exist, create new project? (y/N/s) y
Name: a
Organization: a
Version [1.0]:
Scala version [2.7.5]:
sbt version [0.6.0]:
Getting Scala 2.7.5 ...
:: retrieving :: org.scala-tools.sbt#boot-scala
    confs: [default]
    2 artifacts copied, 0 already retrieved (9831kB/355ms)
Getting org.scala-tools.sbt sbt_2.7.5 0.6.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
    confs: [default]
    13 artifacts copied, 0 already retrieved (13451kB/258ms)
[success] Successfully initialized directory structure.
[info] Building project a 1.0 against Scala 2.7.5
[info]    using sbt.DefaultProject with sbt 0.6.0 and Scala 2.7.5
> ~compile
[info]
[info] == compile ==
[info]   Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Nothing to compile.
[info]   Post-analysis: 0 classes.
[info] == compile ==
[success] Successful.
[info]
[info] Total time: 0 s
Waiting for source changes... (press enter to interrupt)




 —Mohamed

On 19/12/09 23:47, Russ Paielli wrote:

I'm trying to migrate over from Python to Scala. I recently looked into build tools for Scala. It appears that buildr is the way to go, but as I started reading up on it, I started wondering why I should even need to bother with a build tool. I'm an aerospace engineer, so I may be naive about this, but one of the great features about Python is that you never need to explicitly build anything. Everything just happens automagically (including the generation of .pyc files when they need to be created or updated). Why can't Scala work that way too?

Yes, I realize that you may want to compile explicitly, so that capability obviously should not be eliminated, but I personally never need or want to do that. I prefer to just let things get built automagically as needed.

Russ P.

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

Russ P.


On Sat, Dec 19, 2009 at 5:59 PM, Warren Henning <warren.henning@gmail.com> wrote:

Neither of the previous responses addressed the OP.

Scala has a different execution model from Python. It can be run
interpreted but normally it is first compiled to Java bytecodes. Scala
was designed from the beginning to have maximal Java interop, so it
shares the same compilation model. This has certain benefits: without
special libraries like numpy, Scala will almost always be much faster
than Python.

Scala and Python aren't better or worse than one another, just
different. Use what works for you.

Warren

On Sat, Dec 19, 2009 at 3:47 PM, Russ Paielli <russ.paielli@gmail.com> wrote:
> I'm trying to migrate over from Python to Scala. I recently looked into
> build tools for Scala. It appears that buildr is the way to go, but as I
> started reading up on it, I started wondering why I should even need to
> bother with a build tool. I'm an aerospace engineer, so I may be naive about
> this, but one of the great features about Python is that you never need to
> explicitly build anything. Everything just happens automagically (including
> the generation of .pyc files when they need to be created or updated). Why
> can't Scala work that way too?
>
> Yes, I realize that you may want to compile explicitly, so that capability
> obviously should not be eliminated, but I personally never need or want to
> do that. I prefer to just let things get built automagically as needed.
>
> Russ P.
>
> --
> http://RussP.us
>
>

--
http://RussP.us

Normally with Java and Scala, the user gets the object files, already
compiled. It's, perhaps, less convenient in development but I suppose
that you might be able to create a custom class loader that auto
compiles at need.

On Sat, Dec 19, 2009 at 11:01 PM, Russ Paielli wrote:
> Thanks for the explanation, but I still don't understand why Scala can't
> automatically build. Python automatically determines when .pyc files are out
> of date (or nonexistent) and builds them. Why can't Scala do the same for
> .class files? It knows what it needs, so why should I have to tell it?
> Couldn't it be a compiler option or something?

On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

Scala references classes.  There is not a one-to-one correspondence between source code files and class names.  Thus, it's not possible to determine what source code file to load when a particular class is referenced.

Further, Scala has a statically enforced type system.  Thus, in order to determine if a particular method exists on a given class that is referenced in another class, one would have to load and compile the referenced class which will reference other classes, etc., so you're going to wind up loading many classes in order to load the source code of a single class and make sure that the loaded class is correct within the bounds of Scala's static type system.

So, even if you could map from class name to source file, you would still go through the same compilation process on many many classes when you load the first class.

 

Russ P.


On Sat, Dec 19, 2009 at 5:59 PM, Warren Henning <warren.henning@gmail.com> wrote:

Neither of the previous responses addressed the OP.

Scala has a different execution model from Python. It can be run
interpreted but normally it is first compiled to Java bytecodes. Scala
was designed from the beginning to have maximal Java interop, so it
shares the same compilation model. This has certain benefits: without
special libraries like numpy, Scala will almost always be much faster
than Python.

Scala and Python aren't better or worse than one another, just
different. Use what works for you.

Warren

On Sat, Dec 19, 2009 at 3:47 PM, Russ Paielli <russ.paielli@gmail.com> wrote:
> I'm trying to migrate over from Python to Scala. I recently looked into
> build tools for Scala. It appears that buildr is the way to go, but as I
> started reading up on it, I started wondering why I should even need to
> bother with a build tool. I'm an aerospace engineer, so I may be naive about
> this, but one of the great features about Python is that you never need to
> explicitly build anything. Everything just happens automagically (including
> the generation of .pyc files when they need to be created or updated). Why
> can't Scala work that way too?
>
> Yes, I realize that you may want to compile explicitly, so that capability
> obviously should not be eliminated, but I personally never need or want to
> do that. I prefer to just let things get built automagically as needed.
>
> Russ P.
>
> --
> http://RussP.us
>
>

--
http://RussP.us

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

On Saturday December 19 2009, Russ Paielli wrote:
> Thanks for the explanation, but I still don't understand why Scala
> can't automatically build. Python automatically determines when .pyc
> files are out of date (or nonexistent) and builds them. Why can't
> Scala do the same for .class files? It knows what it needs, so why
> should I have to tell it? Couldn't it be a compiler option or
> something?
>
> Russ P.

One might well ask, why can't people trim unnecessary quoted material in
their email replies?

Anyway, SBT has a mode that autonomously monitors source files watching
for them to be modified and when it detects their mod time has changed,
recompiles as necessary.

Randall Schulz

Because Scala is not a build tool, it is a compiler.
There's plenty of automatic compilation around. You have it with Maven, you have it with SBT, you have it with Eclipse, you have it with Netbeans, you have it with IntelliJ, you have it with Play framework... 
On Sun, Dec 20, 2009 at 2:01 AM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

Russ P.


On Sat, Dec 19, 2009 at 5:59 PM, Warren Henning <warren.henning@gmail.com> wrote:

Neither of the previous responses addressed the OP.

Scala has a different execution model from Python. It can be run
interpreted but normally it is first compiled to Java bytecodes. Scala
was designed from the beginning to have maximal Java interop, so it
shares the same compilation model. This has certain benefits: without
special libraries like numpy, Scala will almost always be much faster
than Python.

Scala and Python aren't better or worse than one another, just
different. Use what works for you.

Warren

On Sat, Dec 19, 2009 at 3:47 PM, Russ Paielli <russ.paielli@gmail.com> wrote:
> I'm trying to migrate over from Python to Scala. I recently looked into
> build tools for Scala. It appears that buildr is the way to go, but as I
> started reading up on it, I started wondering why I should even need to
> bother with a build tool. I'm an aerospace engineer, so I may be naive about
> this, but one of the great features about Python is that you never need to
> explicitly build anything. Everything just happens automagically (including
> the generation of .pyc files when they need to be created or updated). Why
> can't Scala work that way too?
>
> Yes, I realize that you may want to compile explicitly, so that capability
> obviously should not be eliminated, but I personally never need or want to
> do that. I prefer to just let things get built automagically as needed.
>
> Russ P.
>
> --
> http://RussP.us
>
>

--
http://RussP.us

--
Daniel C. Sobral

I travel to the future all the time.

All good comments from other posters.  I'll mention in passing that continuous compilation will be part of Buildr's next version (1.4.0) and is currently available as patch/branch on Github.

alex


On Sat, Dec 19, 2009 at 3:47 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

I'm trying to migrate over from Python to Scala. I recently looked into build tools for Scala. It appears that buildr is the way to go, but as I started reading up on it, I started wondering why I should even need to bother with a build tool. I'm an aerospace engineer, so I may be naive about this, but one of the great features about Python is that you never need to explicitly build anything. Everything just happens automagically (including the generation of .pyc files when they need to be created or updated). Why can't Scala work that way too?

Yes, I realize that you may want to compile explicitly, so that capability obviously should not be eliminated, but I personally never need or want to do that. I prefer to just let things get built automagically as needed.

Russ P.

--
http://RussP.us

On Sun, Dec 20, 2009 at 2:01 AM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

I haven’t seen mentioned the -savecompiled option to the `scala’ executable:
       -savecompiled              Save this compiled version of scripts in order to speed up  later  executions  of  the              same  script.   When running a script, save the compiled version of in a file with the              same name as the script but with an extension of .jar.  On subsequent runs of the same              script, the pre-compiled .jar file will be used if it is newer than the script file.
which seems pretty well along the lines of what Russ has in mind.

On Sat, 2009-12-19 at 23:10 -0200, Daniel Sobral wrote:
> Use either maven or sbt. Maven has a mvn scala::cc, which will keep
> compiling everything. SBT has something similar.

Or Gradle -- it now has a Scala plugin.
>
> On Sat, Dec 19, 2009 at 9:47 PM, Russ Paielli
> wrote:
> I'm trying to migrate over from Python to Scala. I recently
> looked into build tools for Scala. It appears that buildr is
> the way to go, but as I started reading up on it, I started
> wondering why I should even need to bother with a build tool.
> I'm an aerospace engineer, so I may be naive about this, but
> one of the great features about Python is that you never need
> to explicitly build anything. Everything just happens
> automagically (including the generation of .pyc files when
> they need to be created or updated). Why can't Scala work that
> way too?
>
> Yes, I realize that you may want to compile explicitly, so
> that capability obviously should not be eliminated, but I
> personally never need or want to do that. I prefer to just let
> things get built automagically as needed.
>
> Russ P.
>
> --
> http://RussP.us
>
>
>
>

Thanks for the explanation. I played around with Ada a few years ago, and I seem to recall that the Ada compiler was part of a comprehensive build system that eliminated the need for "make" or any other build tool. To my way of thinking, that gave it a more complete and "solid" feel than if some third-party program was needed. If that is not possible in Scala or Java, I think that's a shame.

Let me just make a suggestion. If Scala requires a separate build tool that cannot be properly integrated with the compiler, then why not at least provide a default build tool as part of the standard Scala "distribution"? When someone is first starting with Scala (or any language), they don't want to spend time researching which build tool to use, then reading up on how to install and use it. They just want to start programming. If someone wants to use another build tool, they would still be free to do so, but if someone just wants immediate access to one that works, he should have it, without having to find one somewhere "out there."

By the way, having a choice of 37 different build tools is just confusing when all you want is one that works well and is well designed and supported. That's what I think, anyway.

Russ P.


On Sat, Dec 19, 2009 at 9:07 PM, David Pollak <feeder.of.the.bears@gmail.com> wrote:

On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

Scala references classes.  There is not a one-to-one correspondence between source code files and class names.  Thus, it's not possible to determine what source code file to load when a particular class is referenced.

Further, Scala has a statically enforced type system.  Thus, in order to determine if a particular method exists on a given class that is referenced in another class, one would have to load and compile the referenced class which will reference other classes, etc., so you're going to wind up loading many classes in order to load the source code of a single class and make sure that the loaded class is correct within the bounds of Scala's static type system.

So, even if you could map from class name to source file, you would still go through the same compilation process on many many classes when you load the first class.

Am Montag, 21. Dezember 2009 01:54:51 schrieb Russ Paielli:
[...]
> Let me just make a suggestion. If Scala requires a separate build tool that
> cannot be properly integrated with the compiler, then why not at least
> provide a default build tool as part of the standard Scala "distribution"?
> When someone is first starting with Scala (or any language), they don't
> want to spend time researching which build tool to use, then reading up on
> how to install and use it. They just want to start programming. If someone
> wants to use another build tool, they would still be free to do so, but if
> someone just wants immediate access to one that works, he should have it,
> without having to find one somewhere "out there."

I went through the same experience. It has had a bad and a good side:
bad:
- I spent a lot of time (compared to the remaining "programming time") looking
for some tools to help me on my way to scala
- It was frustrating to some extend and nearly turned me away from scala
(partially my fault: I tried to start with scala 2.8 which was a bad idea at
that time)
good:
- I was forced to take a look at some of them (buildr, maven, eclipse, sbt)
and found the one to suit my needs (sbt in my case). If something had been
prebundled I would probably never have looked for something different.

But I agree - it would be quite nice to have a bundle selection. Sort of
"scala installer with either eclipse or sbt or maven...". But I suppose that
this kind of "service distributions" will emerge as scala gets a wider
adoption.

> By the way, having a choice of 37 different build tools is just confusing
> when all you want is one that works well and is well designed and
> supported. That's what I think, anyway.

While initially confusing and distracting I think it is a necessary step of
language ecosystem "evolution". The "fitter" tools will survive and prosper.
Some will find their specific niches - the rest will vanish (and their authors
hopefully joining in their forces and expertise to some remaining or new
challenges).

Just my opinion.
Greetings Bernd

On Sun, Dec 20, 2009 at 4:54 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation. I played around with Ada a few years ago, and I seem to recall that the Ada compiler was part of a comprehensive build system that eliminated the need for "make" or any other build tool. To my way of thinking, that gave it a more complete and "solid" feel than if some third-party program was needed. If that is not possible in Scala or Java, I think that's a shame.

Let me just make a suggestion. If Scala requires a separate build tool that cannot be properly integrated with the compiler, then why not at least provide a default build tool as part of the standard Scala "distribution"? When someone is first starting with Scala (or any language), they don't want to spend time researching which build tool to use, then reading up on how to install and use it. They just want to start programming. If someone wants to use another build tool, they would still be free to do so, but if someone just wants immediate access to one that works, he should have it, without having to find one somewhere "out there."

By the way, having a choice of 37 different build tools is just confusing when all you want is one that works well and is well designed and supported. That's what I think, anyway.

Lift apps use Maven by default.  We only support build-related issues in Maven.  We get kicked in the teeth regularly for being "Maven bigots."  At the end of the day, there's no optimal situation.  However, the JVM eco-system is rich, like the editor eco-system.  So, you may choose "something obvious" (sbt is the most obvious to me) and use it until it no longer works for you.  That day may never come (I still use Emacs).  Or you may find that Ant, Maven, Buildr, etc. does a better job for your use case.

But, at the end of the day, choice is a curse for those who want a simple path.  A simple limited path is a curse for those who have a need for something off that path.
 

Russ P.


On Sat, Dec 19, 2009 at 9:07 PM, David Pollak <feeder.of.the.bears@gmail.com> wrote:

On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli <russ.paielli@gmail.com> wrote:

Thanks for the explanation, but I still don't understand why Scala can't automatically build. Python automatically determines when .pyc files are out of date (or nonexistent) and builds them. Why can't Scala do the same for .class files? It knows what it needs, so why should I have to tell it? Couldn't it be a compiler option or something?

Scala references classes.  There is not a one-to-one correspondence between source code files and class names.  Thus, it's not possible to determine what source code file to load when a particular class is referenced.

Further, Scala has a statically enforced type system.  Thus, in order to determine if a particular method exists on a given class that is referenced in another class, one would have to load and compile the referenced class which will reference other classes, etc., so you're going to wind up loading many classes in order to load the source code of a single class and make sure that the loaded class is correct within the bounds of Scala's static type system.

So, even if you could map from class name to source file, you would still go through the same compilation process on many many classes when you load the first class.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

I am old enough to have used Ada 83 and Ada 95 quite a lot. I got caught
by surprise as I discovered that Java compilation is not nearly as straight-
forward as what I had gotten used to: complete dependency analysis.
Same for Scala (of course).

The net result was that I ended op with outdated class files even though
the compiler thought that everything had been brought up-to-date. My
standard solution is to do a "mvn clean" BEFORE EVERY RECOMPILE.
Hardly efficient, but the closest I can get to what Ada environments used
to offer in long forgotten times.

And yes, I think that this should be part of a tool specification, as it used
to be some 30 years ago ;-) But then, we don't always progress, as you
can observe in Europe now where everything gets to a grinding halt under
circumstances that were also quite common some 30 yers ago ;-)

Best,

Job Honig

> On Sun, Dec 20, 2009 at 4:54 PM, Russ Paielli wrote:
> > Thanks for the explanation. I played around with Ada a few years ago, and
> > I seem to recall that the Ada compiler was part of a comprehensive build
> > system that eliminated the need for "make" or any other build tool. To my
> > way of thinking, that gave it a more complete and "solid" feel than if
> > some third-party program was needed. If that is not possible in Scala or
> > Java, I think that's a shame.
> >
> > Let me just make a suggestion. If Scala requires a separate build tool
> > that cannot be properly integrated with the compiler, then why not at
> > least provide a default build tool as part of the standard Scala
> > "distribution"? When someone is first starting with Scala (or any
> > language), they don't want to spend time researching which build tool to
> > use, then reading up on how to install and use it. They just want to
> > start programming. If someone wants to use another build tool, they would
> > still be free to do so, but if someone just wants immediate access to one
> > that works, he should have it, without having to find one somewhere "out
> > there."
> >
> > By the way, having a choice of 37 different build tools is just confusing
> > when all you want is one that works well and is well designed and
> > supported. That's what I think, anyway.
>
> Lift apps use Maven by default. We only support build-related issues in
> Maven. We get kicked in the teeth regularly for being "Maven bigots." At
> the end of the day, there's no optimal situation. However, the JVM
> eco-system is rich, like the editor eco-system. So, you may choose
> "something obvious" (sbt is the most obvious to me) and use it until it no
> longer works for you. That day may never come (I still use Emacs). Or you
> may find that Ant, Maven, Buildr, etc. does a better job for your use case.
>
> But, at the end of the day, choice is a curse for those who want a simple
> path. A simple limited path is a curse for those who have a need for
> something off that path.
>
> > Russ P.
> >
> >
> >
> > On Sat, Dec 19, 2009 at 9:07 PM, David Pollak <
> >
> > feeder.of.the.bears@gmail.com> wrote:
> >> On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli
wrote:
> >>> Thanks for the explanation, but I still don't understand why Scala
> >>> can't automatically build. Python automatically determines when .pyc
> >>> files are out of date (or nonexistent) and builds them. Why can't Scala
> >>> do the same for .class files? It knows what it needs, so why should I
> >>> have to tell it? Couldn't it be a compiler option or something?
> >>
> >> Scala references classes. There is not a one-to-one correspondence
> >> between source code files and class names. Thus, it's not possible to
> >> determine what source code file to load when a particular class is
> >> referenced.
> >>
> >> Further, Scala has a statically enforced type system. Thus, in order to
> >> determine if a particular method exists on a given class that is
> >> referenced in another class, one would have to load and compile the
> >> referenced class which will reference other classes, etc., so you're
> >> going to wind up loading many classes in order to load the source code
> >> of a single class and make sure that the loaded class is correct within
> >> the bounds of Scala's static type system.
> >>
> >> So, even if you could map from class name to source file, you would
> >> still go through the same compilation process on many many classes when
> >> you load the first class.

What about situations where you don't have all your dependencies up front? In some cases your dependencies haven't even been conceived of yet, not even written. For e.g., a pluggable system with third party plugin developers.

No, I much prefer being able to bind something in late, it is much more powerful. However, I can see how, for simple applications that don't need such power it could turn tedious.

I think David Pollack hit the nail in the head though with this:
"But, at the end of the day, choice is a curse for those who want a simple path.  A simple limited path is a curse for those who have a need for something off that path".

Me? I prefer the extra power that comes with choice. Even if the initial learning curve is painful.

Ishaaq



2009/12/22 Job Honig <joho@xs4all.nl>

I am old enough to have used Ada 83 and Ada 95 quite a lot.  I got caught
by surprise as I discovered that Java compilation is not nearly as straight-
forward as what I had gotten used to: complete dependency analysis.
Same for Scala (of course).

The net result was that I ended op with outdated class files even though
the compiler thought that everything had been brought up-to-date.  My
standard solution is to do a "mvn clean"  BEFORE EVERY RECOMPILE.
Hardly efficient, but the closest I can get to what Ada environments used
to offer in long forgotten times.

And yes, I think that this should be part of a tool specification, as it used
to be some 30 years ago ;-)  But then, we don't always progress, as you
can observe in Europe now where everything gets to a grinding halt under
circumstances that were also quite common some 30 yers ago ;-)

Best,

Job Honig

> On Sun, Dec 20, 2009 at 4:54 PM, Russ Paielli <russ.paielli@gmail.com>wrote:
> > Thanks for the explanation. I played around with Ada a few years ago, and
> > I seem to recall that the Ada compiler was part of a comprehensive build
> > system that eliminated the need for "make" or any other build tool. To my
> > way of thinking, that gave it a more complete and "solid" feel than if
> > some third-party program was needed. If that is not possible in Scala or
> > Java, I think that's a shame.
> >
> > Let me just make a suggestion. If Scala requires a separate build tool
> > that cannot be properly integrated with the compiler, then why not at
> > least provide a default build tool as part of the standard Scala
> > "distribution"? When someone is first starting with Scala (or any
> > language), they don't want to spend time researching which build tool to
> > use, then reading up on how to install and use it. They just want to
> > start programming. If someone wants to use another build tool, they would
> > still be free to do so, but if someone just wants immediate access to one
> > that works, he should have it, without having to find one somewhere "out
> > there."
> >
> > By the way, having a choice of 37 different build tools is just confusing
> > when all you want is one that works well and is well designed and
> > supported. That's what I think, anyway.
>
> Lift apps use Maven by default.  We only support build-related issues in
> Maven.  We get kicked in the teeth regularly for being "Maven bigots."  At
> the end of the day, there's no optimal situation.  However, the JVM
> eco-system is rich, like the editor eco-system.  So, you may choose
> "something obvious" (sbt is the most obvious to me) and use it until it no
> longer works for you.  That day may never come (I still use Emacs).  Or you
> may find that Ant, Maven, Buildr, etc. does a better job for your use case.
>
> But, at the end of the day, choice is a curse for those who want a simple
> path.  A simple limited path is a curse for those who have a need for
> something off that path.
>
> > Russ P.
> >
> >
> >
> > On Sat, Dec 19, 2009 at 9:07 PM, David Pollak <
> >
> > feeder.of.the.bears@gmail.com> wrote:
> >> On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli
<russ.paielli@gmail.com>wrote:
> >>> Thanks for the explanation, but I still don't understand why Scala
> >>> can't automatically build. Python automatically determines when .pyc
> >>> files are out of date (or nonexistent) and builds them. Why can't Scala
> >>> do the same for .class files? It knows what it needs, so why should I
> >>> have to tell it? Couldn't it be a compiler option or something?
> >>
> >> Scala references classes.  There is not a one-to-one correspondence
> >> between source code files and class names.  Thus, it's not possible to
> >> determine what source code file to load when a particular class is
> >> referenced.
> >>
> >> Further, Scala has a statically enforced type system.  Thus, in order to
> >> determine if a particular method exists on a given class that is
> >> referenced in another class, one would have to load and compile the
> >> referenced class which will reference other classes, etc., so you're
> >> going to wind up loading many classes in order to load the source code
> >> of a single class and make sure that the loaded class is correct within
> >> the bounds of Scala's static type system.
> >>
> >> So, even if you could map from class name to source file, you would
> >> still go through the same compilation process on many many classes when
> >> you load the first class.

AFAIK, sbt solves the problem of outdated class files for almost every
practical scenarios. I am not completely sure though, I remove class
files if I want to go for 100% sure, but since using sbt I have never
run into problems anymore.

On 12/21/09, Job Honig wrote:
> I am old enough to have used Ada 83 and Ada 95 quite a lot. I got caught
> by surprise as I discovered that Java compilation is not nearly as straight-
> forward as what I had gotten used to: complete dependency analysis.
> Same for Scala (of course).
>
> The net result was that I ended op with outdated class files even though
> the compiler thought that everything had been brought up-to-date. My
> standard solution is to do a "mvn clean" BEFORE EVERY RECOMPILE.
> Hardly efficient, but the closest I can get to what Ada environments used
> to offer in long forgotten times.
>
> And yes, I think that this should be part of a tool specification, as it used
> to be some 30 years ago ;-) But then, we don't always progress, as you
> can observe in Europe now where everything gets to a grinding halt under
> circumstances that were also quite common some 30 yers ago ;-)
>
> Best,
>
>
> Job Honig
>
>
> > On Sun, Dec 20, 2009 at 4:54 PM, Russ Paielli wrote:
> > > Thanks for the explanation. I played around with Ada a few years ago, and
> > > I seem to recall that the Ada compiler was part of a comprehensive build
> > > system that eliminated the need for "make" or any other build tool. To my
> > > way of thinking, that gave it a more complete and "solid" feel than if
> > > some third-party program was needed. If that is not possible in Scala or
> > > Java, I think that's a shame.
> > >
> > > Let me just make a suggestion. If Scala requires a separate build tool
> > > that cannot be properly integrated with the compiler, then why not at
> > > least provide a default build tool as part of the standard Scala
> > > "distribution"? When someone is first starting with Scala (or any
> > > language), they don't want to spend time researching which build tool to
> > > use, then reading up on how to install and use it. They just want to
> > > start programming. If someone wants to use another build tool, they would
> > > still be free to do so, but if someone just wants immediate access to one
> > > that works, he should have it, without having to find one somewhere "out
> > > there."
> > >
> > > By the way, having a choice of 37 different build tools is just confusing
> > > when all you want is one that works well and is well designed and
> > > supported. That's what I think, anyway.
> >
> > Lift apps use Maven by default. We only support build-related issues in
> > Maven. We get kicked in the teeth regularly for being "Maven bigots." At
> > the end of the day, there's no optimal situation. However, the JVM
> > eco-system is rich, like the editor eco-system. So, you may choose
> > "something obvious" (sbt is the most obvious to me) and use it until it no
> > longer works for you. That day may never come (I still use Emacs). Or you
> > may find that Ant, Maven, Buildr, etc. does a better job for your use case.
> >
> > But, at the end of the day, choice is a curse for those who want a simple
> > path. A simple limited path is a curse for those who have a need for
> > something off that path.
> >
> > > Russ P.
> > >
> > >
> > >
> > > On Sat, Dec 19, 2009 at 9:07 PM, David Pollak <
> > >
> > > feeder.of.the.bears@gmail.com> wrote:
> > >> On Sat, Dec 19, 2009 at 8:01 PM, Russ Paielli
> wrote:
> > >>> Thanks for the explanation, but I still don't understand why Scala
> > >>> can't automatically build. Python automatically determines when .pyc
> > >>> files are out of date (or nonexistent) and builds them. Why can't Scala
> > >>> do the same for .class files? It knows what it needs, so why should I
> > >>> have to tell it? Couldn't it be a compiler option or something?
> > >>
> > >> Scala references classes. There is not a one-to-one correspondence
> > >> between source code files and class names. Thus, it's not possible to
> > >> determine what source code file to load when a particular class is
> > >> referenced.
> > >>
> > >> Further, Scala has a statically enforced type system. Thus, in order to
> > >> determine if a particular method exists on a given class that is
> > >> referenced in another class, one would have to load and compile the
> > >> referenced class which will reference other classes, etc., so you're
> > >> going to wind up loading many classes in order to load the source code
> > >> of a single class and make sure that the loaded class is correct within
> > >> the bounds of Scala's static type system.
> > >>
> > >> So, even if you could map from class name to source file, you would
> > >> still go through the same compilation process on many many classes when
> > >> you load the first class.
>
>

On Monday December 21 2009, Christian Szegedy wrote:
> AFAIK, sbt solves the problem of outdated class files for almost
> every practical scenarios. I am not completely sure though, I remove
> class files if I want to go for 100% sure, but since using sbt I have
> never run into problems anymore.

SBT is conservative in the sense that it will sometimes compile files
that don't strictly require it. To date I have not witnessed it fail to
compile a file that requires it. In practice, it's not a big problem,
if you use SBT in a non-batch modes, either interactive or the
continuous monitoring mode. In batch mode, when no information is saved
from one build to another, SBT is a lot slower (than it is in
interactive mode / continuous build mode).

Randall Schulz

On Monday 21 December 2009, Randall R Schulz wrote:
> On Monday December 21 2009, Christian Szegedy wrote:
> > AFAIK, sbt solves the problem of outdated class files for almost
> > every practical scenarios. I am not completely sure though, I remove
> > class files if I want to go for 100% sure, but since using sbt I have
> > never run into problems anymore.
>
> SBT is conservative in the sense that it will sometimes compile files
> that don't strictly require it. To date I have not witnessed it fail to
> compile a file that requires it. In practice, it's not a big problem,
> if you use SBT in a non-batch modes, either interactive or the
> continuous monitoring mode. In batch mode, when no information is saved
> from one build to another, SBT is a lot slower (than it is in
> interactive mode / continuous build mode).

Right, sbt is quite conservative in order to try to be as reliable as
possible. There is one case that I know of that is not handled, but I've not
run into it in practice.

sbt does preserve information between builds. You should be able to confirm
this by checking the number of files recompiled. The slowdown you notice in
batch mode is due to a cold jvm. In my experience compilation is 2-3x slower
cold.

I'd like to improve partial recompilation in sbt so that modifying a file
without changing the public API (method signatures, type members,
linearization, ...) requires only a very small number of files to be recompiled
but maintains correctness. Ideally, it would be that file only that is
recompiled, but I think it has to be the strongly connected component in the
dependency graph containing that file.

I'm interested to know if this would be a substantial improvement in actual
projects compared to transitive dependencies. That is, what are the sizes of
the strongly connected components in source dependency graphs of real
projects? Are they much smaller than the weakly connected components?

I'd be interested to see the output of

sbt graph-src && sccmap -v target/scala_2.7.7/graph/sources/dependencies

for any project that wants to share this information. For the main sbt
project, the above shows that 65% of sources are in trivial strongly connected
components and the remaining 35% are mostly concentrated in a single large
component.

-Mark

On Mon, Dec 21, 2009 at 3:22 PM, Ishaaq Chandy <ishaaq@gmail.com> wrote:

What about situations where you don't have all your dependencies up front? In some cases your dependencies haven't even been conceived of yet, not even written. For e.g., a pluggable system with third party plugin developers.

No, I much prefer being able to bind something in late, it is much more powerful. However, I can see how, for simple applications that don't need such power it could turn tedious.

I think David Pollack hit the nail in the head though with this:
"But, at the end of the day, choice is a curse for those who want a simple path.  A simple limited path is a curse for those who have a need for something off that path".

Me? I prefer the extra power that comes with choice. Even if the initial learning curve is painful.

I don't understand why it's an either/or situation. How about a simple default that can be overridden when necessary for more complicated situations? I just want to conveniently update my .class files when they are out of date, and I want to do it with a simple command that does not require any installation, configuration, or user guide. Python seems to be able to do it, and so does the Ada gnat compiler. If Java and Scala can't do it, that sure seems like a design flaw to me. But maybe I'm just full of beans.

Russ P.

On Monday December 21 2009, Mark Harrah wrote:
> ...
>
> Right, sbt is quite conservative in order to try to be as reliable as
> possible. There is one case that I know of that is not handled, but
> I've not run into it in practice.
>
> sbt does preserve information between builds. You should be able to
> confirm this by checking the number of files recompiled. The
> slowdown you notice in batch mode is due to a cold jvm. In my
> experience compilation is 2-3x slower cold.

While I restart IDEA daily, I tend to restart SBT only very rarely,
so the cold-start performance isn't an issue for me.

> ...
>
> I'd be interested to see the output of
>
> sbt graph-src && sccmap -v target/scala_2.7.7/graph/sources/dependencies

I tried this, but it doesn't seem to work
(note that I'm using Scla 2.7.4):

% sbt graph-src && sccmap -v target/scala_2.7.4/graph/sources/dependencies
[info] Building project Rho 1.0 using ProjectRho
[info] with sbt 0.5.4 and Scala 2.7.4
[info]
[info] == compile ==
[info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Nothing to compile.
[info] Post-analysis: 2916 classes.
[info] == compile ==
[info]
[info] == graph-src ==
[info] == graph-src ==
[success] Successful.
[info]
[info] Total time: 1 s
[success] Build completed successfully.
[info]
[info] Total build time: 2 s
Can't open target/scala_2.7.4/graph/sources/dependencies

> ...
>
> -Mark

Randall Schulz

On Monday December 21 2009, Russ Paielli wrote:
> > ...
>
> I don't understand why it's an either/or situation. How about a
> simple default that can be overridden when necessary for more
> complicated situations? I just want to conveniently update my .class
> files when they are out of date, and I want to do it with a simple
> command that does not require any installation, configuration, or
> user guide. Python seems to be able to do it, and so does the Ada
> gnat compiler. If Java and Scala can't do it, that sure seems like a
> design flaw to me. But maybe I'm just full of beans.

Ponies, too? Rainbows? With a pot of gold at the end?

You're an engineer, yes? Why would you resist installing a build tool
and writing a build configuration? To me, your perspective seems that
of a hobbyist programmer.

For my money, the language is worth dealing with a minimal toolset.
However, I don't feel it's suitable for dabbling—it takes a commitment.
Maybe in the future more mature tools will lower the barrier to entry,
but nonetheless, I find the currently minimal nature of Scala tooling
not be a deterrent to serious use.

Perhaps IDEA would suit you. The free Community Edition includes the
Scala plug-in and it has a one-button project build function.

> Russ P.

Randall Schulz

On Mon, 2009-12-21 at 21:05 -0800, Russ Paielli wrote:

> I don't understand why it's an either/or situation. How about a simple
> default that can be overridden when necessary for more complicated
> situations? I just want to conveniently update my .class files when
> they are out of date, and I want to do it with a simple command that
> does not require any installation, configuration, or user guide.
> Python seems to be able to do it, and so does the Ada gnat compiler.
> If Java and Scala can't do it, that sure seems like a design flaw to
> me. But maybe I'm just full of beans.
>
Java, and I guess Scala, come from a Fortran, C, C++ mindset with regard
to source code and system construction. Make -> Ant -> Maven (and any
mix of SCons, Gradle, SBT, Buildr, Rake, Rant, etc. -- it seems that
every language now has to have a build system written in it).

Ada was evolving during the early CASE (aka IDE) period which allowed it
to take a different view. Also Ada tends to be monocultural regarding
language.

Eclipse has a continuous compilation mode and so is probably the
standard bearer of "just do it when it is needed" à la Python.

Of course Python is different from Java/Scala in that launching of
Python is via Python. Launching of Java/Scala/Groovy can be from any
other language.

So I am not sure it is a design flaw, but a systemic issue. The Java
Platform now supports systems written in any mix of Java, Scala, Groovy,
Clojure, Jython, JRuby, BeanShell, Boo, etc. It is an open platform
where the platform does not track the source for a given class, so there
is no way of internally tracking update requirements. A given system
must specify in some way all these source relationships. So we end up
back at build system for each project.

So the upshot is that Python and Ada are, pragmatically, special cases
that means they can do the automated compilation as a language feature
rather than a project feature.

On Tuesday 22 December 2009, Randall R Schulz wrote:
> On Monday December 21 2009, Mark Harrah wrote:
> > ...
> >
> > I'd be interested to see the output of
> >
> > sbt graph-src && sccmap -v
> > target/scala_2.7.7/graph/sources/dependencies
>
> I tried this, but it doesn't seem to work
> (note that I'm using Scla 2.7.4):
>
> % sbt graph-src && sccmap -v target/scala_2.7.4/graph/sources/dependencies
> [info] Building project Rho 1.0 using ProjectRho
> [info] with sbt 0.5.4 and Scala 2.7.4
> [info]
> [info] == compile ==
> [info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0
> removed. [info] Compiling main sources...
> [info] Nothing to compile.
> [info] Post-analysis: 2916 classes.
> [info] == compile ==
> [info]
> [info] == graph-src ==
> [info] == graph-src ==
> [success] Successful.
> [info]
> [info] Total time: 1 s
> [success] Build completed successfully.
> [info]
> [info] Total build time: 2 s
> Can't open target/scala_2.7.4/graph/sources/dependencies

Sorry, for sbt 0.5.6, remove the scala_2.7.4.

-Mark