ant scalac/fsc - time machine?

OK, have filed new ticket with a test case:

https://lampsvn.epfl.ch/trac/scala/ticket/2320

On Wednesday 02 September 2009 00:51:59 Andrew Gaydenko wrote:
> Say, I have this ant target definition (have tried with both scalac
> and fsc tasks):
>
>
> destdir="${dist.dir}/classes"
> classpathref="http://www.scala-lang.org/zgs.lib.path#"
> force="always"
> target="jvm-1.5"
> encoding="UTF-8"
> deprecation="on">
>
>
>
> Plase, look at the output of three consequent calls of the task - *without*
> any code change (below). As you see, 2d and 3d (identical) times are foxy
> enough to find what to compile. I have another projects with the same
> "effect" (but with 4 and 6 files "from future").
>
> Have I missed something? Or - is it a bug?
>
>
> Andrew
>
> //------
>
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
> [scalac] Compiling 86 source files to
> /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 40 seconds
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
> [scalac] Compiling 3 source files to
> /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 8 seconds
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
> [scalac] Compiling 3 source files to
> /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 8 seconds
>

Last time I checked, this is because the ant tasks use ant's uptodate
checking, which compares a source file to a class file with the same name.
If the class file does not exist or is older than the source file, it
recompiles the sources. In your case, you have HData.scala with object
Another in it. Therefore, you don't get an HData.class and so ant recompiles
everytime.

I believe you will also find that removing a source file will not force a
recompile.

The solutions to this require storing additional information on disk for each
compile. Maven uses a single timestamp and sbt stores the hash for each
source. The single timestamp doesn't solve the removing the source file
issue, but storing it for every source would.

Thanks,
Mark

On Wednesday 02 September 2009, Andrew Gaydenko wrote:
> OK, have filed new ticket with a test case:
>
> https://lampsvn.epfl.ch/trac/scala/ticket/2320
>
> On Wednesday 02 September 2009 00:51:59 Andrew Gaydenko wrote:
> > Say, I have this ant target definition (have tried with both scalac
> > and fsc tasks):
> >
> >
> > > destdir="${dist.dir}/classes"
> > classpathref="http://www.scala-lang.org/zgs.lib.path#"
> > force="always"
> > target="jvm-1.5"
> > encoding="UTF-8"
> > deprecation="on">
> >
> >
> >
> > Plase, look at the output of three consequent calls of the task -
> > *without* any code change (below). As you see, 2d and 3d (identical)
> > times are foxy enough to find what to compile. I have another projects
> > with the same "effect" (but with 4 and 6 files "from future").
> >
> > Have I missed something? Or - is it a bug?
> >
> >
> > Andrew
> >
> > //------
> >
> > $ ant -f myBuild.xml tryFsc
> > Buildfile: myBuild.xml
> >
> > tryFsc:
> > [scalac] Compiling 86 source files to
> > /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
> >
> > BUILD SUCCESSFUL
> > Total time: 40 seconds
> > $ ant -f myBuild.xml tryFsc
> > Buildfile: myBuild.xml
> >
> > tryFsc:
> > [scalac] Compiling 3 source files to
> > /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
> >
> > BUILD SUCCESSFUL
> > Total time: 8 seconds
> > $ ant -f myBuild.xml tryFsc
> > Buildfile: myBuild.xml
> >
> > tryFsc:
> > [scalac] Compiling 3 source files to
> > /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
> >
> > BUILD SUCCESSFUL
> > Total time: 8 seconds

I was at loss at what you were complaining about when I first read your message. Ant is not capable of correctly incrementally compiling Scala. It's not a bug, it's a fundamental incompatibility between premises of Ant and Scala, and one of the main reasons why SBT was created.

On Wed, Sep 2, 2009 at 9:18 AM, Andrew Gaydenko <a@gaydenko.com> wrote:

OK, have filed new ticket with a test case:

https://lampsvn.epfl.ch/trac/scala/ticket/2320


On Wednesday 02 September 2009 00:51:59 Andrew Gaydenko wrote:
> Say, I have this ant target definition (have tried with both scalac
> and fsc tasks):
>
>     <target name="tryFsc" description="compile zgs">
>         <scalac srcdir="${zgs.dir}/src"
>                 destdir="${dist.dir}/classes"
>                 classpathref="zgs.lib.path"
>                 force="always"
>                 target="jvm-1.5"
>                 encoding="UTF-8"
>                 deprecation="on">
>         </scalac>
>     </target>
>
> Plase, look at the output of three consequent calls of the task - *without*
> any code change (below). As you see, 2d and 3d (identical) times are foxy
> enough to find what to compile. I have another projects with the same
> "effect" (but with 4 and 6 files "from future").
>
> Have I missed something? Or - is it a bug?
>
>
> Andrew
>
> //------
>
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
>    [scalac] Compiling 86 source files to
>  /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 40 seconds
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
>    [scalac] Compiling 3 source files to
>  /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 8 seconds
> $ ant -f myBuild.xml tryFsc
> Buildfile: myBuild.xml
>
> tryFsc:
>    [scalac] Compiling 3 source files to
>  /wrk/usr/netbeans/projects/zgs/alog/alogdist/classes
>
> BUILD SUCCESSFUL
> Total time: 8 seconds
>

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

On Wednesday 02 September 2009 17:08:02 Mark Harrah wrote:
> Last time I checked, this is because the ant tasks use ant's uptodate
> checking, which compares a source file to a class file with the same name.
> If the class file does not exist or is older than the source file, it
> recompiles the sources. In your case, you have HData.scala with object
> Another in it. Therefore, you don't get an HData.class and so ant
> recompiles everytime.
>
> I believe you will also find that removing a source file will not force a
> recompile.
>
> The solutions to this require storing additional information on disk for
> each compile. Maven uses a single timestamp and sbt stores the hash for
> each source. The single timestamp doesn't solve the removing the source
> file issue, but storing it for every source would.
>
> Thanks,
> Mark

On Wednesday 02 September 2009 17:14:59 Daniel Sobral wrote:
> I was at loss at what you were complaining about when I first read your
> message. Ant is not capable of correctly incrementally compiling Scala.
> It's not a bug, it's a fundamental incompatibility between premises of Ant
> and Scala, and one of the main reasons why SBT was created.

Thanks for the clarification!

Were there attempts to modify existing scala ant tasks with, say, saving
timestamp-related information in something like ~/.scalatasks?

On Wed, Sep 2, 2009 at 10:46 AM, Andrew Gaydenko <a@gaydenko.com> wrote:

On Wednesday 02 September 2009 17:14:59 Daniel Sobral wrote:
> I was at loss at what you were complaining about when I first read your
> message. Ant is not capable of correctly incrementally compiling Scala.
>  It's not a bug, it's a fundamental incompatibility between premises of Ant
>  and Scala, and one of the main reasons why SBT was created.

Thanks for the clarification!

Were there attempts to modify existing scala ant tasks with, say, saving
timestamp-related information in something like ~/.scalatasks?

That's more than I can say. Maybe the SBT web pages have more information about what are the fundamental problems with ANT?


--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

On Wednesday 02 September 2009, Daniel Sobral wrote:
> On Wed, Sep 2, 2009 at 10:46 AM, Andrew Gaydenko wrote:
> > On Wednesday 02 September 2009 17:14:59 Daniel Sobral wrote:
> > > I was at loss at what you were complaining about when I first read your
> > > message. Ant is not capable of correctly incrementally compiling Scala.
> > > It's not a bug, it's a fundamental incompatibility between premises of
> >
> > Ant
> >
> > > and Scala, and one of the main reasons why SBT was created.
> >
> > Thanks for the clarification!
> >
> > Were there attempts to modify existing scala ant tasks with, say, saving
> > timestamp-related information in something like ~/.scalatasks?
>
> That's more than I can say. Maybe the SBT web pages have more information
> about what are the fundamental problems with ANT?

I have documented how sbt goes about partial recompilation and why here:
http://code.google.com/p/simple-build-tool/wiki/ChangeDetectionAndTestin...

The part that applies to this thread is detecting which sources have changed.

Thanks,
Mark