Fwd: Forking ant Scalac task

Once more, to the list this time!

---------- Forwarded message ----------
From: Josh Suereth <joshua.suereth@gmail.com>
Date: Sat, Mar 28, 2009 at 7:58 AM
Subject: Re: [scala-internals] Forking ant Scalac task
To: Lukas Rytz <lukas.rytz@epfl.ch>


TBH it's more because I'm not 100% comfortable with pulling the classpath off the antclassloader.  It tested fine on my versions of ant, but I wanted a workaround in case of issues.
As a side benefit it would also bring the scalac task more in line with javac + scalacfork.

Sent from my iPhone
On Mar 28, 2009, at 3:00 AM, Lukas Rytz <lukas.rytz@epfl.ch> wrote:

Hi Josh,
thanks for the patch, looks good to me.Why did you include the compilerPath part? We use that in the sabbus.* tasks becausewhen building the compiler, we need to tell ant which compiler to use (use starr to build locker, use locker to build quick). But I don't think anybody else is interested in that.On the other hand, we could also get rid of the sabbus.* tasks when the official onesare up to date.
Lukas

On Sat, Mar 28, 2009 at 03:57, Josh Suereth <joshua.suereth@gmail.comjoshua.suereth@gmail.com> wrote:

Ah, I realized a little explanation may help with usage.  Here's my build.xml test case.  You'll notice the ability to pass in a diffrent scala compiler if desired (although I have no idea why you would so desire in ant, as the task comes from a scala compiler), and the ability to pass jvm args.  Simply specify <fork>true</fork> and you're all forked up!

<project name="Test Scalac Ant Task" default="compile">
  <path id="scalac.classpath">
    <fileset dir="../scala/build/pack/lib">
      <include name="*.jar"/>
    </fileset>
  </path>
 
  <taskdef resource="scala/tools/ant/antlib.xml">
    <classpath refid="scalac.classpath"/>
  </taskdef>
 
  <target name="compileWithArgs">
   
    <mkdir dir="bin.args"/>
    <scalac srcdir="src" destdir="bin.args" classpathref="scalac.classpath" force="changed" fork="true" jvmargs="-Xmx512M -Xms64M">
      <include name="**/*.scala" />
      <include name="**/*.java" />
    </scalac>   
  </target>
 
  <target name="compileWithPath" depends="compileWithArgs">
   
    <mkdir dir="bin.path"/>
    <scalac srcdir="src" destdir="bin.path" classpathref="scalac.classpath" force="changed" fork="true" compilerpathref="scalac.classpath">
      <include name="**/*.scala" />
      <include name="**/*.java" />
    </scalac>   
  </target>
 
  <target name="compile" depends="compileWithPath">
    <mkdir dir="bin"/>
    <scalac srcdir="src" destdir="bin" classpathref="scalac.classpath" force="changed" fork="true" >
      <include name="**/*.scala" />
      <include name="**/*.java" />
     </scalac>
  </target>
</project>

On Fri, Mar 27, 2009 at 10:55 PM, Josh Suereth <joshua.suereth@gmail.comjoshua.suereth@gmail.com> wrote:

Here's my proposed forking patch for the scalac ant task.  If no one objects, I'll commit it into trunk.  I believe the ant tasks could use a little bit of clean up.  Perhaps when I upgrade the maven plugin to handle the new support for "arguments in a file", I'll also add this to the ant tasks.  I still recommend trying to find someone to maintain these.

-Josh



On Fri, Mar 27, 2009 at 7:25 AM, Josh Suereth <joshua.suereth@gmail.comjoshua.suereth@gmail.com> wrote:

I'll try to tackle the fork/memory options in the official version.  However, like I mentioned earlier, it would be ideal if the task were maintained by someone.  I'm trying not to stretch myself too thin.

-Josh

On Fri, Mar 27, 2009 at 5:45 AM, Lukas Rytz <lukas.rytz@epfl.ch> wrote:

Yes, it would be great if somebody would do some work on the ant tasks.

There are two versions of scalac ant tasks, the "official" and documented ones,
and the ones we use to build the compiler (e.g. scalacfork). Scalacfork was
implemented because we also encountered problems when building the
compiler. It would be very useful for others to have a fork option in the official
version.

Lukas


On Fri, Mar 27, 2009 at 01:05, Josh Suereth <joshua.suereth@gmail.comjoshua.suereth@gmail.com> wrote:

It seems like perhaps someone should step up and maintain the ant plugin.  If I have time I may try to bring the ant plugin and the maven plugin in synch with regard to forking options.  However I have no plans to add features to the ant plugin.

Sent from my iPhone

On Mar 26, 2009, at 6:33 PM, Bill Venners <bill@artima.combill@artima.com> wrote:

Hi All,

We've been struggling with a similar problem in the ScalaTest build,
though it was simply the maximum memory that I needed to configure (to
about 1.25 Gig). I ended up using fsc, because it was the only way I
could get the max memory configured. But then I had to solve the
problem of when you call fsc from ant, it can't find the temp file
with the port number. I solved these by running a little shell script,
but at the JavaPosse roundup, about 10 people were trying to get the
ScalaTest build to work and it was a real pain. One guy could never
get fsc to work at all from ant. It is really a pain.

What we really want to do is fork a scalac process off, and be able to
pass the memory configuration that way. Right now we're trying to for
a java process off that calls scalac, and our current problem is
getting the classpath set up correctly. So we don't yet have a good
solution. I'm not sure if we tried the scalacfork task yet, though we
were made aware of it. So any advice here would be appreciated.

Thanks.

Bill

On Thu, Mar 26, 2009 at 10:25 AM, Ilya Sergey <ilya.sergey@jetbrains.comilya.sergey@jetbrains.com> wrote:

Hello, all.

Trying to compile big projects written in Scala (like IntelliJ plugin or
Scala language itself) on the TeamCity CI server I've occurred a problem.
For compilation of programs like ones mentioned above  very deep stack is
necessary (otherwise compiler fails with SOE). But compiling Scala project
with appropriate `scalac' ant task I have inevitably to give such property
(say, -Xss64m) to that JVM, on which this _ant_ task will be run. In the
particular case of running on a continuous integration server, all threads,
spawned on the same VM will obtain the same stack size. Unfortunatelly, due
to some issuses, there are a lot of such threads (for diffirents aims,
related to logging etc.). And, of course, we're instantly running out of
memory.
Ok, I may use ScalaFork task (which is still undocumented by some reasons),
but it doesn't work with java files, som I'm not able to compile joint
Scala/Java project. Am I missing something?
Thanks.

Kind regards,
Ilya