Re: [scala-devel] Re: SBT build

On Tue, 20 Dec 2011 12:52:57 +0100
Adriaan Moors wrote:

> I'm switching to the SBT build on my laptop now.
>
> Part of my usual workflow is to compile quick and then run that quick
> compiler on some source file, usually with -Xprint:typer or some other
> options. Is there a way to do all this (preferably automatically on file
> change) from the SBT console?

Depending on how Josh has things set up, you might be able to do:

~ run-main scala.tools.nsc.Main -Xprint:typer path/to/Source.scala

~ for triggering on source changes. 'run-main' accepts the main class to run and its arguments.

-Mark

> thanks!
> adriaan
>
> On Thu, Dec 15, 2011 at 8:02 PM, Eugene Burmako wrote:
>
> > Works finely, takes 625s. No idea though whether this is a lot or okay.
> >
> >
> > On 15 December 2011 19:18, Josh Suereth wrote:
> >
> >> You can try:
> >>
> >> testsuite/run-partest-single pos *
> >>
> >> This will run all the pos tests, which shouldn't take too long.
> >>
> >> On Thu, Dec 15, 2011 at 12:52 PM, Eugene Burmako wrote:
> >>
> >>> Btw, 512m permgen and 256m codecache seem to be sweet spots. Increasing
> >>> them does nothing, decreasing them degrades the performance. Speaking of
> >>> partest, I'll test it later one of these days - even a single run takes
> >>> non-trivial time.
> >>>
> >>>
> >>> On 15 December 2011 17:22, Josh Suereth wrote:
> >>>
> >>>> Neato. It's the Reserved Code Cache and PermGen that are important for
> >>>> the SBT build. Most of the other caches can be GC'd in 0.11.2.
> >>>>
> >>>> I would love to see how partest does at those settings as well.
> >>>>
> >>>> - Josh
> >>>>
> >>>> On Thu, Dec 15, 2011 at 10:27 AM, Eugene Burmako <
> >>>> eugene.burmako@epfl.ch> wrote:
> >>>>
> >>>>> Amazingly, SBT can do fine with less memory than ant!!
> >>>>>
> >>>>> I figured that out when I tried to reduce Xms/Xmx in small steps. I
> >>>>> was astonished to see the following results:
> >>>>>
> >>>>> 4g 559s
> >>>>> 3g 592s
> >>>>> 2.5g 557s
> >>>>> 2g 548s
> >>>>> 1.5g 557s
> >>>>> 1g 559s
> >>>>>
> >>>>> To be sure that everything is as good as it looks, I verified the
> >>>>> amount of memory that got freed after I closed sbt via Resource Monitor
> >>>>> tool bundled with Windows. After the 1g run, closing sbt freed 1.5g of
> >>>>> memory (I guess, it's 1g for the heap + 512m for permgen, though I'm a noob
> >>>>> in JVM, so I might be misinterpreting the result). Oh, I'm in love! ♡
> >>>>>
> >>>>> My SBT script is as follows:
> >>>>>
> >>>>> set SCRIPT_DIR=%~dp0
> >>>>> set MEM_SETTINGS_MAIN=-Xms1024M -Xmx1024M
> >>>>> set MEM_SETTINGS_OTHER=-Xss128M -XX:MaxPermSize=512m
> >>>>> -XX:ReservedCodeCacheSize=256m -XX:+CMSClassUnloadingEnabled
> >>>>> set JVM_SETTINGS_OTHER=-XX:+UseParallelGC
> >>>>> set SBT_VERSION=0.11.2
> >>>>> java %MEM_SETTINGS_MAIN% %MEM_SETTINGS_OTHER% %JVM_SETTINGS_OTHER%
> >>>>> -Djline.WindowsTerminal.directConsole=false -jar
> >>>>> "%SCRIPT_DIR%sbt-launch-%SBT_VERSION%.jar" %*
> >>>>>
> >>>>> P.S. Setting Xms/Xms to 512m degrades the run time so badly, that I
> >>>>> get tired of waiting. Setting MaxPermSize to 256m degrades run time by 10%.
> >>>>> Setting ReservedCodeCacheSize to 128m degrades run time by 5%. Turning off
> >>>>> CMSClassUnloadingEnabled degrades run time by 5%. Turning off UseParallelGC
> >>>>> degrades run time by 5% (all the tests in P.S. have been done in comparison
> >>>>> with the baseline result of the script provided above).
> >>>>>
> >>>>> On 15 December 2011 11:43, Eugene Burmako wrote:
> >>>>>
> >>>>>> Hi folks,
> >>>>>>
> >>>>>> I'd like to take a few minutes of your time and advertise the SBT
> >>>>>> builder. This night I tried the latest version of SBT infrastructure, and,
> >>>>>> I must admit, I'm very surprised. I heard of horrible memory consumption
> >>>>>> and inferior speed of SBT, but it turned out to be almost the exact
> >>>>>> opposite.
> >>>>>>
> >>>>>> First of all, SBT is noticeably faster. On my laptop "sbt clean
> >>>>>> compile" finishes in 592s ("ant clean compile" takes about 11 minutes). And
> >>>>>> this is even taking into account that SBT builds all the small stuff (fjbg,
> >>>>>> jline, etc) that ant skips. Also, incremental builds are real awesome. In a
> >>>>>> few of my experiments with incremental compilation sbt beats ant by a
> >>>>>> factor of 1.5x or even 2x (20ish seconds vs 40ish seconds for ant).
> >>>>>>
> >>>>>> Secondly, memory consumption is not that big of a problem. I indeed
> >>>>>> had to increase Xms/Xmx (and especially -XX:MaxPermSize) values for SBT -
> >>>>>> my comfort zone with ant was 2G, here it is at 3G. Also, giving SBT more
> >>>>>> memory makes it even faster (bringing Xmx to 4G made clean compile complete
> >>>>>> in 559s). But personally for me even 3G is tolerable. I also wonder what
> >>>>>> kind of speed degradation SBT will suffer if I lower memory limit to 2.5G.
> >>>>>> That needs further experimenting.
> >>>>>>
> >>>>>> All in all, personally to me the SBT builder seems superior to the
> >>>>>> Ant one. It needs ~1G more RAM, but, in return, it provides higher
> >>>>>> compilation speeds. better functionality and is also cross-platform. If you
> >>>>>> can spare more memory, you can get even better speed.
> >>>>>>
> >>>>>> Come one, give it a try!
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Eugene
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >

On Tue, Dec 20, 2011 at 11:04 PM, Mark Harrah <dmharrah@gmail.com> wrote:

On Tue, 20 Dec 2011 12:52:57 +0100
Adriaan Moors <adriaan.moors@epfl.ch> wrote:

> I'm switching to the SBT build on my laptop now.
>
> Part of my usual workflow is to compile quick and then run that quick
> compiler on some source file, usually with -Xprint:typer or some other
> options. Is there a way to do all this (preferably automatically on file
> change) from the SBT console?

Depending on how Josh has things set up, you might be able to do:

 ~ run-main scala.tools.nsc.Main -Xprint:typer path/to/Source.scala

~ for triggering on source changes.  'run-main' accepts the main class to run and its arguments.

Alternatively, to to recompile a single file with SBT's regular compiler, adorned with additional options:
https://github.com/retronym/scratch/blob/master/20111022/sbt.loghttps://github.com/retronym/scratch/blob/master/20111022/project/CompileQuickPlugin.scala
-jason 

That would be quick-compiler/run-main ....

On Dec 20, 2011 8:04 AM, "Mark Harrah" <dmharrah@gmail.com> wrote:

On Tue, 20 Dec 2011 12:52:57 +0100
Adriaan Moors <adriaan.moors@epfl.ch> wrote:

> I'm switching to the SBT build on my laptop now.
>
> Part of my usual workflow is to compile quick and then run that quick
> compiler on some source file, usually with -Xprint:typer or some other
> options. Is there a way to do all this (preferably automatically on file
> change) from the SBT console?

Depending on how Josh has things set up, you might be able to do:

 ~ run-main scala.tools.nsc.Main -Xprint:typer path/to/Source.scala

~ for triggering on source changes.  'run-main' accepts the main class to run and its arguments.

-Mark

> thanks!
> adriaan
>
> On Thu, Dec 15, 2011 at 8:02 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
>
> > Works finely, takes 625s. No idea though whether this is a lot or okay.
> >
> >
> > On 15 December 2011 19:18, Josh Suereth <joshua.suereth@gmail.com> wrote:
> >
> >> You can try:
> >>
> >> testsuite/run-partest-single pos *
> >>
> >> This will run all the pos tests, which shouldn't take too long.
> >>
> >> On Thu, Dec 15, 2011 at 12:52 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>
> >>> Btw, 512m permgen and 256m codecache seem to be sweet spots. Increasing
> >>> them does nothing, decreasing them degrades the performance. Speaking of
> >>> partest, I'll test it later one of these days - even a single run takes
> >>> non-trivial time.
> >>>
> >>>
> >>> On 15 December 2011 17:22, Josh Suereth <joshua.suereth@gmail.com>wrote:
> >>>
> >>>> Neato.  It's the Reserved Code Cache and PermGen that are important for
> >>>> the SBT build.   Most of the other caches can be GC'd in 0.11.2.
> >>>>
> >>>> I would love to see how partest does at those settings as well.
> >>>>
> >>>> - Josh
> >>>>
> >>>> On Thu, Dec 15, 2011 at 10:27 AM, Eugene Burmako <
> >>>> eugene.burmako@epfl.ch> wrote:
> >>>>
> >>>>> Amazingly, SBT can do fine with less memory than ant!!
> >>>>>
> >>>>> I figured that out when I tried to reduce Xms/Xmx in small steps. I
> >>>>> was astonished to see the following results:
> >>>>>
> >>>>> 4g 559s
> >>>>> 3g 592s
> >>>>> 2.5g 557s
> >>>>> 2g 548s
> >>>>> 1.5g 557s
> >>>>> 1g 559s
> >>>>>
> >>>>> To be sure that everything is as good as it looks, I verified the
> >>>>> amount of memory that got freed after I closed sbt via Resource Monitor
> >>>>> tool bundled with Windows. After the 1g run, closing sbt freed 1.5g of
> >>>>> memory (I guess, it's 1g for the heap + 512m for permgen, though I'm a noob
> >>>>> in JVM, so I might be misinterpreting the result). Oh, I'm in love! ♡
> >>>>>
> >>>>> My SBT script is as follows:
> >>>>>
> >>>>> set SCRIPT_DIR=%~dp0
> >>>>> set MEM_SETTINGS_MAIN=-Xms1024M -Xmx1024M
> >>>>> set MEM_SETTINGS_OTHER=-Xss128M -XX:MaxPermSize=512m
> >>>>> -XX:ReservedCodeCacheSize=256m -XX:+CMSClassUnloadingEnabled
> >>>>> set JVM_SETTINGS_OTHER=-XX:+UseParallelGC
> >>>>> set SBT_VERSION=0.11.2
> >>>>> java %MEM_SETTINGS_MAIN% %MEM_SETTINGS_OTHER% %JVM_SETTINGS_OTHER%
> >>>>> -Djline.WindowsTerminal.directConsole=false -jar
> >>>>> "%SCRIPT_DIR%sbt-launch-%SBT_VERSION%.jar" %*
> >>>>>
> >>>>> P.S. Setting Xms/Xms to 512m degrades the run time so badly, that I
> >>>>> get tired of waiting. Setting MaxPermSize to 256m degrades run time by 10%.
> >>>>> Setting ReservedCodeCacheSize to 128m degrades run time by 5%. Turning off
> >>>>> CMSClassUnloadingEnabled degrades run time by 5%. Turning off UseParallelGC
> >>>>> degrades run time by 5% (all the tests in P.S. have been done in comparison
> >>>>> with the baseline result of the script provided above).
> >>>>>
> >>>>> On 15 December 2011 11:43, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>>>>
> >>>>>> Hi folks,
> >>>>>>
> >>>>>> I'd like to take a few minutes of your time and advertise the SBT
> >>>>>> builder. This night I tried the latest version of SBT infrastructure, and,
> >>>>>> I must admit, I'm very surprised. I heard of horrible memory consumption
> >>>>>> and inferior speed of SBT, but it turned out to be almost the exact
> >>>>>> opposite.
> >>>>>>
> >>>>>> First of all, SBT is noticeably faster. On my laptop "sbt clean
> >>>>>> compile" finishes in 592s ("ant clean compile" takes about 11 minutes). And
> >>>>>> this is even taking into account that SBT builds all the small stuff (fjbg,
> >>>>>> jline, etc) that ant skips. Also, incremental builds are real awesome. In a
> >>>>>> few of my experiments with incremental compilation sbt beats ant by a
> >>>>>> factor of 1.5x or even 2x (20ish seconds vs 40ish seconds for ant).
> >>>>>>
> >>>>>> Secondly, memory consumption is not that big of a problem. I indeed
> >>>>>> had to increase Xms/Xmx (and especially -XX:MaxPermSize) values for SBT -
> >>>>>> my comfort zone with ant was 2G, here it is at 3G. Also, giving SBT more
> >>>>>> memory makes it even faster (bringing Xmx to 4G made clean compile complete
> >>>>>> in 559s). But personally for me even 3G is tolerable. I also wonder what
> >>>>>> kind of speed degradation SBT will suffer if I lower memory limit to 2.5G.
> >>>>>> That needs further experimenting.
> >>>>>>
> >>>>>> All in all, personally to me the SBT builder seems superior to the
> >>>>>> Ant one. It needs ~1G more RAM, but, in return, it provides higher
> >>>>>> compilation speeds. better functionality and is also cross-platform. If you
> >>>>>> can spare more memory, you can get even better speed.
> >>>>>>
> >>>>>> Come one, give it a try!
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Eugene
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >

Does anyone object to my moving the SBT build into master?  I think it's ready for folk to start hammering on it again.  I think feature-wise it's good enough to cut over and I'd like to focus on fixing any issues folk might have before migrating completely to SBT.
- Josh

On Tue, Dec 20, 2011 at 9:57 AM, Josh Suereth wrote:
> Does anyone object to my moving the SBT build into master?  I think it's
> ready for folk to start hammering on it again.  I think feature-wise it's
> good enough to cut over and I'd like to focus on fixing any issues folk
> might have before migrating completely to SBT.

No objection from me. I already deleted the sbt 0.7 build to make space.

Yes, please go ahead, Josh. Would be good to give it a spin.

Cheers

after a successful compile, I did:
> quick-compiler/run-main scala.tools.nsc.Main -Xprint:typer test/files/pos/virtpatmat_alts_subst.scala 
<snip FOUR git rev-list & git log invocations..> 
[info] Running scala.tools.nsc.Main -Xprint:typer test/files/pos/virtpatmat_alts_subst.scalaerror: scala.reflect.internal.MissingRequirementError: object scala not found.
I assume the error is caused by the compiler's classpath is not being set correctly, but couldn't figure out how to set it via run or run-main (yes, I did rtfm)
also, I'd really like locker to be locked by default
cheersadriaan
On Tue, Dec 20, 2011 at 3:38 PM, Josh Suereth <joshua.suereth@gmail.com> wrote:

That would be quick-compiler/run-main ....

On Dec 20, 2011 8:04 AM, "Mark Harrah" <dmharrah@gmail.com> wrote:

On Tue, 20 Dec 2011 12:52:57 +0100
Adriaan Moors <adriaan.moors@epfl.ch> wrote:

> I'm switching to the SBT build on my laptop now.
>
> Part of my usual workflow is to compile quick and then run that quick
> compiler on some source file, usually with -Xprint:typer or some other
> options. Is there a way to do all this (preferably automatically on file
> change) from the SBT console?

Depending on how Josh has things set up, you might be able to do:

 ~ run-main scala.tools.nsc.Main -Xprint:typer path/to/Source.scala

~ for triggering on source changes.  'run-main' accepts the main class to run and its arguments.

-Mark

> thanks!
> adriaan
>
> On Thu, Dec 15, 2011 at 8:02 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
>
> > Works finely, takes 625s. No idea though whether this is a lot or okay.
> >
> >
> > On 15 December 2011 19:18, Josh Suereth <joshua.suereth@gmail.com> wrote:
> >
> >> You can try:
> >>
> >> testsuite/run-partest-single pos *
> >>
> >> This will run all the pos tests, which shouldn't take too long.
> >>
> >> On Thu, Dec 15, 2011 at 12:52 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>
> >>> Btw, 512m permgen and 256m codecache seem to be sweet spots. Increasing
> >>> them does nothing, decreasing them degrades the performance. Speaking of
> >>> partest, I'll test it later one of these days - even a single run takes
> >>> non-trivial time.
> >>>
> >>>
> >>> On 15 December 2011 17:22, Josh Suereth <joshua.suereth@gmail.com>wrote:
> >>>
> >>>> Neato.  It's the Reserved Code Cache and PermGen that are important for
> >>>> the SBT build.   Most of the other caches can be GC'd in 0.11.2.
> >>>>
> >>>> I would love to see how partest does at those settings as well.
> >>>>
> >>>> - Josh
> >>>>
> >>>> On Thu, Dec 15, 2011 at 10:27 AM, Eugene Burmako <
> >>>> eugene.burmako@epfl.ch> wrote:
> >>>>
> >>>>> Amazingly, SBT can do fine with less memory than ant!!
> >>>>>
> >>>>> I figured that out when I tried to reduce Xms/Xmx in small steps. I
> >>>>> was astonished to see the following results:
> >>>>>
> >>>>> 4g 559s
> >>>>> 3g 592s
> >>>>> 2.5g 557s
> >>>>> 2g 548s
> >>>>> 1.5g 557s
> >>>>> 1g 559s
> >>>>>
> >>>>> To be sure that everything is as good as it looks, I verified the
> >>>>> amount of memory that got freed after I closed sbt via Resource Monitor
> >>>>> tool bundled with Windows. After the 1g run, closing sbt freed 1.5g of
> >>>>> memory (I guess, it's 1g for the heap + 512m for permgen, though I'm a noob
> >>>>> in JVM, so I might be misinterpreting the result). Oh, I'm in love! ♡
> >>>>>
> >>>>> My SBT script is as follows:
> >>>>>
> >>>>> set SCRIPT_DIR=%~dp0
> >>>>> set MEM_SETTINGS_MAIN=-Xms1024M -Xmx1024M
> >>>>> set MEM_SETTINGS_OTHER=-Xss128M -XX:MaxPermSize=512m
> >>>>> -XX:ReservedCodeCacheSize=256m -XX:+CMSClassUnloadingEnabled
> >>>>> set JVM_SETTINGS_OTHER=-XX:+UseParallelGC
> >>>>> set SBT_VERSION=0.11.2
> >>>>> java %MEM_SETTINGS_MAIN% %MEM_SETTINGS_OTHER% %JVM_SETTINGS_OTHER%
> >>>>> -Djline.WindowsTerminal.directConsole=false -jar
> >>>>> "%SCRIPT_DIR%sbt-launch-%SBT_VERSION%.jar" %*
> >>>>>
> >>>>> P.S. Setting Xms/Xms to 512m degrades the run time so badly, that I
> >>>>> get tired of waiting. Setting MaxPermSize to 256m degrades run time by 10%.
> >>>>> Setting ReservedCodeCacheSize to 128m degrades run time by 5%. Turning off
> >>>>> CMSClassUnloadingEnabled degrades run time by 5%. Turning off UseParallelGC
> >>>>> degrades run time by 5% (all the tests in P.S. have been done in comparison
> >>>>> with the baseline result of the script provided above).
> >>>>>
> >>>>> On 15 December 2011 11:43, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>>>>
> >>>>>> Hi folks,
> >>>>>>
> >>>>>> I'd like to take a few minutes of your time and advertise the SBT
> >>>>>> builder. This night I tried the latest version of SBT infrastructure, and,
> >>>>>> I must admit, I'm very surprised. I heard of horrible memory consumption
> >>>>>> and inferior speed of SBT, but it turned out to be almost the exact
> >>>>>> opposite.
> >>>>>>
> >>>>>> First of all, SBT is noticeably faster. On my laptop "sbt clean
> >>>>>> compile" finishes in 592s ("ant clean compile" takes about 11 minutes). And
> >>>>>> this is even taking into account that SBT builds all the small stuff (fjbg,
> >>>>>> jline, etc) that ant skips. Also, incremental builds are real awesome. In a
> >>>>>> few of my experiments with incremental compilation sbt beats ant by a
> >>>>>> factor of 1.5x or even 2x (20ish seconds vs 40ish seconds for ant).
> >>>>>>
> >>>>>> Secondly, memory consumption is not that big of a problem. I indeed
> >>>>>> had to increase Xms/Xmx (and especially -XX:MaxPermSize) values for SBT -
> >>>>>> my comfort zone with ant was 2G, here it is at 3G. Also, giving SBT more
> >>>>>> memory makes it even faster (bringing Xmx to 4G made clean compile complete
> >>>>>> in 559s). But personally for me even 3G is tolerable. I also wonder what
> >>>>>> kind of speed degradation SBT will suffer if I lower memory limit to 2.5G.
> >>>>>> That needs further experimenting.
> >>>>>>
> >>>>>> All in all, personally to me the SBT builder seems superior to the
> >>>>>> Ant one. It needs ~1G more RAM, but, in return, it provides higher
> >>>>>> compilation speeds. better functionality and is also cross-platform. If you
> >>>>>> can spare more memory, you can get even better speed.
> >>>>>>
> >>>>>> Come one, give it a try!
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Eugene
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >

Ah... -usejavacp might fix that.  I don't think run-main defaults any settings and you don't have a proper scala home on that project.

As for locker locking by default... I can probably hack that in tomorrow or so.   I'm personally a fan of the current explicit lock behavior so iremember when I've locked it, but my preferences are not those of everyone.

On Dec 26, 2011 12:45 PM, "Adriaan Moors" <adriaan.moors@epfl.ch> wrote:

after a successful compile, I did:
> quick-compiler/run-main scala.tools.nsc.Main -Xprint:typer test/files/pos/virtpatmat_alts_subst.scala 
<snip FOUR git rev-list & git log invocations..> 
[info] Running scala.tools.nsc.Main -Xprint:typer test/files/pos/virtpatmat_alts_subst.scalaerror: scala.reflect.internal.MissingRequirementError: object scala not found.
I assume the error is caused by the compiler's classpath is not being set correctly, but couldn't figure out how to set it via run or run-main (yes, I did rtfm)
also, I'd really like locker to be locked by default
cheersadriaan
On Tue, Dec 20, 2011 at 3:38 PM, Josh Suereth <joshua.suereth@gmail.com> wrote:

That would be quick-compiler/run-main ....

On Dec 20, 2011 8:04 AM, "Mark Harrah" <dmharrah@gmail.com> wrote:

On Tue, 20 Dec 2011 12:52:57 +0100
Adriaan Moors <adriaan.moors@epfl.ch> wrote:

> I'm switching to the SBT build on my laptop now.
>
> Part of my usual workflow is to compile quick and then run that quick
> compiler on some source file, usually with -Xprint:typer or some other
> options. Is there a way to do all this (preferably automatically on file
> change) from the SBT console?

Depending on how Josh has things set up, you might be able to do:

 ~ run-main scala.tools.nsc.Main -Xprint:typer path/to/Source.scala

~ for triggering on source changes.  'run-main' accepts the main class to run and its arguments.

-Mark

> thanks!
> adriaan
>
> On Thu, Dec 15, 2011 at 8:02 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
>
> > Works finely, takes 625s. No idea though whether this is a lot or okay.
> >
> >
> > On 15 December 2011 19:18, Josh Suereth <joshua.suereth@gmail.com> wrote:
> >
> >> You can try:
> >>
> >> testsuite/run-partest-single pos *
> >>
> >> This will run all the pos tests, which shouldn't take too long.
> >>
> >> On Thu, Dec 15, 2011 at 12:52 PM, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>
> >>> Btw, 512m permgen and 256m codecache seem to be sweet spots. Increasing
> >>> them does nothing, decreasing them degrades the performance. Speaking of
> >>> partest, I'll test it later one of these days - even a single run takes
> >>> non-trivial time.
> >>>
> >>>
> >>> On 15 December 2011 17:22, Josh Suereth <joshua.suereth@gmail.com>wrote:
> >>>
> >>>> Neato.  It's the Reserved Code Cache and PermGen that are important for
> >>>> the SBT build.   Most of the other caches can be GC'd in 0.11.2.
> >>>>
> >>>> I would love to see how partest does at those settings as well.
> >>>>
> >>>> - Josh
> >>>>
> >>>> On Thu, Dec 15, 2011 at 10:27 AM, Eugene Burmako <
> >>>> eugene.burmako@epfl.ch> wrote:
> >>>>
> >>>>> Amazingly, SBT can do fine with less memory than ant!!
> >>>>>
> >>>>> I figured that out when I tried to reduce Xms/Xmx in small steps. I
> >>>>> was astonished to see the following results:
> >>>>>
> >>>>> 4g 559s
> >>>>> 3g 592s
> >>>>> 2.5g 557s
> >>>>> 2g 548s
> >>>>> 1.5g 557s
> >>>>> 1g 559s
> >>>>>
> >>>>> To be sure that everything is as good as it looks, I verified the
> >>>>> amount of memory that got freed after I closed sbt via Resource Monitor
> >>>>> tool bundled with Windows. After the 1g run, closing sbt freed 1.5g of
> >>>>> memory (I guess, it's 1g for the heap + 512m for permgen, though I'm a noob
> >>>>> in JVM, so I might be misinterpreting the result). Oh, I'm in love! ♡
> >>>>>
> >>>>> My SBT script is as follows:
> >>>>>
> >>>>> set SCRIPT_DIR=%~dp0
> >>>>> set MEM_SETTINGS_MAIN=-Xms1024M -Xmx1024M
> >>>>> set MEM_SETTINGS_OTHER=-Xss128M -XX:MaxPermSize=512m
> >>>>> -XX:ReservedCodeCacheSize=256m -XX:+CMSClassUnloadingEnabled
> >>>>> set JVM_SETTINGS_OTHER=-XX:+UseParallelGC
> >>>>> set SBT_VERSION=0.11.2
> >>>>> java %MEM_SETTINGS_MAIN% %MEM_SETTINGS_OTHER% %JVM_SETTINGS_OTHER%
> >>>>> -Djline.WindowsTerminal.directConsole=false -jar
> >>>>> "%SCRIPT_DIR%sbt-launch-%SBT_VERSION%.jar" %*
> >>>>>
> >>>>> P.S. Setting Xms/Xms to 512m degrades the run time so badly, that I
> >>>>> get tired of waiting. Setting MaxPermSize to 256m degrades run time by 10%.
> >>>>> Setting ReservedCodeCacheSize to 128m degrades run time by 5%. Turning off
> >>>>> CMSClassUnloadingEnabled degrades run time by 5%. Turning off UseParallelGC
> >>>>> degrades run time by 5% (all the tests in P.S. have been done in comparison
> >>>>> with the baseline result of the script provided above).
> >>>>>
> >>>>> On 15 December 2011 11:43, Eugene Burmako <eugene.burmako@epfl.ch>wrote:
> >>>>>
> >>>>>> Hi folks,
> >>>>>>
> >>>>>> I'd like to take a few minutes of your time and advertise the SBT
> >>>>>> builder. This night I tried the latest version of SBT infrastructure, and,
> >>>>>> I must admit, I'm very surprised. I heard of horrible memory consumption
> >>>>>> and inferior speed of SBT, but it turned out to be almost the exact
> >>>>>> opposite.
> >>>>>>
> >>>>>> First of all, SBT is noticeably faster. On my laptop "sbt clean
> >>>>>> compile" finishes in 592s ("ant clean compile" takes about 11 minutes). And
> >>>>>> this is even taking into account that SBT builds all the small stuff (fjbg,
> >>>>>> jline, etc) that ant skips. Also, incremental builds are real awesome. In a
> >>>>>> few of my experiments with incremental compilation sbt beats ant by a
> >>>>>> factor of 1.5x or even 2x (20ish seconds vs 40ish seconds for ant).
> >>>>>>
> >>>>>> Secondly, memory consumption is not that big of a problem. I indeed
> >>>>>> had to increase Xms/Xmx (and especially -XX:MaxPermSize) values for SBT -
> >>>>>> my comfort zone with ant was 2G, here it is at 3G. Also, giving SBT more
> >>>>>> memory makes it even faster (bringing Xmx to 4G made clean compile complete
> >>>>>> in 559s). But personally for me even 3G is tolerable. I also wonder what
> >>>>>> kind of speed degradation SBT will suffer if I lower memory limit to 2.5G.
> >>>>>> That needs further experimenting.
> >>>>>>
> >>>>>> All in all, personally to me the SBT builder seems superior to the
> >>>>>> Ant one. It needs ~1G more RAM, but, in return, it provides higher
> >>>>>> compilation speeds. better functionality and is also cross-platform. If you
> >>>>>> can spare more memory, you can get even better speed.
> >>>>>>
> >>>>>> Come one, give it a try!
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Eugene
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >