memory limit

If you are running the interpreter, you can set the JAVA_OPTS environment variable.  For example, in bash,

  JAVA_OPTS=-Xmx4G scala

  scala> Runtime.getRuntime.maxMemory
  res0: Long = 3817799680

This also works when calling a compiled file.

Alternatively, you can run the class directly with java:

  java -Xmx4G -cp /usr/local/scala/lib/scala-library.jar:. MyClass

(obviously, use the correct path to your scala-library.jar, and scala-swing.jar if you use swing, etc.) and then you can put whatever options you like on the command-line.

  --Rex

On Thu, Oct 28, 2010 at 3:18 PM, Jiansen He <jiansenhe@googlemail.com> wrote:

Hi all,

I'm writing a small program which will count repeated string sequence in a text file.

My program heavily rely on List to processing intermediate results.  It works fine for small input size, but will raise exceptions for large input size.

Is it possible that the runtime has not allocated enough memory for my program?  If so, how could I request for more memory?


Jiansen

Thank you.

I would more like to run my program through command line.

concordance_0_1.scala is the file where I put my main method through a singleton concordance_0_1 object.

Usually, I compile it by   scalac concordance_0_1.scala, and run by   scala concordance_0_1 2 myText.txt.

I tried java -Xmx4G -cp /usr/local/scala/lib/scala-library.jar:.concordance_0_1, but got the same result as I run java -help.  Perhaps this is due to the -X option.
When I tried java -Xmx4G -cp /usr/local/scala/lib/scala-library.jar:.concordance_0_1 2 myText.txt, it raised java.lang.ClassNotFoundException

Any further suggestions?

Jiansen

You don't have spaces in the right place.  I will write a new line each place there should be a space
  java
  -Xmx4G
  -cp
  /usr/local/scala/lib/scala-library.jar:.
  concordance_0_1
  2
  myText.txt

Also, make sure you use the correct path to your scala-library.jar.  It almost certainly is NOT at /usr/local/scala/lib.  Mine isn't--that was just an example.

Finally, if you're running on a 32 bit machine (or with a 32 bit JVM), 4G is certainly too much memory for the JVM.  Try 1G or 2G.  (If you want more resolution, use e.g. 1500M (for megabytes).)

  --Rex

On Thu, Oct 28, 2010 at 3:48 PM, Jiansen He <jiansenhe@googlemail.com> wrote:

Thank you.

I would more like to run my program through command line.

concordance_0_1.scala is the file where I put my main method through a singleton concordance_0_1 object.

Usually, I compile it by   scalac concordance_0_1.scala, and run by   scala concordance_0_1 2 myText.txt.

I tried java -Xmx4G -cp /usr/local/scala/lib/scala-library.jar:.concordance_0_1, but got the same result as I run java -help.  Perhaps this is due to the -X option.
When I tried java -Xmx4G -cp /usr/local/scala/lib/scala-library.jar:.concordance_0_1 2 myText.txt, it raised java.lang.ClassNotFoundException

Any further suggestions?

Jiansen

Thanks again.  I can run the program in your way.

But it seems that I'm too confident about the correctness of my program :-(


I got following exception:

dyn-95-175-140-17:src jiansenhe$ java -mx4G -cp /Users/jiansenhe/scala/lib/scala-library.jar:. concordance_0_1 1 TMM.txt
File loaded in 0.162 secs
Exception in thread "main" java.lang.StackOverflowError
    at scala.collection.immutable.List.length(List.scala:45)
    at scala.collection.SeqLike$class.size(SeqLike.scala:221)
    at scala.collection.immutable.List.size(List.scala:45)
    at concordance_0_1$.gen_(concordance_0_1.scala:36)
    at concordance_0_1$.gen_(concordance_0_1.scala:37)
    at concordance_0_1$.gen_(concordance_0_1.scala:37)
    ...many same exceptions  ..

Line 32 to line 38 are cited as follows:
32     def gen_(i:Int, len:Int, l:List[String]): List[(List [String], Int)] = l match {
33         case Nil => Nil
34         case h::t =>
35 //          println("i="+i+" l.size=" + l.size + " len=" + len + " l.take(len)=" +l.take(len))
36             if (len>l.size) Nil
37             else insert((l.take(len),i), gen_(i+1, len, t))
38     }

After putting back the commend in Line 35, I got follows:

...
i=7000 l.size=4354 len=1 l.take(len)=List(say)
i=7001 l.size=4353 len=1 l.take(len)=List(a)
Exception in thread "main" java.lang.StackOverflowError
    at scala.runtime.AbstractFunction1.<init>(AbstractFunction1.scala:16)
    at scala.collection.TraversableOnce$$anonfun$addString$1.<init>(TraversableOnce.scala:502)
    at scala.collection.TraversableOnce$class.addString(TraversableOnce.scala:502)
    at scala.collection.immutable.List.addString(List.scala:45)
    at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:467)
    at scala.collection.immutable.List.mkString(List.scala:45)
    at scala.collection.TraversableLike$class.toString(TraversableLike.scala:708)
    at scala.collection.SeqLike$class.toString(SeqLike.scala:869)
    at scala.collection.immutable.List.toString(List.scala:45)
    at java.lang.String.valueOf(String.java:2826)
    at scala.collection.mutable.StringBuilder.append(StringBuilder.scala:178)
    at concordance_0_1$.gen_(concordance_0_1.scala:35)
    at concordance_0_1$.gen_(concordance_0_1.scala:37)
    at concordance_0_1$.gen_(concordance_0_1.scala:37)
    at concordance_0_1$.gen_(concordance_0_1.scala:37)
   ...many same exceptions...


The gen_ function is used as follows:
  gen_(0, len, strings) will generate all sequences at length len, together with index of each sequence.


Any ideas where could go wrong?

Ideally, a list in scala might have infinite length, right?  Besides, the exception is raised really soon. 

Jiansen

On 10/28/2010 01:50 PM, Jiansen He wrote:
> Thanks again. I can run the program in your way.
>
> But it seems that I'm too confident about the correctness of my program :-(
>
>
> I got following exception:
>
> dyn-95-175-140-17:src jiansenhe$ java -mx4G -cp
> /Users/jiansenhe/scala/lib/scala-library.jar:. concordance_0_1 1 TMM.txt
> File loaded in 0.162 secs
> Exception in thread "main" java.lang.StackOverflowError
> [snip]

A stack overflow in Java/Scala is often the result of a recursive
function gone awry (e.g. infinite recursion). In this case, it's
probably just that your list is too large for the available stack space.

There are a couple ways to fix this:
1. increase stack size (I think its the -Xss java parameter)
2. rewrite your function using iteration rather than recursion
3. rewrite your function using tail recursion

> Any ideas where could go wrong?
>
> Ideally, a list in scala might have infinite length, right? Besides, the
> exception is raised really soon.

I assume you mean that a list can have arbitrary length, which is true.
It cannot have infinite length, but a Stream can.

Thanks Aaron.

Since I'm actually running a benchmark for different languages, I'm not going to change the code to tail recursive style. 

My program works fine after using -Xss java parameters.

Thanks again and again

Jiansen

Sorry for forgetting thank you, Rex.

My apologize~~

Jiansen

On Fri, Oct 29, 2010 at 8:58 AM, Jiansen He <jiansenhe@googlemail.com> wrote:

Thanks Aaron.

Since I'm actually running a benchmark for different languages, I'm not going to change the code to tail recursive style. 

My program works fine after using -Xss java parameters.

Thanks again and again

Jiansen