Another compiler crash :(

I have been using Scala since 2007. I have seen the type-checker crash
hundreds of times. Every time I had a type/inferencing error. I bet you
do too. Just sayin'

On 31/05/11 09:07, Lex wrote:
> I don't have any more time or patience tracking down and reporting
> compiler bugs.
> The crash has something to do with accessing private members. So it's
> not crashing on valid code. There will be no follow ups, hope it
> helps.
>
> [exec] Exception in thread "main" scala.MatchError: of class class scala.tools.nsc.typechecker.Infer$Inferencer$AccessError>
> (of class scala.tools.nsc.typechecker.Infer$Inferencer$AccessError)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:754)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$4.apply(Trees.scala:777)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$4.apply(Trees.scala:776)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.atOwner(Trees.scala:899)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:38)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:31)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.atOwner(SuperAccessors.scala:257)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:775)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:891)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:889)
> [exec] at scala.collection.immutable.List.loop$1(List.scala:117)
> [exec] at scala.collection.immutable.List.mapConserve(List.scala:133)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformStats(Trees.scala:889)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:799)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:891)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:889)
> [exec] at scala.collection.immutable.List.loop$1(List.scala:117)
> [exec] at scala.collection.immutable.List.mapConserve(List.scala:133)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformStats(Trees.scala:889)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:799)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:821)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:791)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:891)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:889)
> [exec] at scala.collection.immutable.List.loop$1(List.scala:117)
> [exec] at scala.collection.immutable.List.mapConserve(List.scala:133)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformStats(Trees.scala:889)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:799)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$1$$anonfun$apply$1.apply(SuperAccessors.scala:47)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$1$$anonfun$apply$1.apply(SuperAccessors.scala:47)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.scala$tools$nsc$typechecker$SuperAccessors$SuperAccTransformer$$withInvalidOwner(SuperAccessors.scala:263)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$1.apply(SuperAccessors.scala:47)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$1.apply(SuperAccessors.scala:45)
> [exec] at scala.Tuple2$Zipped$$anonfun$map$1.apply(Tuple2.scala:65)
> [exec] at scala.Tuple2$Zipped$$anonfun$map$1.apply(Tuple2.scala:63)
> [exec] at
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> [exec] at scala.collection.immutable.List.foreach(List.scala:45)
> [exec] at scala.Tuple2$Zipped.map(Tuple2.scala:63)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transformArgs(SuperAccessors.scala:45)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:237)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:873)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:873)
> [exec] at scala.collection.immutable.List.loop$1(List.scala:117)
> [exec] at scala.collection.immutable.List.mapConserve(List.scala:133)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformTrees(Trees.scala:873)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$4.apply(SuperAccessors.scala:185)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer$$anonfun$4.apply(SuperAccessors.scala:185)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.atOwner(Trees.scala:899)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:38)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:31)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.atOwner(SuperAccessors.scala:257)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:185)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformTemplate(Trees.scala:875)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$2.apply(Trees.scala:767)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$2.apply(Trees.scala:766)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.atOwner(Trees.scala:899)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:38)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:31)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.atOwner(SuperAccessors.scala:257)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:765)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:53)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:172)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:891)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:889)
> [exec] at scala.collection.immutable.List.loop$1(List.scala:117)
> [exec] at scala.collection.immutable.List.mapConserve(List.scala:133)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformStats(Trees.scala:889)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$1.apply(Trees.scala:761)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer$$anonfun$transform$1.apply(Trees.scala:761)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.atOwner(Trees.scala:899)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:38)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:31)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.atOwner(SuperAccessors.scala:257)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:760)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.scala$tools$nsc$transform$TypingTransformers$TypingTransformer$$super$transform(TypingTransformers.scala:49)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$2.apply(TypingTransformers.scala:51)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$2.apply(TypingTransformers.scala:51)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.atOwner(Trees.scala:899)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:38)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:31)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.atOwner(SuperAccessors.scala:257)
> [exec] at
> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:51)
> [exec] at
> scala.tools.nsc.typechecker.SuperAccessors$SuperAccTransformer.transform(SuperAccessors.scala:243)
> [exec] at
> scala.tools.nsc.ast.Trees$Transformer.transformUnit(Trees.scala:892)
> [exec] at
> scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30)
> [exec] at
> scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply(Global.scala:326)
> [exec] at
> scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply(Global.scala:326)
> [exec] at
> scala.tools.nsc.reporters.Reporter.withSource(Reporter.scala:47)
> [exec] at
> scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:326)
> [exec] at
> scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:294)
> [exec] at
> scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:294)
> [exec] at scala.collection.Iterator$class.foreach(Iterator.scala:652)
> [exec] at
> scala.collection.mutable.ListBuffer$$anon$1.foreach(ListBuffer.scala:311)
> [exec] at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:294)
> [exec] at scala.tools.nsc.Global$Run.compileSources(Global.scala:949)
> [exec] at scala.tools.nsc.Global$Run.compile(Global.scala:1034)
> [exec] at scala.tools.nsc.Main$.process(Main.scala:106)
> [exec] at scala.tools.nsc.Main$.main(Main.scala:123)
> [exec] at scala.tools.nsc.Main.main(Main.scala)

He,
why does the following code not compile?

trait Ord[T] {

  def <(x : T, y : T): Boolean

}

object Main {

  def main(args : Array[String]) : Unit = {

    

    def lt[T](x : T, y : T)(implicit o : Ord[T]) : T = if(o.<(x, y)) x else y }

    

    { // doesn't compile, which is OK

      //lt(1, 2) 

    }    

    { // OK

      implicit object intOrd extends Ord[Int] {  def <(x : Int, y : Int) = x < y  }

      assert(lt(1, 2) == 1)

    }

    

    { // doesn't compile, which is OK

      //lt(List(1, 2, 2), List(1, 2, 3)) 

    }

    

    {

      implicit object intOrd extends Ord[Int] {  def <(x : Int, y : Int) = x < y  }

      

      implicit def listOrd[T : Ord] = new Ord[List[T]] {

        def <(xs : List[T], ys : List[T]) : Boolean = (xs, ys) match {

          case (_, Nil)           => false

          case (Nil, _)           => true

          case (x :: xs, y :: ys) =>

            { // does not compile, WHY????

              val ev = implicitly[Ord[T]]

              import ev._

            

              x < y || x == y && xs < ys

            //^^^^^

            }

            { // does not compile, WHY????

              implicitly[Ord[T]].<(x, y) || x == y && xs < ys

              //                                      ^^^^^^^

            }

            { // works, but feels awkward

              implicitly[Ord[T]].<(x, y) || x == y && <(xs, ys)

            }

        }

      }

      assert(lt(List(1, 2, 2), List(1, 2, 3)) == List(1, 2, 2))      

    }

        

  }

}

What am I missing here?

Thanks,

  /nm

On my phone, so can't post an example, but it looks like you are missing some kind of wrapper to give a value the '<' method. For a good example of this, see MA.scala in scalaz.

Derek Williams

On Aug 15, 2011 9:28 AM, "Normen Müller" <normen.mueller@googlemail.com> wrote:
> He,
>
> why does the following code not compile?
>
> trait Ord[T] {
>
> def <(x : T, y : T): Boolean
>
> }
>
>
> object Main {
>
> def main(args : Array[String]) : Unit = {
>
>
>
> def lt[T](x : T, y : T)(implicit o : Ord[T]) : T = if(o.<(x, y)) x elsey }
>
>
>
> { // doesn't compile, which is OK
>
> //lt(1, 2)
>
> }
>
> { // OK
>
> implicit object intOrd extends Ord[Int] { def <(x : Int, y : Int) = x
> < y }
>
> assert(lt(1, 2) == 1)
>
> }
>
>
>
> { // doesn't compile, which is OK
>
> //lt(List(1, 2, 2), List(1, 2, 3))
>
> }
>
>
>
> {
>
> implicit object intOrd extends Ord[Int] { def <(x : Int, y : Int) = x
> < y }
>
>
>
> implicit def listOrd[T : Ord] = new Ord[List[T]] {
>
> def <(xs : List[T], ys : List[T]) : Boolean = (xs, ys) match {
>
> case (_, Nil) => false
>
> case (Nil, _) => true
>
> case (x :: xs, y :: ys) =>
>
> { // does not compile, WHY????
>
> val ev = implicitly[Ord[T]]
>
> import ev._
>
>
>
> x < y || x == y && xs < ys
>
> //^^^^^
>
> }
>
> { // does not compile, WHY????
>
> implicitly[Ord[T]].<(x, y) || x == y && xs < ys
>
> // ^^^^^^^
>
> }
>
> { // works, but feels awkward
>
> implicitly[Ord[T]].<(x, y) || x == y && <(xs, ys)
>
> }
>
> }
>
> }
>
> assert(lt(List(1, 2, 2), List(1, 2, 3)) == List(1, 2, 2))
>
> }
>
>
>
> }
>
> }
>
>
> What am I missing here?
>
>
> Thanks,
>
> /nm

On Mon, Aug 15, 2011 at 08:22:22AM -0700, Normen Müller wrote:
> { // does not compile, WHY????
>
> x < y || x == y && xs < ys
>
> //^^^^^

This calls the < method on x with the single argument y, for an object
of unconstrained abstract type T which does not have such a method (which
is incidentally exactly what the compiler error told you), and there is no
implicit in scope that provides such a method. Ord only provides a two
parameter < method.

> What am I missing here?

While haskell desugars binary operators to two argument functions,
scala desugars them to one argument methods.

Maybe you want:

trait Ord[T] {
def <(that : T) : Boolean
}

// both <% and the return type are necessary
implicit def listOrd[T <% Ord[T]](xs : List[T]) : Ord[List[T]] = new Ord[List[T]] {
def <(ys : List[T]) : Boolean = (xs, ys) match {
case (_, Nil) => false
case (Nil, _) => true
case (x :: xs, y :: ys) => {
x < y || x == y && xs < ys
}
}
}

- Florian.

On Aug 15, 2011, at 7:56 PM, Florian Hars wrote:

> On Mon, Aug 15, 2011 at 08:22:22AM -0700, Normen Müller wrote:
>> { // does not compile, WHY????
>>
>> x < y || x == y && xs < ys
>>
>> //^^^^^
>
>
> This calls the < method on x with the single argument y, for an object
> of unconstrained abstract type T which does not have such a method (which
> is incidentally exactly what the compiler error told you), and there is no
> implicit in scope that provides such a method. Ord only provides a two
> parameter < method.

Damn! You're so right! THANX!!!

>> What am I missing here?
>
> While haskell desugars binary operators to two argument functions,
> scala desugars them to one argument methods.
>
> Maybe you want:
>
> trait Ord[T] {
> def <(that : T) : Boolean
> }
>
> // both <% and the return type are necessary
> implicit def listOrd[T <% Ord[T]](xs : List[T]) : Ord[List[T]] = new Ord[List[T]] {
> def <(ys : List[T]) : Boolean = (xs, ys) match {
> case (_, Nil) => false
> case (Nil, _) => true
> case (x :: xs, y :: ys) => {
> x < y || x == y && xs < ys
> }
> }
> }

Yep!

Cheers,
/nm

def lt(x: Int, y: Int): Boolean

x lt y // Does not compile
lt(x,y) // Does compile.
You want
<(x,y) || ...
At least to be a typeclass/typetrait.

On Mon, Aug 15, 2011 at 2:10 PM, Normen Müller <normen.mueller@googlemail.com> wrote:

On Aug 15, 2011, at 7:56 PM, Florian Hars wrote:

> On Mon, Aug 15, 2011 at 08:22:22AM -0700, Normen Müller wrote:
>>           { // does not compile, WHY????
>>
>>              x < y || x == y && xs < ys
>>
>>            //^^^^^
>
>
> This calls the < method on x with the single argument y, for an object
> of unconstrained abstract type T which does not have such a method (which
> is incidentally exactly what the compiler error told you), and there is no
> implicit in scope that provides such a method. Ord only provides a two
> parameter < method.

Damn! You're so right! THANX!!!

>> What am I missing here?
>
> While haskell desugars binary operators to two argument functions,
> scala desugars them to one argument methods.
>
> Maybe you want:
>
> trait Ord[T] {
>  def <(that : T) : Boolean
> }
>
> // both <% and the return type are necessary
> implicit def listOrd[T <% Ord[T]](xs : List[T]) : Ord[List[T]] = new Ord[List[T]] {
>  def <(ys : List[T]) : Boolean = (xs, ys) match {
>    case (_, Nil)           => false
>    case (Nil, _)           => true
>    case (x :: xs, y :: ys) => {
>      x < y || x == y && xs < ys
>    }
>  }
> }

Yep!

Cheers,
 /nm

Dear all,
I have enjoyed using Scala to refactor a java project, where I was doing numerical optimization...

However, I experienced a significant performant loss inside the optimizer, which can be imputed only to the cost function evaluation.

The changed I did are the followings:
- Surround a java native array of doubles with a class whose avoid direct indexing of array creating bugs.
- I replaced a method who was iterating over a java collection with a "scala parallel collection" foreach.

I am still doing some profiling, but can you guess where does the performance loss come frome? Maybe difficulties to inline the procedure? Maybe I should make some final?

Thank you for your support.

Best Regards

Edmondo

What?
You want us to first imagine what the code looks like, and then imagine what your benchmark looks like, then imagine what Scala version you're using, then imagine what compiler settings you're using, then imagine what OS you're on, then imagine Java version and vendor, then imagine what hardware you're on, then imagine the bottleneck?

If I could do that I'd play all lotteries in the world.

Cheers,
V

On Sep 30, 2011 11:06 AM, "Edmondo Porcu" <edmondo.porcu@gmail.com> wrote:
> Dear all,
> I have enjoyed using Scala to refactor a java project, where I was doing
> numerical optimization...
>
> However, I experienced a significant performant loss inside the optimizer,
> which can be imputed only to the cost function evaluation.
>
> The changed I did are the followings:
> - Surround a java native array of doubles with a class whose avoid direct
> indexing of array creating bugs.
> - I replaced a method who was iterating over a java collection with a "scala
> parallel collection" foreach.
>
> I am still doing some profiling, but can you guess where does the
> performance loss come frome? Maybe difficulties to inline the procedure?
> Maybe I should make some final?
>
> Thank you for your support.
>
> Best Regards
>
> Edmondo

On Fri, Sep 30, 2011 at 11:06 AM, Edmondo Porcu wrote:
> Dear all,
> I have enjoyed using Scala to refactor a java project, where I was doing
> numerical optimization...
>
> However, I experienced a significant performant loss inside the optimizer,
> which can be imputed only to the cost function evaluation.
>
> The changed I did are the followings:
> - Surround a java native array of doubles with a class whose avoid direct
> indexing of array creating bugs.
> - I replaced a method who was iterating over a java collection with a "scala
> parallel collection" foreach.
>
> I am still doing some profiling, but can you guess where does the
> performance loss come frome? Maybe difficulties to inline the procedure?
> Maybe I should make some final?

As Victor said, you should publish a small benchmark, and we can offer
specific advice.

In general, boxing of primitives can have a serious impact on the
performance of numerical code. You can use scalac -Xprint:jvm to see
these. For example:

scala -Xprint:jvm -e "val x: Int = Array(1).head; x"

Main$$anon$1.this.x = scala.Int.unbox(scala.this.Predef.intArrayOps(scala.
Array.apply(1, scala.this.Predef.wrapIntArray(Array[Int]{}))).head());
Main$$anon$1.this.x();

-jason

-jason

Dear Viktor,
you are right , and here what I can tell:

I am working with Scala 2.9.1 and jdk 7b17 windows 64bit, and I am using Windows Vista 64 bits.

I have done a profiling sesson which told me that 13.6% of my whole cpu time was spent on scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.sync and about 7% on
scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.start

It appears my collection is too small to take advantage of the parallelization...

Am I right?



2011/9/30 √iktor Ҡlang <viktor.klang@gmail.com>

What?
You want us to first imagine what the code looks like, and then imagine what your benchmark looks like, then imagine what Scala version you're using, then imagine what compiler settings you're using, then imagine what OS you're on, then imagine Java version and vendor, then imagine what hardware you're on, then imagine the bottleneck?

If I could do that I'd play all lotteries in the world.

Cheers,
V

On Sep 30, 2011 11:06 AM, "Edmondo Porcu" <edmondo.porcu@gmail.com> wrote:
> Dear all,
> I have enjoyed using Scala to refactor a java project, where I was doing
> numerical optimization...
>
> However, I experienced a significant performant loss inside the optimizer,
> which can be imputed only to the cost function evaluation.
>
> The changed I did are the followings:
> - Surround a java native array of doubles with a class whose avoid direct
> indexing of array creating bugs.
> - I replaced a method who was iterating over a java collection with a "scala
> parallel collection" foreach.
>
> I am still doing some profiling, but can you guess where does the
> performance loss come frome? Maybe difficulties to inline the procedure?
> Maybe I should make some final?
>
> Thank you for your support.
>
> Best Regards
>
> Edmondo

Contact me off list if you want to talk about this in a less public setting. I do a lot of numerical optimization in scala and may be able to help.

marc
@splittingfield

Dear Marc,
I would be glad to discuss with you on these topics!

Do you have a skype account?

Best Regards

2011/10/3 marc <millstone@gmail.com>

Contact me off list if you want to talk about this in a less public setting. I do a lot of numerical optimization in scala and may be able to help.

marc
@splittingfield

Dear all,
I am encountering a tedious problem in Scala.

I am calling a third party library which receives as a parameter a java object.

This object however has to have certain characteristics in order for the method to succeed, as you do for example in Hibernate, where you can call session.persist() on every java.lang.Object but that will fail unless the object is annotated for example with @Entity.

My use case is the same, and I need to call a third party java library like that.

gigaSpace.write(curve, Lease.FOREVER,UpdateModifiers.UPDATE_OR_WRITE |UpdateModifiers.NO_RETURN_VALUE);

However, here i want to call gigaSpace.write(object, long, int) , while scala convert automatically that to a tuple, and calls

gigaspaces.write (Tuple3(object,long,int)) and this fails.

How can I avoid my parameters to be converted into a tuple ?

Thank you for your precious help

Best Regards


Edmondo

It appears to be a known problem, any suggested workaround?

http://scala-programming-language.1934581.n4.nabble.com/scalac-mind-your-own-tuple-business-td2285496.html

Best regards

---------- Forwarded message ----------
From: Edmondo Porcu <edmondo.porcu@gmail.com>
Date: 2011/10/4
Subject: Scala tuples and method call ! help :(
To: scala-user <scala-user@googlegroups.com>


Dear all,
I am encountering a tedious problem in Scala.

I am calling a third party library which receives as a parameter a java object.

This object however has to have certain characteristics in order for the method to succeed, as you do for example in Hibernate, where you can call session.persist() on every java.lang.Object but that will fail unless the object is annotated for example with @Entity.

My use case is the same, and I need to call a third party java library like that.

gigaSpace.write(curve, Lease.FOREVER,UpdateModifiers.UPDATE_OR_WRITE |UpdateModifiers.NO_RETURN_VALUE);

However, here i want to call gigaSpace.write(object, long, int) , while scala convert automatically that to a tuple, and calls

gigaspaces.write (Tuple3(object,long,int)) and this fails.

How can I avoid my parameters to be converted into a tuple ?

Thank you for your precious help

Best Regards


Edmondo

Is Lease.FOREVER of type Long? If it isn't, you can write Lease.FOREVER : Long.

Scala is going for a tuple3 because the types you passed do not match
the types expected by write, _and_ there's a write(Object obj)
overload.

On Tue, Oct 4, 2011 at 06:00, Edmondo Porcu wrote:
> Dear all,
> I am encountering a tedious problem in Scala.
>
> I am calling a third party library which receives as a parameter a java
> object.
>
> This object however has to have certain characteristics in order for the
> method to succeed, as you do for example in Hibernate, where you can call
> session.persist() on every java.lang.Object but that will fail unless the
> object is annotated for example with @Entity.
>
> My use case is the same, and I need to call a third party java library like
> that.
>
> gigaSpace.write(curve, Lease.FOREVER,UpdateModifiers.UPDATE_OR_WRITE
> |UpdateModifiers.NO_RETURN_VALUE);
>
> However, here i want to call gigaSpace.write(object, long, int) , while
> scala convert automatically that to a tuple, and calls
>
> gigaspaces.write (Tuple3(object,long,int)) and this fails.
>
> How can I avoid my parameters to be converted into a tuple ?
>
> Thank you for your precious help
>
> Best Regards
>
>
> Edmondo
>
>
>

Hi,
On Sep 30, 2011, at 11:34 AM, Edmondo Porcu wrote:

I am working with Scala 2.9.1 and jdk 7b17 windows 64bit, and I am using Windows Vista 64 bits.

I have done a profiling sesson which told me that 13.6% of my whole cpu time was spent on scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.sync and about 7% on
scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.start

It appears my collection is too small to take advantage of the parallelization...

Given your profiling info, it seems the parallel collections runtime is not using the fork/join pool to execute tasks, but falls back to the slower thread-pool task support. The reason is that it doesn't correctly detect that it can use the fork/join pool on JDK 7. This should be fixed as soon as possible.
On JDK 6 your code is probably significantly faster and scales better.
Cheers, Philipp

2011/9/30 √iktor Ҡlang <viktor.klang@gmail.com>

What?
You want us to first imagine what the code looks like, and then imagine what your benchmark looks like, then imagine what Scala version you're using, then imagine what compiler settings you're using, then imagine what OS you're on, then imagine Java version and vendor, then imagine what hardware you're on, then imagine the bottleneck?

If I could do that I'd play all lotteries in the world.

Cheers,
V

On Sep 30, 2011 11:06 AM, "Edmondo Porcu" <edmondo.porcu@gmail.com> wrote:
> Dear all,
> I have enjoyed using Scala to refactor a java project, where I was doing
> numerical optimization...
>
> However, I experienced a significant performant loss inside the optimizer,
> which can be imputed only to the cost function evaluation.
>
> The changed I did are the followings:
> - Surround a java native array of doubles with a class whose avoid direct
> indexing of array creating bugs.
> - I replaced a method who was iterating over a java collection with a "scala
> parallel collection" foreach.
>
> I am still doing some profiling, but can you guess where does the
> performance loss come frome? Maybe difficulties to inline the procedure?
> Maybe I should make some final?
>
> Thank you for your support.
>
> Best Regards
>
> Edmondo

-- Co-author, "Actors in Scala" (Artima Inc, 2011) Postdoc, EPFL and Stanford University

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround
Chris

From: philipp.haller@epfl.ch
To: edmondo.porcu@gmail.com
CC: viktor.klang@gmail.com; scala-user@googlegroups.com
Subject: Re: [scala-user] Scala performance :(
Date: Tue, 4 Oct 2011 14:01:42 +0000

Hi,
On Sep 30, 2011, at 11:34 AM, Edmondo Porcu wrote:

I am working with Scala 2.9.1 and jdk 7b17 windows 64bit, and I am using Windows Vista 64 bits.

I have done a profiling sesson which told me that 13.6% of my whole cpu time was spent on scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.sync and about 7% on
scala.collection.parallel.ThreadPoolTasks$TaskImpl$class.start

It appears my collection is too small to take advantage of the parallelization...

Given your profiling info, it seems the parallel collections runtime is not using the fork/join pool to execute tasks, but falls back to the slower thread-pool task support. The reason is that it doesn't correctly detect that it can use the fork/join pool on JDK 7. This should be fixed as soon as possible.
On JDK 6 your code is probably significantly faster and scales better.
Cheers, Philipp

2011/9/30 √iktor Ҡlang <viktor.klang@gmail.com>

What?
You want us to first imagine what the code looks like, and then imagine what your benchmark looks like, then imagine what Scala version you're using, then imagine what compiler settings you're using, then imagine what OS you're on, then imagine Java version and vendor, then imagine what hardware you're on, then imagine the bottleneck?
If I could do that I'd play all lotteries in the world.
Cheers,
V
On Sep 30, 2011 11:06 AM, "Edmondo Porcu" <edmondo.porcu@gmail.com> wrote:
> Dear all,
> I have enjoyed using Scala to refactor a java project, where I was doing
> numerical optimization...
>
> However, I experienced a significant performant loss inside the optimizer,
> which can be imputed only to the cost function evaluation.
>
> The changed I did are the followings:
> - Surround a java native array of doubles with a class whose avoid direct
> indexing of array creating bugs.
> - I replaced a method who was iterating over a java collection with a "scala
> parallel collection" foreach.
>
> I am still doing some profiling, but can you guess where does the
> performance loss come frome? Maybe difficulties to inline the procedure?
> Maybe I should make some final?
>
> Thank you for your support.
>
> Best Regards
>
> Edmondo

-- Co-author, "Actors in Scala" (Artima Inc, 2011) Postdoc, EPFL and Stanford University

That's right, there was a long-int misunderstanding which mislead the compiler . my fault
Thanks

2011/10/4 Daniel Sobral <dcsobral@gmail.com>

Is Lease.FOREVER of type Long? If it isn't, you can write Lease.FOREVER : Long.

Scala is going for a tuple3 because the types you passed do not match
the types expected by write, _and_ there's a write(Object obj)
overload.

On Tue, Oct 4, 2011 at 06:00, Edmondo Porcu <edmondo.porcu@gmail.com> wrote:
> Dear all,
> I am encountering a tedious problem in Scala.
>
> I am calling a third party library which receives as a parameter a java
> object.
>
> This object however has to have certain characteristics in order for the
> method to succeed, as you do for example in Hibernate, where you can call
> session.persist() on every java.lang.Object but that will fail unless the
> object is annotated for example with @Entity.
>
> My use case is the same, and I need to call a third party java library like
> that.
>
> gigaSpace.write(curve, Lease.FOREVER,UpdateModifiers.UPDATE_OR_WRITE
> |UpdateModifiers.NO_RETURN_VALUE);
>
> However, here i want to call gigaSpace.write(object, long, int) , while
> scala convert automatically that to a tuple, and calls
>
> gigaspaces.write (Tuple3(object,long,int)) and this fails.
>
> How can I avoid my parameters to be converted into a tuple ?
>
> Thank you for your precious help
>
> Best Regards
>
>
> Edmondo
>
>
>



--
Daniel C. Sobral

I travel to the future all the time.

On Tue, Oct 4, 2011 at 11:23, Edmondo Porcu wrote:
> That's right, there was a long-int misunderstanding which mislead the
> compiler . my fault

Don't take all blame upon you. The compiler definitely did something
very unexpected instead of producing a compilation error (or just type
widening). With so many places where I'd like auto-tupling or
auto-detupling, it seems the only place it works is the one I don't
like it in.

> Thanks
>
> 2011/10/4 Daniel Sobral
>>
>> Is Lease.FOREVER of type Long? If it isn't, you can write Lease.FOREVER :
>> Long.
>>
>> Scala is going for a tuple3 because the types you passed do not match
>> the types expected by write, _and_ there's a write(Object obj)
>> overload.
>>
>> On Tue, Oct 4, 2011 at 06:00, Edmondo Porcu
>> wrote:
>> > Dear all,
>> > I am encountering a tedious problem in Scala.
>> >
>> > I am calling a third party library which receives as a parameter a java
>> > object.
>> >
>> > This object however has to have certain characteristics in order for the
>> > method to succeed, as you do for example in Hibernate, where you can
>> > call
>> > session.persist() on every java.lang.Object but that will fail unless
>> > the
>> > object is annotated for example with @Entity.
>> >
>> > My use case is the same, and I need to call a third party java library
>> > like
>> > that.
>> >
>> > gigaSpace.write(curve, Lease.FOREVER,UpdateModifiers.UPDATE_OR_WRITE
>> > |UpdateModifiers.NO_RETURN_VALUE);
>> >
>> > However, here i want to call gigaSpace.write(object, long, int) , while
>> > scala convert automatically that to a tuple, and calls
>> >
>> > gigaspaces.write (Tuple3(object,long,int)) and this fails.
>> >
>> > How can I avoid my parameters to be converted into a tuple ?
>> >
>> > Thank you for your precious help
>> >
>> > Best Regards
>> >
>> >
>> > Edmondo
>> >
>> >
>> >
>>
>>
>>
>> --
>> Daniel C. Sobral
>>
>> I travel to the future all the time.
>
>

On Tue, Oct 4, 2011 at 9:23 AM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround

Wow, fragile like browser detection. Isn't there a way to find out if the FJ pool is available, instead of resorting to vendor names?

reflection?

Am 04.10.2011 16:29, schrieb Nils Kilden-Pedersen:

CABDULvXEH1_N037v4ewhXQ5pEHBuJGOkbSZ+r3rWyPd+WT1CHw [at] mail [dot] gmail [dot] com" type="cite"> On Tue, Oct 4, 2011 at 9:23 AM, Chris Marshall <oxbow_lakes [at] hotmail [dot] com" rel="nofollow">oxbow_lakes@hotmail.com> wrote:

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround

Wow, fragile like browser detection. Isn't there a way to find out if the FJ pool is available, instead of resorting to vendor names?

On Tue, Oct 4, 2011 at 7:21 PM, HamsterofDeath <h-star@gmx.de> wrote:

reflection?

"Whenever someone suggests reflection, Tony gives them a virtual slap in the face" - Old Jungle Proverb
 

Am 04.10.2011 16:29, schrieb Nils Kilden-Pedersen:

On Tue, Oct 4, 2011 at 9:23 AM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround

Wow, fragile like browser detection. Isn't there a way to find out if the FJ pool is available, instead of resorting to vendor names?

--
Viktor Klang

Akka Tech LeadTypesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

On Tue, Oct 4, 2011 at 3:30 PM, Daniel Sobral wrote:
> Don't take all blame upon you. The compiler definitely did something
> very unexpected instead of producing a compilation error (or just type
> widening). With so many places where I'd like auto-tupling or
> auto-detupling, it seems the only place it works is the one I don't
> like it in.

Yes, I wish it wasn't there at all, personally.

Best,
Ismael

Yes, reflection.

On Tue, Oct 4, 2011 at 12:21 PM, HamsterofDeath <h-star@gmx.de> wrote:

reflection?

Am 04.10.2011 16:29, schrieb Nils Kilden-Pedersen:

On Tue, Oct 4, 2011 at 9:23 AM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround

Wow, fragile like browser detection. Isn't there a way to find out if the FJ pool is available, instead of resorting to vendor names?

no rule is always correct.
the only true paradox.

Am 04.10.2011 19:25, schrieb √iktor Ҡlang:

NpPa7iSuc4L4K0e_zRn_ovhBvnaWsui0tFo3_ng [at] mail [dot] gmail [dot] com" type="cite">

On Tue, Oct 4, 2011 at 7:21 PM, HamsterofDeath <h-star [at] gmx [dot] de" rel="nofollow">h-star@gmx.de> wrote:

reflection?

"Whenever someone suggests reflection, Tony gives them a virtual slap in the face" - Old Jungle Proverb
 

Am 04.10.2011 16:29, schrieb Nils Kilden-Pedersen:

On Tue, Oct 4, 2011 at 9:23 AM, Chris Marshall <oxbow_lakes [at] hotmail [dot] com" target="_blank" rel="nofollow">oxbow_lakes@hotmail.com> wrote:

Note that the use of a simple system property setting means you get the FJ pool. See this issue I raised a while back: https://issues.scala-lang.org/browse/SI-4960 which includes a workaround

Wow, fragile like browser detection. Isn't there a way to find out if the FJ pool is available, instead of resorting to vendor names?

--
Viktor Klang

Akka Tech Lead Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang