scala.util.Random.nextInt bug?

Some time ago (don't remembed exact version) Scala library had a bug
where Random constuctor initialized seed by current time only.

You should do:

===
val r = new scala.util.Random
val rnd = r.nextInt(_: Int)
for (i <- 1 to 100) print(rnd(100) + "\t"))
===

It is anyway better.

P. S. Next time send a version of Scala and JDK you are using.

S.

On Fri, Nov 20, 2009 at 04:23, Riobard wrote:
> Hi,
>
> I was trying to implement Fisher–Yates shuffle in Scala and hit this strange problem. I'm not sure if it is a bug, or I misunderstood something:
>
>    val rnd = (new scala.util.Random).nextInt(_: Int)
>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>
> usually I get something like this
>
> 85      85      85      10      10      10      10      10      10      10      11
> 0       10      10      10      10      10      10      10      10      10      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      59      59      59      55
> 9       59      59      59      59      59      59      59      59      59      55
> 9       59      59      59      59      59      59      59      59      87      88
> 7       87      87      87      87      87      87      87      87      87      8
>
> The result doesn't seem to be very random. Using java.util.Random doesn't not have this problem, though:
>
>    val rnd = (new java.util.Random).nextInt(_: Int)
>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>
>
> 71      13      9       0       40      31      84      46      44      83      21
> 2       18      76      65      21      78      17      91      65      5       27
> 3       29      4       78      28      20      59      24      60      45      33
> 5       65      46      33      44      98      30      29      67      56      78
> 4       16      8       99      63      72      58      49      42      45      46
> 8       74      99      53      91      34      73      17      96      65      23
> 9       88      29      13      50      46      80      14      44      99      75
> 7       88      84      18      15      4       48      83      38      68      68
> 1       49      96      87      33      24      72      51      86      71      1
>
> I'm using Scala 2.7.7 on OS X. Please help!
>
>
>
> - Rio

Ah, I see. But is there any plan to "fix" this in the next release? My current version is Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_15)

Thanks!

On 2009-11-20, at 2:32 AM, Stepan Koltsov wrote:

> Some time ago (don't remembed exact version) Scala library had a bug
> where Random constuctor initialized seed by current time only.
>
> You should do:
>
> ===
> val r = new scala.util.Random
> val rnd = r.nextInt(_: Int)
> for (i <- 1 to 100) print(rnd(100) + "\t"))
> ===
>
> It is anyway better.
>
> P. S. Next time send a version of Scala and JDK you are using.
>
> S.
>
> On Fri, Nov 20, 2009 at 04:23, Riobard wrote:
>> Hi,
>>
>> I was trying to implement Fisher–Yates shuffle in Scala and hit this strange problem. I'm not sure if it is a bug, or I misunderstood something:
>>
>> val rnd = (new scala.util.Random).nextInt(_: Int)
>> for (i <- 1 to 100) print(rnd(100) + "\t"))
>>
>> usually I get something like this
>>
>> 85 85 85 10 10 10 10 10 10 10 11
>> 0 10 10 10 10 10 10 10 10 10 33
>> 4 34 34 34 34 34 34 34 34 34 33
>> 4 34 34 34 34 34 34 34 34 34 33
>> 4 34 34 34 34 34 34 34 34 34 33
>> 4 34 34 34 34 34 34 59 59 59 55
>> 9 59 59 59 59 59 59 59 59 59 55
>> 9 59 59 59 59 59 59 59 59 87 88
>> 7 87 87 87 87 87 87 87 87 87 8
>>
>> The result doesn't seem to be very random. Using java.util.Random doesn't not have this problem, though:
>>
>> val rnd = (new java.util.Random).nextInt(_: Int)
>> for (i <- 1 to 100) print(rnd(100) + "\t"))
>>
>>
>> 71 13 9 0 40 31 84 46 44 83 21
>> 2 18 76 65 21 78 17 91 65 5 27
>> 3 29 4 78 28 20 59 24 60 45 33
>> 5 65 46 33 44 98 30 29 67 56 78
>> 4 16 8 99 63 72 58 49 42 45 46
>> 8 74 99 53 91 34 73 17 96 65 23
>> 9 88 29 13 50 46 80 14 44 99 75
>> 7 88 84 18 15 4 48 83 38 68 68
>> 1 49 96 87 33 24 72 51 86 71 1
>>
>> I'm using Scala 2.7.7 on OS X. Please help!
>>
>>
>>
>> - Rio

Can you explain why the original code:
 val rnd = (new scala.util.Random).nextInt(_: Int)
Invokes the constructor upon every call of the rnd function? I don't understand it. This code seems to be creating a function value of a method of a particular object. It is very surprizing, at least here, that (new scala.util.Random) keeps reexecuting, unless we replace that with a variable. Is this a particular scala feature? I don't recall reading about it.
Dimitris

2009/11/20 Stepan Koltsov <stepancheg@mx1.ru>

Some time ago (don't remembed exact version) Scala library had a bug
where Random constuctor initialized seed by current time only.

You should do:

===
val r = new scala.util.Random
val rnd = r.nextInt(_: Int)
for (i <- 1 to 100) print(rnd(100) + "\t"))
===

It is anyway better.

P. S. Next time send a version of Scala and JDK you are using.

S.

On Fri, Nov 20, 2009 at 04:23, Riobard <me@riobard.com> wrote:
> Hi,
>
> I was trying to implement Fisher–Yates shuffle in Scala and hit this strange problem. I'm not sure if it is a bug, or I misunderstood something:
>
>    val rnd = (new scala.util.Random).nextInt(_: Int)
>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>
> usually I get something like this
>
> 85      85      85      10      10      10      10      10      10      10      11
> 0       10      10      10      10      10      10      10      10      10      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      34      34      34      33
> 4       34      34      34      34      34      34      59      59      59      55
> 9       59      59      59      59      59      59      59      59      59      55
> 9       59      59      59      59      59      59      59      59      87      88
> 7       87      87      87      87      87      87      87      87      87      8
>
> The result doesn't seem to be very random. Using java.util.Random doesn't not have this problem, though:
>
>    val rnd = (new java.util.Random).nextInt(_: Int)
>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>
>
> 71      13      9       0       40      31      84      46      44      83      21
> 2       18      76      65      21      78      17      91      65      5       27
> 3       29      4       78      28      20      59      24      60      45      33
> 5       65      46      33      44      98      30      29      67      56      78
> 4       16      8       99      63      72      58      49      42      45      46
> 8       74      99      53      91      34      73      17      96      65      23
> 9       88      29      13      50      46      80      14      44      99      75
> 7       88      84      18      15      4       48      83      38      68      68
> 1       49      96      87      33      24      72      51      86      71      1
>
> I'm using Scala 2.7.7 on OS X. Please help!
>
>
>
> - Rio

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

2009/11/20 Geoff Reedy <geoff@programmer-monk.net>

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
>  val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

Thanks for your answer. I don't think though that the reason is technical in nature, of course the compiler could introduce a variable for the object reference, to match the code Koltsov showed. This looks very strange: new scala.util.Random is on the left, in parens(!), a reader is inclined to assume that this is executed first, the value of the expression in parens is computed, and upon that, there is a method-to-function conversion. I can't believe that a left-most expression in parens is *not* executed first, without even any right-associative operator in action!
Dimitris

> val rnd = (new scala.util.Random).nextInt(_: Int)

The whole q'n is kinda schizophrenic. You use `Random.nextInt' in a way
no other human would do and then propose the existence of a compiler
err. Maybe it would help to read---and understand---the signatur of `Ra
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a
> method of a particular object. It is very surprizing, at least here,
> that (new scala.util.Random) keeps reexecuting, unless we replace that
> with a variable. Is this a particular scala feature? I don't recall
> reading about it.
>
> Dimitris
>
> 2009/11/20 Stepan Koltsov >
>
> Some time ago (don't remembed exact version) Scala library had a bug
> where Random constuctor initialized seed by current time only.
>
> You should do:
>
> ===
> val r = new scala.util.Random
> val rnd = r.nextInt(_: Int)
> for (i <- 1 to 100) print(rnd(100) + "\t"))
> ===
>
> It is anyway better.
>
> P. S. Next time send a version of Scala and JDK you are using.
>
> S.
>
> On Fri, Nov 20, 2009 at 04:23, Riobard > wrote:
> > Hi,
> >
> > I was trying to implement Fisher–Yates shuffle in Scala and hit
> this strange problem. I'm not sure if it is a bug, or I
> misunderstood something:
> >
> > val rnd = (new scala.util.Random).nextInt(_: Int)
> > for (i <- 1 to 100) print(rnd(100) + "\t"))
> >
> > usually I get something like this
> >
> > 85 85 85 10 10 10 10 10
> 10 10 11
> > 0 10 10 10 10 10 10 10
> 10 10 33
> > 4 34 34 34 34 34 34 34
> 34 34 33
> > 4 34 34 34 34 34 34 34
> 34 34 33
> > 4 34 34 34 34 34 34 34
> 34 34 33
> > 4 34 34 34 34 34 34 59
> 59 59 55
> > 9 59 59 59 59 59 59 59
> 59 59 55
> > 9 59 59 59 59 59 59 59
> 59 87 88
> > 7 87 87 87 87 87 87 87
> 87 87 8
> >
> > The result doesn't seem to be very random. Using java.util.Random
> doesn't not have this problem, though:
> >
> > val rnd = (new java.util.Random).nextInt(_: Int)
> > for (i <- 1 to 100) print(rnd(100) + "\t"))
> >
> >
> > 71 13 9 0 40 31 84 46
> 44 83 21
> > 2 18 76 65 21 78 17 91
> 65 5 27
> > 3 29 4 78 28 20 59 24
> 60 45 33
> > 5 65 46 33 44 98 30 29
> 67 56 78
> > 4 16 8 99 63 72 58 49
> 42 45 46
> > 8 74 99 53 91 34 73 17
> 96 65 23
> > 9 88 29 13 50 46 80 14
> 44 99 75
> > 7 88 84 18 15 4 48 83
> 38 68 68
> > 1 49 96 87 33 24 72 51
> 86 71 1
> >
> > I'm using Scala 2.7.7 on OS X. Please help!
> >
> >
> >
> > - Rio
>
>

On Fri, Nov 20, 2009 at 04:45, Riobard wrote:
> Ah, I see. But is there any plan to "fix" this in the next release? My current version is Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_15)

I think it must be fixed.

Patch is trivial:

===
Index: Random.scala
===================================================================

If you use java.secure.Random, it's significantly better.  There's a minor start-up penalty (the seeding is based on machine runtime characteristics), but then the performance is the same.

On Thu, Nov 19, 2009 at 6:49 PM, Stepan Koltsov <stepancheg@mx1.ru> wrote:

On Fri, Nov 20, 2009 at 04:45, Riobard <me@riobard.com> wrote:
> Ah, I see. But is there any plan to "fix" this in the next release? My current version is Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_15)

I think it must be fixed.

Patch is trivial:

===
Index: Random.scala
===================================================================

On Fri, Nov 20, 2009 at 06:28, David Pollak
wrote:
> If you use java.secure.Random, it's significantly better.

It is equally good when you do not need cryptographical strength.

>  There's a minor
> start-up penalty (the seeding is based on machine runtime characteristics),
> but then

> the performance is the same.

Did you make a perf test?

S.

> On Thu, Nov 19, 2009 at 6:49 PM, Stepan Koltsov wrote:
>>
>> On Fri, Nov 20, 2009 at 04:45, Riobard wrote:
>> > Ah, I see. But is there any plan to "fix" this in the next release? My
>> > current version is Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server
>> > VM, Java 1.6.0_15)
>>
>> I think it must be fixed.
>>
>> Patch is trivial:
>>
>> ===
>> Index: Random.scala
>> ===================================================================
>> --- Random.scala        (revision 19735)
>> +++ Random.scala        (working copy)
>> @@ -24,7 +24,7 @@
>>   def this(seed: Int) = this(seed.toLong)
>>
>>   /** Creates a new random number generator. */
>> -  def this() = this(compat.Platform.currentTime)
>> +  def this() = this(new java.util.Random())
>>
>>   /** Returns the next pseudorandom, uniformly distributed boolean value
>>    *  from this random number generator's sequence.
>> ===
>>
>> But why are you going to initialize Random again and again? It could
>> be expensive.
>>
>> Just rewrite as:
>>
>> ===
>> val rnd = {
>>  val r = new scala.util.Random
>>  r.nextInt(_: Int)
>> }
>> ===
>>
>> S.
>>
>> > On 2009-11-20, at 2:32 AM, Stepan Koltsov wrote:
>> >
>> >> Some time ago (don't remembed exact version) Scala library had a bug
>> >> where Random constuctor initialized seed by current time only.
>> >>
>> >> You should do:
>> >>
>> >> ===
>> >> val r = new scala.util.Random
>> >> val rnd = r.nextInt(_: Int)
>> >> for (i <- 1 to 100) print(rnd(100) + "\t"))
>> >> ===
>> >>
>> >> It is anyway better.
>> >>
>> >> P. S. Next time send a version of Scala and JDK you are using.
>> >>
>> >> S.
>> >>
>> >> On Fri, Nov 20, 2009 at 04:23, Riobard wrote:
>> >>> Hi,
>> >>>
>> >>> I was trying to implement Fisher–Yates shuffle in Scala and hit this
>> >>> strange problem. I'm not sure if it is a bug, or I misunderstood something:
>> >>>
>> >>>    val rnd = (new scala.util.Random).nextInt(_: Int)
>> >>>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>> >>>
>> >>> usually I get something like this
>> >>>
>> >>> 85      85      85      10      10      10      10      10      10
>> >>>  10      11
>> >>> 0       10      10      10      10      10      10      10      10
>> >>>  10      33
>> >>> 4       34      34      34      34      34      34      34      34
>> >>>  34      33
>> >>> 4       34      34      34      34      34      34      34      34
>> >>>  34      33
>> >>> 4       34      34      34      34      34      34      34      34
>> >>>  34      33
>> >>> 4       34      34      34      34      34      34      59      59
>> >>>  59      55
>> >>> 9       59      59      59      59      59      59      59      59
>> >>>  59      55
>> >>> 9       59      59      59      59      59      59      59      59
>> >>>  87      88
>> >>> 7       87      87      87      87      87      87      87      87
>> >>>  87      8
>> >>>
>> >>> The result doesn't seem to be very random. Using java.util.Random
>> >>> doesn't not have this problem, though:
>> >>>
>> >>>    val rnd = (new java.util.Random).nextInt(_: Int)
>> >>>    for (i <- 1 to 100) print(rnd(100) + "\t"))
>> >>>
>> >>>
>> >>> 71      13      9       0       40      31      84      46      44
>> >>>  83      21
>> >>> 2       18      76      65      21      78      17      91      65
>> >>>  5       27
>> >>> 3       29      4       78      28      20      59      24      60
>> >>>  45      33
>> >>> 5       65      46      33      44      98      30      29      67
>> >>>  56      78
>> >>> 4       16      8       99      63      72      58      49      42
>> >>>  45      46
>> >>> 8       74      99      53      91      34      73      17      96
>> >>>  65      23
>> >>> 9       88      29      13      50      46      80      14      44
>> >>>  99      75
>> >>> 7       88      84      18      15      4       48      83      38
>> >>>  68      68
>> >>> 1       49      96      87      33      24      72      51      86
>> >>>  71      1
>> >>>
>> >>> I'm using Scala 2.7.7 on OS X. Please help!
>> >>>
>> >>>
>> >>>
>> >>> - Rio
>> >
>> >
>
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Surf the harmonics
>

On Thursday November 19 2009, David Pollak wrote:
> If you use java.secure.Random, it's significantly better. There's a
> minor start-up penalty (the seeding is based on machine runtime
> characteristics), but then the performance is the same.

Two questions:

1) Where is that defined? I don't see it in the 1.6 API docs. There
appears to be only on class named Random, java.util.Random.

2) Don't secure random number generators draw on a local entropy source,
usually provided via the the operating system? Isn't it best not to use
that unless you have an actual cryptographic need?

Randall Schulz

It's there
http://java.sun.com/javase/6/docs/api/java/security/SecureRandom.html

As some of you know, I used to implement java.security.*. Among many of
the marketing myths that we were privy to, it was a running joke that
the filth had convinced the public that "SecureRandom was more secure."

Amazing, it still is.

Randall R Schulz wrote:
> On Thursday November 19 2009, David Pollak wrote:
>
>> If you use java.secure.Random, it's significantly better. There's a
>> minor start-up penalty (the seeding is based on machine runtime
>> characteristics), but then the performance is the same.
>>
>
> Two questions:
>
> 1) Where is that defined? I don't see it in the 1.6 API docs. There
> appears to be only on class named Random, java.util.Random.
>
> 2) Don't secure random number generators draw on a local entropy source,
> usually provided via the the operating system? Isn't it best not to use
> that unless you have an actual cryptographic need?
>
>
> Randall Schulz
>
>

On Thu, Nov 19, 2009 at 7:45 PM, Riobard <me@riobard.com> wrote:

Ah, I see. But is there any plan to "fix" this in the next release?

If it had been "fixed" then you wouldn't have noticed that you were using it wrong.

I don't see the actual error by implementing the constructor call with the current time. You are obviously using the class wrong as you are creating lots of objects that are initialized with the same seed. So do not do this or provide your own seed value. I do not think that implementing it so that the default seed is always seeded differently is the correct way of patching it.

just one example that comes to mind is when doing testing you might want to have a random number generator that always provides the same value.
And do not forget the functional nature of Scala. When using random in a functional setting you want it to return always the same value as long as the input parameters are the same.

As a consequence I'd say works as designed no need to fix.
Regards
Stefan

2009/11/20 Nils Kilden-Pedersen <nilskp@gmail.com>

On Thu, Nov 19, 2009 at 7:45 PM, Riobard <me@riobard.com> wrote:

Ah, I see. But is there any plan to "fix" this in the next release?

If it had been "fixed" then you wouldn't have noticed that you were using it wrong.

On Fri, 2009-11-20 at 09:57 +0100, Stefan Langer wrote:
> I don't see the actual error by implementing the constructor call with
> the current time. You are obviously using the class wrong as you are
> creating lots of objects that are initialized with the same seed. So
> do not do this or provide your own seed value. I do not think that
> implementing it so that the default seed is always seeded differently
> is the correct way of patching it.

Well, that is how it works in 2.8.0 as it just calls the constructor for
java.util.Random without parameters, which in turn does:

public Random() { this(++seedUniquifier + System.nanoTime()); }

> just one example that comes to mind is when doing testing you might
> want to have a random number generator that always provides the same
> value.

In that case you should provide the seed.

> And do not forget the functional nature of Scala. When using random in
> a functional setting you want it to return always the same value as
> long as the input parameters are the same.

For the case where you pass no parameters, it has never been functional
as there is no input. Also, the result would be different in subsequent
invocations as time passed.

> As a consequence I'd say works as designed no need to fix.

It was considered a bug and has been fixed in 2.8.0.

Best,
Ismael

On 2009-11-20, at 10:14 AM, Ismael Juma wrote:

> On Fri, 2009-11-20 at 09:57 +0100, Stefan Langer wrote:
>> I don't see the actual error by implementing the constructor call with
>> the current time. You are obviously using the class wrong as you are
>> creating lots of objects that are initialized with the same seed. So
>> do not do this or provide your own seed value. I do not think that
>> implementing it so that the default seed is always seeded differently
>> is the correct way of patching it.
>
> Well, that is how it works in 2.8.0 as it just calls the constructor for
> java.util.Random without parameters, which in turn does:
>
> public Random() { this(++seedUniquifier + System.nanoTime()); }
>
>> just one example that comes to mind is when doing testing you might
>> want to have a random number generator that always provides the same
>> value.
>
> In that case you should provide the seed.
>
>> And do not forget the functional nature of Scala. When using random in
>> a functional setting you want it to return always the same value as
>> long as the input parameters are the same.
>
> For the case where you pass no parameters, it has never been functional
> as there is no input. Also, the result would be different in subsequent
> invocations as time passed.
>
>> As a consequence I'd say works as designed no need to fix.
>
> It was considered a bug and has been fixed in 2.8.0.

Thank goodness it's fixed.

IMHO, there is no point of using only time as the default seed because if you call twice too closely in time, you'll get the the same seed. As illustrated in the code below:

// Fisher-Yates shuffle
// http://en.wikipedia.org/wiki/Fisher–Yates_shuffle
def shuffle[T](seq: Seq[T]): Seq[T] = {
val arr = seq.toArray
val gen = new java.util.Random
// Don't use scala.util.Random. It uses only current time as seed
// which will lead to not random behaviors if called very often
// It's said to be fixed in Scala 2.8
val rnd = gen.nextInt(_:Int)
for (n <- 1 until arr.length reverse) {
val k = rnd(n+1)
val t = arr(k)
arr(k) = arr(n)
arr(n) = t
}
return arr
}

If one uses scala.util.Random in 2.7 to shuffle the same sequence repeatedly, you'll get the same result, which is unexpected.

And random() is not functional in the sense that functional code gives one the same result if one provides the same input.

My 2 cents.
- Rio

On Fri, Nov 20, 2009 at 12:11 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

2009/11/20 Geoff Reedy <geoff@programmer-monk.net>

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
>  val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

Thanks for your answer. I don't think though that the reason is technical in nature, of course the compiler could introduce a variable for the object reference, to match the code Koltsov showed. This looks very strange: new scala.util.Random is on the left, in parens(!), a reader is inclined to assume that this is executed first, the value of the expression in parens is computed, and upon that, there is a method-to-function conversion. I can't believe that a left-most expression in parens is *not* executed first, without even any right-associative operator in action!

  The leftmost expression _is_ executed first, of course. The question is, where is the boundary of the function you created? Suppose we de-anonymized it. Let's see:   def deAnom(x: Int) = {   val random = new scala.util.Random   random.nextInt(x) }   There you have it, the leftmost expression is executed first, and you get exactly the very same result. The problem is, where is the boundary of the function you made?   Let's try something. Suppose you write:    val set1: Int => scala.collection.Set[Int] = (new scala.collection.mutable.HashSet[Int]) + _   Where are the boundaries of the function? Where does it start, where does it end? Clearly, all of it must be part of the function. So, back to your definition:
val rnd = (new scala.util.Random).nextInt(_: Int)
It might seem to you that, because "_" is inside parenthesis, the function would be inside it. This is what usually happens -- and it would cause error here if it did, but Scala treats _ when they are a parameter as a belonging to an outside scope.   So, how would you do what you wanted? Well, I propose this:   val rnd = ({rand: scala.util.Random => rand.nextInt(_: Int)}) (new scala.util.Random)   Here we have a clear separation of each step. We have a function of scala.util.Random => Int => Int, and then we pass the first parameter to it, yield a new function, of Int => int. The parameter we passed will not be recomputed, because it is _outside_ the definition of the new function.

--
Daniel C. Sobral

Veni, vidi, veterni.

val rnd = (new scala.util.Random).nextInt(_: Int)
if the leftmost expression was executed first, then this would be equivalent to:
val tmp_valueOfLeftMostExpressionExecutedFirst = (new scala.util.Random) val rnd = tmp_valueOfLeftMostExpressionExecutedFirst.nextInt(_: Int) //simple substitution of an expression with its computed value, "trivial"
But you say that the boundary of the created function matters. But the function creation is on the right, so if it matters *first*, that's what is executed first after all. 
Anyway, I'll try to look up the rule for this. It would be nice if there was some explanation by whoever knows the reasoning behind this decision though, what makes it particularly beneficial to include it. I guess if one wanted the current interpretation, he could always write this instead:
val rnd = (x: Int) => new scala.util.Random().nextInt(x)
Which doesn't look bad, so at least in this case the gain of this feature isn't apparent. Any example where it is?
Dimitris
2009/11/20 Daniel Sobral <dcsobral@gmail.com>

On Fri, Nov 20, 2009 at 12:11 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

2009/11/20 Geoff Reedy <geoff@programmer-monk.net>

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
>  val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

Thanks for your answer. I don't think though that the reason is technical in nature, of course the compiler could introduce a variable for the object reference, to match the code Koltsov showed. This looks very strange: new scala.util.Random is on the left, in parens(!), a reader is inclined to assume that this is executed first, the value of the expression in parens is computed, and upon that, there is a method-to-function conversion. I can't believe that a left-most expression in parens is *not* executed first, without even any right-associative operator in action!

  The leftmost expression _is_ executed first, of course. The question is, where is the boundary of the function you created? Suppose we de-anonymized it. Let's see:   def deAnom(x: Int) = {   val random = new scala.util.Random   random.nextInt(x) }   There you have it, the leftmost expression is executed first, and you get exactly the very same result. The problem is, where is the boundary of the function you made?   Let's try something. Suppose you write:    val set1: Int => scala.collection.Set[Int] = (new scala.collection.mutable.HashSet[Int]) + _   Where are the boundaries of the function? Where does it start, where does it end? Clearly, all of it must be part of the function. So, back to your definition:
val rnd = (new scala.util.Random).nextInt(_: Int)
It might seem to you that, because "_" is inside parenthesis, the function would be inside it. This is what usually happens -- and it would cause error here if it did, but Scala treats _ when they are a parameter as a belonging to an outside scope.   So, how would you do what you wanted? Well, I propose this:   val rnd = ({rand: scala.util.Random => rand.nextInt(_: Int)}) (new scala.util.Random)   Here we have a clear separation of each step. We have a function of scala.util.Random => Int => Int, and then we pass the first parameter to it, yield a new function, of Int => int. The parameter we passed will not be recomputed, because it is _outside_ the definition of the new function.

--
Daniel C. Sobral

Veni, vidi, veterni.

On Fri, Nov 20, 2009 at 11:57, Stefan Langer
wrote:
> I don't see the actual error by implementing the constructor call with the
> current time.

Two Random instances (initialized in different threads, or in
different modules) could yield same values. It is a problem.

S.

> You are obviously using the class wrong as you are creating
> lots of objects that are initialized with the same seed. So do not do this
> or provide your own seed value. I do not think that implementing it so that
> the default seed is always seeded differently is the correct way of patching
> it.
>
> just one example that comes to mind is when doing testing you might want to
> have a random number generator that always provides the same value.
> And do not forget the functional nature of Scala. When using random in a
> functional setting you want it to return always the same value as long as
> the input parameters are the same.
>
> As a consequence I'd say works as designed no need to fix.
> Regards
> Stefan
>
> 2009/11/20 Nils Kilden-Pedersen
>>
>> On Thu, Nov 19, 2009 at 7:45 PM, Riobard wrote:
>>>
>>> Ah, I see. But is there any plan to "fix" this in the next release?
>>
>> If it had been "fixed" then you wouldn't have noticed that you were using
>> it wrong.
>
>

Why do you keep removing THE left most part of it, "val rnd ="? Let me rewrite it preserving the ordering:   val rnd = {   val tmp_valueOfLeftMostExpressionExecutedFirst = (new scala.util.Random)   tmp_valueOfLeftMostExpressionExecutedFirst.nextInt(_ :Int) }   There. Order preserved, works as expected.   Now, "function created on the right". No, the PARAMETER of the function was used on the right, but that's NOT where the function was declared. It was declared in the innermost scope: ie, everything after "=".
Now, as for the gain, try this:   List(1,2,3,4).foldLeft(_+_).   It is much smaller than   List(1,2,3,4).foldLeft((x$1: Int, x$2: Int) => x$1 + x$2)
There are plenty of places where using underscore is beneficial.   On Fri, Nov 20, 2009 at 11:40 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

val rnd = (new scala.util.Random).nextInt(_: Int)
if the leftmost expression was executed first, then this would be equivalent to:
val tmp_valueOfLeftMostExpressionExecutedFirst = (new scala.util.Random) val rnd = tmp_valueOfLeftMostExpressionExecutedFirst.nextInt(_: Int) //simple substitution of an expression with its computed value, "trivial"
But you say that the boundary of the created function matters. But the function creation is on the right, so if it matters *first*, that's what is executed first after all. 
Anyway, I'll try to look up the rule for this. It would be nice if there was some explanation by whoever knows the reasoning behind this decision though, what makes it particularly beneficial to include it. I guess if one wanted the current interpretation, he could always write this instead:
val rnd = (x: Int) => new scala.util.Random().nextInt(x)
Which doesn't look bad, so at least in this case the gain of this feature isn't apparent. Any example where it is?
Dimitris
2009/11/20 Daniel Sobral <dcsobral@gmail.com>

On Fri, Nov 20, 2009 at 12:11 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

2009/11/20 Geoff Reedy <geoff@programmer-monk.net>

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
>  val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

Thanks for your answer. I don't think though that the reason is technical in nature, of course the compiler could introduce a variable for the object reference, to match the code Koltsov showed. This looks very strange: new scala.util.Random is on the left, in parens(!), a reader is inclined to assume that this is executed first, the value of the expression in parens is computed, and upon that, there is a method-to-function conversion. I can't believe that a left-most expression in parens is *not* executed first, without even any right-associative operator in action!

  The leftmost expression _is_ executed first, of course. The question is, where is the boundary of the function you created? Suppose we de-anonymized it. Let's see:   def deAnom(x: Int) = {   val random = new scala.util.Random   random.nextInt(x) }   There you have it, the leftmost expression is executed first, and you get exactly the very same result. The problem is, where is the boundary of the function you made?   Let's try something. Suppose you write:    val set1: Int => scala.collection.Set[Int] = (new scala.collection.mutable.HashSet[Int]) + _   Where are the boundaries of the function? Where does it start, where does it end? Clearly, all of it must be part of the function. So, back to your definition:
val rnd = (new scala.util.Random).nextInt(_: Int)
It might seem to you that, because "_" is inside parenthesis, the function would be inside it. This is what usually happens -- and it would cause error here if it did, but Scala treats _ when they are a parameter as a belonging to an outside scope.   So, how would you do what you wanted? Well, I propose this:   val rnd = ({rand: scala.util.Random => rand.nextInt(_: Int)}) (new scala.util.Random)   Here we have a clear separation of each step. We have a function of scala.util.Random => Int => Int, and then we pass the first parameter to it, yield a new function, of Int => int. The parameter we passed will not be recomputed, because it is _outside_ the definition of the new function.

--
Daniel C. Sobral

Veni, vidi, veterni.

--
Daniel C. Sobral

Veni, vidi, veterni.

2009/11/20 Daniel Sobral <dcsobral@gmail.com>

Why do you keep removing THE left most part of it, "val rnd ="? Let me rewrite it preserving the ordering:

It's an assignment, the assignment will execute last. The next expression to be executed is where the strangeness occurs.

  val rnd = {   val tmp_valueOfLeftMostExpressionExecutedFirst = (new scala.util.Random)   tmp_valueOfLeftMostExpressionExecutedFirst.nextInt(_ :Int) } 

  There. Order preserved, works as expected.

Of course you can rewrite the code. I also did the inverse previously, i.e.:
val rnd = (x: Int) => new scala.util.Random().nextInt(x)
But that's besides the point. The confusion is that the original code: 
val rnd = (new scala.util.Random).nextInt(_: Int)
Looks like the leftmost expression in parens should be evaluated first. "Evaluated" to be interpreted as "substituted with its value". This is not the case obviously - if one replaces that expression with its value (i.e. with an intermediate val) *it's not the same program any more*. This is what you did in your code above, and it's not the same program. I would expect something as fundamental as replacing an expression with its value *not* to alter the meaning of the program.
Also, still on strangeness, observe the difference between:
val rnd = (new scala.util.Random).nextInt(_: Int)
and
val rnd = (new scala.util.Random).nextInt _
Yikes!
(For those who won't try this out: the latter *does* evaluate the Random instance immediately, not once per invocation, while the former, as we already discussed, creates a new Random instance each time).
Now, can you still argue that in both of the latter two cases, the left-most expression is executed first? I find that hard to imagine.
Dimitris

  Now, "function created on the right". No, the PARAMETER of the function was used on the right, but that's NOT where the function was declared. It was declared in the innermost scope: ie, everything after "=".
Now, as for the gain, try this:   List(1,2,3,4).foldLeft(_+_).   It is much smaller than   List(1,2,3,4).foldLeft((x$1: Int, x$2: Int) => x$1 + x$2)
There are plenty of places where using underscore is beneficial.

 

  On Fri, Nov 20, 2009 at 11:40 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

val rnd = (new scala.util.Random).nextInt(_: Int)
if the leftmost expression was executed first, then this would be equivalent to:
val tmp_valueOfLeftMostExpressionExecutedFirst = (new scala.util.Random) val rnd = tmp_valueOfLeftMostExpressionExecutedFirst.nextInt(_: Int) //simple substitution of an expression with its computed value, "trivial"
But you say that the boundary of the created function matters. But the function creation is on the right, so if it matters *first*, that's what is executed first after all. 
Anyway, I'll try to look up the rule for this. It would be nice if there was some explanation by whoever knows the reasoning behind this decision though, what makes it particularly beneficial to include it. I guess if one wanted the current interpretation, he could always write this instead:
val rnd = (x: Int) => new scala.util.Random().nextInt(x)
Which doesn't look bad, so at least in this case the gain of this feature isn't apparent. Any example where it is?
Dimitris
2009/11/20 Daniel Sobral <dcsobral@gmail.com>

On Fri, Nov 20, 2009 at 12:11 AM, Dimitris Andreou <jim.andreou@gmail.com> wrote:

2009/11/20 Geoff Reedy <geoff@programmer-monk.net>

On Fri, Nov 20, 2009 at 03:48:01AM +0200, Dimitris Andreou said
> Can you explain why the original code:
>
>  val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Invokes the constructor upon every call of the rnd function? I don't
> understand it. This code seems to be creating a function value of a method
> of a particular object. It is very surprizing, at least here, that (new
> scala.util.Random) keeps reexecuting, unless we replace that with a
> variable. Is this a particular scala feature? I don't recall reading about
> it.

This is due to the way that the _ way of defining a function is
desugared. I cannot recall the exact rules but in this case the
equivalent long-hand code is

val rnd = {x: Int => (new scala.util.Random).nextInt(x)}

the only other reasonable expansion would be

val rnd = (new scala.util.Random).nextInt({x: Int => x})

which is obviously a type error.

Thanks for your answer. I don't think though that the reason is technical in nature, of course the compiler could introduce a variable for the object reference, to match the code Koltsov showed. This looks very strange: new scala.util.Random is on the left, in parens(!), a reader is inclined to assume that this is executed first, the value of the expression in parens is computed, and upon that, there is a method-to-function conversion. I can't believe that a left-most expression in parens is *not* executed first, without even any right-associative operator in action!

  The leftmost expression _is_ executed first, of course. The question is, where is the boundary of the function you created? Suppose we de-anonymized it. Let's see:   def deAnom(x: Int) = {   val random = new scala.util.Random   random.nextInt(x) }   There you have it, the leftmost expression is executed first, and you get exactly the very same result. The problem is, where is the boundary of the function you made?   Let's try something. Suppose you write:    val set1: Int => scala.collection.Set[Int] = (new scala.collection.mutable.HashSet[Int]) + _   Where are the boundaries of the function? Where does it start, where does it end? Clearly, all of it must be part of the function. So, back to your definition:
val rnd = (new scala.util.Random).nextInt(_: Int)
It might seem to you that, because "_" is inside parenthesis, the function would be inside it. This is what usually happens -- and it would cause error here if it did, but Scala treats _ when they are a parameter as a belonging to an outside scope.   So, how would you do what you wanted? Well, I propose this:   val rnd = ({rand: scala.util.Random => rand.nextInt(_: Int)}) (new scala.util.Random)   Here we have a clear separation of each step. We have a function of scala.util.Random => Int => Int, and then we pass the first parameter to it, yield a new function, of Int => int. The parameter we passed will not be recomputed, because it is _outside_ the definition of the new function.

--
Daniel C. Sobral

Veni, vidi, veterni.

--
Daniel C. Sobral

Veni, vidi, veterni.

On Fri, Nov 20, 2009 at 04:42:41PM +0200, Dimitris Andreou wrote:
> But that's besides the point. The confusion is that the original code:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Looks like the leftmost expression in parens should be evaluated
> first.

I find this thread interesting because while scala is not without its
surprises for me, all of this is exactly as I'd expect it.

I suggest your intuition fails because you view the above line as having
a "leftmost expression" when in fact it only has "an expression" and,
should you care to parse it further, "subexpressions". You haven't
given scala any reason to see the line above as a sequence of
expressions with stored intermediate values. The type of rnd is

Int => Int

so if anything it would be surprising to find out that somehow a
scala.util.Random is being stored somewhere, in an expression with that
type and without your ever declaring any storage.

> This is not the case obviously - if one replaces that expression with
> its value (i.e. with an intermediate val) *it's not the same program
> any more*.

Well yeah, because you're moving bits out of the body of a function by
doing that. It'd be surprising if that didn't change the program.

> Also, still on strangeness, observe the difference between:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> and
>
> val rnd = (new scala.util.Random).nextInt _
>
> Yikes!

Why is that yikes? Those are two different underscores you know. The
first is "placeholder syntax", and the second is method to function
conversion. In this case they don't even refer to the same method.

Hi Paul!

2009/11/20 Paul Phillips <paulp@improving.org>

On Fri, Nov 20, 2009 at 04:42:41PM +0200, Dimitris Andreou wrote:
> But that's besides the point. The confusion is that the original code:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> Looks like the leftmost expression in parens should be evaluated
> first.

I find this thread interesting because while scala is not without its
surprises for me, all of this is exactly as I'd expect it.

I suggest your intuition fails because you view the above line as having
a "leftmost expression" when in fact it only has "an expression"

and,
should you care to parse it further, "subexpressions".   

You haven't
given scala any reason to see the line above as a sequence of
expressions with stored intermediate values.  

Alright, so where in the following line I give a reason to scala to see it "as a sequence of expressions with stored intermediate values"?
val rnd = (new scala.util.Random).nextInt _  

The type of rnd is

 Int => Int

so if anything it would be surprising to find out that somehow a
scala.util.Random is being stored somewhere, in an expression with that
type and without your ever declaring any storage.

Again, where did I declare a storage in "(new scala.util.Random).nextInt _ " ?

> This is not the case obviously - if one replaces that expression with
> its value (i.e. with an intermediate val) *it's not the same program
> any more*.

Well yeah, because you're moving bits out of the body of a function by
doing that.  It'd be surprising if that didn't change the program.

> Also, still on strangeness, observe the difference between:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> and
>
> val rnd = (new scala.util.Random).nextInt _
>
> Yikes!

Why is that yikes? Those are two different underscores you know.  The
first is "placeholder syntax", and the second is method to function
conversion.  In this case they don't even refer to the same method.

Anyway, so the rule is that the placeholder syntax nests the expression that comes before it? I can live with that (I'll just append a tiny exception to a remote part of my brains :)). Still, I don't understand (any examples?) what this interpretation buys compared to the other option, of evaluating the left expression first, which was chosen instead in method-to-function conversion. I can well imagine method-to-function conversion nesting its left expression too, what would be the problem in that case?
Thanks!Dimitris 

--
Paul Phillips      | One way is to make it so simple that there are
In Theory          | obviously no deficiencies. And the other way is to make
Empiricist         | it so complicated that there are no obvious deficiencies.
slap pi uphill!    |     -- Hoare

On Fri, Nov 20, 2009 at 05:23:03PM +0200, Dimitris Andreou wrote:
> Alright, so where in the following line I give a reason to scala to
> see it "as a sequence of expressions with stored intermediate values"?

val rnd = (new scala.util.Random).nextInt _

That underscore means to create a function object based on the method
you are calling. The method you are calling is nextInt on that specific
scala.util.Random instance you just created, and so that's what the
newly created function object does too.

val rnd = (new scala.util.Random).nextInt(_: Int)

THAT underscore means that the entire expression is an anonymous
function expressing with placeholder syntax. The function created
evaluates that entire expression with the given Int argument. Since the
expression creates a new scala.util.Random object, the newly created
function does too.

The re-use of _ in many contexts takes some getting used to, but this is
not an illogical arrangement.

2009/11/20 Paul Phillips <paulp@improving.org>

> Also, still on strangeness, observe the difference between:
>
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> and
>
> val rnd = (new scala.util.Random).nextInt _
>
> Yikes!

Why is that yikes? Those are two different underscores you know.  The
first is "placeholder syntax", and the second is method to function
conversion.  In this case they don't even refer to the same method.

(Sorry about that, lets assume scala.util.Random had only a nextInt(Int) method, I didn't mean to involve overloading, my intention was that both vals referred to the same method) 

2009/11/20 Paul Phillips <paulp@improving.org>

On Fri, Nov 20, 2009 at 05:23:03PM +0200, Dimitris Andreou wrote:
> Alright, so where in the following line I give a reason to scala to
> see it "as a sequence of expressions with stored intermediate values"?

val rnd = (new scala.util.Random).nextInt _

That underscore means to create a function object based on the method
you are calling.  The method you are calling is nextInt on that specific
scala.util.Random instance you just created, and so that's what the
newly created function object does too.

Pail, I understand the "what" of this code interpretation, I'm asking about the "why". I see both options (nesting the left expression, or evaluating it and storing it) viable for both placeholder syntax and method-to-function conversion (are they not? It seems they are). I'm just curious in why the designers of scala chose one interpretation for the first case, and the second interpretation for the second case. From the point of view of the casual observer (that would be me), this would create a language with one rule less, e.g. one could lookup "method-to-function conversion" and see merely this description "it's the same as if you put placeholders for all parameters of a method". I'm *sure* there are good reasons why this design route was discarded, they are just not obvious to me and I'm curious to learn them. Hopefully this is clear enough? Or is it that these two cases were considered very distanced from each other, so consistency between them didn't matter?
Regards,Dimitris 

val rnd = (new scala.util.Random).nextInt(_: Int)

THAT underscore means that the entire expression is an anonymous
function expressing with placeholder syntax.  The function created
evaluates that entire expression with the given Int argument.  Since the
expression creates a new scala.util.Random object, the newly created
function does too.

The re-use of _ in many contexts takes some getting used to, but this is
not an illogical arrangement.

--
Paul Phillips      | We act as though comfort and luxury were the chief
Everyman           | requirements of life, when all that we need to make us
Empiricist         | really happy is something to be enthusiastic about.
slap pi uphill!    |     -- Charles Kingsley

On 2009-11-20, at 4:45 PM, Paul Phillips wrote:

> On Fri, Nov 20, 2009 at 05:23:03PM +0200, Dimitris Andreou wrote:
>> Alright, so where in the following line I give a reason to scala to
>> see it "as a sequence of expressions with stored intermediate values"?
>
> val rnd = (new scala.util.Random).nextInt _
>
> That underscore means to create a function object based on the method
> you are calling. The method you are calling is nextInt on that specific
> scala.util.Random instance you just created, and so that's what the
> newly created function object does too.

This is the behavior I originally intended to use: I want a shorter name to a function which generates a random integer 0<=rnd
> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> THAT underscore means that the entire expression is an anonymous
> function expressing with placeholder syntax. The function created
> evaluates that entire expression with the given Int argument. Since the
> expression creates a new scala.util.Random object, the newly created
> function does too.
>
> The re-use of _ in many contexts takes some getting used to, but this is
> not an illogical arrangement.

I think this is very easy to get confused, at least for beginners like me :|

On Fri, Nov 20, 2009 at 12:42 PM, Dimitris Andreou <jim.andreou@gmail.com> wrote:
 

2009/11/20 Daniel Sobral <dcsobral@gmail.com>

Why do you keep removing THE left most part of it, "val rnd ="? Let me rewrite it preserving the ordering:

 
It's an assignment, the assignment will execute last. The next expression to be executed is where the strangeness occurs.

 
I see. I think I understand what the problem is. You see "_" as being some kind of "delay this part" annotation on an expression, while it actually is a "this is a function, and this is one of the parameters to it.

 
So, what happens is this:
 
val rnd = (new scala.util.Random).nextInt(_: Int)
 
<scala> mmm, there's a "_" in a place where it means function parameter. Where are the function boundaries?
 
To compute the function boundary, it (Scala) finds the scope of the innermost expression. For example:
 
(x + (1 + _)) // Innermost expression is "(1 + _)"
x; 1 + _; y // Innermost expression is "1 + _"
{ x
 1 + _ } // Innermost expression is "1 + _"
1.(_) // Innermost expression is "1.(_)"
1.(x + _) // Innermost expresison is "x + _"
 
The last two results are surprising. Scala does not consider a parameter to be an "expression" if it is just an "_". If you convert to operator style notation, things are clearer.
 
So, you have "val rnd = (new scala.util.Random).nextInt(_: Int)", which is equivalent, barring type annotations, to "val rnd = (new scala.util.Random) nextInt _". Scala has to find the innermost expression in that, which happens to be the RHS. It then converts it to:

 
val rnd = ((x$1: Int) => (new scala.util.Random).nextInt(x$1)
 
There's no trick to it. Is "(new scala.util.Random)" evaluated first? Yes, it sure is. But evaluation is a run-time thing, and what you wrote is a _function literal_. The question is only what is inside your function, and what is outside, and "(new scala.util.Random)" is inside the function.

 
You are thinking about the semantics of the code, what happens at execution time. But function literals are just syntax: there is no trick to it, except understanding where the function starts, and where the function ends, but the function is _exactly_ what you are seeing.

 
Consider things like order of execution. Here are some examples:
 
x + y * _
x + _ * y
_ + x * y

 
x :: y :: _x :: _ :: y
_ :: x :: y
 
The order of execution changes in each of these, but they have one thing in common: they are all expressions. For all of them, the scope of the function literal they represent is the entire expression.
 
 

  

Looks like the leftmost expression in parens should be evaluated first. "Evaluated" to be interpreted as "substituted with its value". This is not the case obviously - if one replaces that expression with its value (i.e. with an intermediate val) *it's not the same program any more*. This is what you did in your code above, and it's not the same program. I would expect something as fundamental as replacing an expression with its value *not* to alter the meaning of the program.

 
Evaluation happens at run-time, not compile time. The process of declaring a function literal is a compile-time process. I did that because, as a matter of fact, the following expressions are identical:
 
(new scala.util.Random).nextInt(_:Int)
(x$1: Int) => (new scala.util.Random).nextInt(x$1)
{ (x$1: Int) => (new scala.util.Random).nextInt(x$1) }
 
And the last one is the non-suggared version of it. When you write "_" to represent a function parameter, you are placing {} around an expression, and turning it into a function.
 
 

 
Also, still on strangeness, observe the difference between:

 val rnd = (new scala.util.Random).nextInt(_: Int)

 
and

 
val rnd = (new scala.util.Random).nextInt _

 
Yikes!

 
As Paul said, they are different things. And the following would be the same as the first, not the second:
 
val md = (new scala.util.Random) nextInt _
 
That is a function literal, same as the first one. The second example you shown is the syntax for "give me a reference for this function", and not "create a new function".
 
--
Daniel C. Sobral
 
Veni, vidi, veterni.
 

Many thanks Daniel for taking the time and being thorough, that was very helpful. Those were nice examples, and you certainly caught me off guard with this:
val rnd = (new scala.util.Random).nextInt _ being different to this:val rnd = (new scala.util.Random) nextInt _
I would hardly notice that. 
I also realized what "Pail" :) meant when he mentioned "_" usages potentially being confusing. I'll be careful with this in the future.
Regards,Dimitris
PS: Paul, come on, you should be thankful I didn't do a *second* typo in Pail! :)

2009/11/20 Daniel Sobral <dcsobral@gmail.com>

On Fri, Nov 20, 2009 at 12:42 PM, Dimitris Andreou <jim.andreou@gmail.com> wrote:  

2009/11/20 Daniel Sobral <dcsobral@gmail.com>

Why do you keep removing THE left most part of it, "val rnd ="? Let me rewrite it preserving the ordering:

  It's an assignment, the assignment will execute last. The next expression to be executed is where the strangeness occurs.

  I see. I think I understand what the problem is. You see "_" as being some kind of "delay this part" annotation on an expression, while it actually is a "this is a function, and this is one of the parameters to it.   So, what happens is this:   val rnd = (new scala.util.Random).nextInt(_: Int)   <scala> mmm, there's a "_" in a place where it means function parameter. Where are the function boundaries?   To compute the function boundary, it (Scala) finds the scope of the innermost expression. For example:   (x + (1 + _)) // Innermost expression is "(1 + _)" x; 1 + _; y // Innermost expression is "1 + _" { x  1 + _ } // Innermost expression is "1 + _" 1.(_) // Innermost expression is "1.(_)" 1.(x + _) // Innermost expresison is "x + _"   The last two results are surprising. Scala does not consider a parameter to be an "expression" if it is just an "_". If you convert to operator style notation, things are clearer.

  So, you have "val rnd = (new scala.util.Random).nextInt(_: Int)", which is equivalent, barring type annotations, to "val rnd = (new scala.util.Random) nextInt _". Scala has to find the innermost expression in that, which happens to be the RHS. It then converts it to:   val rnd = ((x$1: Int) => (new scala.util.Random).nextInt(x$1)   There's no trick to it. Is "(new scala.util.Random)" evaluated first? Yes, it sure is. But evaluation is a run-time thing, and what you wrote is a _function literal_. The question is only what is inside your function, and what is outside, and "(new scala.util.Random)" is inside the function.   You are thinking about the semantics of the code, what happens at execution time. But function literals are just syntax: there is no trick to it, except understanding where the function starts, and where the function ends, but the function is _exactly_ what you are seeing.   Consider things like order of execution. Here are some examples:   x + y * _ x + _ * y _ + x * y   x :: y :: _x :: _ :: y _ :: x :: y   The order of execution changes in each of these, but they have one thing in common: they are all expressions. For all of them, the scope of the function literal they represent is the entire expression.    

   Looks like the leftmost expression in parens should be evaluated first. "Evaluated" to be interpreted as "substituted with its value". This is not the case obviously - if one replaces that expression with its value (i.e. with an intermediate val) *it's not the same program any more*. This is what you did in your code above, and it's not the same program. I would expect something as fundamental as replacing an expression with its value *not* to alter the meaning of the program.

  Evaluation happens at run-time, not compile time. The process of declaring a function literal is a compile-time process. I did that because, as a matter of fact, the following expressions are identical:   (new scala.util.Random).nextInt(_:Int) (x$1: Int) => (new scala.util.Random).nextInt(x$1) { (x$1: Int) => (new scala.util.Random).nextInt(x$1) }   And the last one is the non-suggared version of it. When you write "_" to represent a function parameter, you are placing {} around an expression, and turning it into a function.    

  Also, still on strangeness, observe the difference between:  val rnd = (new scala.util.Random).nextInt(_: Int)   and   val rnd = (new scala.util.Random).nextInt _   Yikes!

  As Paul said, they are different things. And the following would be the same as the first, not the second:   val md = (new scala.util.Random) nextInt _   That is a function literal, same as the first one. The second example you shown is the syntax for "give me a reference for this function", and not "create a new function".   -- Daniel C. Sobral   Veni, vidi, veterni.  

On Fri, Nov 20, 2009 at 06:09:51PM +0200, Dimitris Andreou wrote:
> Pail, I understand the "what" of this code interpretation, I'm asking
> about the "why".

Pail can't help you with whys, he's not been around nearly long enough
to divine motivations. Pail can only comment on what he finds
surprising and what he doesn't, while trying to remain focused on
preventing overmuch water from accumulating at the bottom of the boat.

On Fri, Nov 20, 2009 at 4:45 PM, Paul Phillips wrote:
> val rnd = (new scala.util.Random).nextInt _
>
> That underscore means to create a function object based on the method
> you are calling.  The method you are calling is nextInt on that specific
> scala.util.Random instance you just created, and so that's what the
> newly created function object does too.

I got to know that as the partial evaluation syntax. This implies that
some values are closed over.

> val rnd = (new scala.util.Random).nextInt(_: Int)
>
> THAT underscore means that the entire expression is an anonymous
> function expressing with placeholder syntax.  The function created
> evaluates that entire expression with the given Int argument.  Since the
> expression creates a new scala.util.Random object, the newly created
> function does too.

When I started using Scala, this syntax was still very new sugar for x
=> ... which you wouldn't suspect of capturing values. So, although I
stayed clear of any false expectations, it's good to internalize these
differences to make proper use of them...

Thanks for making this so perfectly clear.

> Pail can't help you with whys, he's not been around nearly long enough
> to divine motivations. Pail can only comment on what he finds
> surprising and what he doesn't, while trying to remain focused on
> preventing overmuch water from accumulating at the bottom of the boat.

Now you take an obious typo---I resides next to the U---as a commence
for a pun. Not my kinda style, ``Pail."

>> Pail can't help you with whys, he's not been around nearly long enough
>> to divine motivations. Pail can only comment on what he finds
>> surprising and what he doesn't, while trying to remain focused on
>> preventing overmuch water from accumulating at the bottom of the boat.
>
> Now you take an obious typo---I resides next to the U---as a commence
> for a pun. Not my kinda style, ``Pail."

To elaborate on this: If a newbie coming to Scala meets Paul Phillips on
this list, this know-it-all, explains-nothing, I think many newbies
might be defeated and simply be afraid of this omnipotent guy and Poker
millionaire from San Francisco. (I wish I had that money, so this is not
a discussion of envy. Cudos!) But newbies. in general, don't have much
luck here---whenever they pose an allegedly dumb question, God's ire
strikes down on `em immediately.

He's from San Francisco? I too am envious now.

These mailing lists are actually quite newbie-friendly, in my opinion.
Most of us were Scala newbies not long ago, that probably helps. I
think you'll find the newbies have less patience than Paul has.

2009/11/20 Philip Köster :
>>> Pail can't help you with whys, he's not been around nearly long enough to
>>> divine motivations.  Pail can only comment on what he finds surprising and
>>> what he doesn't, while trying to remain focused on preventing overmuch water
>>> from accumulating at the bottom of the boat.
>>
>>  Now you take an obious typo---I resides next to the U---as a commence for
>> a pun. Not my kinda style, ``Pail."
>
> To elaborate on this: If a newbie coming to Scala meets Paul Phillips on
> this list, this know-it-all, explains-nothing, I think many newbies might be
> defeated and simply be afraid of this omnipotent guy and Poker millionaire
> from San Francisco. (I wish I had that money, so this is not a discussion of
> envy. Cudos!) But newbies. in general, don't have much luck here---whenever
> they pose an allegedly dumb question, God's ire strikes down on `em
> immediately.
>

Wow, I hadn't realize that poker thing. "Cudos" indeed! (By the way, that's a greek word, and we don't even have a "c" letter)
2009/11/20 Philip Köster <philip.koester@web.de>

Pail can't help you with whys, he's not been around nearly long enough to divine motivations.  Pail can only comment on what he finds surprising and what he doesn't, while trying to remain focused on preventing overmuch water from accumulating at the bottom of the boat.

 Now you take an obious typo---I resides next to the U---as a commence for a pun. Not my kinda style, ``Pail."

To elaborate on this: If a newbie coming to Scala meets Paul Phillips on this list, this know-it-all, explains-nothing, I think many newbies might be defeated and simply be afraid of this omnipotent guy and Poker millionaire from San Francisco. (I wish I had that money, so this is not a discussion of envy. Cudos!) But newbies. in general, don't have much luck here---whenever they pose an allegedly dumb question, God's ire strikes down on `em immediately.

On Friday November 20 2009, Dimitris Andreou wrote:
> Wow, I hadn't realize that poker thing. "Cudos" indeed! (By the way,
> that's a greek word, and we don't even have a "c" letter)

That may explain why we spell it "kudos."

Kappa-upsilon-delta-omega-sigma?

RRS

>>>>> "Ricky" == Ricky Clarkson writes:

Ricky> He's from San Francisco? I too am envious now. These mailing
Ricky> lists are actually quite newbie-friendly, in my opinion. Most
Ricky> of us were Scala newbies not long ago, that probably helps. I
Ricky> think you'll find the newbies have less patience than Paul has.

I think we are quite friendly to newbies asking questions. We are not
always so friendly to newbies confidently making dubious assertions.
(I leave it to the reader to decide if that's a good or a bad thing.)

On 20/11/2009 18:17, Philip Köster wrote:
> But newbies. in general, don't have much
> luck here---whenever they pose an allegedly dumb question, God's ire
> strikes down on `em immediately.

I'm sure that this kind of message should just be ignored, but well...
I'm rather new on that list, found it really welcoming and friendly,
most of the time discussion are made between gentlemen (and sometimes
Lolcoders, but that's ok ;).

So, several things here:
- please, don't mix Scala and programming related topic with privacy.
- don't make public assertion upon people here (even if you know them).
- you want nice answers ? try to ask good questions and do your
homeworks. This faq is a nice help
http://catb.org/~esr/faqs/smart-questions.html

Again, this list was really pleasant as a newbie.

On Fri, Nov 20, 2009 at 5:17 PM, Philip Köster <philip.koester@web.de> wrote:

Pail can't help you with whys, he's not been around nearly long enough to divine motivations.  Pail can only comment on what he finds surprising and what he doesn't, while trying to remain focused on preventing overmuch water from accumulating at the bottom of the boat.

 Now you take an obious typo---I resides next to the U---as a commence for a pun. Not my kinda style, ``Pail."

To elaborate on this: If a newbie coming to Scala meets Paul Phillips on this list, this know-it-all, explains-nothing, I think many newbies might be defeated and simply be afraid of this omnipotent guy and Poker millionaire from San Francisco. (I wish I had that money, so this is not a discussion of envy. Cudos!) But newbies. in general, don't have much luck here---whenever they pose an allegedly dumb question, God's ire strikes down on `em immediately.

Are you suggesting that Paul==god?
Given his enviable track record on getting equality right, I suspect that this too, could be made to give the expected answer :)