Actors/Futures

On Fri, Aug 21, 2009 at 08:21:46AM -0600, Bastian, Mark wrote:
> val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f

Range strikes again.

This isn't doing what you think it's doing. Try this:

for (i <- (0 to Runtime.getRuntime.availableProcessors).toList)

And here I will again predict that this behavior is going to change
eventually so we might as well deal with it now. (That's what I said
about the relative package problem way back when, so my skills as a
scala prophet are unquestionable!) It's simply too big a trap to leave
in a place where innocent newbies can get at it.

On Fri, Aug 21, 2009 at 08:21:46AM -0600, Bastian, Mark wrote:
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f

Range strikes again.

This isn't doing what you think it's doing.  Try this:

  for (i <- (0 to Runtime.getRuntime.availableProcessors).toList)

And here I will again predict that this behavior is going to change
eventually so we might as well deal with it now.  (That's what I said
about the relative package problem way back when, so my skills as a
scala prophet are unquestionable!) It's simply too big a trap to leave
in a place where innocent newbies can get at it.

--
Paul Phillips      | Simplicity and elegance are unpopular because
In Theory          | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
ha! spill, pupil   |     -- Dijkstra

On Fri, Aug 21, 2009 at 08:21:46AM -0600, Bastian, Mark wrote:
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f

Range strikes again.

This isn't doing what you think it's doing.  Try this:

  for (i <- (0 to Runtime.getRuntime.availableProcessors).toList)

And here I will again predict that this behavior is going to change
eventually so we might as well deal with it now.  (That's what I said
about the relative package problem way back when, so my skills as a
scala prophet are unquestionable!) It's simply too big a trap to leave
in a place where innocent newbies can get at it.

--
Paul Phillips      | Simplicity and elegance are unpopular because
In Theory          | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
ha! spill, pupil   |     -- Dijkstra

Hi,

Paul is right about the range issue--the second `!!` send was only
executed after you received the first reply.

Otherwise, the use of `future` is redundant here--you already use `!!`
and let the actor do the computation. So, here it is:

object ScalaFuturesApp3 {
def main(args : Array[String]) {
val futures = for(i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
yield createActor !! "future"
val t = System.nanoTime
//This blocks.
val res = for(f <- futures) {
f()
println((System.nanoTime - t) * 1.0E-9)
}
//If this is much greater than 1, you probably
//aren't doing it right.
println((System.nanoTime - t) * 1.0E-9)
}

def createActor = actor {
react {
case "future" =>
println("started")
Thread.sleep(1000)
println("DONE")
reply()
}
}
}

If you don't need messaging, you can use `future` as follows:

val futures = for (i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
yield future { Thread.sleep(1000); println("DONE") }
val t = System.nanoTime
val res = for(f <- futures) {
f()
println((System.nanoTime - t) * 1.0E-9)
}

Cheers,
Philipp

Bastian, Mark wrote:
> Hi Folks,
>
> I’ve been scouring the web and “Programming in Scala” for examples on
> how !! and futures are used within the Actors framework and have come up
> empty. Based on the examples I’ve found, I’ve worked up this example:
>
> object ScalaFuturesApp {
> def main(args : Array[String]) : Unit =
> {
> val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
> yield f
> val t = System.nanoTime
> for(f <- futures) f()
> //If this is is much greater than the number of processors, you probably
> //aren't doing it right.
> println((System.nanoTime - t) * 1.0E-9)
> }
>
> def f = future { Thread.sleep(1000) }
> }
>
> Unsurprisingly, it doesn’t execute the futures in parallel (I am not
> surprised because a) the docs state futures suspend execution in the
> current actor; and b) I don’t see any actors in this code). However, the
> few examples I’ve found do just this. For examples:
> http://stackoverflow.com/questions/1149505/using-scala-actor-framework-a...
> http://www.nabble.com/Beginner-question-about-actors---concurrent-progra...
>
> So then I tried the following based on the last link, only adding some
> actors in the mix. It appears that the yield f() statements never get
> executed. What I would like to see from this trivial example is a
> something where all of the futures execute in parallel. Something along
> the lines of taking about a second or two, dumping a bunch of “DONE”
> statements, then printing the total execution time.
>
> object ScalaFuturesApp2 {
> def main(args : Array[String]) : Unit =
> {
> val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
> yield createActor !! "future"
> val t = System.nanoTime
> //This should block, but I don't think it does.
> val res = for(f <- futures) yield f()
> //If this is is much greater than the number of processors, you probably
> //aren't doing it right.
> println((System.nanoTime - t) * 1.0E-9)
> }
>
> def createActor = actor
> {
> loop {
> react {
> case "future" => reply(f)
> }
> }
> }
>
> def f = future { Thread.sleep(1000); println("DONE") }
> }
>
> Thanks for any help you can provide.
>
> -Mark

Hi Folks,

I’ve been scouring the web and “Programming in Scala” for examples on how !! and futures are used within the Actors framework and have come up empty. Based on the examples I’ve found, I’ve worked up this example:

object ScalaFuturesApp {
  def main(args : Array[String]) : Unit =
  {
    val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f

Hi,

Paul is right about the range issue--the second `!!` send was only
executed after you received the first reply.

Otherwise, the use of `future` is redundant here--you already use `!!`
and let the actor do the computation. So, here it is:

object ScalaFuturesApp3 {
   def main(args : Array[String]) {
     val futures = for(i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
                   yield createActor !! "future"
     val t = System.nanoTime
     //This blocks.
     val res = for(f <- futures) {
       f()
       println((System.nanoTime - t) * 1.0E-9)
     }
     //If this is much greater than 1, you probably
     //aren't doing it right.
     println((System.nanoTime - t) * 1.0E-9)
   }

   def createActor = actor {
     react {
       case "future" =>
         println("started")
         Thread.sleep(1000)
         println("DONE")
         reply()
     }
   }
}

If you don't need messaging, you can use `future` as follows:

   val futures = for (i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
     yield future { Thread.sleep(1000); println("DONE") }
   val t = System.nanoTime
   val res = for(f <- futures) {
     f()
     println((System.nanoTime - t) * 1.0E-9)
   }

Cheers,
Philipp


Bastian, Mark wrote:
> Hi Folks,
>
> I’ve been scouring the web and “Programming in Scala” for examples on
> how !! and futures are used within the Actors framework and have come up
> empty. Based on the examples I’ve found, I’ve worked up this example:
>
> object ScalaFuturesApp {
>   def main(args : Array[String]) : Unit =
>   {
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
> yield f
>    val t = System.nanoTime
>    for(f <- futures) f()
>     //If this is is much greater than the number of processors, you probably
>    //aren't doing it right.
>    println((System.nanoTime - t) * 1.0E-9)
>   }
>  
>   def f = future { Thread.sleep(1000) }
> }
>
> Unsurprisingly, it doesn’t execute the futures in parallel (I am not
> surprised because a) the docs state futures suspend execution in the
> current actor; and b) I don’t see any actors in this code). However, the
> few examples I’ve found do just this. For examples:
> http://stackoverflow.com/questions/1149505/using-scala-actor-framework-as-fork-join-computation
> http://www.nabble.com/Beginner-question-about-actors---concurrent-programming-td23506032.html
>
> So then I tried the following based on the last link, only adding some
> actors in the mix. It appears that the yield f() statements never get
> executed.  What I would like to see from this trivial example is a
> something where all of the futures execute in parallel. Something along
> the lines of taking about a second or two, dumping a bunch of “DONE”
> statements, then printing the total execution time.
>
> object ScalaFuturesApp2 {
>   def main(args : Array[String]) : Unit =
>   {
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
>       yield createActor !! "future"
>    val t = System.nanoTime
>    //This should block, but I don't think it does.
>    val res = for(f <- futures) yield f()
>     //If this is is much greater than the number of processors, you probably
>    //aren't doing it right.
>    println((System.nanoTime - t) * 1.0E-9)
>   }
>  
>   def createActor = actor
>  {
>     loop {
>       react {
>         case "future" => reply(f)
>       }
>     }
>   }
>  
>   def f = future { Thread.sleep(1000); println("DONE") }
> }
>
> Thanks for any help you can provide.
>
> -Mark

Range is lazy.  When you use for/yield on a range the expression in the yield is not evaluated, until needed.  Like so

case class Foo(x : Int) {
  println("created Foo" + x)
}
 
def test {
 val foos = for(x <- 1 to 10) yield Foo(x)
 println("done")
}

This program only prints "done", never creating any Foos

scala> test
done

This small change forces them to be evaluated

def test {
 val foos = (for(x <- 1 to 10) yield Foo(x)).force // note the ".force"
 println("done")
}

scala> test
created Foo1
created Foo2
created Foo3
created Foo4
created Foo5
created Foo6
created Foo7
created Foo8
created Foo9
created Foo10
done

In your case what's happening is that your futures aren't being created until you demand a value, at which point the future is created, started, and the current thread blocks waiting for the result.  Add a "force"

On Fri, Aug 21, 2009 at 7:21 AM, Bastian, Mark <mbastia [at] sandia [dot] gov" rel="nofollow">mbastia@sandia.gov> wrote:

Hi Folks,

I’ve been scouring the web and “Programming in Scala” for examples on how !! and futures are used within the Actors framework and have come up empty. Based on the examples I’ve found, I’ve worked up this example:

object ScalaFuturesApp {
  def main(args : Array[String]) : Unit =
  {
    val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f

 

Thanks for all of the help! Let me offer up what I would consider the world’s simplest Fork-Join example based on your help:

object ForkJoinExample {
  def main(args : Array[String]) : Unit =
  {
    //Fork
   val futures = for(i <- (1 to Runtime.getRuntime.availableProcessors).toList)
      yield future[Int] { Thread.sleep(1000); 1 }
    val t = System.nanoTime
   //Join
   val numProcs = (for(f <- futures) yield f()).reduceRight(_+_)
    //If this is is much greater than 1, you probably aren't doing it right.
   println((System.nanoTime - t) * 1.0E-9)
    println("You have " + numProcs + " processors.")
  }
}

I apologize if I sound dense, but I am still not sure what the utility of the !! operator is given that the above example seems to pretty much do what you need with futures. Would you use !! to run the future on a different thread or maybe to have the actor be a producer of futures rather than your local thread?

Thanks,
Mark

On 8/21/09 9:29 AM, "Philipp Haller" <philipp [dot] haller [at] epfl [dot] ch" target="_blank" rel="nofollow">philipp.haller@epfl.ch> wrote:

Hi,

Paul is right about the range issue--the second `!!` send was only
executed after you received the first reply.

Otherwise, the use of `future` is redundant here--you already use `!!`
and let the actor do the computation. So, here it is:

object ScalaFuturesApp3 {
   def main(args : Array[String]) {
     val futures = for(i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
                   yield createActor !! "future"
     val t = System.nanoTime
     //This blocks.
     val res = for(f <- futures) {
       f()
       println((System.nanoTime - t) * 1.0E-9)
     }
     //If this is much greater than 1, you probably
     //aren't doing it right.
     println((System.nanoTime - t) * 1.0E-9)
   }

   def createActor = actor {
     react {
       case "future" =>
         println("started")
         Thread.sleep(1000)
         println("DONE")
         reply()
     }
   }
}

If you don't need messaging, you can use `future` as follows:

   val futures = for (i <- (1 to
Runtime.getRuntime.availableProcessors).toList)
     yield future { Thread.sleep(1000); println("DONE") }
   val t = System.nanoTime
   val res = for(f <- futures) {
     f()
     println((System.nanoTime - t) * 1.0E-9)
   }

Cheers,
Philipp


Bastian, Mark wrote:
> Hi Folks,
>
> I’ve been scouring the web and “Programming in Scala” for examples on
> how !! and futures are used within the Actors framework and have come up
> empty. Based on the examples I’ve found, I’ve worked up this example:
>
> object ScalaFuturesApp {
>   def main(args : Array[String]) : Unit =
>   {
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
> yield f
>    val t = System.nanoTime
>    for(f <- futures) f()
>     //If this is is much greater than the number of processors, you probably
>    //aren't doing it right.
>    println((System.nanoTime - t) * 1.0E-9)
>   }
>  
>   def f = future { Thread.sleep(1000) }
> }
>
> Unsurprisingly, it doesn’t execute the futures in parallel (I am not
> surprised because a) the docs state futures suspend execution in the
> current actor; and b) I don’t see any actors in this code). However, the
> few examples I’ve found do just this. For examples:
> http://stackoverflow.com/questions/1149505/using-scala-actor-framework-as-fork-join-computation
> http://www.nabble.com/Beginner-question-about-actors---concurrent-programming-td23506032.html
>
> So then I tried the following based on the last link, only adding some
> actors in the mix. It appears that the yield f() statements never get
> executed.  What I would like to see from this trivial example is a
> something where all of the futures execute in parallel. Something along
> the lines of taking about a second or two, dumping a bunch of “DONE”
> statements, then printing the total execution time.
>
> object ScalaFuturesApp2 {
>   def main(args : Array[String]) : Unit =
>   {
>     val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors)
>       yield createActor !! "future"
>    val t = System.nanoTime
>    //This should block, but I don't think it does.
>    val res = for(f <- futures) yield f()
>     //If this is is much greater than the number of processors, you probably
>    //aren't doing it right.
>    println((System.nanoTime - t) * 1.0E-9)
>   }
>  
>   def createActor = actor
>  {
>     loop {
>       react {
>         case "future" => reply(f)
>       }
>     }
>   }
>  
>   def f = future { Thread.sleep(1000); println("DONE") }
> }
>
> Thanks for any help you can provide.
>
> -Mark

Thanks, makes perfect sense (the reason for the behavior, not the design of the behavior itself. That’s um, unexpected.).

On 8/21/09 9:55 AM, "James Iry" <jamesiry [at] gmail [dot] com" target="_blank" rel="nofollow">jamesiry@gmail.com> wrote:

Range is lazy.  When you use for/yield on a range the expression in the yield is not evaluated, until needed.  Like so

case class Foo(x : Int) {
  println("created Foo" + x)
}
 
def test {
 val foos = for(x <- 1 to 10) yield Foo(x)
 println("done")
}

This program only prints "done", never creating any Foos

scala> test
done

This small change forces them to be evaluated

def test {
 val foos = (for(x <- 1 to 10) yield Foo(x)).force // note the ".force"
 println("done")
}

scala> test
created Foo1
created Foo2
created Foo3
created Foo4
created Foo5
created Foo6
created Foo7
created Foo8
created Foo9
created Foo10
done

In your case what's happening is that your futures aren't being created until you demand a value, at which point the future is created, started, and the current thread blocks waiting for the result.  Add a "force"

On Fri, Aug 21, 2009 at 7:21 AM, Bastian, Mark <mbastia [at] sandia [dot] gov" target="_blank" rel="nofollow">mbastia@sandia.gov> wrote:

Hi Folks,

I’ve been scouring the web and “Programming in Scala” for examples on how !! and futures are used within the Actors framework and have come up empty. Based on the examples I’ve found, I’ve worked up this example:

object ScalaFuturesApp {
  def main(args : Array[String]) : Unit =
  {
    val futures = for(i <- 0 to Runtime.getRuntime.availableProcessors) yield f