- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
observable laziness
Tue, 2010-10-12, 01:20
This is more satisfying than it might otherwise be because no compilers
were harmed in the making of this code. It's all off the shelf parts.
scala> import scala.tools.nsc.util.LazyInfo._
import scala.tools.nsc.util.LazyInfo._
scala> class Bippy {
| lazy val x1 = { println("x1") ; 5 }
| lazy val x2 = { println("x2") ; "abcde" }
| }
defined class Bippy
scala> val bip = new Bippy
bip: Bippy = Bippy@1f3c5cc8
scala> val info = bip.lazify
info: scala.tools.nsc.util.LazyInfo[Bippy] = scala.tools.nsc.util.LazyInfo@4e89cb3b
scala> info.lazyNames
res0: List[String] = List(x1, x2)
scala> val o1 = info.observe("x1", _.x1)
o1: info.ObservableLazyVal[Int] = Bippy#x1:
scala> o1.isDone
res1: Boolean = false
scala> o1.get()
x1
res2: Int = 5
scala> o1.isDone
res3: Boolean = true
scala> val o2 = info.observe("x2", _.x2)
o2: info.ObservableLazyVal[java.lang.String] = Bippy#x2:
scala> bip.x2
x2
res4: java.lang.String = abcde
scala> o2.isDone
res5: Boolean = true
scala> info.lazyNames
res6: List[String] = List()
scala> info.forcedNames
res7: List[String] = List(x1, x2)
Tue, 2010-10-12, 02:17
#2
Re: observable laziness
On Mon, Oct 11, 2010 at 08:48:20PM -0400, Jorge Ortiz wrote:
> If I could add extra "hooks" to how lazy vals get accessed, I would be
> so, so, so happy.
What would the interface look like in your fantasy scenario?
> Foursquare uses Lift. Lift stores stuff in the session. Most of the
> time, Lift is storing closures in the session. A lot of the time,
> Scala closures capture an $outer variable that points to pretty much
> everything the closure could possibly point to.
I'm sure you're aware of the tickets, but for completeness.
https://lampsvn.epfl.ch/trac/scala/ticket/1419
"don't generate unneeded references to outer objects in inner classes"
That one is two years old, not too promising.
https://lampsvn.epfl.ch/trac/scala/ticket/2617
""$outer" variable in closures causes objects to unnecessarily remain referenced"
That one gambled by micromanaging and busted out with a wontfix.
And then there are related ones such as:
https://lampsvn.epfl.ch/trac/scala/ticket/743
"generated java field in object left referencing stale data"
Which plays out as a bit of tragic theater:
Changed 2 years ago by dragos
You are right, there is indeed a problem. Let me first explain what happens: [...]
Changed 2 years ago by dragos
* owner changed from dragos to odersky
Well, I'm not so sure there is an easy fix for this issue...
Changed 2 years ago by odersky
* status changed from new to assigned
* milestone changed from next_bugfix to postponed
It's indeed not so simple. We need to solve it, but can't do it right now.
Tue, 2010-10-12, 05:07
#3
Re: observable laziness
Is there anywhere I can see the implementation for LazyInfo and ObservableLazyVal?
Ideally this to work for both lazy vals that defined inside methods as well as lazy vals defined inside classes.
I'd have to think about ideal interface... Right now it looks like
lazy val leaky = 5
vs
val notLeaky /* : RequestCache[Int] */ = RequestCache { 5 }
This interface is as clean as it can get at the definition-site. At the access-site I either need an ugly explicit method call or a flaky implicit one (neither is ideal). This looks like I would pay a definition-site cost for the benefit of clean access-sites. (Probably a price I'd be willing to pay)
In terms of mechanics, I would need two hooks. One that lets me submit an (ObservableLazyVal[_] => Unit) that gets called when the lazy val is first accessed (registers the lazy val with the request, so the request knows what to clean up when it's done), and a method on ObservableLazyVal[_] that lets me "clear" the lazy val (null it out, reset state so it gets recomputed on next access).
--j
On Mon, Oct 11, 2010 at 9:13 PM, Paul Phillips <paulp@improving.org> wrote:
Ideally this to work for both lazy vals that defined inside methods as well as lazy vals defined inside classes.
I'd have to think about ideal interface... Right now it looks like
lazy val leaky = 5
vs
val notLeaky /* : RequestCache[Int] */ = RequestCache { 5 }
This interface is as clean as it can get at the definition-site. At the access-site I either need an ugly explicit method call or a flaky implicit one (neither is ideal). This looks like I would pay a definition-site cost for the benefit of clean access-sites. (Probably a price I'd be willing to pay)
In terms of mechanics, I would need two hooks. One that lets me submit an (ObservableLazyVal[_] => Unit) that gets called when the lazy val is first accessed (registers the lazy val with the request, so the request knows what to clean up when it's done), and a method on ObservableLazyVal[_] that lets me "clear" the lazy val (null it out, reset state so it gets recomputed on next access).
--j
On Mon, Oct 11, 2010 at 9:13 PM, Paul Phillips <paulp@improving.org> wrote:
On Mon, Oct 11, 2010 at 08:48:20PM -0400, Jorge Ortiz wrote:
> If I could add extra "hooks" to how lazy vals get accessed, I would be
> so, so, so happy.
What would the interface look like in your fantasy scenario?
> Foursquare uses Lift. Lift stores stuff in the session. Most of the
> time, Lift is storing closures in the session. A lot of the time,
> Scala closures capture an $outer variable that points to pretty much
> everything the closure could possibly point to.
I'm sure you're aware of the tickets, but for completeness.
https://lampsvn.epfl.ch/trac/scala/ticket/1419
"don't generate unneeded references to outer objects in inner classes"
That one is two years old, not too promising.
https://lampsvn.epfl.ch/trac/scala/ticket/2617
""$outer" variable in closures causes objects to unnecessarily remain referenced"
That one gambled by micromanaging and busted out with a wontfix.
And then there are related ones such as:
https://lampsvn.epfl.ch/trac/scala/ticket/743
"generated java field in object left referencing stale data"
Which plays out as a bit of tragic theater:
Changed 2 years ago by dragos
You are right, there is indeed a problem. Let me first explain what happens: [...]
Changed 2 years ago by dragos
* owner changed from dragos to odersky
Well, I'm not so sure there is an easy fix for this issue...
Changed 2 years ago by odersky
* status changed from new to assigned
* milestone changed from next_bugfix to postponed
It's indeed not so simple. We need to solve it, but can't do it right now.
--
Paul Phillips | Adultery is the application of democracy to love.
Future Perfect | -- H. L. Mencken
Empiricist |
ha! spill, pupil |----------* http://www.improving.org/paulp/ *----------
�
Tue, 2010-10-12, 06:17
#4
Re: observable laziness
On Mon, Oct 11, 2010 at 11:57:09PM -0400, Jorge Ortiz wrote:
> Is there anywhere I can see the implementation for LazyInfo and
> ObservableLazyVal?
Long as we understand I wrote this in a couple hours this afternoon and
am not suggesting it's complete, sure. Here it is right up to this
second right... now.
> In terms of mechanics, I would need two hooks. One that lets me submit
> an (ObservableLazyVal[_] => Unit) that gets called when the lazy val
> is first accessed (registers the lazy val with the request, so the
> request knows what to clean up when it's done), and a method on
> ObservableLazyVal[_] that lets me "clear" the lazy val (null it out,
> reset state so it gets recomputed on next access).
These things seem outside the scope of what I've done here. It is not a
technical difficulty to generate different code for the lazy val logic,
but I'm sure it would be a hard sell. To have something run when the
lazy val unthunks will require either that, wrapping it, or if you are
really, really sick, you could poll it with the code above. If I had
your requirements I assume I would just reimplement them the way I
wanted them.
There's no code in there to reset the lazy bit because I didn't go into
this with apocalyptic chaos as a specific goal. You can see it wouldn't
be difficult assuming jvm access restrictions don't stand in the way,
but rebooting lazy vals is when I smile and nod and sloooowly back away.
Tue, 2010-10-12, 09:37
#5
Re: observable laziness
On Tue, Oct 12, 2010 at 5:57 AM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:
Is there anywhere I can see the implementation for LazyInfo and ObservableLazyVal?
Ideally this to work for both lazy vals that defined inside methods as well as lazy vals defined inside classes.
I'd have to think about ideal interface... Right now it looks like
lazy val leaky = 5
vs
val notLeaky /* : RequestCache[Int] */ = RequestCache { 5 }
This interface is as clean as it can get at the definition-site. At the access-site I either need an ugly explicit method call or a flaky implicit one (neither is ideal). This looks like I would pay a definition-site cost for the benefit of clean access-sites. (Probably a price I'd be willing to pay)
In terms of mechanics, I would need two hooks. One that lets me submit an (ObservableLazyVal[_] => Unit) that gets called when the lazy val is first accessed (registers the lazy val with the request, so the request knows what to clean up when it's done), and a method on ObservableLazyVal[_] that lets me "clear" the lazy val (null it out, reset state so it gets recomputed on next access).
I don't think this is desirable. What you describe here looks like a caching mechanism, which should not be hooked to a language feature. What exactly is wrong with using your own classes? Syntax doesn't look that bad either, and I'm sure whoever reads your code afterwards will be happy to know lazy values are exactly what they are supposed to be.
I'll have a look again at the $outer pointer tickets. That sounds like the real issue here, and maybe we can help with a more general solution (but it does indeed look hard :).
iulian
--j
On Mon, Oct 11, 2010 at 9:13 PM, Paul Phillips <paulp@improving.org> wrote:
On Mon, Oct 11, 2010 at 08:48:20PM -0400, Jorge Ortiz wrote:
> If I could add extra "hooks" to how lazy vals get accessed, I would be
> so, so, so happy.
What would the interface look like in your fantasy scenario?
> Foursquare uses Lift. Lift stores stuff in the session. Most of the
> time, Lift is storing closures in the session. A lot of the time,
> Scala closures capture an $outer variable that points to pretty much
> everything the closure could possibly point to.
I'm sure you're aware of the tickets, but for completeness.
https://lampsvn.epfl.ch/trac/scala/ticket/1419
"don't generate unneeded references to outer objects in inner classes"
That one is two years old, not too promising.
https://lampsvn.epfl.ch/trac/scala/ticket/2617
""$outer" variable in closures causes objects to unnecessarily remain referenced"
That one gambled by micromanaging and busted out with a wontfix.
And then there are related ones such as:
https://lampsvn.epfl.ch/trac/scala/ticket/743
"generated java field in object left referencing stale data"
Which plays out as a bit of tragic theater:
Changed 2 years ago by dragos
You are right, there is indeed a problem. Let me first explain what happens: [...]
Changed 2 years ago by dragos
* owner changed from dragos to odersky
Well, I'm not so sure there is an easy fix for this issue...
Changed 2 years ago by odersky
* status changed from new to assigned
* milestone changed from next_bugfix to postponed
It's indeed not so simple. We need to solve it, but can't do it right now.
--
Paul Phillips | Adultery is the application of democracy to love.
Future Perfect | -- H. L. Mencken
Empiricist |
ha! spill, pupil |----------* http://www.improving.org/paulp/ *----------
�
--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais
Really, really want.
If I could add extra "hooks" to how lazy vals get accessed, I would be so, so, so happy.
Background:
Foursquare uses Lift. Lift stores stuff in the session. Most of the time, Lift is storing closures in the session. A lot of the time, Scala closures capture an $outer variable that points to pretty much everything the closure could possibly point to. If this closure gets stored in the session, a lot of stuff is prevented from being garbage collected for the duration of the session (15-30minutes). This gets ugly pretty quickly, as you're jamming the session full of closures that have pointers to big result sets from database queries which can't be garbage collected.
We've written a RequestCache class, which is basically a lazy val, except: 1) on creation it gets registered in a ThreadLocal variable, 2) at the end of the request it gets nulled out, and 3) if it's ever needed again it can recompute itself (by keeping a closure, hah!).
It's a total hack, and it has some sharp edges, but it works well enough and has alleviated some memory issues we had.
In order to access it, there's an implicit RequestCache[T] => T (yuck, I know), but if it could be layered on as a "hook" to lazy vals, it'd be even easier and more convenient to use.
--j
On Mon, Oct 11, 2010 at 8:20 PM, Paul Phillips <paulp@improving.org> wrote: