This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Random Interpreter Hackery

18 replies
Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Hey guys,
I'm trying to pull the bytes of a class defined in the interpreter using the classloader.  
My current code is as follows:
lazy val classLoaderClass : Option[Class[_]] = try {       Some(Class.forName("scala.tools.nsc.interpreter.AbstractFileClassLoader"))  } catch {    case ex : ClassNotFoundException => None  }  private def isReplClassLoader(cl : ClassLoader) = {     if (cl.getClass.getName == "scala.tools.nsc.interpreter.AbstractFileClassLoader") true    else classLoaderClass.map(_.isAssignableFrom(cl.getClass)) getOrElse false  }
val obj = new Function0[Unit] { def apply() { println("Hi") } }val cll = obj.getClass.getClassLoader.asInstanceOf[ScalaClassLoader]val bytes = cll.findBytesForClassName(obj.getClass.getCanonicalName) Console.println("Found " + cl.getName + " with classfile size = " + bytes.length + " from cl " + cll)
I keep getting back empty bytes.   I noticed that classes are built in a virtual directory called "(memory)".  I also know that each line gets placed into a package file in lineN style.
Is there a way for me to use the findBytesForClassName to recover the original bytes of the class?  I'm trying to directly serialize REPL objects for parallelization to other JVMs and I can't use RMI.

Thanks for the help!- Josh
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/10/11 5:03 AM, Josh Suereth wrote:
> Hey guys,
>
> I'm trying to pull the bytes of a class defined in the interpreter using
> the classloader.

Look at Javap.scala and then ILoop.scala. The :javap command does
exactly what you want. The key is pathToFlatName.

private object javap extends Javap(intp.classLoader, new
IMain.ReplStrippingWriter(intp)) {
override def tryClass(path: String): Array[Byte] = {
// Look for Foo first, then Foo$, but if Foo$ is given explicitly,
// we have to drop the $ to find object Foo, then tack it back onto
// the end of the flattened name.
def className = intp pathToFlatName path
def moduleName = (intp pathToFlatName path.stripSuffix("$")) + "$"

val bytes = super.tryClass(className)
if (bytes.nonEmpty) bytes
else super.tryClass(moduleName)
}
}

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
Is there a way I can pull in the interpreter via an implicit or something?  I'm writing code in a library that's compiled.   Later I run the interpreter against the library and construct a pipeline of operations to perform.  I want to serialize these classes, with the interpreter classes, and send them over the wire (in this case, just my hard disk) to the execution engine.
Ideally, I'd like to hinge a few high-priority implicits to see if the interpreter is available, and if so add the extra hacks to serialize the interpreter classes along with the objects.   The unfortunate bit is that now I don't have access to the interpreter at all, so I have no idea if a given class needs to be serialized specially except by looking at its classLoader.
In any case, looks like I have to make a few hacks to unwind all of these method calls so that I can construct a full path sans an interpreter object.
Thanks for the help! - Josh

On Wed, Mar 9, 2011 at 11:52 PM, Paul Phillips <paulp@improving.org> wrote:
On 3/10/11 5:03 AM, Josh Suereth wrote:
Hey guys,

I'm trying to pull the bytes of a class defined in the interpreter using
the classloader.

Look at Javap.scala and then ILoop.scala.  The :javap command does exactly what you want.  The key is pathToFlatName.

 private object javap extends Javap(intp.classLoader, new IMain.ReplStrippingWriter(intp)) {
   override def tryClass(path: String): Array[Byte] = {
     // Look for Foo first, then Foo$, but if Foo$ is given explicitly,
     // we have to drop the $ to find object Foo, then tack it back onto
     // the end of the flattened name.
     def className  = intp pathToFlatName path
     def moduleName = (intp pathToFlatName path.stripSuffix("$")) + "$"

     val bytes = super.tryClass(className)
     if (bytes.nonEmpty) bytes
     else super.tryClass(moduleName)
   }
 }

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
BTW - I'm doing this on 2.8.1. I just looked at the 2.8.1 source code and my did you clean things up.  This looks a bit hairy-er in 2.8.1.

On Thu, Mar 10, 2011 at 10:28 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
Is there a way I can pull in the interpreter via an implicit or something?  I'm writing code in a library that's compiled.   Later I run the interpreter against the library and construct a pipeline of operations to perform.  I want to serialize these classes, with the interpreter classes, and send them over the wire (in this case, just my hard disk) to the execution engine.
Ideally, I'd like to hinge a few high-priority implicits to see if the interpreter is available, and if so add the extra hacks to serialize the interpreter classes along with the objects.   The unfortunate bit is that now I don't have access to the interpreter at all, so I have no idea if a given class needs to be serialized specially except by looking at its classLoader.
In any case, looks like I have to make a few hacks to unwind all of these method calls so that I can construct a full path sans an interpreter object.
Thanks for the help! - Josh

On Wed, Mar 9, 2011 at 11:52 PM, Paul Phillips <paulp@improving.org> wrote:
On 3/10/11 5:03 AM, Josh Suereth wrote:
Hey guys,

I'm trying to pull the bytes of a class defined in the interpreter using
the classloader.

Look at Javap.scala and then ILoop.scala.  The :javap command does exactly what you want.  The key is pathToFlatName.

 private object javap extends Javap(intp.classLoader, new IMain.ReplStrippingWriter(intp)) {
   override def tryClass(path: String): Array[Byte] = {
     // Look for Foo first, then Foo$, but if Foo$ is given explicitly,
     // we have to drop the $ to find object Foo, then tack it back onto
     // the end of the flattened name.
     def className  = intp pathToFlatName path
     def moduleName = (intp pathToFlatName path.stripSuffix("$")) + "$"

     val bytes = super.tryClass(className)
     if (bytes.nonEmpty) bytes
     else super.tryClass(moduleName)
   }
 }


Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
Ok, before I go insane:  Is there a way to take the string:  "line8$object$$iw$$iw$$iw$$iw$$anonfun$1" and convert it into a classname that will get the bits from the AbstractFileClassLoader if I do not have an instance of any other Repl class?  It seems like the interpreter is wrapping things an awful lot to allow overrides to work.  I'm ok with that.  Removing all the layers of convenience in the REPL classes is what's taking me a while, and everytime I think I have it all figured out, it doesn't seem to work.
- Josh

On Thu, Mar 10, 2011 at 11:17 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
BTW - I'm doing this on 2.8.1. I just looked at the 2.8.1 source code and my did you clean things up.  This looks a bit hairy-er in 2.8.1.

On Thu, Mar 10, 2011 at 10:28 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
Is there a way I can pull in the interpreter via an implicit or something?  I'm writing code in a library that's compiled.   Later I run the interpreter against the library and construct a pipeline of operations to perform.  I want to serialize these classes, with the interpreter classes, and send them over the wire (in this case, just my hard disk) to the execution engine.
Ideally, I'd like to hinge a few high-priority implicits to see if the interpreter is available, and if so add the extra hacks to serialize the interpreter classes along with the objects.   The unfortunate bit is that now I don't have access to the interpreter at all, so I have no idea if a given class needs to be serialized specially except by looking at its classLoader.
In any case, looks like I have to make a few hacks to unwind all of these method calls so that I can construct a full path sans an interpreter object.
Thanks for the help! - Josh

On Wed, Mar 9, 2011 at 11:52 PM, Paul Phillips <paulp@improving.org> wrote:
On 3/10/11 5:03 AM, Josh Suereth wrote:
Hey guys,

I'm trying to pull the bytes of a class defined in the interpreter using
the classloader.

Look at Javap.scala and then ILoop.scala.  The :javap command does exactly what you want.  The key is pathToFlatName.

 private object javap extends Javap(intp.classLoader, new IMain.ReplStrippingWriter(intp)) {
   override def tryClass(path: String): Array[Byte] = {
     // Look for Foo first, then Foo$, but if Foo$ is given explicitly,
     // we have to drop the $ to find object Foo, then tack it back onto
     // the end of the flattened name.
     def className  = intp pathToFlatName path
     def moduleName = (intp pathToFlatName path.stripSuffix("$")) + "$"

     val bytes = super.tryClass(className)
     if (bytes.nonEmpty) bytes
     else super.tryClass(moduleName)
   }
 }



Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
For posterity, I thought I'd share the strange hack I had to use to serialize the class with the object.   The code is pretty lame at this point, but check out the annotateClass method.
For some reason, the findBytesForClass method is failing, but using reflection hacks I can rip out the VirtualDirectory and pull the class file from it by hand.  I'm not all that familiar with the classloader to know why this is the case, and I know that this mechanism will break horribly in 2.9 due to the new Interpreter.   Any suggestions for how to handle that are appreciated ;)
- Josh
----------------------My Lame Hack-----------------------

class InterpreterObjectOutputStream(delegate : ObjectOutputStream) extends ObjectOutputStream(delegate) {   lazy val classLoaderClass = classOf[AbstractFileClassLoader]  private def isReplClassLoader(cl : ClassLoader) = {    if (cl == null || cl.getClass == null) false    else if (cl.getClass.getName == "scala.tools.nsc.interpreter.AbstractFileClassLoader") true     else classLoaderClass.isAssignableFrom(cl.getClass)  }  @throws(classOf[IOException])  override protected def annotateClass(cl : Class[_]) : Unit = {    val extramagic = isReplClassLoader(cl.getClassLoader)     // Mark whether we had to serialize the class.    writeBoolean(extramagic)    if (extramagic) {      val cll = cl.getClassLoader.asInstanceOf[AbstractFileClassLoader]
      // TODO(joshuasuereth): Find a less hacky solution.      val bytes = {        val tmp = cll.findBytesForClassName(cl.getName)        if (tmp.length == 0) {          val tmp = cll.getClass.getDeclaredField("root")           tmp.setAccessible(true)  // Yippie for violating runtime protection.  Hope no one uses java security.          val dir = tmp.get(cll).asInstanceOf[VirtualDirectory]          val fileName = cl.getName + ".class"           val result = dir.iterator.find(_.name == fileName)          Console.println("Found result: " + result)          result.get.toByteArray  // TODO handle missing data better         } else tmp      }      Console.println("Serializing " + cl.getName + " with size = " + bytes.length + " from cl " + cll)      if (bytes.length == 0) throw new IOException("Could not find REPL class for: " + cl.getName)       writeInt(bytes.length)      write(bytes)    }  }}
On Thu, Mar 10, 2011 at 12:54 PM, Josh Suereth <joshua.suereth@gmail.com> wrote:
Ok, before I go insane:  Is there a way to take the string:  "line8$object$$iw$$iw$$iw$$iw$$anonfun$1" and convert it into a classname that will get the bits from the AbstractFileClassLoader if I do not have an instance of any other Repl class?  It seems like the interpreter is wrapping things an awful lot to allow overrides to work.  I'm ok with that.  Removing all the layers of convenience in the REPL classes is what's taking me a while, and everytime I think I have it all figured out, it doesn't seem to work.
- Josh

On Thu, Mar 10, 2011 at 11:17 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
BTW - I'm doing this on 2.8.1. I just looked at the 2.8.1 source code and my did you clean things up.  This looks a bit hairy-er in 2.8.1.

On Thu, Mar 10, 2011 at 10:28 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
Is there a way I can pull in the interpreter via an implicit or something?  I'm writing code in a library that's compiled.   Later I run the interpreter against the library and construct a pipeline of operations to perform.  I want to serialize these classes, with the interpreter classes, and send them over the wire (in this case, just my hard disk) to the execution engine.
Ideally, I'd like to hinge a few high-priority implicits to see if the interpreter is available, and if so add the extra hacks to serialize the interpreter classes along with the objects.   The unfortunate bit is that now I don't have access to the interpreter at all, so I have no idea if a given class needs to be serialized specially except by looking at its classLoader.
In any case, looks like I have to make a few hacks to unwind all of these method calls so that I can construct a full path sans an interpreter object.
Thanks for the help! - Josh

On Wed, Mar 9, 2011 at 11:52 PM, Paul Phillips <paulp@improving.org> wrote:
On 3/10/11 5:03 AM, Josh Suereth wrote:
Hey guys,

I'm trying to pull the bytes of a class defined in the interpreter using
the classloader.

Look at Javap.scala and then ILoop.scala.  The :javap command does exactly what you want.  The key is pathToFlatName.

 private object javap extends Javap(intp.classLoader, new IMain.ReplStrippingWriter(intp)) {
   override def tryClass(path: String): Array[Byte] = {
     // Look for Foo first, then Foo$, but if Foo$ is given explicitly,
     // we have to drop the $ to find object Foo, then tack it back onto
     // the end of the flattened name.
     def className  = intp pathToFlatName path
     def moduleName = (intp pathToFlatName path.stripSuffix("$")) + "$"

     val bytes = super.tryClass(className)
     if (bytes.nonEmpty) bytes
     else super.tryClass(moduleName)
   }
 }




extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/10/11 7:21 PM, Josh Suereth wrote:
> For some reason, the findBytesForClass method is failing, but using
> reflection hacks I can rip out the VirtualDirectory and pull the class
> file from it by hand. I'm not all that familiar with the classloader to
> know why this is the case, and I know that this mechanism will break
> horribly in 2.9 due to the new Interpreter.

It's not a new interpreter, just the continued evolution of the same
one. But what you are running into is why there is code to do this in
2.9, and if you want to do it in 2.8.1 I don't think there's a more
direct solution than backporting the code I pointed at.

Also, I will shortly be checking in the view code which should cure both
my regression and the longer term performance fail, at least the one
caused by the linearization issue. I am quite interested in your view
performance results (and would like to subscribe to your performance
newsletter with its pi charts and jagged graphs.)

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/10/11 9:54 AM, Josh Suereth wrote:
> Ok, before I go insane: Is there a way to take the string:
> "line8$object$$iw$$iw$$iw$$iw$$anonfun$1" and convert it into a
> classname that will get the bits from the AbstractFileClassLoader if I
> do not have an instance of any other Repl class?

Not sure how that name differs from the name you need for the
classloader. Looking at the code I am appalled at a) how much work I
have done in this area which I have never checked in and b) the
unhelpful tendency of VirtualDirectory and AbstractFile in general to
throw exceptions just because it can. It didn't used to do that here: I
once could directly poke around the repl generated classfiles in
virtualDirectory, but now it just hurls throwables at me.

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
So, in 2.8.1 I can look up that class with the name "(memory)/line8$object$$iw$$iw$$iw$$iw$$anonfun$1.class" on the VirtualDirectory.  If I try to use getBytesForClass then these strings all do not work:
"(memory).line8$object$$iw$$iw$$iw$$iw$$anonfun$1""line8$object$$iw$$iw$$iw$$iw$$anonfun$1" "line8$object.$iw$$iw$$iw$$iw$$anonfun$1"etc. with more random . replacement for giggles.
So yes.  The work you did in trunk is really helping to clean stuff up, but I think I'll stick with my hack for 2.8.1 and use a better solution once it exists ;)
As a side note, how do you feel about providing an implicit scala.tools.nsc.Interpreter (or scala.tools.nsc.interpreter.IMain) object when running inside the REPL?   I'd love to have my API have opt-in implicits that use this desrialization magic *only* when required, i.e. running in the REPL.
- Josh
On Fri, Mar 11, 2011 at 7:37 AM, Paul Phillips <paulp@improving.org> wrote:
On 3/10/11 9:54 AM, Josh Suereth wrote:
Ok, before I go insane:  Is there a way to take the string:
"line8$object$$iw$$iw$$iw$$iw$$anonfun$1" and convert it into a
classname that will get the bits from the AbstractFileClassLoader if I
do not have an instance of any other Repl class?

Not sure how that name differs from the name you need for the classloader.  Looking at the code I am appalled at a) how much work I have done in this area which I have never checked in and b) the unhelpful tendency of VirtualDirectory and AbstractFile in general to throw exceptions just because it can.  It didn't used to do that here: I once could directly poke around the repl generated classfiles in virtualDirectory, but now it just hurls throwables at me.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/11/11 7:03 AM, Josh Suereth wrote:
> So, in 2.8.1 I can look up that class with the name
> "(memory)/line8$object$$iw$$iw$$iw$$iw$$anonfun$1.class" on the
> VirtualDirectory. If I try to use getBytesForClass then these strings
> all do not work:
>
> "(memory).line8$object$$iw$$iw$$iw$$iw$$anonfun$1"
> "line8$object$$iw$$iw$$iw$$iw$$anonfun$1"
> "line8$object.$iw$$iw$$iw$$iw$$anonfun$1"

It works for me if I strip (memory)/ and .class. This is 2.8.1:

scala> repl.virtualDirectory.toList map (_.path stripPrefix "(memory)/"
stripSuffix ".class") foreach (x => println(x + ": " + (try {
repl.classLoader getBytesForClass x size } catch { case x => "" })))

line30$object$$iw$$iw$$iw$$iw$: 681
line3$object$$iw$$iw$$iw$: 353
line52$object$$iw$$iw$$iw$$iw$$anonfun$2: 1502
RequestResult$line49$object$: 1455
line2$object: 672
line39$object$$iw$$iw$$iw$: 355
line46$object$$iw$: 339
line9$object$: 329
[etc etc but no failures]

> As a side note, how do you feel about providing an implicit
> scala.tools.nsc.Interpreter (or scala.tools.nsc.interpreter.IMain)
> object when running inside the REPL?

I like it fine in principle, what I don't like is the idea of having to
support an interface here. It's still a long ways from where I want it
and paulp is kind of beset as it is.

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery


On Fri, Mar 11, 2011 at 10:18 AM, Paul Phillips <paulp@improving.org> wrote:
On 3/11/11 7:03 AM, Josh Suereth wrote:
So, in 2.8.1 I can look up that class with the name
"(memory)/line8$object$$iw$$iw$$iw$$iw$$anonfun$1.class" on the
VirtualDirectory.  If I try to use getBytesForClass then these strings
all do not work:

"(memory).line8$object$$iw$$iw$$iw$$iw$$anonfun$1"
"line8$object$$iw$$iw$$iw$$iw$$anonfun$1"
"line8$object.$iw$$iw$$iw$$iw$$anonfun$1"

It works for me if I strip (memory)/ and .class.  This is 2.8.1:

scala> repl.virtualDirectory.toList map (_.path stripPrefix "(memory)/" stripSuffix ".class") foreach (x => println(x + ": " + (try { repl.classLoader getBytesForClass x size } catch { case x => "<failed>" })))


Ah... I was actuallying tring to pull it from x.getClass.getClassLoader... hmm.... wonder what the difference is between that and repl.classLoader.   When I do x.getClass.getClassLoader == repl.classLoader // true.

 
line30$object$$iw$$iw$$iw$$iw$: 681
line3$object$$iw$$iw$$iw$: 353
line52$object$$iw$$iw$$iw$$iw$$anonfun$2: 1502
RequestResult$line49$object$: 1455
line2$object: 672
line39$object$$iw$$iw$$iw$: 355
line46$object$$iw$: 339
line9$object$: 329
[etc etc but no failures]

As a side note, how do you feel about providing an implicit
scala.tools.nsc.Interpreter (or scala.tools.nsc.interpreter.IMain)
object when running inside the REPL?

I like it fine in principle, what I don't like is the idea of having to support an interface here.  It's still a long ways from where I want it and paulp is kind of beset as it is.

Totally understand.   I actually don't care about the REPL object itself, just if it is implicitly there, I want a different set of behavior to kick in that does not use the REPL directly.  Things like preventing horribly verbose STDOUT printing and such.   I can wait for this sort of magikz.
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/11/11 7:23 AM, Josh Suereth wrote:
> Ah... I was actuallying tring to pull it from
> x.getClass.getClassLoader... hmm.... wonder what the difference is
> between that and repl.classLoader. When I do x.getClass.getClassLoader
> == repl.classLoader // true.

There will be no difference unless you call reset() or something, in
which case a new classLoader is created. Then all the previous objects
will still point at the old one if you use getClass. If they compare
equal then there can't be any difference, as it's the same object.

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
All I can say is that it wasn't working for me and it was pretty vexing, because everything should have worked.  I'll chalk it up to my own incompetence, but I can't seem to get the classfile bytes without going directly at the virtual directory.   At least this is working I can make a sweet demo with a sucky backend.
I'll try to get to those Collections performance charts this weekend so you can be amazed at how one person can make such ugly graphs.
- Josh

On Fri, Mar 11, 2011 at 10:45 AM, Paul Phillips <paulp@improving.org> wrote:
On 3/11/11 7:23 AM, Josh Suereth wrote:
Ah... I was actuallying tring to pull it from
x.getClass.getClassLoader... hmm.... wonder what the difference is
between that and repl.classLoader.   When I do x.getClass.getClassLoader
== repl.classLoader // true.

There will be no difference unless you call reset() or something, in which case a new classLoader is created.  Then all the previous objects will still point at the old one if you use getClass.  If they compare equal then there can't be any difference, as it's the same object.

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
Funny story, I think I have serialize all these class files anyway, just in case the one I'm serializing depends on another... 

On Fri, Mar 11, 2011 at 11:28 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
All I can say is that it wasn't working for me and it was pretty vexing, because everything should have worked.  I'll chalk it up to my own incompetence, but I can't seem to get the classfile bytes without going directly at the virtual directory.   At least this is working I can make a sweet demo with a sucky backend.
I'll try to get to those Collections performance charts this weekend so you can be amazed at how one person can make such ugly graphs.
- Josh

On Fri, Mar 11, 2011 at 10:45 AM, Paul Phillips <paulp@improving.org> wrote:
On 3/11/11 7:23 AM, Josh Suereth wrote:
Ah... I was actuallying tring to pull it from
x.getClass.getClassLoader... hmm.... wonder what the difference is
between that and repl.classLoader.   When I do x.getClass.getClassLoader
== repl.classLoader // true.

There will be no difference unless you call reset() or something, in which case a new classLoader is created.  Then all the previous objects will still point at the old one if you use getClass.  If they compare equal then there can't be any difference, as it's the same object.


spoon
Joined: 2008-07-01,
User offline. Last seen 1 year 21 weeks ago.
Re: Random Interpreter Hackery

On Fri, Mar 11, 2011 at 10:03 AM, Josh Suereth wrote:
> As a side note, how do you feel about providing an implicit
> scala.tools.nsc.Interpreter (or scala.tools.nsc.interpreter.IMain) object
> when running inside the REPL?   I'd love to have my API have opt-in
> implicits that use this desrialization magic *only* when required, i.e.
> running in the REPL.

There is already support for adding new objects via
interpreter.bind(). You can bind objects that are then available
within the session.

Binding a specific object for your application looks better than
opening up the entire interpreter to call itself from within
itself....

Lex

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
Yeah, I'm not actually looking for the entire interpreter interface, just a mechanism of finding out if I'm in the interpreter.
And using interpreter.bind() is less than ideal, but was also my thought for a backup plan.  I'd like to have a zero-config mechanism of allowing users of the library to have REPL-specific functions get used.  Forcing them to use my own REPL class isn't quite zero-config.
- Josh

On Mon, Mar 14, 2011 at 12:28 PM, Lex Spoon <lex@lexspoon.org> wrote:
On Fri, Mar 11, 2011 at 10:03 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
> As a side note, how do you feel about providing an implicit
> scala.tools.nsc.Interpreter (or scala.tools.nsc.interpreter.IMain) object
> when running inside the REPL?   I'd love to have my API have opt-in
> implicits that use this desrialization magic *only* when required, i.e.
> running in the REPL.

There is already support for adding new objects via
interpreter.bind(). You can bind objects that are then available
within the session.

Binding a specific object for your application looks better than
opening up the entire interpreter to call itself from within
itself....

Lex

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/11/11 8:28 AM, Josh Suereth wrote:
> All I can say is that it wasn't working for me and it was pretty vexing,
> because everything should have worked. I'll chalk it up to my own
> incompetence, but I can't seem to get the classfile bytes without going
> directly at the virtual directory.

In case you're not an avid changelog reader:

http://lampsvn.epfl.ch/trac/scala/changeset/24624

% scala -Dscala.repl.power

scala> class Bippus extends Traversable[Int] { def foreach[U](f: Int
=> U) = () }
defined class Bippus

scala> intp.classLoader.getResourceAsStream("Bippus").bytes()
res0: Array[Byte] = Array(-54, -2, -70, ...

scala> res0.size
res1: Int = 23954

scala> case class Bippy(x: Int)
defined class Bippy

// classBytes is shorter way to say the same thing
scala> intp.classLoader.classBytes("Bippy").size
res2: Int = 2356

scala> intp.classLoader.classBytes("Bippy$").size
res3: Int = 1741

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Random Interpreter Hackery
AWESOME!
As an aside... I'm now at the pointer where I need to serialize all dependencies of a given class as well... I.e. all objects, vals and defs defined in the REPL need to have their class-bytes serialized too.  SO I know pull the entire VIrtual Directory and try to share it...
In any case, combined with some ideas I have to pull dependencies, I think this is the perfect solution.  Is this in 2.9?

On Thu, Mar 31, 2011 at 12:51 AM, Paul Phillips <paulp@improving.org> wrote:
On 3/11/11 8:28 AM, Josh Suereth wrote:
All I can say is that it wasn't working for me and it was pretty vexing,
because everything should have worked.  I'll chalk it up to my own
incompetence, but I can't seem to get the classfile bytes without going
directly at the virtual directory.

In case you're not an avid changelog reader:

http://lampsvn.epfl.ch/trac/scala/changeset/24624

 % scala -Dscala.repl.power

 scala> class Bippus extends Traversable[Int] { def foreach[U](f: Int => U) = () }
 defined class Bippus

 scala> intp.classLoader.getResourceAsStream("Bippus").bytes()
 res0: Array[Byte] = Array(-54, -2, -70, ...

 scala> res0.size
 res1: Int = 23954

 scala> case class Bippy(x: Int)
 defined class Bippy

 // classBytes is shorter way to say the same thing
 scala> intp.classLoader.classBytes("Bippy").size
 res2: Int = 2356

 scala> intp.classLoader.classBytes("Bippy$").size
 res3: Int = 1741

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Random Interpreter Hackery

On 3/31/11 6:28 AM, Josh Suereth wrote:
> In any case, combined with some ideas I have to pull dependencies, I
> think this is the perfect solution. Is this in 2.9?

I don't think it's in the RC, but all my commits are still intended for
2.9. (But I don't know if it'll play out that way.)

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland