Synthetic Methods from a compiler plugin - we can do that!

I originally posted this to scala-internals
but now the code is live on github, I think it's of more general appeal...

The project can be found at:
http://github.com/scala-incubator/autoproxy-plugin
note: the annotation in my original post was @with, I've changed it to @proxy so that I can stay in lowercase and avoid a keyword conflict.

So, generating new methods in a plugin, here's what it does...

First, my test code that uses the plugin and requires synthetic methods:

class Foo {
       val prop0 = "prop0"
       def method0 = "method0"
       def method1(arg: String) = "method1(" + arg + ")"
}

class Bar {
       @proxy val myFoo : Foo = new RichFoo
       def barmethod0 = "barmethod0"
}

object Main {
       def main(args : Array[String]) {
               val bar = new Bar
               println(bar.prop1)
               println(bar.method0)
               println(bar.barmethod0)
       }
}


The idea behind the @proxy annotation is that it creates delegates for
all members of the annotated member, thusly:

class Bar {
       @proxy val myFoo : Foo = new RichFoo
       def barmethod0 = "barmethod0"

       // These are synthesized
       def prop0 = myFoo.prop0
       def method0 = myFoo.ethod0
       def method1(arg: String) = myFoo.method1(arg)
}

The annotation can be used on any variety of member: vars, vals, defs and constructor arguments.
Used on a constructor parameter, this is a very clean way to allow dynamic mixins.
i.e. mixins on existing object instances instead of on classes at the point of construction

Still to do:
- add all of Foo's interfaces to Bar.  If Foo is a trait then this means that Bar can be anywhere that an instance of Foo is required.
- lots of testing against various typing scenarios, I really have to see how this interacts if either Foo or Bar is a parameterized type.


And here's how it works:
(feel free to skip this part if you're not concerned about compiler internals)

None of that is especially difficult, code generation is trivial using
the TreeDSL library (thanks again Paul!)

So what's the hard part?

To generate these synthetics I need to know the members of myFoo,
which means that I need to run the typer phase
but... typer will fail on Main, as prop1 and method0 do not yet exist on Bar

It seems pretty obvious that typer will have to be run twice, once to
give me the info I need for creating the synthetics, and again for
where the synthetics are used.


And how I solved it:

1. Create a subclass of Reporter, called SilentReporter
   This simply records if an error occurred and doesn't write to the console

2. Create and run two new phases: earlynamer and earlytyper.
   Identical to namer and typer except that they swap global.reporter
for an instance of SilentReporter

3. Run the gensynthetics phase, which looks for the @proxy annotation
and builds the delegates.
   This phase extends TypingTransformers so that it can call
localTyper.typed() on the generated code

4. Run another typer phase, retypeerrors
   This is another TypingTransformer that:
       looks for nodes with tpe=ErrorType
       resets them to tpe=null
       calls localTyper.typed on them
   Any errors caused by missing synthetics will be fixed at this point
   Any other errors will be correctly reported, as SilentReporter
isn't being used

5. Proceed onto the pre-existing namer and typer phases.
   These do nothing as the tree is already fully annotated and the
symbol table populated

I've created a (public) wave to discuss the plugin here:https://wave.google.com/wave/?pli=1#restored:search:scala,restored:wave:googlewave.com!w%252Ba3G_NBOLC.1
Anyone needing a wave account, we also have a document to request invites here: http://spreadsheets.google.com/ccc?key=0Aqaj_meGwHUOdGREbFNRRnpMODhZLUw4UGxReEJwRVE&hl=en

On Mon, Nov 16, 2009 at 8:30 PM, Kevin Wright <kev.lee.wright@googlemail.com> wrote:

I originally posted this to scala-internals
but now the code is live on github, I think it's of more general appeal...

The project can be found at:
http://github.com/scala-incubator/autoproxy-plugin
note: the annotation in my original post was @with, I've changed it to @proxy so that I can stay in lowercase and avoid a keyword conflict.

So, generating new methods in a plugin, here's what it does...

First, my test code that uses the plugin and requires synthetic methods:

class Foo {
       val prop0 = "prop0"
       def method0 = "method0"
       def method1(arg: String) = "method1(" + arg + ")"
}

class Bar {
       @proxy val myFoo : Foo = new RichFoo
       def barmethod0 = "barmethod0"
}

object Main {
       def main(args : Array[String]) {
               val bar = new Bar
               println(bar.prop1)
               println(bar.method0)
               println(bar.barmethod0)
       }
}


The idea behind the @proxy annotation is that it creates delegates for
all members of the annotated member, thusly:

class Bar {
       @proxy val myFoo : Foo = new RichFoo
       def barmethod0 = "barmethod0"

       // These are synthesized
       def prop0 = myFoo.prop0
       def method0 = myFoo.ethod0
       def method1(arg: String) = myFoo.method1(arg)
}

The annotation can be used on any variety of member: vars, vals, defs and constructor arguments.
Used on a constructor parameter, this is a very clean way to allow dynamic mixins.
i.e. mixins on existing object instances instead of on classes at the point of construction

Still to do:
- add all of Foo's interfaces to Bar.  If Foo is a trait then this means that Bar can be anywhere that an instance of Foo is required.
- lots of testing against various typing scenarios, I really have to see how this interacts if either Foo or Bar is a parameterized type.


And here's how it works:
(feel free to skip this part if you're not concerned about compiler internals)

None of that is especially difficult, code generation is trivial using
the TreeDSL library (thanks again Paul!)

So what's the hard part?

To generate these synthetics I need to know the members of myFoo,
which means that I need to run the typer phase
but... typer will fail on Main, as prop1 and method0 do not yet exist on Bar

It seems pretty obvious that typer will have to be run twice, once to
give me the info I need for creating the synthetics, and again for
where the synthetics are used.


And how I solved it:

1. Create a subclass of Reporter, called SilentReporter
   This simply records if an error occurred and doesn't write to the console

2. Create and run two new phases: earlynamer and earlytyper.
   Identical to namer and typer except that they swap global.reporter
for an instance of SilentReporter

3. Run the gensynthetics phase, which looks for the @proxy annotation
and builds the delegates.
   This phase extends TypingTransformers so that it can call
localTyper.typed() on the generated code

4. Run another typer phase, retypeerrors
   This is another TypingTransformer that:
       looks for nodes with tpe=ErrorType
       resets them to tpe=null
       calls localTyper.typed on them
   Any errors caused by missing synthetics will be fixed at this point
   Any other errors will be correctly reported, as SilentReporter
isn't being used

5. Proceed onto the pre-existing namer and typer phases.
   These do nothing as the tree is already fully annotated and the
symbol table populated