- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
Writing a compiler plugin which adds code
Sat, 2009-10-10, 18:29
Hello,
I'm trying to write a compiler plugin, which generates some code.
So far I'm trying the simplest thing: adding a "1" constant to the
beginning of each method.
I implemented the skeleton of the plugin as described on the tutorial
page (http://www.scala-lang.org/node/140), using the Transform trait
in my component (as in the examples).
As I'm generating code, I wanted my plugin to be executed as early as
possible, so I specified "runsAfter" as "namer" (I want my generated
code to be typed etc, so I guess the phase must be before "typer").
In my transformation method I have the following:
tree match {
case t @ global.Block(stats, expr) => global.copy.Block(t,
global.Literal(global.Constant(1)) :: stats, expr)
case t => super.transform(t)
}
so that should add a new literal (constant 1) to the statements of
each block. However, when I'm trying to compile a simple class with
one method, using the plugin, I get an error that the method is
duplicated:
example/src/test/scala/Example1.scala:14: error: ambiguous reference
to overloaded definition,
both method test in object Example1 of type (java.lang.Object)Int
and method test in object Example1 of type (java.lang.Object)Int
match argument types (Null) and expected result type Unit
test(null)
^
example/src/test/scala/Example1.scala:8: error: method test is defined
twice
def test(parameter : Object): Int = {
^
two errors found
I suspect that maybe I have to run "namer" on the code I generate by
hand - but how?
Sat, 2009-10-10, 20:27
#2
Re: Writing a compiler plugin which adds code
typer is set to runsRightAfter namer (in 2.8 at least), so immediately
after typer is the earliest stage where a plugin can operate. At this
point, the AST is essentially the same as the mental model you have
when reading the source, so it's a good place to start with code
generation.
As you understand the phases better, it often becomes more efficient
to implement plugins after later phases. For example, you can safely
ignore a lot of stuff (like inner classes) after the lambda-lift and
explicit-outer phases, and if you can work after erasure then you can
often avoid even more work.
As paul suggests, because you're running after typer you'll need to
re-invoke it again for any generated AST, via typer.typed(...)
As this is brought into scope via global and takes a by-name parameter
it can be used as:
typed {
... generate AST here ....
}
.
It's often the case that phases can go back and forth, not executing
in a strict linear sequence, this is especially true of the
interaction between namer and typer, and of generated code. Once you
get a feel of how earlier phases can be revisited by later ones then
things become much more understandable.
You'll also want to look at some good examples. Both namer and typer
use code generation for case classes and for the @BeanProperty
attribute, there's a syntheticMethods class that handles much of the
rest of case classes. The specialized class also generates a lot of
code, but its a bit of a monster and not the best place to start.
good luck!
:)
On Sat, Oct 10, 2009 at 7:51 PM, Paul Phillips wrote:
> On Sat, Oct 10, 2009 at 07:29:30PM +0200, Adam Warski wrote:
>> As I'm generating code, I wanted my plugin to be executed as early as
>> possible, so I specified "runsAfter" as "namer" (I want my generated
>> code to be typed etc, so I guess the phase must be before "typer").
>
> No... you don't think the 15 phases which come after typer abandon
> typing, do you? I wouldn't try to run before typer. You call methods in
> a typer to get typed trees - see the "typer typed " throughout the
> compiler.
>
> --
> Paul Phillips | Eschew mastication.
> Future Perfect |
> Empiricist |
> pp: i haul pills |----------* http://www.improving.org/paulp/ *----------
>
Mon, 2009-10-12, 08:27
#3
Re: Writing a compiler plugin which adds code
Hello,
thanks for the answers
I suspected that it's possible to generate code after namer, but
wasn't sure how to re-run the phases ;)
> typer is set to runsRightAfter namer (in 2.8 at least), so immediately
> after typer is the earliest stage where a plugin can operate. At this
> point, the AST is essentially the same as the mental model you have
> when reading the source, so it's a good place to start with code
> generation.
I'm still using 2.7, so I don't get this yet.
> As paul suggests, because you're running after typer you'll need to
> re-invoke it again for any generated AST, via typer.typed(...)
Don't I have to re-invoke namer also? There's unfortunately no
"namer" val in global ;)
The problem I'm having now is that in my generated code I'm using a
parameter of the method, and the typer doesn't seem to be able to find
it. So either I need to run namer before typer or I'm missing
something else.
I've tried creating a typer with the proper owner set (the method in
which my code will be enclosed), but this doesn't help also:
global.typer.atOwner(defDef.symbol).typed {
generatedTree
}
Well, anyway, I'll continue investigating.
> You'll also want to look at some good examples. Both namer and typer
> use code generation for case classes and for the @BeanProperty
> attribute, there's a syntheticMethods class that handles much of the
> rest of case classes. The specialized class also generates a lot of
> code, but its a bit of a monster and not the best place to start.
Thanks, I'll check those :)
Mon, 2009-10-12, 10:07
#4
Re: Writing a compiler plugin which adds code
Hello,
> Don't I have to re-invoke namer also? There's unfortunately no
> "namer" val in global ;)
I found namer, but unfortunately that didn't change the exception at
all. I still get
Exception in thread "main" scala.tools.nsc.symtab.Types$TypeError: not
found: value parameter
(parameter is my very original name for the method parameter :) )
The way I do naming and typing now is:
val typer = global.typer.atOwner(tree, tree.symbol) // tree is a
DefDef (method)
typer.namer.enterSym(generatedTree)
val typedGeneratedTree = typer.typed(generatedTree)
And at the invocation of "typed" I get the exception.
Any obvious phase that I'm missing? :)
Mon, 2009-10-12, 10:27
#5
Re: Writing a compiler plugin which adds code
On Mon, Oct 12, 2009 at 9:58 AM, Adam Warski wrote:
> Hello,
>
>> Don't I have to re-invoke namer also? There's unfortunately no "namer"
>> val in global ;)
No, namer is all about taking the parsed/tokenised source and building
the symbol table, which you can manipulate directly in a plugin.
I'm a bit stuck to give good examples, as I'm also new to the compiler
and built all my experience on 2.8, so I really don't know if I'd be
pointing you to something that's totally different under 2.7
> I found namer, but unfortunately that didn't change the exception at all. I
> still get
>
> Exception in thread "main" scala.tools.nsc.symtab.Types$TypeError: not
> found: value parameter
>
> (parameter is my very original name for the method parameter :) )
>
> The way I do naming and typing now is:
>
> val typer = global.typer.atOwner(tree, tree.symbol) // tree is a DefDef
> (method)
> typer.namer.enterSym(generatedTree)
> val typedGeneratedTree = typer.typed(generatedTree)
>
> And at the invocation of "typed" I get the exception.
> Any obvious phase that I'm missing? :)
>
> --
> Adam
>
Wed, 2009-10-14, 08:17
#6
Re: Writing a compiler plugin which adds code
> No, namer is all about taking the parsed/tokenised source and building
> the symbol table, which you can manipulate directly in a plugin.
So you say I should set the symbols myself? Unfortunately I don't
quite know what symbols to put where.
Although I could just copy what is generated by the parser, if I try
to compile the code I generate in a normal file ... but it seems that
re-invoking namer would be easier.
> I'm a bit stuck to give good examples, as I'm also new to the compiler
> and built all my experience on 2.8, so I really don't know if I'd be
> pointing you to something that's totally different under 2.7
Maybe I'll try 2.8 then, as I'm quite stuck with trying to invoke the
typer in the right context, so that it's able to resolve method
parameters.
Wed, 2009-10-14, 09:27
#7
Re: Writing a compiler plugin which adds code
On Wed, Oct 14, 2009 at 8:07 AM, Adam Warski wrote:
>
>> No, namer is all about taking the parsed/tokenised source and building
>> the symbol table, which you can manipulate directly in a plugin.
>
> So you say I should set the symbols myself? Unfortunately I don't quite know
> what symbols to put where.
> Although I could just copy what is generated by the parser, if I try to
> compile the code I generate in a normal file ... but it seems that
> re-invoking namer would be easier.
>
>> I'm a bit stuck to give good examples, as I'm also new to the compiler
>> and built all my experience on 2.8, so I really don't know if I'd be
>> pointing you to something that's totally different under 2.7
>
> Maybe I'll try 2.8 then, as I'm quite stuck with trying to invoke the typer
> in the right context, so that it's able to resolve method parameters.
>
> --
> Adam
>
Well, 2.8 is going to be released any day now, honest!
Then take a look at how namers and typers are handling the
BeanProperty annotation, that's definitely the easiest example of
synthesised methods that i found in the current codebase.
Thu, 2009-10-15, 19:37
#8
Re: Writing a compiler plugin which adds code
> Well, 2.8 is going to be released any day now, honest!
>
> Then take a look at how namers and typers are handling the
> BeanProperty annotation, that's definitely the easiest example of
> synthesised methods that i found in the current codebase.
I still can't get it to work ;). I'm quite sure that it's some stupid
mistake I make, as very often in such cases.
Anyway, for now I made a work-around of running the plugin after the
"parser" phase, which makes it essentially the first plugin executed,
before namer and typer. Then, all of the code I generate is handled by
them normally.
If you're interested, you can see the results here: http://www.warski.org/blog/?p=124
:)
Adam
On Sat, Oct 10, 2009 at 07:29:30PM +0200, Adam Warski wrote:
> As I'm generating code, I wanted my plugin to be executed as early as
> possible, so I specified "runsAfter" as "namer" (I want my generated
> code to be typed etc, so I guess the phase must be before "typer").
No... you don't think the 15 phases which come after typer abandon
typing, do you? I wouldn't try to run before typer. You call methods in
a typer to get typed trees - see the "typer typed " throughout the
compiler.