name->typer, runsAfter vs runsRightAfter

Kevin Wright wrote:
> There are a couple of cases where the namers phase is used to generate
> methods on an existing class/trait, so that they will be available early for
> the typer to do its thing.
> Mixin methods are one such case, as are getXXX/setXXX methods generated by
> the @BeanProperty annotation. Ideally, a compiler plugin would be able to
> generate methods in a similar fashion, and have them appear in the Scala
> signature so that they can actually be used, and can be
> browsed/auto-completed in the eclipse IDE plugin.
>
>
> The problem seems to be that method generation is best performed before the
> typer phase. This can't happen as typer is set to runRightAfter namer
> instead of runAfter namer. Is there any reason why this has to be the case,
> and why I can't create an intermediate phase with my plugin?

The basic reason is that the tree that the namer generates is not
complete, only the top most names have been resolved/traversed this is
then given to the typer and whenever the typer traverses into the
structure of the tree the namer is called on that particular subtree to
name another level. But these calls of the namer are explicit and would
in a plugin case not call the plugin!
So with the current design a plugin that is installed between namer and
typer will only be called once! And the tree that is has to work on is
not complete/valid.
If a plugin should be allowed between namer and typer the explicit calls
in typer should be replaced with a wrapper that would not only call the
namer but also all plugins that are registered between namer and typer!
This was the reason for putting a runsRightAfter between namer and
typer.

/Anders

Ack! not so useful :(
I feel that I'm frustratingly close to this one, as my generated methods are in the class file and visible and callable (and work correctly!) from Java.
The only other examples I know of code being generated this way are mixins and @BeanProperty, both of which use namer to do so - which made it look like the obvious culprit
Back to the drawing board then, I'm sure it's something to do with the signature...

On Fri, Sep 25, 2009 at 11:30 AM, Anders Bach Nielsen <andersbach.nielsen@epfl.ch> wrote:

Kevin Wright wrote:
> There are a couple of cases where the namers phase is used to generate
> methods on an existing class/trait, so that they will be available early for
> the typer to do its thing.
> Mixin methods are one such case, as are getXXX/setXXX methods generated by
> the @BeanProperty annotation.  Ideally, a compiler plugin would be able to
> generate methods in a similar fashion, and have them appear in the Scala
> signature so that they can actually be used, and can be
> browsed/auto-completed in the eclipse IDE plugin.
>
>
> The problem seems to be that method generation is best performed before the
> typer phase.  This can't happen as typer is set to runRightAfter namer
> instead of runAfter namer.  Is there any reason why this has to be the case,
> and why I can't create an intermediate phase with my plugin?

The basic reason is that the tree that the namer generates is not
complete, only the top most names have been resolved/traversed this is
then given to the typer and whenever the typer traverses into the
structure of the tree the namer is called on that particular subtree to
name another level. But these calls of the namer are explicit and would
in a plugin case not call the plugin!
So with the current design a plugin that is installed between namer and
typer will only be called once! And the tree that is has to work on is
not complete/valid.
If a plugin should be allowed between namer and typer the explicit calls
in typer should be replaced with a wrapper that would not only call the
namer but also all plugins that are registered between namer and typer!
This was the reason for putting a runsRightAfter between namer and
typer.

/Anders

--
Anders Bach Nielsen            |   http://www.cs.au.dk/~abachn/
University of Aarhus           |   abachn@cs.au.dk
-
 BOFH Excuse #717171: Too many interrupts

On Fri, Sep 25, 2009 at 1:12 PM, Kevin Wright
wrote:
> Ack! not so useful :(
> I feel that I'm frustratingly close to this one, as my generated methods are
> in the class file and visible and callable (and work correctly!) from Java.
> The only other examples I know of code being generated this way are mixins
> and @BeanProperty, both of which use namer to do so - which made it look
> like the obvious culprit
> Back to the drawing board then, I'm sure it's something to do with the
> signature...

I know this may be a silly question but are you calling from scala
code within the same compilation run? If so you might want to try a
separate run. These are all problems I had and previously raised on
the relevant groups, it seems that only a handful of people outside of
epfl want to extend actual compilation this way.

I tried in a separate compilation run; so the generated code was in a library, which I then tried to use from a second project, still no luck :(
At the moment I'm using a TypeTransformer and matching on Template in the AST, and think I've got all the bases covered:
- Using TreeDSL to insert the new method into the AST- Ensuring symbol info is correctly set for every element in my generated tree fragment- wrapping the whole definition in a typed{ } block
I'm still investigating:
- Do I need to set any additional flags on my phase- Should I be using an InfoTransformer instead of a TypeTransformer- Do I need to transform at the ClassDef or CompilationUnit level instead of the Template



On Fri, Sep 25, 2009 at 2:38 PM, Chris Twiner <chris.twiner@gmail.com> wrote:

On Fri, Sep 25, 2009 at 1:12 PM, Kevin Wright
<kev.lee.wright@googlemail.com> wrote:
> Ack! not so useful :(
> I feel that I'm frustratingly close to this one, as my generated methods are
> in the class file and visible and callable (and work correctly!) from Java.
> The only other examples I know of code being generated this way are mixins
> and @BeanProperty, both of which use namer to do so - which made it look
> like the obvious culprit
> Back to the drawing board then, I'm sure it's something to do with the
> signature...

I know this may be a silly question but are you calling from scala
code within the same compilation run?  If so you might want to try a
separate run.  These are all problems I had and previously raised on
the relevant groups, it seems that only a handful of people outside of
epfl want to extend actual compilation this way.

On Fri, Sep 25, 2009 at 03:06:41PM +0100, Kevin Wright wrote:
> - Using TreeDSL to insert the new method into the AST

Neat, I'm glad that is being used. However, be aware that the entirety
of TreeDSL was written by paulp and its mechanisms are only as correct
as I was able to infer from existing code. It must be somewhat correct
since I rewrote a couple whole phases using it, but don't think it's
some kind of reliable fully vetted black box (not that there is ANY code
in the compiler which fits that description...)

I was using Mixin as a reference here, so that's obviously one of your re-written phases, and mixin does seem to work :)
I was also able to check that TreeDSL was doing the same thing that I did by hand before discovering it... This has definitely been a journey in producing less and less code to achieve the same results; in the first cut I was manually typing everything, but that's before I discovered typed{ }, it got a bit "interesting"!
I'm still looking into how the signatures are generated and then backtracking that to see what I need to push into trees/symbols/types to get the necessary.
I also need to: - figure out why some phases use a local typer instead of the instance in global (which is what I'm currently doing)- work my way through the specialized phase, to see if that gives me a deeper understanding of what I'm missing




On Fri, Sep 25, 2009 at 3:38 PM, Paul Phillips <paulp@improving.org> wrote:

On Fri, Sep 25, 2009 at 03:06:41PM +0100, Kevin Wright wrote:
> - Using TreeDSL to insert the new method into the AST

Neat, I'm glad that is being used.  However, be aware that the entirety
of TreeDSL was written by paulp and its mechanisms are only as correct
as I was able to infer from existing code.  It must be somewhat correct
since I rewrote a couple whole phases using it, but don't think it's
some kind of reliable fully vetted black box (not that there is ANY code
in the compiler which fits that description...)

--
Paul Phillips      | The most dangerous man to any government is the man who
Stickler           | is able to think things out [...] Almost inevitably he
Empiricist         | comes to the conclusion that the government he lives under
pull his pi pal!   | is dishonest, insane, intolerable.   -- H. L. Mencken