How to simplify passing macro context around?

On Sun, Feb 19, 2012 at 10:33 AM, Eugene Burmako <eugene.burmako@epfl.ch> wrote:

You might wish to pass _context implicitly, but this hits a wall. On
the one hand, implicit parameters must come after explicit parameters.
But, on the other hand, you need that wannabe-implicit context to
define path-dependent types of your explicit parameters, so it has to
come first in the parameter list.

I'd also like to lift this restriction. (https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/typechecker/Namers.scala#L1454)
In principle, I think it would suffice to enforce the dependence graph for arguments is cycle-free.This would mean you could write:
def foo(tree: context.Tree)(implicit context: Context): context.Tree
I tend to think of implicit arguments as the value-level counterpart of type parameters,so you could think of the above as:
def foo[val context: Context](tree: context.Tree): context.Tree
cheersadriaan

On Sun, Feb 19, 2012 at 10:20 AM, Adriaan Moors wrote:
> On Sun, Feb 19, 2012 at 10:33 AM, Eugene Burmako
> wrote:
>>
>> You might wish to pass _context implicitly, but this hits a wall. On
>> the one hand, implicit parameters must come after explicit parameters.
>> But, on the other hand, you need that wannabe-implicit context to
>> define path-dependent types of your explicit parameters, so it has to
>> come first in the parameter list.
>
> I'd also like to lift this restriction.
> (https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/...)
>
> In principle, I think it would suffice to enforce the dependence graph for
> arguments is cycle-free.
> This would mean you could write:
>
> def foo(tree: context.Tree)(implicit context: Context): context.Tree

Yes! Please! Want! Now!

Cheers,

Miles

Frankly, I am not super concerned about macros being hard to write. I
am more concerned that they are too easy to write and people will go
overboard writing crazy stuff with them. So, maybe a high bar for
writing macros is even desirable. I certainly would not want to bend
over backwards to give more convenience for macro writers.

On Sun, Feb 19, 2012 at 11:24 AM, martin odersky wrote:
> Frankly, I am not super concerned about macros being hard to write. I
> am more concerned that they are too easy to write and people will go
> overboard writing crazy stuff with them. So, maybe a high bar for
> writing macros is even desirable. I certainly would not want to bend
> over backwards to give more convenience for macro writers.
>
>  -- Martin

The difficulty has nothing to do with macros per se -- it's to do with
the cake pattern. The same problems will affect people using the
reflection API, or those who, perhaps after reading PiS, have adopted
this pattern in their own codebase. Surely we want to make it easier
for all of these people to keep their code well factored.

Dependent method types go a long way, but the overly-conservative
restriction on the direction of the dependencies still stands in the
way.

-jason

On Sun, Feb 19, 2012 at 10:33 AM, Eugene Burmako wrote:
> Retronym's recent experiments with macros nicely illustrate the
> problem:
> https://github.com/retronym/macrocosm/blob/95a0505126b4035cf58a379e0dc43....

> This begs for a question. How do we fight bakery of doom here?

I've just checked in a slightly different approach:

https://github.com/retronym/macrocosm/commit/f0197ab2

The important bits:

implicit def Util(context: Context) = new Util[context.type](context)

class Util[C <: Context with Singleton](val context: C) {
import context._

def id(a: Tree): Tree = a
// more method here
}

-jason

On Sun, Feb 19, 2012 at 11:50 AM, Jason Zaugg wrote:
> On Sun, Feb 19, 2012 at 10:33 AM, Eugene Burmako wrote:
>> Retronym's recent experiments with macros nicely illustrate the
>> problem:
>> https://github.com/retronym/macrocosm/blob/95a0505126b4035cf58a379e0dc43....
>
>> This begs for a question. How do we fight bakery of doom here?
>
> I've just checked in a slightly different approach:
>
> https://github.com/retronym/macrocosm/commit/f0197ab2
>
> The important bits:
>
>  implicit def Util(context: Context) = new Util[context.type](context)
>
>  class Util[C <: Context with Singleton](val context: C) {
>    import context._
>
>    def id(a: Tree): Tree = a
>    // more method here
>  }
>
> -jason

Yes, that looks more like it. We can always refactor by creating
intermediate objects.

Cheers

On Sun, Feb 19, 2012 at 11:40 AM, Jason Zaugg wrote:
> On Sun, Feb 19, 2012 at 11:24 AM, martin odersky wrote:
>> Frankly, I am not super concerned about macros being hard to write. I
>> am more concerned that they are too easy to write and people will go
>> overboard writing crazy stuff with them. So, maybe a high bar for
>> writing macros is even desirable. I certainly would not want to bend
>> over backwards to give more convenience for macro writers.
>>
>>  -- Martin
>
> The difficulty has nothing to do with macros per se -- it's to do with
> the cake pattern. The same problems will affect people using the
> reflection API, or those who, perhaps after reading PiS, have adopted
> this pattern in their own codebase. Surely we want to make it easier
> for all of these people to keep their code well factored.
>
> Dependent method types go a long way, but the overly-conservative
> restriction on the direction of the dependencies still stands in the
> way.
>
What else can we do? (without throwing more syntax at the problem).

On Sun, Feb 19, 2012 at 11:51 AM, martin odersky wrote:
>> The difficulty has nothing to do with macros per se -- it's to do with
>> the cake pattern. The same problems will affect people using the
>> reflection API, or those who, perhaps after reading PiS, have adopted
>> this pattern in their own codebase. Surely we want to make it easier
>> for all of these people to keep their code well factored.
>>
>> Dependent method types go a long way, but the overly-conservative
>> restriction on the direction of the dependencies still stands in the
>> way.
>>
> What else can we do? (without throwing more syntax at the problem).

Just as Adriaan suggested: loosen the check in Namers [1] to allow for:

def foo(a: b.x)(implicit b: B)

-jason

[1] https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/...)

On Sun, Feb 19, 2012 at 11:56 AM, Jason Zaugg wrote:
> On Sun, Feb 19, 2012 at 11:51 AM, martin odersky wrote:
>>> The difficulty has nothing to do with macros per se -- it's to do with
>>> the cake pattern. The same problems will affect people using the
>>> reflection API, or those who, perhaps after reading PiS, have adopted
>>> this pattern in their own codebase. Surely we want to make it easier
>>> for all of these people to keep their code well factored.
>>>
>>> Dependent method types go a long way, but the overly-conservative
>>> restriction on the direction of the dependencies still stands in the
>>> way.
>>>
>> What else can we do? (without throwing more syntax at the problem).
>
> Just as Adriaan suggested: loosen the check in Namers [1] to allow for:
>
>  def foo(a: b.x)(implicit b: B)
>
And what would the resulting types look like?

MethodType(a: b.x, MethodType(b: B, ...)

I think we'd introduce some serious scoping problems here!

On Sun, Feb 19, 2012 at 12:10 PM, martin odersky <martin.odersky@epfl.ch> wrote:

On Sun, Feb 19, 2012 at 11:56 AM, Jason Zaugg <jzaugg@gmail.com> wrote:
> On Sun, Feb 19, 2012 at 11:51 AM, martin odersky <martin.odersky@epfl.ch> wrote:
>>> The difficulty has nothing to do with macros per se -- it's to do with
>>> the cake pattern. The same problems will affect people using the
>>> reflection API, or those who, perhaps after reading PiS, have adopted
>>> this pattern in their own codebase. Surely we want to make it easier
>>> for all of these people to keep their code well factored.
>>>
>>> Dependent method types go a long way, but the overly-conservative
>>> restriction on the direction of the dependencies still stands in the
>>> way.
>>>
>> What else can we do? (without throwing more syntax at the problem).
>
> Just as Adriaan suggested: loosen the check in Namers [1] to allow for:
>
>  def foo(a: b.x)(implicit b: B)
>
And what would the resulting types look like?

MethodType(a: b.x, MethodType(b: B, ...)

I think we'd introduce some serious scoping problems here!

yes, the forward references do complicate matters
I think one solution could be to introduce the value-level equivalent of context.undetparams (when you encounter a dependency on an undetermined argument while typing an application, you treat it like an undetermined type parameter and replace it by the type selection on the actual argument, once the latter has been supplied -- eta expansion of partial applications would be tricky, though...)
I have a similar approach in mind for uniformising constraint propagation in implicit coercions and implicit arg application [https://issues.scala-lang.org/browse/SI-4699]

adriaan

ps: for the record, I do not propose we add this to 2.10 -- just speculating on how we could implement it

Submitted an improvement to the issue tracker: https://issues.scala-lang.org/browse/SI-5502.

On Feb 19, 12:33 pm, Adriaan Moors wrote:
> On Sun, Feb 19, 2012 at 12:10 PM, martin odersky wrote:
>
>
>
>
>
> > On Sun, Feb 19, 2012 at 11:56 AM, Jason Zaugg wrote:
> > > On Sun, Feb 19, 2012 at 11:51 AM, martin odersky
> > wrote:
> > >>> The difficulty has nothing to do with macros per se -- it's to do with
> > >>> the cake pattern. The same problems will affect people using the
> > >>> reflection API, or those who, perhaps after reading PiS, have adopted
> > >>> this pattern in their own codebase. Surely we want to make it easier
> > >>> for all of these people to keep their code well factored.
>
> > >>> Dependent method types go a long way, but the overly-conservative
> > >>> restriction on the direction of the dependencies still stands in the
> > >>> way.
>
> > >> What else can we do? (without throwing more syntax at the problem).
>
> > > Just as Adriaan suggested: loosen the check in Namers [1] to allow for:
>
> > >  def foo(a: b.x)(implicit b: B)
>
> > And what would the resulting types look like?
>
> > MethodType(a: b.x, MethodType(b: B, ...)
>
> > I think we'd introduce some serious scoping problems here!
>
> yes, the forward references do complicate matters
>
> I think one solution could be to introduce the value-level equivalent of
> context.undetparams (when you encounter a dependency on an undetermined
> argument while typing an application, you treat it like an undetermined
> type parameter and replace it by the type selection on the actual argument,
> once the latter has been supplied -- eta expansion of partial applications
> would be tricky, though...)
>
> I have a similar approach in mind for uniformising constraint propagation
> in implicit coercions and implicit arg application [https://issues.scala-lang.org/browse/SI-4699]
>
> adriaan
>
> ps: for the record, I do not propose we add this to 2.10 -- just
> speculating on how we could implement it

On Sun, Feb 19, 2012 at 2:50 AM, Jason Zaugg <jzaugg@gmail.com> wrote:

The important bits:

 implicit def Util(context: Context) = new Util[context.type](context)

 class Util[C <: Context with Singleton](val context: C) {
   import context._

   def id(a: Tree): Tree = a
   // more method here
 }

This looks like what I did here a couple months ago:
  https://github.com/scala/scala/commit/d05881e42e
As it says in the commit message, "Amazingly, it seems to work."The important bits:
  /** Latest attempt to work around the challenge of foo.global.Type    *  not being seen as the same type as bar.global.Type even though   *  the globals are the same.  Dependent method types to the rescue.   */  def mkManifestToType[T <: Global](global: T) = {     import global._    import definitions._        /** We can't use definitions.manifestToType directly because we're passing     *  it to map and the compiler refuses to perform eta expansion on a method      *  with a dependent return type.  (Can this be relaxed?) To get around this     *  I have this forwarder which widens the type and then cast the result back     *  to the dependent type.      */    def manifestToType(m: OptManifest[_]): Global#Type =      definitions.manifestToType(m)        class AppliedTypeFromManifests(sym: Symbol) {      def apply[M](implicit m1: Manifest[M]): Type =         appliedType(sym.typeConstructor, List(m1) map (x => manifestToType(x).asInstanceOf[Type]))
      def apply[M1, M2](implicit m1: Manifest[M1], m2: Manifest[M2]): Type =        appliedType(sym.typeConstructor, List(m1, m2) map (x => manifestToType(x).asInstanceOf[Type]))     }        (sym: Symbol) => new AppliedTypeFromManifests(sym)  }

Oh yeah, an example of what that's for:

scala> intp("scala.collection.Map")[Int, Int]res0: $r.global.Type = scala.collection.Map[Int,Int]
Drink that one in for a while.

You can also do this:

scala> intp.types[Map[_,_]].apply[Int, Int]res0: $r.global.Type = scala.collection.immutable.Map[Int,Int]
It'd be cool if you could do this, and you almost can because I fixed consecutive type application, but:
scala> intp.types[Map[_,_]][Int, Int]<console>:32: error: method types: (implicit evidence$3: ClassManifest[Map[_, _]])$r.intp.global.Symbol does not take type parameters.               intp.types[Map[_,_]][Int, Int]                                  ^
Those darn implicit parameters again.