Desirable behaviour for scala.Dynamic

Hi Jordi,

Dynamic is not yet cast in stone. We are experimenting with variations
in some real use cases, and encourage others to do the same.
Personally, I don't think that having only one applyDynamic method is
a burdensome restriction, however.

Cheers

On Sat, Oct 1, 2011 at 6:22 AM, martin odersky wrote:
> Personally, I don't think that having only one applyDynamic method is
> a burdensome restriction, however.

One could think of it as encouragement to write us a nice reusable
multimethods implementation.

On Sat, Oct 1, 2011 at 6:04 AM, Jordi Salvat i Alabart
> Otherwise Dynamic classes will be limited to one single applyDynamic method

Hi.

Paul set me up for a difficult one...

2011/10/1 Paul Phillips <paulp@improving.org>

[...]
One could think of it as encouragement to write us a nice reusable
multimethods implementation.

After sweating profusely and bumping my head against a wall several times trying to accomplish a non-local rewrite of the tree inside the Typer, inspiration finally arrived. I concluded I was trying to solve the wrong problem: I believe the solution I've come up with works more nicely together with Scala's features.

Instead of rewriting x.meth(args) to x.applyDynamic(meth, args) or x.applyDynamic(meth)(args), I am just rewriting it to x.applyDynamic(args).

Then it's up to the author of the applyDynamic method to have it take an implicit method of type scala.Dynamic.MethodName which, if left to resolve to the provided method scala.Dynamic.callingName, will take the value "meth".

This is the basic test I've been using during development:

object Test extends App with Dynamic {
  def applyDynamic(b: Boolean)(implicit m: Dynamic.MethodName) = "Boolean: "+m+"("+b+")"
  def applyDynamic(i: Int)(implicit m: Dynamic.MethodName) = "Int: "+m+"("+i+")"

  println(this.anInt(1))
  println(this.aBoolean(false))
}

which should of course print out:

Int: anInt(1)
Boolean: aBoolean(false)

The definition of trait Dynamic doesn't change (still empty), but it now has a companion Module:

object Dynamic {
  class MethodName(private val name: String) {
    override def toString = name
  }

  /**This method magically returns the calling name used to invoke an applyDynamic method via the
   * scala.Dynamic mechanism.
   *
   * If invoked directly (without magic), it just throws an AssertionError.
   */
  implicit def callingName: MethodName = {
    throw new AssertionError("The scala.Dynamic mechanism failed to provide the calling method's name.")
  }
}

Makes sense? Thoughts?

I realize that I am no longer bugfixing, but actually evolving a language feature (even if experimental). What's the process to get this accomplished (assuming you like this solution)? Should I prepare an SID, or is it enough to propose the change via github?

--
Thanks,

Jordi.

On 2 October 2011 19:50, Jordi Salvat i Alabart <jordi.salvat.i.alabart@gmail.com> wrote:

After sweating profusely and bumping my head against a wall several times trying to accomplish a non-local rewrite of the tree inside the Typer, inspiration finally arrived. I concluded I was trying to solve the wrong problem: I believe the solution I've come up with works more nicely together with Scala's features.

Instead of rewriting x.meth(args) to x.applyDynamic(meth, args) or x.applyDynamic(meth)(args), I am just rewriting it to x.applyDynamic(args).

Then it's up to the author of the applyDynamic method to have it take an implicit method of type scala.Dynamic.MethodName which, if left to resolve to the provided method scala.Dynamic.callingName, will take the value "meth".

Jordi,
What I'm missing from the discussion is justification for being able to define overloaded applyDynamic implementations. Can you give compelling example when it's needed?
Also note that applyDynamic is already fairly tricky concept/hack (there's no signature defined for it) and adding even more complexity to it (through method overloading) doesn't seem like a good idea.

--
Grzegorz Kossakowski

These are my two real reasons for working on this solution:

1/ Not having overloading looks like an irregularity that needs documenting -- in fact, I initially believed it to be a bug with the implementation of the feature. Seen this way, adding multimethods simplifies the language (by making it more regular) rather than complicating it.

2/ Paul challenged me :-)

Not quite strong as a justification... So let me try to work out one on the spot.

Drawing from the uses in http://squirrelsewer.blogspot.com/2011/02/scalas-upcoming-dynamic-capabilities.html :

1. Integration with dynamic languages on the JVM.

This probably only needs applyDynamic(Any*)

2. DSLs for accessing data where there's already no type safety, ie JSON and XML (written by @jorgeortiz85, I think)

A DSL to create JSON objects (programmatically, NOT through serialization of scala/java objects) could define five six-parameter methods, one for each data type in JSON: http://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example

I've twitted Jorge (hola, tocayo!) in case he wants to chime in.

3. DSL for reflection. Reflection libraries are ridiculously verbose, and like JSON/XML/etc libraries, offer little static typing benefits anyway.

Not sure about this one. Anyone?

So 1 out of 3.

--
Salut,

Jordi.


2011/10/2 Grzegorz Kossakowski <grzegorz.kossakowski@gmail.com>

On 2 October 2011 19:50, Jordi Salvat i Alabart <jordi.salvat.i.alabart@gmail.com> wrote:

After sweating profusely and bumping my head against a wall several times trying to accomplish a non-local rewrite of the tree inside the Typer, inspiration finally arrived. I concluded I was trying to solve the wrong problem: I believe the solution I've come up with works more nicely together with Scala's features.

Instead of rewriting x.meth(args) to x.applyDynamic(meth, args) or x.applyDynamic(meth)(args), I am just rewriting it to x.applyDynamic(args).

Then it's up to the author of the applyDynamic method to have it take an implicit method of type scala.Dynamic.MethodName which, if left to resolve to the provided method scala.Dynamic.callingName, will take the value "meth".

Jordi,
What I'm missing from the discussion is justification for being able to define overloaded applyDynamic implementations. Can you give compelling example when it's needed?
Also note that applyDynamic is already fairly tricky concept/hack (there's no signature defined for it) and adding even more complexity to it (through method overloading) doesn't seem like a good idea.

--
Grzegorz Kossakowski

On Sun, Oct 2, 2011 at 10:50 AM, Jordi Salvat i Alabart
wrote:
> I realize that I am no longer bugfixing, but actually evolving a language
> feature (even if experimental). What's the process to get this accomplished
> (assuming you like this solution)? Should I prepare an SID, or is it enough
> to propose the change via github?

We're well past casual githubbery now.

There was talk that method manifests were being worked upon and you
are starting to intersect with that (or at least so I imagine, never
having seen the method manifests.) I also have this vague feeling that
they were dropped. I wish I knew for real, because I was kind of
counting on the features I imagined they were bringing.

<I'm first going on a little tangent before talking about overloading applyDynamic>
method manifests have landed in virtualized scalac, where we're still githubbing casually [*], since it's a fork of the language (it will eventually be integrated of course)
They're called SourceInfo (since 15a4806c).
One of the fixes you propose was already in virtualized scalac, by the way (not resolving a missing applyDynamic method using applyDynamic).
We've also had to differentiate selectDynamic and applyDynamic (because in one of the DSLs we're working on, foo() and foo must be distinguishable).
</tangent>

Not having overloading looks like an irregularity that needs documenting

True, that's why there's that first argument list -- problem solved (as you know, only the first argument list is taken into account when doing overload resolution).

Java-style overloading is a bad idea in all but trivial cases, if you ask me, and I don't think it deals well with your last man that was still standing out of those 3 (nonetheless interesting individuals).
The main problem is that you often don't know statically what kind of data you're dealing with, so you have to do run-time type tests anyway.
When you *do* know statically what kind of data you're dealing with (in Jorge's XML example, or when embedding a sql-like DSL for known DB schema's),  Java-style static overloading is too rigid, as you really need type-level relations (such as CanBuildFrom), or dedicated compiler support (which we're working on for row-like datatypes, which come in handy in the examples mentioned above, as well as in our embedded JS DSL).
I'm writing up documentation about all this this week -- I'll follow up with a link as soon as first drafts are available.
One more thought: how about just having selectDynamic that takes a name and an expected type (expressed as an HList type argument that specifies argument types + a result type), and some implicit magic to do the resolution based on that? 
def selectDynamic[Expected](name: String)(implicit resolution: CanSelectFrom[name.type, Expected]): resolution.Result
We're currently doing something like this in the compiler, but specialised for row-types (see topic/reify_new in my github repo -- it's also merged into virtualized-master)
cheersadriaan
[*] we do have continuous integration running, and nightly builds are being pushed to scala-tools [**] [**] well, we're still refining the process -- we just setup jenkins last Friday

On Mon, Oct 3, 2011 at 3:17 AM, Paul Phillips <paulp@improving.org> wrote:

On Sun, Oct 2, 2011 at 10:50 AM, Jordi Salvat i Alabart
<jordi.salvat.i.alabart@gmail.com> wrote:
> I realize that I am no longer bugfixing, but actually evolving a language
> feature (even if experimental). What's the process to get this accomplished
> (assuming you like this solution)? Should I prepare an SID, or is it enough
> to propose the change via github?

We're well past casual githubbery now.

There was talk that method manifests were being worked upon and you
are starting to intersect with that (or at least so I imagine, never
having seen the method manifests.) I also have this vague feeling that
they were dropped.  I wish I knew for real, because I was kind of
counting on the features I imagined they were bringing.

2011/10/3 Adriaan Moors <adriaan.moors@epfl.ch>

method manifests have landed in virtualized scalac, where we're still githubbing casually [*], since it's a fork of the language (it will eventually be integrated of course)

 I wasn't even aware of the virtualized scalac effort. Sorry for being new. :-)

 

Java-style overloading is a bad idea in all but trivial cases, ...

You may be right, but it is there -- and the current implementation of Dynamic hides that 1st parameter list, so it generates an irregularity, which is undesirable even if not a big problem.
 

I'm writing up documentation about all this this week -- I'll follow up with a link as soon as first drafts are available.

Thanks. That will be very interesting reading.

In the meantime, I'll just move on with my original project (which was about the support for XML literals) -- I just got side-tracked into scala.Dynamic because I need it and it was so problem-prone.

--
Salut,

Jordi.