merging scala-virtualized into trunk, hidden behind -Yvirtualize

Hi,

I'd like to give everyone a chance to review this patch: https://github.com/adriaanm/scala/compare/master...topic%2Fvirt before I submit a pull request to master (tonight).
It brings virtualization to master, but scalac's behaviour should be unmodified unless -Yvirtualize is specified (except for some additions to manifests that are always on: SourceLocation/SourceContext, RefinedManifest)
Please ignore the build.xml, src/build/pack.xml changes; everything else is good to go:  - I've tested that lockers built from a) master and b) topic/virt produce identical bytecode for quick.bin   - the test suite passes as well of course, and it builds a dist

the functionality in this commit that's unleashed by -Xexperimental:  - more refined applyDynamic/selectDynamic/updateDynamic
the following functionality is behind -Yvirtualize:  - virtualization of if/then/else, while, return, mutable variables, new Row{...} (to be renamed to Struct)  - external methods (bar.foo(...) --> infix_foo(bar, ...))   - statically typed applyDynamic/... for rows


cheersadriaan

On Wed, Jan 18, 2012 at 06:15, Adriaan Moors wrote:
> Hi,
>
>   - external methods (bar.foo(...) --> infix_foo(bar, ...))

I couldn't find (or identify) any test case for the infix stuff. Is it
missing or did *I* miss it?

I looked over the patch and it seems good to me. Thanks for putting this together!
My main worry at this point is that we will have a lot more external DSL users very soon and we need to provide them with a reasonably stable environment. Following trunk may be too much of a moving target for them.
But since we have a patch now, it should be easy to apply it on top of the 2.9.x branch, too, right? Anybody who uses 2.9.2-SNAPSHOT would get 2.9.1 + epsilon + virtualization. That sounds highly desirable to me.
Is it possible to use Eclipse with 2.9.2-SNAPSHOT, too?

For trunk, I think it will be important to have Jenkins do additional nightly build + test cycles with -Yvirtualize enabled everywhere to make sure we catch any strange interactions with new commits.
Cheers,- Tiark

On Jan 18, 2012, at 9:15 AM, Adriaan Moors wrote:

Hi,

I'd like to give everyone a chance to review this patch: https://github.com/adriaanm/scala/compare/master...topic%2Fvirt before I submit a pull request to master (tonight).
It brings virtualization to master, but scalac's behaviour should be unmodified unless -Yvirtualize is specified (except for some additions to manifests that are always on: SourceLocation/SourceContext, RefinedManifest)
Please ignore the build.xml, src/build/pack.xml changes; everything else is good to go:  - I've tested that lockers built from a) master and b) topic/virt produce identical bytecode for quick.bin   - the test suite passes as well of course, and it builds a dist

the functionality in this commit that's unleashed by -Xexperimental:  - more refined applyDynamic/selectDynamic/updateDynamic
the following functionality is behind -Yvirtualize:  - virtualization of if/then/else, while, return, mutable variables, new Row{...} (to be renamed to Struct)  - external methods (bar.foo(...) --> infix_foo(bar, ...))   - statically typed applyDynamic/... for rows


cheersadriaan

You didn't miss anything. Infix methods are unlikely to become an official feature; they should be superseded by inlined implicit classes.

Similarly, dsl scope methods and __new (structs) are likely to be replaced by macros and some more generic compiler support, as soon as those are in place.

The uncontroversial bits of this patch will land in trunk, but the rest will mature in topic/virt for a while longer.

--adriaan

On 18 Jan 2012, at 15:27, Daniel Sobral wrote:

> On Wed, Jan 18, 2012 at 06:15, Adriaan Moors wrote:
>> Hi,
>>
>> - external methods (bar.foo(...) --> infix_foo(bar, ...))
>
> I couldn't find (or identify) any test case for the infix stuff. Is it
> missing or did *I* miss it?
>

On Jan 18, 2012, at 7:04 PM, Adriaan Moors wrote:

> You didn't miss anything. Infix methods are unlikely to become an official feature; they should be superseded by inlined implicit classes.

implicit conversions (and by extension inline implicit classes) can not replace infix_ methods:

1) they don't allow you to override-pimp methods (like clone or toString on Rep[T])
2) they don't expose methods as top-level traits members, so cake-style composition requires an indirection
3) they have different overloading semantics: infix_+(a,b) behaves like plus(a,b), not like plus(a)(b)

> Similarly, dsl scope methods and __new (structs) are likely to be replaced by macros and some more generic compiler support, as soon as those are in place.
>
> The uncontroversial bits of this patch will land in trunk, but the rest will mature in topic/virt for a while longer.

I'm mystified. Are we going to have a -Yvirtualize flag or not? Keeping a separate scala-virtualized repository as before is totally fine with me but we should avoid fragmentation. So I think it should be either or.

- Tiark

>
> --adriaan
>
> On 18 Jan 2012, at 15:27, Daniel Sobral wrote:
>
>> On Wed, Jan 18, 2012 at 06:15, Adriaan Moors wrote:
>>> Hi,
>>>
>>> - external methods (bar.foo(...) --> infix_foo(bar, ...))
>>
>> I couldn't find (or identify) any test case for the infix stuff. Is it
>> missing or did *I* miss it?
>>

Just wanted to add that infix_ methods will be immensely powerful as macros because they can retroactively macro-fy methods of existing classes (in the current scope):

object DownWithForeach {
macro def infix_foreach[T](xs: Iterable[T], f: T => Unit) = // expand as while loop + iterator
}

import DownWithForeach._

HashMap(....) foreach { x => ... } // will compile as while loop + iterator
List(...) foreach { x => ... } // same here

cheers,
- Tiark

On Jan 18, 2012, at 7:58 PM, Tiark Rompf wrote:

> On Jan 18, 2012, at 7:04 PM, Adriaan Moors wrote:
>
>> You didn't miss anything. Infix methods are unlikely to become an official feature; they should be superseded by inlined implicit classes.
>
> implicit conversions (and by extension inline implicit classes) can not replace infix_ methods:
>
> 1) they don't allow you to override-pimp methods (like clone or toString on Rep[T])
> 2) they don't expose methods as top-level traits members, so cake-style composition requires an indirection
> 3) they have different overloading semantics: infix_+(a,b) behaves like plus(a,b), not like plus(a)(b)
>
>> Similarly, dsl scope methods and __new (structs) are likely to be replaced by macros and some more generic compiler support, as soon as those are in place.
>>
>> The uncontroversial bits of this patch will land in trunk, but the rest will mature in topic/virt for a while longer.
>
> I'm mystified. Are we going to have a -Yvirtualize flag or not? Keeping a separate scala-virtualized repository as before is totally fine with me but we should avoid fragmentation. So I think it should be either or.
>
> - Tiark
>
>
>>
>> --adriaan
>>
>> On 18 Jan 2012, at 15:27, Daniel Sobral wrote:
>>
>>> On Wed, Jan 18, 2012 at 06:15, Adriaan Moors wrote:
>>>> Hi,
>>>>
>>>> - external methods (bar.foo(...) --> infix_foo(bar, ...))
>>>
>>> I couldn't find (or identify) any test case for the infix stuff. Is it
>>> missing or did *I* miss it?
>>>
>>> --
>>> Daniel C. Sobral
>>>
>>> I travel to the future all the time.
>

Tiark,

I think the current policy should be that trunk will only accept
contributions that make sense in trunk or that are very easy to
support. We want to avoid large swathes of code that make only sense
for virtualized, because they impose a tax on everyone maintaining
trunk. And, of course, we should try to avoid forking the language, so
if it is at all possible that we get the necessary behavior of
virtualized without infix methods we should try that.

Cheers

On Wed, Jan 18, 2012 at 07:04:33PM +0100, Adriaan Moors wrote:
> You didn't miss anything. Infix methods are unlikely to become an
> official feature; they should be superseded by inlined implicit
> classes.

Is there documentation about how inlined implicit classes will work?
I'm hoping it means something like:

a + b --> new GenericOps(a).+(b) --> GenericOps.+(a, b)

Would inlined implicit classes avoid creating the anonymous GenericOps
object (assuming it didn't do any work in its constructor)?

Yes, I think we all agree on the policy. The merge proposal caught me by surprise and I assumed you had signed off on it.
- Tiark

On Jan 18, 2012, at 9:26 PM, martin odersky wrote:

> Tiark,
>
> I think the current policy should be that trunk will only accept
> contributions that make sense in trunk or that are very easy to
> support. We want to avoid large swathes of code that make only sense
> for virtualized, because they impose a tax on everyone maintaining
> trunk. And, of course, we should try to avoid forking the language, so
> if it is at all possible that we get the necessary behavior of
> virtualized without infix methods we should try that.
>
> Cheers
>

On Wed, Jan 18, 2012 at 10:24 PM, Tiark Rompf wrote:
> Yes, I think we all agree on the policy. The merge proposal caught me by surprise and I assumed you had signed off on it.

No, it caught me by surprise as well. But it triggered a very
productive afternoon with Adriaan where we went over all the diffs
between trunk and virtualized.

Cheers

I apologise for my unorthodox ways, but -- as Martin said -- I think we made progress. --a

On 18 Jan 2012, at 22:48, martin odersky wrote:

> On Wed, Jan 18, 2012 at 10:24 PM, Tiark Rompf wrote:
>> Yes, I think we all agree on the policy. The merge proposal caught me by surprise and I assumed you had signed off on it.
>
> No, it caught me by surprise as well. But it triggered a very
> productive afternoon with Adriaan where we went over all the diffs
> between trunk and virtualized.
>
> Cheers
>

I apologise for my unorthodox ways, but -- as Martin said -- I think we made progress. --a

I applaud the fact that this discussion happened in open. --g