sbt 0.11.3 eta or fix for sbt on master?

On 18 January 2012 09:51, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

Hi,
What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.
If an "official" fix isn't expected today/tomorrow, I'll work on a workaround today myself.

Shouldn't we just revert the breaking change from the Scala master?
I should also say that removing methods from places like definitions should not be taken lightly. For example, Scala+GWT another project that got broken in substantial way by this change. 
We should have a lightweight deprecation procedure that allow all projects working with Scala compiler internals to adapt to changes. It doesn't need to be formal (e.g. we'll remove deprecated methods after 2 releases) but giving people a month or two to catch up is a lot nicer than just breaking everybody around (especially because I haven't seen any discussion on that particular change before it arrived in master).
--
Grzegorz Kossakowski

I agree fully.
For the short term, I think I fixed it at https://github.com/adriaanm/xsbt/tree/0.11.3 -- pull from there, and run sbt build-all, then require version 0.11.3 in your own project. I'm still in the process of verifying on virtualization-lms-core, but I manually compiled the compiler/inferface with scalac from master.
cheersadriaan

On Wed, Jan 18, 2012 at 12:35 PM, Grzegorz Kossakowski <grzegorz.kossakowski@gmail.com> wrote:

On 18 January 2012 09:51, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

Hi,
What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.
If an "official" fix isn't expected today/tomorrow, I'll work on a workaround today myself.

Shouldn't we just revert the breaking change from the Scala master?
I should also say that removing methods from places like definitions should not be taken lightly. For example, Scala+GWT another project that got broken in substantial way by this change. 
We should have a lightweight deprecation procedure that allow all projects working with Scala compiler internals to adapt to changes. It doesn't need to be formal (e.g. we'll remove deprecated methods after 2 releases) but giving people a month or two to catch up is a lot nicer than just breaking everybody around (especially because I haven't seen any discussion on that particular change before it arrived in master).
--
Grzegorz Kossakowski

Awh, I'm really feeling the SBT love.
Sorry guys, I've been in native packaging hell recently.  GOOD NEWS is for the new Scala release our installers can be far more awesome.
BAD NEWS is I haven't been paying attention to what's happened.
I'm chatting with Mark about how soon we could cut a release.  I'm probably going to go into SBT and see if I can work out a setting so that you can just update the compiler/interface without upgrading SBT itself.   We'll see what happens.
- Josh

On Wed, Jan 18, 2012 at 6:46 AM, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

I agree fully.
For the short term, I think I fixed it at https://github.com/adriaanm/xsbt/tree/0.11.3 -- pull from there, and run sbt build-all, then require version 0.11.3 in your own project. I'm still in the process of verifying on virtualization-lms-core, but I manually compiled the compiler/inferface with scalac from master.
cheersadriaan

On Wed, Jan 18, 2012 at 12:35 PM, Grzegorz Kossakowski <grzegorz.kossakowski@gmail.com> wrote:

On 18 January 2012 09:51, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

Hi,
What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.
If an "official" fix isn't expected today/tomorrow, I'll work on a workaround today myself.

Shouldn't we just revert the breaking change from the Scala master?
I should also say that removing methods from places like definitions should not be taken lightly. For example, Scala+GWT another project that got broken in substantial way by this change. 
We should have a lightweight deprecation procedure that allow all projects working with Scala compiler internals to adapt to changes. It doesn't need to be formal (e.g. we'll remove deprecated methods after 2 releases) but giving people a month or two to catch up is a lot nicer than just breaking everybody around (especially because I haven't seen any discussion on that particular change before it arrived in master).
--
Grzegorz Kossakowski

On Wed, Jan 18, 2012 at 12:51 AM, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.

As an FYI only, it's a simple change on sbt's side (that's what I did, but I have to build sbt with some frequency anyway.) It isn't that easy to fix on scala's side, here's an excerpt from my explanation.  The reason these methods are under attack is there is an unbelievable amount of wasteful creating, converting (implicitly and explicitly), reconverting, assembling, disassembling, duplicating among strings, names, termnames, and typenames, and it'll never be fixed as long as there are implicits from everything to everything else.
This also illustrates a larger challenge with implicits and Predef.
---
Yeah, I can't fix it, it has to fix in sbt. (This is why all possible
compiler code should come out of sbt.) The problem is that names need
all the same kinds of manipulations that strings and sequences do, and
there is already an implicit conversion from strings to sequences in
Predef.  So to put efficient sequence methods on Name with expected
names (like say a "dropRight" which calls substring on the underlying
string, rather than going through conversions from Name to Seq to
Name) I have to make up new names for every method if there is a
String -> Name implicit in scope, because dropRight on a String could
apply to either.

I'm not going to make up new names for every method if only in the
interests of eventual correctness.  The problem can be seen in
practice via the 40 ambiguities which arise if one puts back the
absent implicit.

To futureproof itself, sbt should not utilize any implicit conversions
between names and strings.

On 18 January 2012 15:17, Paul Phillips <paulp@improving.org> wrote:

On Wed, Jan 18, 2012 at 12:51 AM, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.

As an FYI only, it's a simple change on sbt's side (that's what I did, but I have to build sbt with some frequency anyway.) It isn't that easy to fix on scala's side, here's an excerpt from my explanation.  The reason these methods are under attack is there is an unbelievable amount of wasteful creating, converting (implicitly and explicitly), reconverting, assembling, disassembling, duplicating among strings, names, termnames, and typenames, and it'll never be fixed as long as there are implicits from everything to everything else.
This also illustrates a larger challenge with implicits and Predef.

While I don't disagree with removing those implicits in general (I can see why they are causing lots of trouble) I just don't like they simply disappeared one day without any kind of deprecation period.
If we had them deprecated first for a 2-3 weeks and removed after that period I'd be more than happy. Otherwise we get a situation that Eclipse IDE didn't publish its nightly builds for a week now just because somewhere deep in sbt we need a couple of lines to be changed.
To sum up: I don't complain about the change per se (I think it's going in a good direction) but I'm not very happy that it broke quite a few projects and maintainers of those projects where not warned in any way beforehand so they could avoid the breakage.

--
Grzegorz Kossakowski

I'm looking at ways to hack SBT so that we can release new compiler interface with SBT, or force it to use one of our own design.  
The biggest issue I see is that Scala (the project devs) don't trust SBT to stay up-to-date with trunk and SBT doesn't trust Scala to fix bugs in the interface quick enough for its users.   Something's gotta give here.
I definitely do *not* think the removed implicits should be added back.  If we're going to force folks to use Namers and Names instead of strings for efficiency we should avoid String like the plague, so we don't end up with millions of `$1` strings in HEAP when profiling applications like we do currently.
However, we need a better solution for dealing with the compiler interface.   I'm almost wondering if we can build our own fork of the source in the scala build.
- Josh

On Wed, Jan 18, 2012 at 9:17 AM, Paul Phillips <paulp@improving.org> wrote:

On Wed, Jan 18, 2012 at 12:51 AM, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

What's the status of resolving SBT's breakage on scala master? It's blocking quite a few of us.

As an FYI only, it's a simple change on sbt's side (that's what I did, but I have to build sbt with some frequency anyway.) It isn't that easy to fix on scala's side, here's an excerpt from my explanation.  The reason these methods are under attack is there is an unbelievable amount of wasteful creating, converting (implicitly and explicitly), reconverting, assembling, disassembling, duplicating among strings, names, termnames, and typenames, and it'll never be fixed as long as there are implicits from everything to everything else.
This also illustrates a larger challenge with implicits and Predef.
---
Yeah, I can't fix it, it has to fix in sbt. (This is why all possible
compiler code should come out of sbt.) The problem is that names need
all the same kinds of manipulations that strings and sequences do, and
there is already an implicit conversion from strings to sequences in
Predef.  So to put efficient sequence methods on Name with expected
names (like say a "dropRight" which calls substring on the underlying
string, rather than going through conversions from Name to Seq to
Name) I have to make up new names for every method if there is a
String -> Name implicit in scope, because dropRight on a String could
apply to either.

I'm not going to make up new names for every method if only in the
interests of eventual correctness.  The problem can be seen in
practice via the 40 ambiguities which arise if one puts back the
absent implicit.

To futureproof itself, sbt should not utilize any implicit conversions
between names and strings.

On Wed, Jan 18, 2012 at 6:30 AM, Grzegorz Kossakowski <grzegorz.kossakowski@gmail.com> wrote:

To sum up: I don't complain about the change per se (I think it's going in a good direction) but I'm not very happy that it broke quite a few projects and maintainers of those projects where not warned in any way beforehand so they could avoid the breakage.

You're right.  It's really quite a challenge not breaking things when I have to consider the entire compiler as possible surface area.  There is very little if anything which is less than public (including things like vars in Symbol!) and there's no API at all.  I periodically try building some projects against local builds, but it is an indescribable time sink in most cases.  Even in the best case, where there is an easy to figure out push-button build against snapshots, something is likely to arise which requires me to debug either eighty pages of maven output or someone's sbt build, tasks which I find of comparable difficulty.
Another tool I don't have but would use if I had it is something which can look at the state of the tree before and after a commit and tell me exactly what has changed in terms of signatures.   Because I do try to leave a deprecated trail, but like with most things which depend on a person rather than code, it is an error-prone process.
If you can assist me in getting any pieces of this puzzle in place which would help alert me to the impact of changes without requiring constant intervention or adjustment on my part (because I already lose what seems like half my day just trying to verify the code right in front of me) it would be very helpful.
I still think all of us, sbt especially, should be pursuing the elimination of low-level compiler code in places other than the compiler.  I mean, the line in sbt which broke involved the use of this:

    final val LOCAL_CHILD: NameType                    = "<local child>"
This should be at the very bottom level of implementation details.  I am not saying the code didn't have to be written the way it was at the time, I'm saying we should aggressively work toward a sane interface with a supported API which covers less than 100K lines of code.