Eclipse plugin health?

Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean

On Wed, Dec 17, 2008 at 6:29 PM, Sean McDirmid wrote:
> I've been pretty busy at work. I was just wondering how the Eclipse plugin
> was holding on after 2.7.1/2 was released. I should pretend that little
> activity is probably good news. However being the pessimist I am, I would
> take a lack of complaints about the plugin as a sign of hopelessness and
> despair :)

Well, you could, you know ... try it and see?

Cheers,

Miles

On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak wrote:
> For starters, a working forward-delete key might be a nice "feature". :-)

Works for me and I don't see your ticket for this in Trac ...

Cheers,

Miles

>>a working forward-delete key might be a nice "feature".  :-)

I presume that by "forward-delete key" you mean, pressing the "delete" key should delete the character to the right of the caret, leaving the caret in the same place.

This works properly for me in the scala 2.7.2final eclipse editor.  Please let me know if you meant something else.

Sam Reid

Miles Sabin wrote:

30961e500812171044w4a1beebuf4a4614f040069d [at] mail [dot] gmail [dot] com" type="cite">

On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak  wrote:
  

For starters, a working forward-delete key might be a nice "feature".  :-)
    

Works for me and I don't see your ticket for this in Trac ...

Cheers,


Miles

  

On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak <djspiewak@gmail.com> wrote:
> For starters, a working forward-delete key might be a nice "feature".  :-)

Works for me and I don't see your ticket for this in Trac ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

I tried out the Eclipse plugin as of 12-15-2008, known as "scala eclipse
plugin 2.7.2.final". I'm new to scala and to eclipse, but here are the
features I noticed:

1. Able to handle mixed scala/java projects.
2. Good syntax highlighting
3. 'Open declaration' works for both Java and Scala source (though this
only worked through the context menu, not through F3 keystroke)
4. Easy to set up and run a scala application
5. Error highlighting
6. Debugger looks like it's working properly

High priority features that aren't there or I couldn't figure out:
1. Automatic imports.
2. Rename/refactor
3. Find usages
4. Autocomplete for some scenarios
5. Autocomplete by type

In a few cases, the compiler and the editor became out of sync; the code
would compile, but the editor complained and wouldn't continue. When I
got into this scenario, restarting eclipse seemed to sync up the editor.

Let me know how it goes with you,
Sam Reid

Sean McDirmid wrote:
> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend
> that little activity is probably good news. However being the
> pessimist I am, I would take a lack of complaints about the plugin as
> a sign of hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't
> even fired up the plugin myself for over a couple of months now.
>
> Sean

I thought I read that there was some behind the scenes getting help
from the AspectJ developers?

On 12/17/08, Daniel Spiewak wrote:
> Assumed it was a known issue that just hadn't been reported (single-dev
> syndrome) and so I didn't want to spam your ticket queue. :-)
>
> Here ya go! https://lampsvn.epfl.ch/trac/scala/ticket/1588
>
> Daniel
>
> On Wed, Dec 17, 2008 at 12:44 PM, Miles Sabin wrote:
>
>> On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak
>> wrote:
>> > For starters, a working forward-delete key might be a nice "feature".
>> :-)
>>
>> Works for me and I don't see your ticket for this in Trac ...
>>
>> Cheers,
>>
>>
>> Miles
>>
>> --
>> Miles Sabin
>> tel: +44 (0)1273 720 779
>> mobile: +44 (0)7813 944 528
>> skype: milessabin
>>
>

Well, I don't have the feeling it has become more usable. Maybe it's my
projects, but I have to close and reopen and clean build my projects
more than ever. I cannot open editors, eclipse throws errors at me when
I want to open a context menu of a Scala file, the editor screws up my
source code (not on disk, just the display), renaming a file (from the
Scala submenu) doesn't really work when it's currently open (getting a
nice smearing effect) and incremental compilation too often doesn't work
so I end up cleaning projects more than necessary. Unfortunately, I
don't have much time to investigate eclipse bugs and they seem so
obvious to me that I actually believe that someone must have noticed
them before.

Ingo

Sean McDirmid wrote:
> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend that
> little activity is probably good news. However being the pessimist I am,
> I would take a lack of complaints about the plugin as a sign of
> hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't even
> fired up the plugin myself for over a couple of months now.
>
> Sean

On Wed, Dec 17, 2008 at 6:56 PM, Naftoli Gugenheim wrote:
> I thought I read that there was some behind the scenes getting help
> from the AspectJ developers?

Yes, that's correct. As of 2.8.0 the plugin's JDT integration will be
using the JDT weaving feature developed by the AJDT people,

http://wiki.eclipse.org/JDT_weaving_features

This will dramatically improve the depth and robustness of the JDT
integration over 2.7.2 (and the upcoming 2.7.3). This should be
landing on trunk by the end of the week.

Cheers,

Miles

This is more my bad. I've been slacking on my goal of stabilizing the
plugin on windows. Just use Linux from now on :)

I have been able to use the plugin with a maven plugin successfully on
windows. My issues are more with trying to develop/bugfix the plugin
itself on windows.

I'm of the opinion that unless I care enough to look at the issue,
then I shouldn't complain too loudly. That being said, I have a lot
of confidence in miles ability to work out bugs. I just wish more
people would help out..

On Dec 17, 2008, at 1:44 PM, "Miles Sabin" wrote:

> On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak
> wrote:
>> For starters, a working forward-delete key might be a nice
>> "feature". :-)
>
> Works for me and I don't see your ticket for this in Trac ...
>
> Cheers,
>
>
> Miles
>

I just wanted to add some comments, I've been using the plugin fairly
frequently for almost a year. I have, I assume, a fairly unique setup
of using OSGi / PDE exclusively.

Sam's list is a great starting point for my comments

On Wed, Dec 17, 2008 at 7:44 PM, Samuel Robert Reid wrote:
> I tried out the Eclipse plugin as of 12-15-2008, known as "scala eclipse
> plugin 2.7.2.final". I'm new to scala and to eclipse, but here are the
> features I noticed:
>
> 1. Able to handle mixed scala/java projects.
> 2. Good syntax highlighting
> 3. 'Open declaration' works for both Java and Scala source (though this only
> worked through the context menu, not through F3 keystroke)
> 4. Easy to set up and run a scala application
> 5. Error highlighting

can't agree more

> 6. Debugger looks like it's working properly

it works up to the point of the expressions view. That would be the
best possible addition to the plugin debug experience imo.

> High priority features that aren't there or I couldn't figure out:
> 1. Automatic imports.

I don't really mind an absence of this, I think that due to my
approach of "import where you need to" it probably can't help much.

> 2. Rename/refactor

I don't use this nearly as much working with scala as I do in java but
the times I've needed rename, search replace have worked fine. The
only other refactoring items I'd like is extract function / constant.
Those would be great.

> 3. Find usages

Total godsend if this was introduced. Features like the outline (and
resulting ctrl-o usage) were absolutely excellent, find usages I
thought would have derived from this, but seems not to have been.

> 4. Autocomplete for some scenarios
> 5. Autocomplete by type

for some reason the autocomplete seems patchy, but often its a simple
close editor, clean and reopen.

What I'd add to the wish list is that under pde / osgi usage, if I add
classes, or functions to classes etc to one project they aren't
visible in the other projects until I do a clean / rebuild on the
providing project. This alone would save me 60% of the rebuilds.

Having said that the plugin is 20 times better (arbritary number
pulled from thin air) over vim/nedit/emacs/ant combinations, but thats
the reason I'm using eclipse in the first place.

Great work from Miles (and of course Sean), they have both been
extremely helpful in adding small fixes and helping with problems I've
had. I have high hopes for the future of the plugin.

many thanks,
Chris

On Wed, Dec 17, 2008 at 6:50 PM, Daniel Spiewak wrote:
> Assumed it was a known issue that just hadn't been reported (single-dev
> syndrome) and so I didn't want to spam your ticket queue. :-)
>
> Here ya go! https://lampsvn.epfl.ch/trac/scala/ticket/1588

Ahh ... you're on a Mac ...

I'm sure I've mentioned this before, but I don't have a Mac to test
the plugin on, so contributions (of bug reports, patches) from Mac
users are especially important.

Cheers,

Miles

Well, to be honest, the lack of comments is more toward despair than contentment on my part.  I do use the plugin now and again, but it's so unreliable (particularly with lots of case classes) that I just can't apply it to anything serious.  Don't get me wrong, I was impressed with the 2.7.2 release!  The stability you guys achieved was light-years ahead of what things had been in some of the snapshot builds, but I still don't see it as a tool that I can use for anything non-trivial.  My silence has been mostly due to the fact that previous remarks on the instability have been mostly ineffective.

What I think the plugin really needs is a couple people to really "own" it.  Developers who can sit down and hammer out why the editor is so flaky and how it can be repaired.  Miles has been great (is he gone now?), but he was always focused on bringing the plugin's features to a level worthy of recognition.  I think that SDT has reached the point where it is useful feature-wise, it just needs a lot more work to make it a reliable tool.  For starters, a working forward-delete key might be a nice "feature".  :-)  Random loss of highlighting and those ever-disturbing erroneous error highlights would also be a couple problems to fix.

Daniel

On Wed, Dec 17, 2008 at 12:29 PM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:

Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean

On Wed, Dec 17, 2008 at 8:04 PM, Miles Sabin wrote:
> On Wed, Dec 17, 2008 at 6:56 PM, Naftoli Gugenheim wrote:
>> I thought I read that there was some behind the scenes getting help
>> from the AspectJ developers?
>
> Yes, that's correct. As of 2.8.0 the plugin's JDT integration will be
> using the JDT weaving feature developed by the AJDT people,
>
> http://wiki.eclipse.org/JDT_weaving_features
>
> This will dramatically improve the depth and robustness of the JDT
> integration over 2.7.2 (and the upcoming 2.7.3). This should be
> landing on trunk by the end of the week.
>
Hi Miles,

just to clarify, as an Xmas present the plugin should get another leap
in functionality? Please drop an email on scala-tools and let us know
when its out. I for one would love to help testing that out.

yours droolingly,
Chris

On Wed, Dec 17, 2008 at 6:29 PM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> I've been pretty busy at work. I was just wondering how the Eclipse plugin
> was holding on after 2.7.1/2 was released. I should pretend that little
> activity is probably good news. However being the pessimist I am, I would
> take a lack of complaints about the plugin as a sign of hopelessness and
> despair :)

Well, you could, you know ... try it and see?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

hi sean, hi miles,

first of all i must say i think the plug-in stability has greately
improved with 2.7.2, but then your assumption about the "silence" is
right, i got a little disappears with the all the remaining problems
in everyday work. i think they are very obvious and there are a lot
(maybe just a few bugs but they are omnipresent) so i didn't yet talk
back....

priority wise, i think the most severe problems are
- fixing mistakes usually produces an avalange of weird unrelated
compiler errors, so the incremental compiling doesn't work at all. i
basically (as i have auto-build enabled) hit "clean" for every error
i fix, this costs a lot of time and energy.
- the editor usually gets screwed up after a while, often in
combination with copy and paste:
- code completion disappears usually pretty quickly (i think i have
a lot of .scala files where code completion isn't even offered even
if i open the editor from scratch)
- the outline either doesn't show the complete outline, or it shows
it but clicking on the items won't bring me to the correct line (but
to like the top of the source file)
- indentation has a lot of problems; i'm not sure but my theory is
that the plug-in tries to look at lines _below_ the line you want to
indent (hitting the tab key), so the line gets shifted to a weird
position, where it should in the first place consider the previous
line. i can have for example three lines of line comments // and
while indention is correct for like the first two lines, the third
line (for example) gets indented completely wrong. also if the
previous line contains more than one statement with a semicolon, like
"gaga.make; gugu.make" the identation of the next line is wrong. this
is a very annoying issue
- missing features: toggle line comments (Cmd+Shift+7 (german
keyboard) in the java plug-in)
- refactor -> rename doesn't work properly (it always ends with an
error dialog, although it starts to convert files i think)

other annoying things
- the debugger is very cool and usefull. however, when you hit the go-
into button (to branch into a method), you often end up in
ClassLoader.class for some reason, this should be skipped i think

minor issues
- sometimes .scala files appear duplicate in the project browser (at
some point the duplicates disappear again i think)

cool features would be
- a preferences of compiler warnings, like in the java-plug in, like
to get warnings about unused local variables, shadowing of class
fields by local variables etc.

thanks, -sciss-

Am 17.12.2008 um 19:29 schrieb Sean McDirmid:

> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend
> that little activity is probably good news. However being the
> pessimist I am, I would take a lack of complaints about the plugin
> as a sign of hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't
> even fired up the plugin myself for over a couple of months now.
>
> Sean

On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> Case classes are a sore point with me, as they got changed in a way in 2.7
> that broke the plugin in a huge way that I couldn't hack around very well.
> I'm still waiting for Martin to do something about this or tell us what we
> should do about it.
>
Frankly, I think this a strong indication that the plugin is tied too
closely to the compiler. If it's going to break everytime we change
some datastructure in the compiler we have tough times ahead of us. I
don't think I can do anything about it; it's up to the Eclipse plugin
developers to make it more robust wrt compiler changes.

Cheers

I believe this is (possibly) what prompted eclipse looong ago to come
up with JDT - something they could control deeply.

I have been using IntelliJ for a while - its plugin seems to be
reasonably stable (those who have tried it on larger apps - please
jump in - my experience is really quite shallow with it) - but the
core things seem to work well (when it crashes - it tends to happen on
startup and can be fixed) - I generally can always do what I need to
do in it, and the usual tools work quite well.

I am not sure how brittle it is with compiler versions though - I have
noticed that sometimes the content awareness is not inline with scalac
- but that doesn't actually stop things from working.

On Thu, Dec 18, 2008 at 9:47 AM, martin odersky <martin.odersky@epfl.ch> wrote:
> On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
>> Case classes are a sore point with me, as they got changed in a way in 2.7
>> that broke the plugin in a huge way that I couldn't hack around very well.
>> I'm still waiting for Martin to do something about this or tell us what we
>> should do about it.
>>
> Frankly, I think this a strong indication that the plugin is tied too
> closely to the compiler. If it's going to break everytime we change
> some datastructure in the compiler we have tough times ahead of us. I
> don't think I can do anything about it; it's up to the Eclipse plugin
> developers to make it more robust wrt compiler changes.
>
> Cheers
>

Sean - I have endless admiration for people who do what you do. I have
a bit of experience with an evolving language and an eclipse IDE tool
to support it - to this day we still can't/won't separate our IDE
release cycles from the language cause its just really hard work !

Is it possible or desirable to have IDE tooling that *mostly* works
and is useful in almost all case? (ie when it comes across a case it
can't deal with, pun intended, it just soldiers on?) - is that a
realistic goal? I guess I haven't done enough to know how much the
short comings of the IDEs would let me down.

On Thu, Dec 18, 2008 at 1:39 PM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> Yes, its the huge tradeoff that we talked about at the beginning of this
> project. On the one hand, if we implement our own compiler (like the
> IntelliJ effort), we have to re-implement the Scala language and we'll have
> to worry about language drift. On the one hand, if we integrate too closely
> with Scalac, then we are at the mercy of the compiler writer to not change
> the architecture too much. We are taking an approach similar to the JDT
> team, but then the team in France knows they really have to support the team
> in Zurich, otherwise they don't have a product. They had a lot of arguments
> when the effort first started because the compiler people in France couldn't
> really understand the design requirements of the IDE people in Zurich.
>
> In my opinion, the case class apply/unapply method issue is very unique: it
> really breaks the mostly clean compiler model since it reaches out across
> trees and does a "if the method doesn't exist in that tree over there, make
> it exist in that tree over there" test that just can't be rationalized when
> trees are being compiled separately. Nothing else in the compiler is really
> written in such a non-modular way (with respect to modularity in the tree
> data structure): tree data structures are mostly processed by themselves way
> where inter-tree communication between trees occurs through scopes in a way
> that we can track and manage. I can deal with that model, I can't deal with
> code that reaches out to another top-level tree directly and by completely
> passes namers/type completion/scopes.
>
> The problem isn't really changes to data structures in the compiler, its
> changes to the underlying design principles that the compiler was written
> under; i.e., huge hacks to support new features that are written in such
> weird ways because the existing compiler infrastructure doesn't support the
> feature very well (also known as compiler rot) are going to kill the IDE.
>
> Anyways, overall, I'm surprised how well the technique works. Of course,
> when you make changes to the compiler someone who understands the compiler
> really well (me) has to make sure that they haven't broken the IDE, but this
> hasn't been a huge time draw. I am worried that no one can really replace me
> in this regard, as learning scalac internals takes a long time and a lot of
> discussion/working with Martin.
>
>
> On Thu, Dec 18, 2008 at 6:47 AM, martin odersky <martin.odersky@epfl.ch>
> wrote:
>>
>> On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean.mcdirmid@gmail.com>
>> wrote:
>> > Case classes are a sore point with me, as they got changed in a way in
>> > 2.7
>> > that broke the plugin in a huge way that I couldn't hack around very
>> > well.
>> > I'm still waiting for Martin to do something about this or tell us what
>> > we
>> > should do about it.
>> >
>> Frankly, I think this a strong indication that the plugin is tied too
>> closely to the compiler. If it's going to break everytime we change
>> some datastructure in the compiler we have tough times ahead of us. I
>> don't think I can do anything about it; it's up to the Eclipse plugin
>> developers to make it more robust wrt compiler changes.
>>
>> Cheers
>>
>> -- Martin
>
>

Hi Sean,

OK let's see how we can get case classes unstuck. I'll describe the
current scheme first:

For setters and getters, namer creates new symbols that refer to the
original field definition, but that have special type completers so
that we know we are dealing with a setter/getter, not a field. Then,
in typerphase, when we see a field, we also generate setter and getter
trees for it and patch them so their symbol fields point to the
peviously generated symbols.

Pre-2.7 we did analogous things with case-class factories. The
disadvantage of this scheme is that it's rather ad hoc and smears work
over namers and typers.

For case class related things in 2.7, I put a new scheme in place
which I thought was more systematic. The idea was that we would then
also migrate setters and getters to the new scheme.

The scheme is to have a `synthetics' collection in CompilationUnit.
When namer finds it has to create a new symbol (for instance for
companion object of a case class, or its apply or unapply method), it
immediately creates an unattributed tree for it. The tree is then
processed as normal, which leads to the correct symbol to be created.
The symbol is entered into the right scope automatically. Then, during
typerphase all symbols in a scope are traversed. If a synthetic symbol
is found, its corresponding tree is incorporated into the result tree.
The big advantage is that it's one localized solution rather than a
dozen hacks, one for each class of synthetic symbols.

Because of this advantage I am reluctant to give the new scheme up.
Can we analyze what needs to be done to make the plugin happy? I mean,
could we maybe create the symbols as described above but have their
positions point to the originating tree instead of the synthetic tree,
much as we do with setters and getters now? Would that already be
enough?

Cheers

Yes, I have the same opinion. Lots of strange things happen like moving
a file will break undo/redo for ALL Scala editors etc. The bugs are so
numerous and feel so obvious that I haven't bothered reporting them. Of
course, as a software developer, I shouldn't assume their obviousness...
So I'll try to report at least one bug a week from now on :)

I still use the plugin though and just try to ignore the pain, mostly
because I strongly prefer Eclipse (and/or don't want to cough up the
dough for IDEA).

Erkki

Ingo Maier wrote:
> Well, I don't have the feeling it has become more usable. Maybe it's
> my projects, but I have to close and reopen and clean build my
> projects more than ever. I cannot open editors, eclipse throws errors
> at me when I want to open a context menu of a Scala file, the editor
> screws up my source code (not on disk, just the display), renaming a
> file (from the Scala submenu) doesn't really work when it's currently
> open (getting a nice smearing effect) and incremental compilation too
> often doesn't work so I end up cleaning projects more than necessary.
> Unfortunately, I don't have much time to investigate eclipse bugs and
> they seem so obvious to me that I actually believe that someone must
> have noticed them before.
>
> Ingo
>
> Sean McDirmid wrote:
>> Hi,
>>
>> I've been pretty busy at work. I was just wondering how the Eclipse
>> plugin was holding on after 2.7.1/2 was released. I should pretend
>> that little activity is probably good news. However being the
>> pessimist I am, I would take a lack of complaints about the plugin as
>> a sign of hopelessness and despair :)
>>
>> Its a bit depressing not being able to program in Scala. I haven't
>> even fired up the plugin myself for over a couple of months now.
>>
>> Sean
>

Hi Sean

I got seriously behind my e-mail over the holidays; that's why I could
respond only now.

> I'm pretty immersed in the compiler so I understand the scheme. The
> getters/setters fields can be kind of difficult to deal with (it took some
> code to figure out) but they only involve one Scala tree, so we can process
> the logic in one tree This is important: if we have to process something in
> two places we are screwed as the trees will eventually not be processed
> together.
>
Hmm. But I don't think it's a good idea to hack the type system or the
compiler to satisfy that criterion. Scala is subtle enough as it is.
So we should try our utmost to make type checking clearer not to make
it more opaque. I think we need to keep apply/unapply in the companion
object.
We can play with lots of things around it: Like, we can let their
completers point to the original tree, or we can even have special
marker symbols for the sake of the Eclipse plugin. But I resist the
suggestion to compromise the integrity of the compiler for the plugins
sake.

Any idea is welcome!

Cheers

I talked to Erik Gamma shortly before Christmas, and complained about
the closedness of the JDT.
His answer was that we should just do a cp -r of it and hack it as we
like. It seems many other projects did the same thing. Is this
something we have considered? Opinions?

Cheers

On Sun, Jan 4, 2009 at 10:49 PM, martin odersky wrote:
> I talked to Erik Gamma shortly before Christmas, and complained about
> the closedness of the JDT.
> His answer was that we should just do a cp -r of it and hack it as we
> like. It seems many other projects did the same thing. Is this
> something we have considered? Opinions?

It's something I've considered, and something the AspectJ people did
at one point.

Aside from the pain and effort involved in maintaining a fork of the
JDT we'd then have to worry about interoperability with all the other
projects which depend on the JDT in some way or another.

Fortunately the AspectJ people came up with something better than a
fork. From the next release of the Eclipse AspectJ tooling they'll be
using Equinox Aspects[1] to retrofit the kind of API that we need to
JDT without having to fork or otherwise modify the JDT.

I've been working with Andrew Eisenberg on using this mechanism in the
Scala plugin for a little while now and I can report that it works
extremely well.

I'd hoped to have it committed to trunk before Xmas but unfortunately
got bogged down with other things ... expect to see something RSN ...

[1] http://www.eclipse.org/equinox/incubator/aspects/

Cheers,

Miles

On Mon, Jan 5, 2009 at 12:16 AM, Miles Sabin wrote:
> On Sun, Jan 4, 2009 at 10:49 PM, martin odersky wrote:
>> I talked to Erik Gamma shortly before Christmas, and complained about
>> the closedness of the JDT.
>> His answer was that we should just do a cp -r of it and hack it as we
>> like. It seems many other projects did the same thing. Is this
>> something we have considered? Opinions?
>
> It's something I've considered, and something the AspectJ people did
> at one point.
>
> Aside from the pain and effort involved in maintaining a fork of the
> JDT we'd then have to worry about interoperability with all the other
> projects which depend on the JDT in some way or another.
>
> Fortunately the AspectJ people came up with something better than a
> fork. From the next release of the Eclipse AspectJ tooling they'll be
> using Equinox Aspects[1] to retrofit the kind of API that we need to
> JDT without having to fork or otherwise modify the JDT.
>
> I've been working with Andrew Eisenberg on using this mechanism in the
> Scala plugin for a little while now and I can report that it works
> extremely well.
>
> I'd hoped to have it committed to trunk before Xmas but unfortunately
> got bogged down with other things ... expect to see something RSN ...
>
Cool! I can't wait. -- Martin

Hi Sean

Hmm. But I don't think it's a good idea to hack the type system or the
compiler to satisfy that criterion. Scala is subtle enough as it is.
So we should try our utmost to make type checking clearer not to make
it more opaque. I think we need to keep apply/unapply in the companion
object.
We can play with lots of things around it: Like, we can let their
completers point to the original tree, or we can even have special
marker symbols for the sake of the Eclipse plugin. But I resist the
suggestion to compromise the integrity of the compiler for the plugins
sake.

Any idea is welcome!

Cheers

 -- Martin

On Tue, Jan 6, 2009 at 6:54 AM, Sean McDirmid wrote:
> I don't know if this is an exclusive IDE issue. I think case classes are
> already kind of messed up implementation wise, where a cleaner
> implementation would probably benefit everyone. By this, I mean, there are
> so many things that can go wrong with the current implementation; e.g., what
> if someone puts a case class and companion object into separate source files
> (I think there is a bug report opened on this). Problems like these are
> going to fester with or without the IDE.

But, Sean, the current design is the result of a focussed effort to
*clean up* case classes.
Maybe we need to put in a couple more guards against mis-use (such as
declaring object and class in different source files) but that's a
separate issue. I maintain that the current design is sound and cannot
think of a better one, unless somebody steps forward and shows me one.
In fact, I'd like to roll setters and getters over to the new scheme.

> I don't think the solution is fork or aspect weave. If one compiler isn't an
> option, the next best solution is a clean implementation of Scala where
> de-sugaring doesn't happen until after type checking. This would make tools
> easier to write. We could de-emphasize performance and reuse the existing
> lexer/parser (just throw out namers and typers), we wouldn't have to go
> beyond type checking. This is still a lot of stuff to implement, and there
> are always drift problems. I'd like to avoid it but I know you'd like to
> de-sugaring as early as possible to make the code generation easier to
> implement.
>
I think forking and aspect weaving are considered for JDT integration,
not for compiler integration.

Cheers

On Tue, Jan 6, 2009 at 6:54 AM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> I don't know if this is an exclusive IDE issue. I think case classes are
> already kind of messed up implementation wise, where a cleaner
> implementation would probably benefit everyone. By this, I mean, there are
> so many things that can go wrong with the current implementation; e.g., what
> if someone puts a case class and companion object into separate source files
> (I think there is a bug report opened on this). Problems like these are
> going to fester with or without the IDE.

But, Sean, the current design is the result of a focussed effort to
*clean up* case classes.
Maybe we need to put in a couple more guards against mis-use (such as
declaring object and class in different source files) but that's a
separate issue. I maintain that the current design is sound and cannot
think of a better one, unless somebody steps forward and shows me one.
In fact, I'd like to roll setters and getters over to the new scheme.

> I don't think the solution is fork or aspect weave. If one compiler isn't an
> option, the next best solution is a clean implementation of Scala where
> de-sugaring doesn't happen until after type checking. This would make tools
> easier to write. We could de-emphasize performance and reuse the existing
> lexer/parser (just throw out namers and typers), we wouldn't have to go
> beyond type checking. This is still a lot of stuff to implement, and there
> are always drift problems. I'd like to avoid it but I know you'd like to
> de-sugaring as early as possible to make the code generation easier to
> implement.
>
I think forking and aspect weaving are considered for JDT integration,
not for compiler integration.

Cheers

 -- Martin

On Wed, Jan 7, 2009 at 1:13 AM, Sean McDirmid wrote:
> I'm afraid of what you will do to getters and setters, if its of the style
> of "if (!overthere.exists) overthere.make_it_exists", or involves some kind
> of global hack map to track info across trees, the IDE is screwed.

I really don't buy that.

I've asked you a couple of times before ... can you explain _why_ the
IDE is screwed in this scenario.

Cheers,

Miles

On Wed, Jan 7, 2009 at 1:13 AM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> I'm afraid of what you will do to getters and setters, if its of the style
> of "if (!overthere.exists) overthere.make_it_exists", or involves some kind
> of global hack map to track info across trees, the IDE is screwed.

I really don't buy that.

I've asked you a couple of times before ... can you explain _why_ the
IDE is screwed in this scenario.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

On Wed, Jan 7, 2009 at 1:43 AM, Sean McDirmid wrote:
> The IDE compiles some trees separately, just like scalac can compile
> files separately.

OK, so what's to prevent the IDE from always attempting to compile
classes and their companion objects together?

Cheers,

Miles

On Wed, Jan 7, 2009 at 1:43 AM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> The IDE compiles some trees separately, just like scalac can compile
> files separately.

OK, so what's to prevent the IDE from always attempting to compile
classes and their companion objects together?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

On Wed, Jan 7, 2009 at 1:54 AM, Sean McDirmid wrote:
> The IDE already tries to do this. If the companion object is processed, it
> will invalidate the case class so the apply method will be added back.
> However, in some cases it seems that the case class is being processed
> before the companion object and then this invalidation is ignored, hence the
> missing apply method.

This is basically just a bug then, no?

BTW, some of your comments suggest you're planning to do some work on
the compiler or the plugin in the not too distant future ... is that
actually the case?

Cheers,

Miles

On Wed, Jan 7, 2009 at 1:54 AM, Sean McDirmid <sean.mcdirmid@gmail.com> wrote:
> The IDE already tries to do this. If the companion object is processed, it
> will invalidate the case class so the apply method will be added back.
> However, in some cases it seems that the case class is being processed
> before the companion object and then this invalidation is ignored, hence the
> missing apply method.

This is basically just a bug then, no?

BTW, some of your comments suggest you're planning to do some work on
the compiler or the plugin in the not too distant future ... is that
actually the case?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

On Wed, Jan 7, 2009 at 2:17 AM, Sean McDirmid wrote:
> Yes, its a bug with no easy resolution.

I'm still not clear on this. So far you've not flagged up anything as
being especially difficult rather than being a matter of time and
effort. I appreciate that you don't have the time right now, but I do,
so I don't see this as too much of a problem. If there's something
else about this particular bug which you've not mentioned, please let
us know, otherwise I really don't see it as something which should
affect the design of the language.

> I'm willing to do maintenance work on the plugin/compiler if its going to
> survive, but I'm not very encouraged right now.

That's fine. Any bug fixing you can do would be very welcome, however
the thing which would help me and the community the most would be some
documentation, even rough notes, of the interface between the compiler
and the plugin, esp. the maintenance of the incremental ASTs.

Cheers,

Miles

On Wed, Jan 7, 2009 at 2:13 AM, Sean McDirmid wrote:
> The clean up was a hack that reached across trees to break separate
> compilation. I'm not sure how you'll fix things with a few extra checks, the
> problem occurs when things are compiled separately so the checks won't know
> something else exists and the user will just scratch their head wondering
> what the heck is going on. Anyways, you are backing yourself into a corner
> with this design and it will be interesting to see how you get out of it,
> but its a scalac issue and not an IDE issue.
> Anyways, all the synthetic code you add during type checking is only their
> to support code generation. If we just suppressed adding the apply method to
> the companion object and add special logic to the type checker to otherwise
> "see" the apply method, the problem would probably go away.

Are you sure? I mean the reason the method tree is added is so that
the compiler will produce a symbol to play with during typechecking.
The tree generation migth be delayed, but not the symbol generation.
In any case, even if we generate the tree in Namers, it would be
fairly straightforward to mark it to be invisible for the Eclipse
plugin. But would that solve the probem?

Also, it would be fairly straightfroward to always typecheck companion
object or class in some order if that's what's required.

Cheers