Scala Refactoring Term Project

Hi Mirko,

On Thu, 2009-09-10 at 22:04 +0200, Mirko Stocker wrote:
> Now, what I'd like to know from you:
>
> - Which refactorings are you missing the most?

Rename and extract variable/method are the ones I use all the time
without even thinking. Inline method/variable is also very useful and
others I use on occasion are change method signature, extract
interface/trait, pull up, push down, convert anonymous class to nested,
convert nested class to top-level and convert local variable to field.

> - How important is having refactoring support?

It's not essential, but basic support is very useful.

> - If there was a simple API with which you could modify your sourcecode, would
> you use it to create your own refactorings or code manipulation tools? (This
> is at most a long-term idea, but imagine something like an intelligent search-
> replace that understands your code).

Is this something like Project Jackpot?

Best,
Ismael

Mirko,

I think this is an excellent idea.

Another use case is code-generation. Currently, Scala does something fairly primitive and hackish to generate all the necessary TupleN and FunctionN classes. It'd be great to, eg, have a code tool that could take a Tuple2 source file and generate Tuple3...N source files.

--j

On Thu, Sep 10, 2009 at 1:04 PM, Mirko Stocker <me@misto.ch> wrote:

Hi all

Reading about people creating whole new IDEs in just six month, I almost feel
bad about this, but I'd still like to announce my Scala Refactoring term
project.

In short, the goal is to create an IDE independent refactoring tool/API for
Scala. The longer version and my proposal can be found on my blog [1].

Now, what I'd like to know from you:

- Which refactorings are you missing the most?
- How important is having refactoring support?
- If there was a simple API with which you could modify your sourcecode, would
you use it to create your own refactorings or code manipulation tools? (This
is at most a long-term idea, but imagine something like an intelligent search-
replace that understands your code).
- And any other comment you might have on the topic :-)

Regards

Mirko

[1] http://misto.ch/scala-refactoring-term-project/

- Which refactorings are you missing the most?

Here's a list of (rather novel) refactorings, gratuitously stolen from a previous thread[1]:

Curry Method (split a parameter list, and the arg lists of all callers).
Uncurry Method (merge split parameter list, including merging the arg lists of callers.  If method is called with partial args, either complain or automatically create a helper method which represents the partial application, and replace partial calls with it.)
Extract Trait (including searching for other classes which can have the same trait extracted.  Tricky with super calls, but not impossible)
Split Trait (splits trait into two traits (putting in self-types if needed), change all extending classes to extend both traits)
Extract Extractor (select a pattern, automatically create an extractor)
Extract Closure (similar to extract method, but creating a function object)
Introduce by-name parameter
Extract type definition (obvious)
Merge nested for-comprehensions into single for-comprehension (and converse)
Split guard from for-comprehension into nested if (and converse)
Convert for-comprehension into map/filter/flatmap chain (and converse)
Wrap parameter as Option (converting null checks, etc.)
Convert instanceOf/asInstance pair to match
Replace case clause with if body to guarded case clause(s) 

- Colin

[1] http://www.nabble.com/Scala%3A-Great-potential%2C-but-not-ready-for-production-td22064598i20.html#a22082676

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Mirko Stocker wrote:
> Hi all
>
> Reading about people creating whole new IDEs in just six month, I almost feel
> bad about this, but I'd still like to announce my Scala Refactoring term
> project.
>
> In short, the goal is to create an IDE independent refactoring tool/API for
> Scala. The longer version and my proposal can be found on my blog [1].
>
> Now, what I'd like to know from you:
>
> - Which refactorings are you missing the most?

All of them. :)

Just kidding - the ones I use most in Java are

Rename field and method
Extract method
Move (classes between packages, mostly)
and occasionally Extract interface

There are undoubtedly some new Scala-specific ones that are possible and that I would use.
Someone posted their ideas for some a little ways back.

> - How important is having refactoring support?

Extremely. I use Scala for hobby projects now, and for the occasional quick tool at work.
If the IDE support was better - and refactoring support is a very big part of that - I
would use Scala at work too. Perhaps more importantly, my coworkers are holding off on
Scala until there's a refactoring IDE.

> - If there was a simple API with which you could modify your sourcecode, would
> you use it to create your own refactorings or code manipulation tools? (This
> is at most a long-term idea, but imagine something like an intelligent search-
> replace that understands your code).

I wouldn't, since I don't know enough about making Eclipse plugins, and I would strongly
prefer a tool that was as well-integrated with my IDE as the Eclipse JDT. However, if
your API would help IDE plugin writers make better plugins more quickly, it would be a big
help.

> - And any other comment you might have on the topic :-)
>
> Regards
>
> Mirko
>
> [1] http://misto.ch/scala-refactoring-term-project/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFKqWHcrZoE3ArapxERCP/AAJ0dv+MCID06AKZiwqX/JlWpnp+JNACglzaJ
702yAnHoobPvRHAkcyfbEIw=
=ay1q
-----END PGP SIGNATURE-----

Mirko Stocker wrote:

> - Which refactorings are you missing the most?

Things I miss the most:

1. rename var/method/class/etc;
2. when I move classes between packages or from/to outer classes, it
requires adjusting names and imports in whole project;
3. change method signature (change class/method generic signature would
be useful too).

> - How important is having refactoring support?

Strict typization does its job... I mean manual refactorings are asspain
but they are incomparably easier than in dynamically typed languages.

So the point is productivity only; errors introduced by simple manual
refactorings are caught by compiler.

> - If there was a simple API with which you could modify your sourcecode, would
> you use it to create your own refactorings or code manipulation tools? (This
> is at most a long-term idea, but imagine something like an intelligent search-
> replace that understands your code).

The idea looks very interesting. But the question about practical use is
probably to those who deal with large projects, especially projects with
lots of computer-generated source code (best candidates for automated
project-specific project-wide refactorings). The only thing I can say is
that API and whole process of creating/installing custom refactoring
must be as simple as possible in all means.

> - And any other comment you might have on the topic :-)

I'm not a pro in this area, but just interesting: Scala IDE plugin
builds syntactic tree in realtime while I edit source code. Do you plan
to build your tool on top of Scala plugin? If no, then how and what
about effeciency?

On Thursday 10 September 2009 22:13:15 Ismael Juma wrote:
> Is this something like Project Jackpot?

Haven't heard of Project Jackpot before, but yes, that sounds similar. But
this is nor part of the plan for this project, just an idea for other things
one might want to do with a refactoring engine.

Thanks for your list of refactorings, I'll collect these and consider them for
implementation.

Cheers

Mirko

On Thursday 10 September 2009 22:20:50 Jorge Ortiz wrote:
> Another use case is code-generation.

Code generation will naturally be part of the project, so I guess this might
be doable as well.

Regards

Mirko

On Thursday 10 September 2009 22:21:48 Colin Bullock wrote:
> Here's a list of (rather novel) refactorings, gratuitously stolen from a
> previous thread[1]:

Thanks!

good luck. i look forward to seeing what you can produce.

where do you plan on hosting the code?

Mirko Stocker wrote:
> Hi all
>
> Reading about people creating whole new IDEs in just six month, I almost feel
> bad about this, but I'd still like to announce my Scala Refactoring term
> project.
>
> In short, the goal is to create an IDE independent refactoring tool/API for
> Scala. The longer version and my proposal can be found on my blog [1].
>
> Now, what I'd like to know from you:
>
> - Which refactorings are you missing the most?
> - How important is having refactoring support?
> - If there was a simple API with which you could modify your sourcecode, would
> you use it to create your own refactorings or code manipulation tools? (This
> is at most a long-term idea, but imagine something like an intelligent search-
> replace that understands your code).
> - And any other comment you might have on the topic :-)
>
> Regards
>
> Mirko
>
> [1] http://misto.ch/scala-refactoring-term-project/
>

On Friday 11 September 2009 05:42:22 Dmitry Grigoriev wrote:
> > - And any other comment you might have on the topic :-)
>
> I'm not a pro in this area, but just interesting: Scala IDE plugin
> builds syntactic tree in realtime while I edit source code. Do you plan
> to build your tool on top of Scala plugin? If no, then how and what
> about effeciency?

Honestly, I can't answer this question at this time.. the project hasn't
really started yet and I've just begun digging into the code. But of course,
if the automated refactoring takes longer than doing it by hand, then nobody
will use it, so I have to make it fast. After all, this shouldn't just be an
academic exercise but something useful.

On Fri, Sep 11, 2009 at 2:01 PM, Mirko Stocker wrote:
> On Friday 11 September 2009 05:42:22 Dmitry Grigoriev wrote:
>> > - And any other comment you might have on the topic :-)
>>
>> I'm not a pro in this area, but just interesting: Scala IDE plugin
>> builds syntactic tree in realtime while I edit source code. Do you plan
>> to build your tool on top of Scala plugin? If no, then how and what
>> about effeciency?
>
> Honestly, I can't answer this question at this time.. the project hasn't
> really started yet and I've just begun digging into the code. But of course,
> if the automated refactoring takes longer than doing it by hand, then nobody
> will use it, so I have to make it fast. After all, this shouldn't just be an
> academic exercise but something useful.

This should work out just fine.

On trunk, the Scala IDE interacts with the presentation and build
compilers via scalac's filesystem abstraction. In the case of open
files being edited, their modified buffers are exposed to the
presentation compiler as virtual files. Something similar should fit
the refactoring scenario very nicely.

Things are different on <= 2.7.6 ... that had a rather arcane and
fragile interface to the compiler, and I don't see any practical way
that a clean IDE-independent refactoring tool could have been
integrated with it.

Cheers,

Miles

On Thu, Sep 10, 2009 at 9:04 PM, Mirko Stocker wrote:
> In short, the goal is to create an IDE independent refactoring tool/API for
> Scala.

I encourage everyone to give as much input and other assistance to
Mirko as possible: this is a really valuable project which will be
enormously beneficial for the entire Scala community, not just for
users of the Scala IDE for Eclipse.

Being IDE-independent, it'll be relatively straightforward for a broad
slice of the community to contribute refactorings and formatters
without having to commit to any particular IDE (accurate refactoring
in Emacs anyone?) or deal with the complexities of developing within
and IDE framework.

Cheers,

Miles

On Friday 11 September 2009 15:06:25 Mohamed Bana wrote:
> where do you plan on hosting the code?

I'll get a virtual server from the university where I have to set up a build
server, Wiki, Bug tracking, etc. I can publish its address once I'm done
setting it up :)