Design of a new Eclipse Scala debugger

On Mon, Nov 29, 2010 at 10:36 AM, Donna Malayeri wrote:
> Iulian and I have made up a list of specific issues that the Scala debugger
> should address, and we would like any feedback that any of you have to offer.
> I'm sure there's things that are missing from our list, and we also welcome
> any (high- or low-level) design ideas you may have.

Sounds like excellent stuff :-)

One thing you've not mentioned, but which I think needs to be
emphasized, is that it's crucial that mixed Scala/Java codebases be
supported well ... it should be possible to step through from Java to
Scala and back whilst maintaining a coherent view of the state of the
execution.

Cheers,

Miles

This looks great!

While the new variables view will certainly be a lot more useful for
Scala programmers, I would suggest leaving an option somewhere to show
the old "raw" view on demand.

Cheers,
Alex

On 29/11/10 11:36, Donna Malayeri wrote:
> All,
>
> Iulian Dragos and I (and probably others!) are starting a project to create a new Scala debugger for Eclipse. I haven't used the Scala Eclipse debugger very much, but as far as I am aware, it is basically just stepping through the Java bytecode and showing fully mangled Scala members, etc. So, invoking closures or accessing members that have getter functions involves stepping through lots of uninteresting code.
>
> Iulian and I have made up a list of specific issues that the Scala debugger should address, and we would like any feedback that any of you have to offer. I'm sure there's things that are missing from our list, and we also welcome any (high- or low-level) design ideas you may have.
>
> ---
>
> ** Variables View
>
> - All names should be unmangled.
> - Nested classes: The outer pointers should be shown as
> EnclosingClass.this
> - Closures: captured variables should appear as 'captured locals', not
> as reference cells
> - Objects with nested objects. The inner object does not directly store a pointer to the outer one, but semantically it would be good to see this. So, there will be a synthetic member representing the outer object, which can then be expanded to its members.
>
> ** Stepping
>
> - Step-into should still skip synthetic, uninteresting methods (such as getters, auxilliary methods, fowarders, etc.)
> - Expression-level stepping has been requested, but it is not clear how to visually indicate the breakpoint location or the current source location. Perhaps the programmer would highlight the expression and click some option as that line has been reached. (We welcome any ideas on how this could be designed.)
>
> ** Pattern Matching
>
> - The debugger should step directly to the matching pattern, and not try to show any intermediate matching steps (particularly since the pattern-matching code may merge multiple cases and do a secondary branch, which does not directly correspond to the source-level view). If the programmer wants to step into a possible 'unapply' method, they can just put a breakpoint on the appropriate case(s).
>
> ** Traits
>
> - Currently, stepping doesn't work properly due to static forwarder methods. This should be fixed so that the forwarders are invisible to the programmer.
>
> ** Local definitions (e.g. def nested within a def)
>
> - The nested def is compiled as another method that takes extra parameters for the captured variables. Instead, the variables view should show all variables in the outer scope as well as any parameters to the nested def.
>
> ** Case classes
>
> - Shouldn't have to step into auto-generated methods (toString, equals, etc)
>
> ** Implicit conversions
>
> - Will have a debug option to a) always step over all implicit conversions, b) step over only the ones in particular user-specified packages (e.g., scala.*), or c) step into all implicit conversions (current behavior).
>
> ** Lazy vals
>
> - Should show up in the variables view as either or the value if it's been initialized. Accessing a lazy val should only step into the getter if the val has not yet been initialized.
>
> ** By name parameters
>
> - In variables view, show. Programmer can step into an expression containing the variable if she wishes to see its evaluation.
>
> ** 'return' statement in a closure
>
> - Should jump to the closing curly of the enclosing function.
>
> ** Stack frames
>
> - The stack frame will contain many uninteresting method calls (getters, forwarders, apply(), etc.) so there will be an option to either filter or grey out the "uninteresting" items.
>
> ** Watch expressions and conditional breakpoints
>
> - Should be able to enter arbitrary Scala expressions for a watch expression or a breakpoint condition.
> - Possible future enhancement: REPL initialized with the current scope and program state
>
>
> .
>
>

> One thing you've not mentioned, but which I think needs to be
> emphasized, is that it's crucial that mixed Scala/Java codebases be
> supported well ... it should be possible to step through from Java to
> Scala and back whilst maintaining a coherent view of the state of the
> execution.

Ah, that is an excellent point. Do you have any suggestions for what this view should look like? Perhaps there should be another view or a view option that shows raw or semi-raw Scala parts, while also clearly identifying which is Java and which is Scala.

Donna

On Mon, Nov 29, 2010 at 11:00 AM, Donna Malayeri wrote:
>> One thing you've not mentioned, but which I think needs to be
>> emphasized, is that it's crucial that mixed Scala/Java codebases be
>> supported well ... it should be possible to step through from Java to
>> Scala and back whilst maintaining a coherent view of the state of the
>> execution.
>
> Ah, that is an excellent point. Do you have any suggestions for what this
> view should look like?  Perhaps there should be another view or a view
> option that shows raw or semi-raw Scala parts, while also clearly identifying
> which is Java and which is Scala.

If it's possible to reuse the existing Java view, but tweak it to
support Scala traces idiomatically that would probably be the best way
to go. It's also the route has the most likelihood of getting fed back
into future work on opening up the JDT for alternative JVM languages.

The ability to toggle back to a raw view would also be very useful.

Cheers,

Miles

Hi,

Wow, that list of features is very impressive, I'm looking forward to it :-)

About the Expression-level stepping, the Java Debugger allows to "step into
selection", maybe one could offer something similar, like a "run to selected
expression", without the possibility to set a breakpoint there. For me, that
would already be enough,

Mirko

On Monday 29 November 2010 11:36:58 Donna Malayeri wrote:
> All,
>
> Iulian Dragos and I (and probably others!) are starting a project to create
> a new Scala debugger for Eclipse. I haven't used the Scala Eclipse
> debugger very much, but as far as I am aware, it is basically just
> stepping through the Java bytecode and showing fully mangled Scala
> members, etc. So, invoking closures or accessing members that have getter
> functions involves stepping through lots of uninteresting code.
>
> Iulian and I have made up a list of specific issues that the Scala debugger
> should address, and we would like any feedback that any of you have to
> offer. I'm sure there's things that are missing from our list, and we also
> welcome any (high- or low-level) design ideas you may have.
>
> ---
>
> ** Variables View
>
> - All names should be unmangled.
> - Nested classes: The outer pointers should be shown as
> EnclosingClass.this
> - Closures: captured variables should appear as 'captured locals', not
> as reference cells
> - Objects with nested objects. The inner object does not directly store a
> pointer to the outer one, but semantically it would be good to see this.
> So, there will be a synthetic member representing the outer object, which
> can then be expanded to its members.
>
> ** Stepping
>
> - Step-into should still skip synthetic, uninteresting methods (such as
> getters, auxilliary methods, fowarders, etc.) - Expression-level stepping
> has been requested, but it is not clear how to visually indicate the
> breakpoint location or the current source location. Perhaps the programmer
> would highlight the expression and click some option as that line has been
> reached. (We welcome any ideas on how this could be designed.)
>
> ** Pattern Matching
>
> - The debugger should step directly to the matching pattern, and not try to
> show any intermediate matching steps (particularly since the
> pattern-matching code may merge multiple cases and do a secondary branch,
> which does not directly correspond to the source-level view). If the
> programmer wants to step into a possible 'unapply' method, they can just
> put a breakpoint on the appropriate case(s).
>
> ** Traits
>
> - Currently, stepping doesn't work properly due to static forwarder
> methods. This should be fixed so that the forwarders are invisible to the
> programmer.
>
> ** Local definitions (e.g. def nested within a def)
>
> - The nested def is compiled as another method that takes extra parameters
> for the captured variables. Instead, the variables view should show all
> variables in the outer scope as well as any parameters to the nested def.
>
> ** Case classes
>
> - Shouldn't have to step into auto-generated methods (toString, equals,
> etc)
>
> ** Implicit conversions
>
> - Will have a debug option to a) always step over all implicit conversions,
> b) step over only the ones in particular user-specified packages (e.g.,
> scala.*), or c) step into all implicit conversions (current behavior).
>
> ** Lazy vals
>
> - Should show up in the variables view as either or the value
> if it's been initialized. Accessing a lazy val should only step into the
> getter if the val has not yet been initialized.
>
> ** By name parameters
>
> - In variables view, show . Programmer can step into an expression
> containing the variable if she wishes to see its evaluation.
>
> ** 'return' statement in a closure
>
> - Should jump to the closing curly of the enclosing function.
>
> ** Stack frames
>
> - The stack frame will contain many uninteresting method calls (getters,
> forwarders, apply(), etc.) so there will be an option to either filter or
> grey out the "uninteresting" items.
>
> ** Watch expressions and conditional breakpoints
>
> - Should be able to enter arbitrary Scala expressions for a watch
> expression or a breakpoint condition. - Possible future enhancement: REPL
> initialized with the current scope and program state

On Mon, Nov 29, 2010 at 11:45 AM, Miles Sabin <miles@milessabin.com> wrote:

On Mon, Nov 29, 2010 at 10:36 AM, Donna Malayeri <lindydonna@gmail.com> wrote:
> Iulian and I have made up a list of specific issues that the Scala debugger
> should address, and we would like any feedback that any of you have to offer.
> I'm sure there's things that are missing from our list, and we also welcome
> any (high- or low-level) design ideas you may have.

Sounds like excellent stuff :-)

One thing you've not mentioned, but which I think needs to be
emphasized, is that it's crucial that mixed Scala/Java codebases be
supported well ... it should be possible to step through from Java to
Scala and back whilst maintaining a coherent view of the state of the
execution.

What exactly do you mean by that? Is there any proposed feature that conflicts with this ideal?
iulian 

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
gtalk: miles@milessabin.com
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

This is excellent news, even for non-Eclipse users. The feature I am particularly interested in seeing is the "step across but stay on same line" (expression-level stepping) when there are multiple evaluations happening on a single line. You state that it is difficult to see how you could put a breakpoint mid-line but this is an issue where something is most certainly better than nothing esp if you allow "Run to cursor" with the cursor part-way through a line.
Indeed the original IBM Java IDE "Visual Age for Java" (RIP) based as it was on the Smalltalk runtime allowed for exactly this functionality. It really is critical in a functional language where chaining statements is so common

Chris
> From: lindydonna@gmail.com
> Subject: [scala-internals] Design of a new Eclipse Scala debugger
> Date: Mon, 29 Nov 2010 11:36:58 +0100
> CC: iulian.dragos@epfl.ch
> To: scala-internals@listes.epfl.ch
>
> All,
>
> Iulian Dragos and I (and probably others!) are starting a project to create a new Scala debugger for Eclipse. I haven't used the Scala Eclipse debugger very much, but as far as I am aware, it is basically just stepping through the Java bytecode and showing fully mangled Scala members, etc. So, invoking closures or accessing members that have getter functions involves stepping through lots of uninteresting code.
>
> ** Stepping
>
> - Step-into should still skip synthetic, uninteresting methods (such as getters, auxilliary methods, fowarders, etc.)
> - Expression-level stepping has been requested, but it is not clear how to visually indicate the breakpoint location or the current source location. Perhaps the programmer would highlight the expression and click some option as that line has been reached. (We welcome any ideas on how this could be designed.)
>

On Mon, Nov 29, 2010 at 11:36:58AM +0100, Donna Malayeri wrote:
> Iulian and I have made up a list of specific issues that the Scala
> debugger should address, and we would like any feedback that any of
> you have to offer. I'm sure there's things that are missing from our
> list, and we also welcome any (high- or low-level) design ideas you
> may have.

I am not familiar with the architecture of the Eclipse debugger--how
likely is it that the debugger you build could be used in a stand alone
way (a la jdb) or be integrated into other editors (e.g. ENSIME for
Emacs)?

Thanks to you and Iulian for working on this--looks very promising!

> I am not familiar with the architecture of the Eclipse debugger--how
> likely is it that the debugger you build could be used in a stand alone
> way (a la jdb) or be integrated into other editors (e.g. ENSIME for
> Emacs)?

Very unlikely. The implementation will build on top of the JDT (Java Development Tools) of Eclipse, and most of the code will have to do with Eclipse-specific integration.

On the other hand, we will make an option to generate class files with additional debug information, so this could be exploited by other debuggers.

Donna

On Mon, Nov 29, 2010 at 6:46 PM, Donna Malayeri <lindydonna@gmail.com> wrote:

> I am not familiar with the architecture of the Eclipse debugger--how
> likely is it that the debugger you build could be used in a stand alone
> way (a la jdb) or be integrated into other editors (e.g. ENSIME for
> Emacs)?

Very unlikely. The implementation will build on top of the JDT (Java Development Tools) of Eclipse, and most of the code will have to do with Eclipse-specific integration.

On the other hand, we will make an option to generate class files with additional debug information, so this could be exploited by other debuggers.

As Donna says, we'll have to generate additional debugging information. We'll be using the standard SourceDebugExtension attribute (JSR-45), so other debuggers could use this information. We'll  probably need a custom section in the SMAP attribute, since the default deals only with line number information coming from different source files (it was designed originally for JSP debugging). Once we figure out the details, we'll produce a SID to document the contents of this additional attribute.
 

Donna

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

On Mon, Nov 29, 2010 at 2:36 AM, Donna Malayeri <lindydonna@gmail.com> wrote:

All,

Iulian Dragos and I (and probably others!) are starting a project to create a new Scala debugger for Eclipse.

Very exciting news! 

- Possible future enhancement: REPL initialized with the current scope and program state

Consider doing this early -- sounds like high value, low effort.  Low hanging fruit.
Cheers,Lee