Scala viability for large scale projects (was Re: Why does scala have "new"?)

Please move to debate.  This is a worthwhile conversation but shouldn't clog the main list.

On Thu, Sep 3, 2009 at 4:16 PM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

I am currently learning Scala with the intent of using it in a large
scale project, and what David Flemström said below is also becoming a
concern for me.

What is the opinion of people on this matter ? Would you objectively
recommend Scala for a massively online game where performance,
flexibility and maintainability are all prime factors in the adoption
of a platform/language ?

Sébastien

On Wed, Sep 2, 2009 at 16:32, David Flemström<david.flemstrom@gmail.com> wrote:

> I thought that was part of the Scala "philosophy", if you will; to reflect on
> the language itself, to analyze options and embrace language features that
> enhance the language. I'm not proposing that the instantiation system of Scala
> should be replaced, but there are things that could be done to improve the
> current situation anyways.
>
> To me, Scala is slowly becoming in a way like what Perl is today. New features
> are added all the time (Implicit methods? Named arguments? Package objects?
> etc) and the language is becoming more and more complex and more obscure
> problems arise because of that. It's not there yet, but that's where it's
> going, slowly but surely.
>
> That's why I brought this discussion up to begin with: Is it really too late
> to bring the language back to the (supposed) root principles it took from
> functional programming, where a language is simple but powerful and not
> complex, broad and filled with workarounds because you have to fight the
> language itself? How many threads are there about "double" implicit
> conversions? How often do questions pop up along the lines of "how does type
> variance work together with existential types and what does the compiler mean
> by '[?] forSome { type ? <% $2 }'"?
>
> I might be seeking a solution to a problem that doesn't exist, but isn't that
> what progress is all about? And wouldn't the language benefit from simplicity?
>
> Thanks. :-)

--
http://erikengbrecht.blogspot.com/

I don't share David's view at all. In fact, I'm getting used to skip over what he is saying, because of how often I simply disagree with his opinions.   As XP teaches, make it as simple as possible with it still working.   David asks "how many times do we see X or Y?", and I honestly haven't seen any of the issues he mentions. Well, variance, perhaps, but the alternative is not having variance -- and we might as well just give up on Object Orientation if we go that way.   "New features all the time". Huh? Scala 2.7 was released quite a while ago. Scala 2.8 is still some ways into the future. ALL the mentioned features are future 2.8 features. And I'll add more: there are very few of them, and all of them target specific problems.

On Thu, Sep 3, 2009 at 5:16 PM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

I am currently learning Scala with the intent of using it in a large
scale project, and what David Flemström said below is also becoming a
concern for me.

What is the opinion of people on this matter ? Would you objectively
recommend Scala for a massively online game where performance,
flexibility and maintainability are all prime factors in the adoption
of a platform/language ?

Sébastien

On Wed, Sep 2, 2009 at 16:32, David Flemström<david.flemstrom@gmail.com> wrote:

> I thought that was part of the Scala "philosophy", if you will; to reflect on
> the language itself, to analyze options and embrace language features that
> enhance the language. I'm not proposing that the instantiation system of Scala
> should be replaced, but there are things that could be done to improve the
> current situation anyways.
>
> To me, Scala is slowly becoming in a way like what Perl is today. New features
> are added all the time (Implicit methods? Named arguments? Package objects?
> etc) and the language is becoming more and more complex and more obscure
> problems arise because of that. It's not there yet, but that's where it's
> going, slowly but surely.
>
> That's why I brought this discussion up to begin with: Is it really too late
> to bring the language back to the (supposed) root principles it took from
> functional programming, where a language is simple but powerful and not
> complex, broad and filled with workarounds because you have to fight the
> language itself? How many threads are there about "double" implicit
> conversions? How often do questions pop up along the lines of "how does type
> variance work together with existential types and what does the compiler mean
> by '[?] forSome { type ? <% $2 }'"?
>
> I might be seeking a solution to a problem that doesn't exist, but isn't that
> what progress is all about? And wouldn't the language benefit from simplicity?
>
> Thanks. :-)

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

I'll add a possibly useful perspective. I've been working (off-and-on) on
adding some more advanced tooling around Scala to IntelliJ IDEA:
intentions, inspections, refactorings, automated Java to Scala translation.
I've done similar work for Java, Groovy and JavaScript, and been privileged
to have my work included by JetBrains in their product. To come up with
automated code inspections for a language, I basically have to scour the web
to find any corner cases in the language that could trip up unwary
developers, and flag any potentially surprising or bug-prone semantics. I
spend a lot of time thinking about how languages can confuse or frustrate
programmers. What I've discovered is that Scala is about as close as any
language has gotten gotcha-free, with the possible exception of Haskell.
Everything has clean, predictable semantics. Pretty much everything can be
nested inside pretty much anything and behave exactly as you would expect.
There are no special cases for different sorts of data or different scopes.
Rules for nesting, shadowing, overriding, encapsulating and importing are
uniform for all constructs. The number of mis-designed library classes is
extremely small. It's not perfect, but pretty much all of Scala's
complexity is due to the inherent complexity of the problems it solves, not
incidentally added by gaps or flaws in the language.

As far as viability for large projects, I'd say that Scala's first gap there
is the size of the available recruiting pool, followed by the maturity of
it's tools. I don't see any language evolution problems worth discussing.

--Dave Griffith

That's good to hear :) With the current activity rate seen on Scala
lists, one can guess that by the time we finish the core
design/artwork and data modeling, the IDE support will have improved a
lot. As for learning the language, the "Programming in Scala" book is
big, but it's very intuitive and its pace is just right for most
everyone.

Sébastien

On Thu, Sep 3, 2009 at 17:27, Dave Griffith wrote:
>
>
> I'll add a possibly useful perspective.  I've been working (off-and-on) on
> adding some more advanced tooling around Scala to IntelliJ IDEA:
> intentions, inspections, refactorings, automated Java to Scala translation.
> I've done similar work for Java, Groovy and JavaScript, and been privileged
> to have my work included by JetBrains in their product.  To come up with
> automated code inspections for a language, I basically have to scour the web
> to find any corner cases in the language that could trip up unwary
> developers, and flag any potentially surprising or bug-prone semantics.  I
> spend a lot of time thinking about how languages can confuse or frustrate
> programmers.  What I've discovered is that Scala is about as close as any
> language has gotten gotcha-free, with the possible exception of Haskell.
> Everything has clean, predictable semantics.  Pretty much everything can be
> nested inside pretty much anything and behave exactly as you would expect.
> There are no special cases for different sorts of data or different scopes.
> Rules for nesting, shadowing, overriding, encapsulating and importing are
> uniform for all constructs.  The number of mis-designed library classes is
> extremely small.  It's not perfect, but pretty much all of Scala's
> complexity is due to the inherent complexity of the problems it solves, not
> incidentally added by gaps or flaws in the language.
>
> As far as viability for large projects, I'd say that Scala's first gap there
> is the size of the available recruiting pool, followed by the maturity of
> it's tools.  I don't see any language evolution problems worth discussing.
>
> --Dave Griffith
> --
> View this message in context: http://www.nabble.com/Scala-viability-for-large-scale-projects-%28was-Re...
> Sent from the Scala - User mailing list archive at Nabble.com.
>
>

On Thursday September 3 2009, Sébastien Lorion wrote:
> That's good to hear :) With the current activity rate seen on Scala
> lists, one can guess that by the time we finish the core
> design/artwork and data modeling, the IDE support will have improved
> a lot. As for learning the language, the "Programming in Scala" book
> is big, but it's very intuitive and its pace is just right for most
> everyone.

While I often defend IDEs as important tools for developers, I don't
think that a good language should be foregone for lack of an IDE that
is as featureful and mature as those currently used for Java
development.

At least for me (at least at this juncture), programming in Scala is a
qualitatively different experience than programming in Java. It's a
much more "thoughtful" process, by which I mean I spend a lot more time
thinking and a lot less time typing, testing and debugging. As I like
to put it, it slows down my programming and speeds up my development. I
think comprehensive IDE support is less important for Scala than it is
for lower-level languages like Java. The only thing I care much about
is the "hyperlinking" that lets me go from use to definition or from
definition to use. Rename refactoring is very useful, of course. IDEA
handles both of these pretty well already.

> Sébastien

Randall Schulz

On Thu, Sep 3, 2009 at 4:16 PM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

Would you objectively
recommend Scala for a massively online game where performance,
flexibility and maintainability are all prime factors in the adoption
of a platform/language ?

Yes, but with the caveat that I only would if I knew that the project would be staffed exclusively by good coders.  By "good" I mean both "can reasonably quickly write code that does what it's supposed to with reasonable performance" and "writes code that makes it relatively easy for other expert users to read".

Scala has much more potential for obfuscation than does Java.  If you can't trust your coders to avoid obfuscation traps ("I think I'll call the 'attack' method =|--- because it looks like a sword!  And 'counterattack' will be ---|= to match!") then it's probably safer to make them use Java.  Scala also has much more potential for clarity than does Java, because you don't have to wade through nearly so much syntactic cruft to get to the point.

I assume you mean you'd be writing the server in Scala.  The client should use a standard graphics engine (assuming that the game would need decent graphics), which generally means C/C++.  (Don't try to write your own graphics engine--takes too long.)

Oh, also, as a practical matter, your builds are likely to take longer with Scala than with Java when you need to rebuild a lot from source.  If you're doing that routinely, I'd suggest that something is wrong, but the different speed is worth keeping in mind.

  --Rex
 

> Scala has much more potential for obfuscation than does Java.  If you can't
> trust your coders to avoid obfuscation traps ("I think I'll call the
> 'attack' method =|--- because it looks like a sword!  And 'counterattack'
> will be ---|= to match!") then it's probably safer to make them use Java.

1. As a manager or team lead you should keep an eye on the code and
catch such things early. I'm not either of those and I take care to
read through everything committed to the projects I work on.
2. Bad alphanumeric names are just as bad as bad symbolic names.

A plus of using Scala with bad coders is that using better idioms
usually results in shorter more readable code, as opposed to Java.
For example, catch (Exception e) is often shorter than doing it right,
and allowing resource leaks when exceptions happen is always shorter
and more readable than doing it right (closing in a finally block).

I remain convinced that most if not all coders need to write bad code
in order to know how to write good code.

Sébastien,  Viability compared to what?
  Scala is immensely better suited to building large systems than Java due to its type system, support for functional paradigms, and much more powerful features for object-oriented composition.  I don't think Ruby and Python are really viable for large-scale systems, and they are slow, anyway, so if you are really concerned with performance then they are off the table.
  Performance wise Scala is about the same as Java for equivalent code.  However, if you're dealing with a lot of primitives, you have to be very careful because idiomatic Scala constructs will lead to tons of boxing and unboxing.  @specialized in 2.8 should do much to alleviate this, and you can always drop down and write Java-like code.
  The real risk with Scala is more a traditional early adopter risk.  The compiler and library are far less mature than for Java.  There are less people who know Scala and the tools are immature.  Scala is open source, and there are no companies to turn to for support contracts.  Progress would slow way down if key contributors decided to go on vacation.
  I wouldn't launch a large Scala project without having at least a couple people skilled enough to at least write patches for the libraries, and preferably someone who could if necessary write patches for the compiler.  I wouldn't want to launch a large project of any sort without a least of a couple individuals with that skill level, but common industry practices seem to disagree with me.
  Anyway, the point is, at the language level Scala will give skilled developers a significant productivity boost over Java.  They'll also lose some productivity due to the immaturity of Scala and its tool chain.  Overall it's most likely a net gain, but in my opinion at some point you will get snagged on something that clearly wouldn't have snagged you with Java.  During that time you may forget how much time Scala has saved you.  Some obnoxious manager almost certainly will.  But that doesn't change the fact that it's a net gain.
-Erik

On Thu, Sep 3, 2009 at 4:16 PM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

I am currently learning Scala with the intent of using it in a large
scale project, and what David Flemström said below is also becoming a
concern for me.

What is the opinion of people on this matter ? Would you objectively
recommend Scala for a massively online game where performance,
flexibility and maintainability are all prime factors in the adoption
of a platform/language ?

Sébastien

On Wed, Sep 2, 2009 at 16:32, David Flemström<david.flemstrom@gmail.com> wrote:

> I thought that was part of the Scala "philosophy", if you will; to reflect on
> the language itself, to analyze options and embrace language features that
> enhance the language. I'm not proposing that the instantiation system of Scala
> should be replaced, but there are things that could be done to improve the
> current situation anyways.
>
> To me, Scala is slowly becoming in a way like what Perl is today. New features
> are added all the time (Implicit methods? Named arguments? Package objects?
> etc) and the language is becoming more and more complex and more obscure
> problems arise because of that. It's not there yet, but that's where it's
> going, slowly but surely.
>
> That's why I brought this discussion up to begin with: Is it really too late
> to bring the language back to the (supposed) root principles it took from
> functional programming, where a language is simple but powerful and not
> complex, broad and filled with workarounds because you have to fight the
> language itself? How many threads are there about "double" implicit
> conversions? How often do questions pop up along the lines of "how does type
> variance work together with existential types and what does the compiler mean
> by '[?] forSome { type ? <% $2 }'"?
>
> I might be seeking a solution to a problem that doesn't exist, but isn't that
> what progress is all about? And wouldn't the language benefit from simplicity?
>
> Thanks. :-)

--
http://erikengbrecht.blogspot.com/

Sébastien Lorion-2 wrote:
>
> With the current activity rate seen on Scala
> lists, one can guess that by the time we finish the core
> design/artwork and data modeling, the IDE support will have improved a
> lot.
>

Probably. Based on the current state of the IDEA plugin, their progress, my
work, and the inherent toolability of Scala, I'd estimate that in about 8-12
months Scala will have better IDE support than any other language, including
Java and C#.

--Dave Griffith

Well, as far as I am concerned, I want a good debugger, intellisense,
inline documentation, go to definition and renaming support. Beyond
that, it's candy I can live without and honestly, don't use much. I
much of our current team could say the same.

How is the multi-thread debugging experience in scala ?

Sébastien

p.s. It would be great if the mailing list software would make it so
that the reply-to is the list email address...

On Thu, Sep 3, 2009 at 18:22, Randall R Schulz wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> That's good to hear :) With the current activity rate seen on Scala
>> lists, one can guess that by the time we finish the core
>> design/artwork and data modeling, the IDE support will have improved
>> a lot. As for learning the language, the "Programming in Scala" book
>> is big, but it's very intuitive and its pace is just right for most
>> everyone.
>
> While I often defend IDEs as important tools for developers, I don't
> think that a good language should be foregone for lack of an IDE that
> is as featureful and mature as those currently used for Java
> development.
>
> At least for me (at least at this juncture), programming in Scala is a
> qualitatively different experience than programming in Java. It's a
> much more "thoughtful" process, by which I mean I spend a lot more time
> thinking and a lot less time typing, testing and debugging. As I like
> to put it, it slows down my programming and speeds up my development. I
> think comprehensive IDE support is less important for Scala than it is
> for lower-level languages like Java. The only thing I care much about
> is the "hyperlinking" that lets me go from use to definition or from
> definition to use. Rename refactoring is very useful, of course. IDEA
> handles both of these pretty well already.
>
>
>> Sébastien
>
>
> Randall Schulz
>

see inline comments:

On Thu, Sep 3, 2009 at 18:51, Rex Kerr wrote:
> On Thu, Sep 3, 2009 at 4:16 PM, Sébastien Lorion
> wrote:
>>
>> Would you objectively
>> recommend Scala for a massively online game where performance,
>> flexibility and maintainability are all prime factors in the adoption
>> of a platform/language ?
>
> Yes, but with the caveat that I only would if I knew that the project would
> be staffed exclusively by good coders.  By "good" I mean both "can
> reasonably quickly write code that does what it's supposed to with
> reasonable performance" and "writes code that makes it relatively easy for
> other expert users to read".
>
> Scala has much more potential for obfuscation than does Java.  If you can't
> trust your coders to avoid obfuscation traps ("I think I'll call the
> 'attack' method =|--- because it looks like a sword!  And 'counterattack'
> will be ---|= to match!") then it's probably safer to make them use Java.
> Scala also has much more potential for clarity than does Java, because you
> don't have to wade through nearly so much syntactic cruft to get to the
> point.
>

That part is well covered. My fear is that the staffing will be hard
as the team expand, as others have pointed out.

> I assume you mean you'd be writing the server in Scala.  The client should
> use a standard graphics engine (assuming that the game would need decent
> graphics), which generally means C/C++.  (Don't try to write your own
> graphics engine--takes too long.)

Yes, the server side only. As for the client side, after making our
business plan and initial project chart, we can easily see that the
budget for a good to excellent game is at the very minimum 5M, and
more like 10M. If we would go that route, we would use an engine such
as the Hero Engine (http://www.heroengine.com). We are not foolish
enough to build one from scratch, and especially in C++ :)

Anyway, we do not have that kind of money and don't want to go the VC
route. So as a stepping stone, we are making our game web based, but
with the caveat that the interface will be really rich and
interactive, and built only with html and toolkits such as jQuery,
mooTools, etc. One thing that dragged us to Scala is the Lift
framework. We find the existing web frameworks a pain to use, both on
Java and .NET and would very much like to avoid them if possible.

> Oh, also, as a practical matter, your builds are likely to take longer with
> Scala than with Java when you need to rebuild a lot from source.  If you're
> doing that routinely, I'd suggest that something is wrong, but the different
> speed is worth keeping in mind.
>
>   --Rex
>

On Thursday September 3 2009, Sébastien Lorion wrote:
> Well, as far as I am concerned, I want a good debugger, intellisense,
> inline documentation, go to definition and renaming support. Beyond
> that, it's candy I can live without and honestly, don't use much. I
> much of our current team could say the same.

Then IDEA will suit you fine. It has all those things.

> How is the multi-thread debugging experience in scala ?

Not really Scala-specific.

IDEA's bugger is not ideal insofar as all the functional objects in
Scala kind of gum up the works and if you write code concisely it seems
hard to get a breakpoint where you want it. When push comes to shove, I
introduce auxiliary val definiitions to get breakpointable lines in the
source code.

> Sébastien
>
> ...

Randall Schulz

On Fri, Sep 4, 2009 at 01:07, Randall R Schulz wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> Well, as far as I am concerned, I want a good debugger, intellisense,
>> inline documentation, go to definition and renaming support. Beyond
>> that, it's candy I can live without and honestly, don't use much. I
>> much of our current team could say the same.
>
> Then IDEA will suit you fine. It has all those things.
>
>
>> How is the multi-thread debugging experience in scala ?
>
> Not really Scala-specific.
>
> IDEA's bugger is not ideal insofar as all the functional objects in
> Scala kind of gum up the works and if you write code concisely it seems
> hard to get a breakpoint where you want it. When push comes to shove, I
> introduce auxiliary val definiitions to get breakpointable lines in the
> source code.

Is that the only issue you think is worth mentioning ? Can you easily
freeze threads and inspect them, including navigable call stack ?

>
>
>> Sébastien
>>
>> ...
>
>
> Randall Schulz
>

On our team, some people are well versed in Java, others in .NET (c#)
and all of us also use Ruby extensively. I find the libraries better
on Java, but the C# language is way ahead on the functional side and
CLR support for generics is much better. Other things to consider is
the OS platform (leading to hosting cost and server stability). With
.NET, it's pretty much Windows and nothing else. Mono is out of the
question for us. Honestly, we would much much prefer running on *BSD
or some other UNIX.

So far in our learning and experimenting, Scala seems like the perfect
choice, as it combines the advantages of Java and C# and then some.
That said, the risk factor is much higher, and having the opinion of
experts on this list is really helpful, especially if it comes from
hand-on experience on projects of the same scale.

Sébastien

On Thu, Sep 3, 2009 at 20:36, Erik Engbrecht wrote:
> Sébastien,
>   Viability compared to what?
>   Scala is immensely better suited to building large systems than Java due
> to its type system, support for functional paradigms, and much more powerful
> features for object-oriented composition.  I don't think Ruby and Python are
> really viable for large-scale systems, and they are slow, anyway, so if you
> are really concerned with performance then they are off the table.
>   Performance wise Scala is about the same as Java for equivalent code.
>  However, if you're dealing with a lot of primitives, you have to be very
> careful because idiomatic Scala constructs will lead to tons of boxing and
> unboxing.  @specialized in 2.8 should do much to alleviate this, and you can
> always drop down and write Java-like code.
>   The real risk with Scala is more a traditional early adopter risk.  The
> compiler and library are far less mature than for Java.  There are less
> people who know Scala and the tools are immature.  Scala is open source, and
> there are no companies to turn to for support contracts.  Progress would
> slow way down if key contributors decided to go on vacation.
>   I wouldn't launch a large Scala project without having at least a couple
> people skilled enough to at least write patches for the libraries, and
> preferably someone who could if necessary write patches for the compiler.  I
> wouldn't want to launch a large project of any sort without a least of a
> couple individuals with that skill level, but common industry practices seem
> to disagree with me.
>   Anyway, the point is, at the language level Scala will give skilled
> developers a significant productivity boost over Java.  They'll also lose
> some productivity due to the immaturity of Scala and its tool chain.
>  Overall it's most likely a net gain, but in my opinion at some point you
> will get snagged on something that clearly wouldn't have snagged you with
> Java.  During that time you may forget how much time Scala has saved you.
>  Some obnoxious manager almost certainly will.  But that doesn't change the
> fact that it's a net gain.
> -Erik
>
> On Thu, Sep 3, 2009 at 4:16 PM, Sébastien Lorion
> wrote:
>>
>> I am currently learning Scala with the intent of using it in a large
>> scale project, and what David Flemström said below is also becoming a
>> concern for me.
>>
>> What is the opinion of people on this matter ? Would you objectively
>> recommend Scala for a massively online game where performance,
>> flexibility and maintainability are all prime factors in the adoption
>> of a platform/language ?
>>
>> Sébastien
>>
>> On Wed, Sep 2, 2009 at 16:32, David Flemström
>> wrote:
>>
>> > I thought that was part of the Scala "philosophy", if you will; to
>> > reflect on
>> > the language itself, to analyze options and embrace language features
>> > that
>> > enhance the language. I'm not proposing that the instantiation system of
>> > Scala
>> > should be replaced, but there are things that could be done to improve
>> > the
>> > current situation anyways.
>> >
>> > To me, Scala is slowly becoming in a way like what Perl is today. New
>> > features
>> > are added all the time (Implicit methods? Named arguments? Package
>> > objects?
>> > etc) and the language is becoming more and more complex and more obscure
>> > problems arise because of that. It's not there yet, but that's where
>> > it's
>> > going, slowly but surely.
>> >
>> > That's why I brought this discussion up to begin with: Is it really too
>> > late
>> > to bring the language back to the (supposed) root principles it took
>> > from
>> > functional programming, where a language is simple but powerful and not
>> > complex, broad and filled with workarounds because you have to fight the
>> > language itself? How many threads are there about "double" implicit
>> > conversions? How often do questions pop up along the lines of "how does
>> > type
>> > variance work together with existential types and what does the compiler
>> > mean
>> > by '[?] forSome { type ? <% $2 }'"?
>> >
>> > I might be seeking a solution to a problem that doesn't exist, but isn't
>> > that
>> > what progress is all about? And wouldn't the language benefit from
>> > simplicity?
>> >
>> > Thanks. :-)
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

What about actors ? Is the experience and ease of debugging about the
same as for threads ?

Sébastien

On Fri, Sep 4, 2009 at 01:07, Randall R Schulz wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> Well, as far as I am concerned, I want a good debugger, intellisense,
>> inline documentation, go to definition and renaming support. Beyond
>> that, it's candy I can live without and honestly, don't use much. I
>> much of our current team could say the same.
>
> Then IDEA will suit you fine. It has all those things.
>
>
>> How is the multi-thread debugging experience in scala ?
>
> Not really Scala-specific.
>
> IDEA's bugger is not ideal insofar as all the functional objects in
> Scala kind of gum up the works and if you write code concisely it seems
> hard to get a breakpoint where you want it. When push comes to shove, I
> introduce auxiliary val definiitions to get breakpointable lines in the
> source code.
>
>
>> Sébastien
>>
>> ...
>
>
> Randall Schulz
>

> Probably. Based on the current state of the IDEA plugin, their progress, my
> work, and the inherent toolability of Scala, I'd estimate that in about 8-12
> months Scala will have better IDE support than any other language, including
> Java and C#.

I'll have what he's having.

--
Ricky Clarkson
Java Programmer, AD Holdings
+44 1565 770804
Skype: ricky_clarkson
Google Talk: ricky.clarkson@gmail.com

On Fri, Sep 4, 2009 at 10:29 AM, Ricky Clarkson <ricky.clarkson@gmail.com> wrote:

> Probably.  Based on the current state of the IDEA plugin, their progress, my
> work, and the inherent toolability of Scala, I'd estimate that in about 8-12
> months Scala will have better IDE support than any other language, including
> Java and C#.

I'll have what he's having.

Save some for me.
 

--
Ricky Clarkson
Java Programmer, AD Holdings
+44 1565 770804
Skype: ricky_clarkson
Google Talk: ricky.clarkson@gmail.com

--
Viktor Klang

Blog: klangism.blogspot.com
Twttr: viktorklang

Lift Committer - liftweb.com
AKKA Committer - akkasource.org
Cassidy - github.com/viktorklang/Cassidy.git
SoftPub founder: http://groups.google.com/group/softpub

Ricky Clarkson wrote:
>
> I'll have what he's having.
>

Yeah, well, it's easier than it sounds, since it's leveraging the already
existing work of the best Java IDE in existence.

--Dave Griffith

Dave Griffith wrote:
> Yeah, well, it's easier than it sounds, since it's leveraging the already
> existing work of the best Java IDE in existence.
>
> --Dave Griffit

On that happy note, what's the consensus on the best version at the
moment for Scala development?

Hi Eric,

On Fri, 2009-09-04 at 11:22 +0100, Eric Bowman wrote:
> On that happy note, what's the consensus on the best version at the
> moment for Scala development?

I don't know what the consensus is, but my opinion is:

- 2.7.5 for production now

- 2.8.0 if the libraries you need are already available for 2.8.0,
you're a few months away from production and you're willing to deal with
the occasional breakage.

Best,
Ismael

Ismael Juma wrote:
>> On that happy note, what's the consensus on the best version at the
>> moment for Scala development?
>>
>
> I don't know what the consensus is, but my opinion is:
>
> - 2.7.5 for production now
>
> - 2.8.0 if the libraries you need are already available for 2.8.0,
> you're a few months away from production and you're willing to deal with
> the occasional breakage.
>

Sorry, I wasn't clear: IDEA 8, or IDEA 9 MEAP?

On Thursday September 3 2009, Sébastien Lorion wrote:
> On Fri, Sep 4, 2009 at 01:07, Randall R Schulz wrote:
> > ...
>
> Is that the only issue you think is worth mentioning ? Can you easily
> freeze threads and inspect them, including navigable call stack ?

I don't deal much in multi-threaded code. The IDEA debugger is thread
aware, but I don't know how refined it is. I do know that you can
choose to have breakpoints and exceptions (those marked to stop in the
debugger) to stop only the thread on which they occurred.

IDEA has a 30-day free trial which is not functionally limited, so you
could try it out.

> What about actors ? Is the experience and ease of debugging about the
> same as for threads ?

I have done no programming with Scala Actors.

Randall Schulz

> What about actors ? Is the experience and ease of debugging about the
> same as for threads ?

I have done no programming with Scala Actors.

That one's a tough call, it really depends on whether you're using react or receive.
With receive there is a 1:1 correspondence between the actor and an underlying thread, so I'd imagine that debugging a receive-based actor is actually easier than "conventional" thread debugging as you get full tool support but shouldn't have to worry about locks/synchronization/etc.
React is tricker, as there are no guarantees on thread affinity (to my knowledge), Hopefully, existing tool support will still help a great deal, but I imagine that in the future we'll be seeing better features to specifically address this space...

Actors are harder to debug because sometimes they are hard to find.  I know that may sound funny, but if you have an event-based actor that is waiting for a message (or has died a horrible death) it is just another object, and finding a random object to inspect its innards can be very challenging.  Even if it is executing, you don't know what thread it is on, if you have a lot of threads going it can still be hard to find.

On Fri, Sep 4, 2009 at 1:36 AM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

What about actors ? Is the experience and ease of debugging about the
same as for threads ?

Sébastien

On Fri, Sep 4, 2009 at 01:07, Randall R Schulz<rschulz@sonic.net> wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> Well, as far as I am concerned, I want a good debugger, intellisense,
>> inline documentation, go to definition and renaming support. Beyond
>> that, it's candy I can live without and honestly, don't use much. I
>> much of our current team could say the same.
>
> Then IDEA will suit you fine. It has all those things.
>
>
>> How is the multi-thread debugging experience in scala ?
>
> Not really Scala-specific.
>
> IDEA's bugger is not ideal insofar as all the functional objects in
> Scala kind of gum up the works and if you write code concisely it seems
> hard to get a breakpoint where you want it. When push comes to shove, I
> introduce auxiliary val definiitions to get breakpointable lines in the
> source code.
>
>
>> Sébastien
>>
>> ...
>
>
> Randall Schulz
>

--
http://erikengbrecht.blogspot.com/

Well, we have experience debugging threads in IDEA with Java, so as
long as what is possible with Scala is roughly the same, we should be
fine on that front. It's not perfect, but it is workable :)

Sébastien

On Fri, Sep 4, 2009 at 09:20, Randall R Schulz wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> On Fri, Sep 4, 2009 at 01:07, Randall R Schulz wrote:
>> > ...
>>
>> Is that the only issue you think is worth mentioning ? Can you easily
>> freeze threads and inspect them, including navigable call stack ?
>
> I don't deal much in multi-threaded code. The IDEA debugger is thread
> aware, but I don't know how refined it is. I do know that you can
> choose to have breakpoints and exceptions (those marked to stop in the
> debugger) to stop only the thread on which they occurred.
>
> IDEA has a 30-day free trial which is not functionally limited, so you
> could try it out.
>
>
>> What about actors ? Is the experience and ease of debugging about the
>> same as for threads ?
>
> I have done no programming with Scala Actors.
>
>
> Randall Schulz
>

So am I right in thinking that it would be useful to also develop a
monitoring console that would keep track of those actors ? We could
then use it to help development and also in production mode to monitor
the servers. Roughly speaking, I am guessing it would be relatively
easy to make if we centralize the creation of actors in a manager
class/library. Is there any current project aimed at this goal ? If
not, that is one part we could release as open source ...

Sébastien

On Fri, Sep 4, 2009 at 09:32, Kevin Wright wrote:
>> > What about actors ? Is the experience and ease of debugging about the
>> > same as for threads ?
>>
>> I have done no programming with Scala Actors.
>>
>
> That one's a tough call, it really depends on whether you're using react or
> receive.
> With receive there is a 1:1 correspondence between the actor and an
> underlying thread, so I'd imagine that debugging a receive-based actor is
> actually easier than "conventional" thread debugging as you get full tool
> support but shouldn't have to worry about locks/synchronization/etc.
> React is tricker, as there are no guarantees on thread affinity (to my
> knowledge), Hopefully, existing tool support will still help a great deal,
> but I imagine that in the future we'll be seeing better features to
> specifically address this space...

Is there any current project aimed at this goal ? If
not, that is one part we could release as open source ...

You may want to take a look at Akka: http://wiki.github.com/jboner/akka. Very neat stuff.

- Colin

Indeed! Thanks for the tip!

Sébastien

On Fri, Sep 4, 2009 at 11:34, Colin Bullock wrote:
>
>> Is there any current project aimed at this goal ? If
>> not, that is one part we could release as open source ...
>
> You may want to take a look at Akka: http://wiki.github.com/jboner/akka.
> Very neat stuff.
>
> - Colin
>

On 09/04/2009 04:00 AM, Eric Bowman wrote:
> Sorry, I wasn't clear: IDEA 8, or IDEA 9 MEAP?
I've been using Maia 10666 since it came out Aug. 21st (the quiet period
probably portends a 9.0 beta) and had pretty good results, except for
some occasional svn quirks. I had been using nightly build 26511 of the
Scala plugin (released Aug. 24th) and just now noticed a new build,
which I'll be trying shortly. :)

-0xe1a

The same could be said of Java's EJB technology.

- Josh

On Fri, Sep 4, 2009 at 9:33 AM, Erik Engbrecht <erik.engbrecht@gmail.com> wrote:

Actors are harder to debug because sometimes they are hard to find.  I know that may sound funny, but if you have an event-based actor that is waiting for a message (or has died a horrible death) it is just another object, and finding a random object to inspect its innards can be very challenging.  Even if it is executing, you don't know what thread it is on, if you have a lot of threads going it can still be hard to find.

On Fri, Sep 4, 2009 at 1:36 AM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

What about actors ? Is the experience and ease of debugging about the
same as for threads ?

Sébastien

On Fri, Sep 4, 2009 at 01:07, Randall R Schulz<rschulz@sonic.net> wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> Well, as far as I am concerned, I want a good debugger, intellisense,
>> inline documentation, go to definition and renaming support. Beyond
>> that, it's candy I can live without and honestly, don't use much. I
>> much of our current team could say the same.
>
> Then IDEA will suit you fine. It has all those things.
>
>
>> How is the multi-thread debugging experience in scala ?
>
> Not really Scala-specific.
>
> IDEA's bugger is not ideal insofar as all the functional objects in
> Scala kind of gum up the works and if you write code concisely it seems
> hard to get a breakpoint where you want it. When push comes to shove, I
> introduce auxiliary val definiitions to get breakpointable lines in the
> source code.
>
>
>> Sébastien
>>
>> ...
>
>
> Randall Schulz
>

--
http://erikengbrecht.blogspot.com/

Great idea for Scala debugger development – find and trace the state of actors...

 

From: Erik Engbrecht [mailto:erik.engbrecht@gmail.com]
Sent: Friday, September 04, 2009 4:33 PM
To: Sébastien Lorion
Cc: Randall R Schulz; scala-user@listes.epfl.ch
Subject: Re: [scala-user] Scala viability for large scale projects (was Re: Why does scala have "new"?)

 

Actors are harder to debug because sometimes they are hard to find.  I know that may sound funny, but if you have an event-based actor that is waiting for a message (or has died a horrible death) it is just another object, and finding a random object to inspect its innards can be very challenging.  Even if it is executing, you don't know what thread it is on, if you have a lot of threads going it can still be hard to find.

On Fri, Sep 4, 2009 at 1:36 AM, Sébastien Lorion <sl@thestrangefactory.com> wrote:

What about actors ? Is the experience and ease of debugging about the
same as for threads ?

Sébastien

On Fri, Sep 4, 2009 at 01:07, Randall R Schulz<rschulz@sonic.net> wrote:
> On Thursday September 3 2009, Sébastien Lorion wrote:
>> Well, as far as I am concerned, I want a good debugger, intellisense,
>> inline documentation, go to definition and renaming support. Beyond
>> that, it's candy I can live without and honestly, don't use much. I
>> much of our current team could say the same.

.hmmessage P { margin:0px; padding:0px } body.hmmessage { font-size: 10pt; font-family:Verdana } Unfortunately IDEA's desbugging support for Scala falls well short of the support it can offer Java. As well as the problem you have already mentioned (about how to put a breakpoint in "densely written" code), the main issues are:

1). Breakpoints do not work at all in nested constructs (a foreach loop, an actor's react block, etc). I assume that this is something to do with how these are actually converted into bytecode - aeons ago, the same problem existed for debugging inner classes in Java. What's needed here is something like the old Smalltalk / Visual Age for Java debugging ability.
2). Inspections/Evaluations do not work at all.
3). Conditional breakpoints do not work at all

You can *just about* use bits of IDEA to debug code, assuming that you can find a place to put a breakpoint in a top-level method.

Chris


> From: sl@thestrangefactory.com
>
> Well, we have experience debugging threads in IDEA with Java, so as
> long as what is possible with Scala is roughly the same, we should be
> fine on that front. It's not perfect, but it is workable :)


View your other email accounts from your Hotmail inbox. Add them now.

It seems that putting a breakpoint on a line is simply the wrong approach in
a dense language like Scala. It's already insufficient for non-trivial Java.
Perhaps what's needed is the ability to put a breakpoint on a given element,
wherever in a line that may occur?

val b = a.map (x => process(x) )
^

How practical that is given the JVM I don't know.

christopher marshall-2 wrote:
>
>
> Unfortunately IDEA's desbugging support for Scala falls well short of the
> support it can offer Java. As well as the problem you have already
> mentioned (about how to put a breakpoint in "densely written" code), the
> main issues are:
>
> 1). Breakpoints do not work at all in nested constructs (a foreach loop,
> an actor's react block, etc). I assume that this is something to do with
> how these are actually converted into bytecode - aeons ago, the same
> problem existed for debugging inner classes in Java. What's needed here is
> something like the old Smalltalk / Visual Age for Java debugging ability.
> 2). Inspections/Evaluations do not work at all.
> 3). Conditional breakpoints do not work at all
>
> You can *just about* use bits of IDEA to debug code, assuming that you can
> find a place to put a breakpoint in a top-level method.
>
> Chris
>
>
>> From: sl@thestrangefactory.com
>>
>> Well, we have experience debugging threads in IDEA with Java, so as
>> long as what is possible with Scala is roughly the same, we should be
>> fine on that front. It's not perfect, but it is workable :)
>
>
> _________________________________________________________________
> View your other email accounts from your Hotmail inbox. Add them now.
> http://clk.atdmt.com/UKM/go/167688463/direct/01/
>

On Mon, 2009-09-07 at 05:08 -0700, Marcus Downing wrote:
> It seems that putting a breakpoint on a line is simply the wrong approach in
> a dense language like Scala. It's already insufficient for non-trivial Java.
> Perhaps what's needed is the ability to put a breakpoint on a given element,
> wherever in a line that may occur?
>
> val b = a.map (x => process(x) )

When dealing with Java code in Eclipse, one can use ctrl+alt+click for
debugging navigation (it works similarly to ctrl-click for code
navigation). That means that even if you can't set the breakpoint at the
exact point you need, you can easily navigate to that point once the
original breakpoint is hit.

It can be quite handy for situations like the one you mentioned.

Best,
Ismael

Hopefully for us that situation will improve quickly because the 3
points you highlight are mostly how we use debugging 80% of the
time...

1) I agree with Marcus when he said that putting a breakpoint on a
line seems like the wrong approach. Breaking on an expression
(assignment, function call, etc.) would be much more convenient that
having to step into the whole line and skipping a whole bunch of code.
That said, no IDE that we use can do that, so it's more like a nice to
have than anything else.
2) Tracing has its uses, but from my experience, if it is the only
option (cannot add watches, etc.), productivity really goes down the
drain when debugging. Fortunately, debugging is a last resort ..
usually :)
3) We can always go around this one by writing temporary code that
throws/catches an exception when a condition is met. (yurk)

Sébastien

On Mon, Sep 7, 2009 at 02:59, christopher
marshall wrote:
> Unfortunately IDEA's desbugging support for Scala falls well short of the
> support it can offer Java. As well as the problem you have already mentioned
> (about how to put a breakpoint in "densely written" code), the main issues
> are:
>
> 1). Breakpoints do not work at all in nested constructs (a foreach loop, an
> actor's react block, etc). I assume that this is something to do with how
> these are actually converted into bytecode - aeons ago, the same problem
> existed for debugging inner classes in Java. What's needed here is something
> like the old Smalltalk / Visual Age for Java debugging ability.
> 2). Inspections/Evaluations do not work at all.
> 3). Conditional breakpoints do not work at all
>
> You can *just about* use bits of IDEA to debug code, assuming that you can
> find a place to put a breakpoint in a top-level method.
>
> Chris
>
>
>> From: sl@thestrangefactory.com
>>
>> Well, we have experience debugging threads in IDEA with Java, so as
>> long as what is possible with Scala is roughly the same, we should be
>> fine on that front. It's not perfect, but it is workable :)
>
>
> ________________________________
> View your other email accounts from your Hotmail inbox. Add them now.