unexpectedly abrupt end for such an innocent mistake

On Fri, Jun 26, 2009 at 02:34:06PM -0700, Meredith Gregory wrote:
> See the trace below. Is this how we'd like this error to be handled?

There was a fierce debate between outputting a brief error message and
spewing a long stack trace, but in the end we went with the stack trace.

(Perhaps you don't realize how "Is this how we'd like this error to be
handled?" comes across.)

Can I give some general encouragement to people to use the bug database
to report bugs rather than sending them to the lists, unless there is
something super unusual and fascinating about that particular bug,
keeping in mind there are hundreds of open tickets which should be no
less important for being a little (or a lot) older.

On Fri, Jun 26, 2009 at 02:34:06PM -0700, Meredith Gregory wrote:
> See the trace below. Is this how we'd like this error to be handled?

There was a fierce debate between outputting a brief error message and
spewing a long stack trace, but in the end we went with the stack trace.

(Perhaps you don't realize how "Is this how we'd like this error to be
handled?" comes across.)

Can I give some general encouragement to people to use the bug database
to report bugs rather than sending them to the lists, unless there is
something super unusual and fascinating about that particular bug,
keeping in mind there are hundreds of open tickets which should be no
less important for being a little (or a lot) older.

--
Paul Phillips      | Where there's smoke, there's mirrors!
Apatheist          |
Empiricist         |
pal, i pill push   |----------* http://www.improving.org/paulp/ *----------

On Fri, Jun 26, 2009 at 02:47:35PM -0700, Meredith Gregory wrote:
> Thanks for all your kind feedback. i wasn't really concerned about the
> long stack trace. i was concerned about the termination of the REPL
> process.

Oh, I realize. My point is that there are endless ways to crash the
repl, and this bug is in no way unique or even unusual, so it should
probably get in line behind all the others. I was making a pretty good
dent in the database for a while there but I decided I'd spent enough
effort on one-bug-at-a-time and am trying to get whole handfuls. This
means that for now they're stacking up again.

On Fri, Jun 26, 2009 at 02:47:35PM -0700, Meredith Gregory wrote:
> Thanks for all your kind feedback. i wasn't really concerned about the
> long stack trace. i was concerned about the termination of the REPL
> process.

Oh, I realize.  My point is that there are endless ways to crash the
repl, and this bug is in no way unique or even unusual, so it should
probably get in line behind all the others.  I was making a pretty good
dent in the database for a while there but I decided I'd spent enough
effort on one-bug-at-a-time and am trying to get whole handfuls.  This
means that for now they're stacking up again.

--
Paul Phillips      | Giving every man a vote has no more made men wise
Everyman           | and free than Christianity has made them good.
Empiricist         |     -- H. L. Mencken
ha! spill, pupil   |----------* http://www.improving.org/paulp/ *----------

> This means that for now they're stacking up again.

so what does this mean? any of:

* scala needs a tighter development proces
* scala is happening too fast
* scala is insufficiently staffed
* software is just plain hard
* all us whiners should be assigned one bug to fix a week.

On Fri, 2009-06-26 at 15:13 -0700, Raoul Duke wrote:
> * scala is insufficiently staffed

This is the main one.

> * software is just plain hard

A bit of this.

> * all us whiners should be assigned one bug to fix a week.

If us whiners actually solved one bug a week instead of just being
assigned one, maybe the problem would go away. :)

Best,
Ismael

On Fri, 2009-06-26 at 15:13 -0700, Meredith Gregory wrote:
> Paul,
>
> Cool. i honestly had no idea if it was intended behavior or not. i
> thought i'd check, because it might not be seen as a bug.

If the compiler or interpreter show a stacktrace that is not caused by
an exception thrown by your code, it's generally safe to assume that
it's a bug.

Best,
Ismael

On Fri, Jun 26, 2009 at 03:13:01PM -0700, Raoul Duke wrote:
> * scala needs a tighter development proces
> * scala is happening too fast
> * scala is insufficiently staffed
> * software is just plain hard
> * all us whiners should be assigned one bug to fix a week.

All open source projects are insufficiently staffed, so that one is
safe. Learning the compiler well enough to fix even simple-ish bugs
took me months, so the barrier to entry is high. It's very difficult to
be sure a bug fix hasn't broken something else without rebuilding and
running the whole test suite. Very slight changes in a trait can force
rebuilding many files because of the extensive interdependencies, and
the ant-based build system has no intelligence at all so you're on your
own if you want to do anything less than a total clean.

In summary, there are a lot of little things that conspire together to
make it a very slow process. And, fixing bugs is pretty much as unsexy
as it gets, so you have to be really into it for its own sake on top of
everything else. I probably now know approximately how to fix every bug
in the database except those revolving around some abstruse aspect of
the type system, but even if I worked on bugs full time, they're opening
faster than I can close them.

Fortunately most of them do things like crash the compiler, or fail to
infer the type when you think they should, or involve other behaviors
which though annoying should not be deal breakers. Scala is still very
useful, but it could be a lot more robust at the edges.

I won't say development is happening too fast, but it's true that having
a bunch of huge patches land on trunk makes life interesting. We don't
need lots of people trying to fix one bug a week -- we need a very small
number of people to invest the time and effort necessary to learn the
compiler, and then for them to fix a few bugs a day.

(Anyone out there with nothing to do for the next year or so, who needs
no paycheck and enjoys closing tickets? Opportunities abound!)

> On Fri, Jun 26, 2009 at 3:31 PM, Paul Phillips wrote:

ok, that sounds like pretty classical less-than-idea software
engineering! more is the pity. i mean, in some sense it is the "good
problems to have" because it means Scala caught on, for which we are
(perhaps not always obviously) grateful -- thank you! -- but i feel
like it is past the point where somebody has to actually step up and
deal with the blankety-blank technical debt, already.

here i am living in a glass house, by the way :-)

sincerely.

On Fri, Jun 26, 2009 at 03:36:57PM -0700, Raoul Duke wrote:
> ok, that sounds like pretty classical less-than-idea software
> engineering! more is the pity. i mean, in some sense it is the "good
> problems to have" because it means Scala caught on, for which we are
> (perhaps not always obviously) grateful -- thank you! -- but i feel
> like it is past the point where somebody has to actually step up and
> deal with the blankety-blank technical debt, already.

I have a handle on a lot of it. I think if I had one more person of
comaprable ability working full time under my guidance, we would cover a
lot of ground in a hurry. I'm sure money can be thrown at this problem,
but I'm not interested in handling logistics or organizing or etc, I am
only interested in writing software.

Paul Phillips wrote on 27.6.2009 1:31:
> It's very difficult to
> be sure a bug fix hasn't broken something else without rebuilding and
> running the whole test suite. Very slight changes in a trait can force
> rebuilding many files because of the extensive interdependencies, and
> the ant-based build system has no intelligence at all so you're on your
> own if you want to do anything less than a total clean.

Does the Scala compiler support compiling multiple source files in
parallel? That should speed up compiling as the number of cores per CPU
increases. Would other build systems be more intelligent than Ant and
compile only the affected files?

Is there some high-level documentation about the compiler's internals,
so that a new developer could quickly get an overview of the
architecture and begin to dig in deeper?

On Mon, Jun 29, 2009 at 01:23:57AM +0300, Esko Luontola wrote:
> Does the Scala compiler support compiling multiple source files in
> parallel?

Not yet.

> That should speed up compiling as the number of cores per CPU
> increases. Would other build systems be more intelligent than Ant and
> compile only the affected files?

It's hard to be less intelligent than ant, but it's non-trivial to
determine what needs recompilation. There is a lot of ongoing work on
this front, and I don't know where it stands at this second.

> Is there some high-level documentation about the compiler's internals,
> so that a new developer could quickly get an overview of the
> architecture and begin to dig in deeper?

There is some, but all of it must be consumed with salt because it's
always dated to a greater or lesser degree. Here is a key document,
which I can improve if I find some time:

https://lampsvn.epfl.ch/trac/scala/wiki/OverviewOfCompilerPhases