Fwd: NPE in presentation compiler - what can I attempt to narrow this down

On Sat, Dec 3, 2011 at 4:44 PM, Chris Twiner <chris.twiner@gmail.com> wrote:

Hi All,

before I raise a ticket (can't find similar stack trace on jira) I'd
like to know how I can narrow this down in order to help it being
fixed.

The attached stack traces are from a build based on the latest 2.9.x
branch out of the github project.

Tried to get it working in Eclipse but it fails very early on and I
can't see any point in continuing with that approach at narrowing
things down:

java.lang.Error
at scala.tools.nsc.symtab.SymbolTable.abort(SymbolTable.scala:35)
at scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:737)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:4273)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedApply$1$1.apply(Typers.scala:3355)
at scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedApply$1$1.apply(Typers.scala:3355)

(thats on the core-tests subproject, which jaxen-tests depends on)

Background: its a test only project in sbt, with the presentation
compiler being run out of ensime.  There are two package objects, that
pull in a number of traits and provide some functions of their own, in
the main project (jaxen-tests in scales-xml).

Using sbt to build it is mostly flawless (except the odd case of
package object issues causing a ;clean;compile).

Cheers,
Chris

Hi Chris,

The compiler outputs some message before crashing: https://github.com/scala/scala/blob/2.9.x/src/compiler/scala/tools/nsc/transform/Erasure.scala#L734 but Eclipse won't show that. So, how to get the error message:

 - start Eclipse from console - it should log errors there
 - clean your project and recompile it
 - check out the messages from the compiler -- look for "exception when typing:"
 - if you still don't see anything like "exception while typing:":
   - enable presentation compiler output in Preferences > Scala > plugininfo [X] Enable logging of the Scala Plugin Info
   - enable verbose from Preferences > Scala > Compiler
   - restart eclipse
   - clean your project and recompile it

If you have the exact error it will be easier to track down the problem.

HTH
Vlad

Hi All, and thanks Vlad for the suggestions.

I finally got eclipse to work with using sbt-eclipse instead of manual
config. So much nicer.

As expected/hoped I get the same npe from the presentation compiler:

java.lang.NullPointerException
at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3920)
at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3921)
at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.apply(Types.scala:3984)
at scala.tools.nsc.symtab.Symbols$Symbol.adaptToNewRun$1(Symbols.scala:822)

Thats the verification part.

Any ideas on how to narrow this bug down before I raise a ticket? I'm
expecting that someone, PaulP for example, has played with this bit of
code (adaptToNewRun) before. I'm obviously starting off with a binary
approach, but a guiding hand would be great.

Cheers,
Chris

On Sat, Dec 3, 2011 at 5:24 PM, Vlad Ureche wrote:
>
> On Sat, Dec 3, 2011 at 4:44 PM, Chris Twiner wrote:
>>
>> Hi All,
>>
>> before I raise a ticket (can't find similar stack trace on jira) I'd
>> like to know how I can narrow this down in order to help it being
>> fixed.
>>
>> The attached stack traces are from a build based on the latest 2.9.x
>> branch out of the github project.
>>
>> Tried to get it working in Eclipse but it fails very early on and I
>> can't see any point in continuing with that approach at narrowing
>> things down:
>>
>> java.lang.Error
>> at scala.tools.nsc.symtab.SymbolTable.abort(SymbolTable.scala:35)
>> at scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:737)
>> at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:4273)
>> at
>> scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedApply$1$1.apply(Typers.scala:3355)
>> at
>> scala.tools.nsc.typechecker.Typers$Typer$$anonfun$typedApply$1$1.apply(Typers.scala:3355)
>>
>> (thats on the core-tests subproject, which jaxen-tests depends on)
>>
>> Background: its a test only project in sbt, with the presentation
>> compiler being run out of ensime.  There are two package objects, that
>> pull in a number of traits and provide some functions of their own, in
>> the main project (jaxen-tests in scales-xml).
>>
>> Using sbt to build it is mostly flawless (except the odd case of
>> package object issues causing a ;clean;compile).
>>
>> Cheers,
>> Chris
>
>
> Hi Chris,
>
> The compiler outputs some message before crashing:
> https://github.com/scala/scala/blob/2.9.x/src/compiler/scala/tools/nsc/t...
> but Eclipse won't show that. So, how to get the error message:
>
>  - start Eclipse from console - it should log errors there
>  - clean your project and recompile it
>  - check out the messages from the compiler -- look for "exception when
> typing:"
>  - if you still don't see anything like "exception while typing:":
>    - enable presentation compiler output in Preferences > Scala > plugininfo
> [X] Enable logging of the Scala Plugin Info
>    - enable verbose from Preferences > Scala > Compiler
>    - restart eclipse
>    - clean your project and recompile it
>
> If you have the exact error it will be easier to track down the problem.
>
> HTH
> Vlad
>
>

On 4 December 2011 15:44, Chris Twiner wrote:
> Hi All, and thanks Vlad for the suggestions.
>
> I finally got eclipse to work with using sbt-eclipse instead of manual
> config.  So much nicer.
>
> As expected/hoped I get the same npe from the presentation compiler:
>
> java.lang.NullPointerException
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3920)
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3921)
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.apply(Types.scala:3984)
> at scala.tools.nsc.symtab.Symbols$Symbol.adaptToNewRun$1(Symbols.scala:822)
>
> Thats the verification part.
>
> Any ideas on how to narrow this bug down before I raise a ticket?  I'm
> expecting that someone, PaulP for example, has played with this bit of
> code (adaptToNewRun) before.  I'm obviously starting off with a binary
> approach, but a guiding hand would be great.

Do yo have small, self-contained test-case for this problem?

That's what i am trying to get but i have no idea which of the 70+ source files could be causing it or if its the project structure or if its just that I am using package objects.

That's what i need the guiding hand for, to help me focus on what kind of thing could cause this.

On Dec 4, 2011 3:53 PM, "Grzegorz Kossakowski" <grzegorz.kossakowski@gmail.com> wrote:

On 4 December 2011 15:44, Chris Twiner <chris.twiner@gmail.com> wrote:
> Hi All, and thanks Vlad for the suggestions.
>
> I finally got eclipse to work with using sbt-eclipse instead of manual
> config.  So much nicer.
>
> As expected/hoped I get the same npe from the presentation compiler:
>
> java.lang.NullPointerException
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3920)
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.adaptToNewRun(Types.scala:3921)
> at scala.tools.nsc.symtab.Types$adaptToNewRunMap$.apply(Types.scala:3984)
> at scala.tools.nsc.symtab.Symbols$Symbol.adaptToNewRun$1(Symbols.scala:822)
>
> Thats the verification part.
>
> Any ideas on how to narrow this bug down before I raise a ticket?  I'm
> expecting that someone, PaulP for example, has played with this bit of
> code (adaptToNewRun) before.  I'm obviously starting off with a binary
> approach, but a guiding hand would be great.

Do yo have small, self-contained test-case for this problem?

--
Grzegorz Kossakowski

On 4 December 2011 16:04, Chris Twiner wrote:
> That's what i am trying to get but i have no idea which of the 70+ source
> files could be causing it or if its the project structure or if its just
> that I am using package objects.
>
> That's what i need the guiding hand for, to help me focus on what kind of
> thing could cause this.

Oh, I see. Have you tried enabling -verbose and inspecting output of
the compiler before the crash?

You should see names of files that compiler is processing.

Can you go into Types.scala and show me the line that causes the
problem? The version I have installed is newer, so I can't see where
exactly the problem is happening.

Thanks

Sure thing:

object adaptToNewRunMap extends TypeMap {
private def adaptToNewRun(pre: Type, sym: Symbol): Symbol = {
if (phase.flatClasses) {
sym
} else if (sym.isModuleClass) { //<---- 3920
adaptToNewRun(pre, sym.sourceModule).moduleClass //<-- 3921
where I assume sym.sourceModule is null
} else if ((pre eq NoPrefix) || (pre eq NoType) || sym.isPackageClass) {
sym
} else {

I've tried to reproduce within eclipse, but I've not yet been
successful. The verbose presentation logs just showed the file I was
loading and the npe.

If it makes sense to just return sym if sym.sourceModule is null, I'll
also give that a try tomorrow.

On Mon, Dec 5, 2011 at 8:42 PM, martin odersky wrote:
> Can you go into Types.scala and show me the line that causes the
> problem? The version I have installed is newer, so I can't see where
> exactly the problem is happening.
>
> Thanks
>
>  -- Martin
>

On Mon, Dec 5, 2011 at 2:45 PM, Chris Twiner wrote:
> Sure thing:
>
>  object adaptToNewRunMap extends TypeMap {
>    private def adaptToNewRun(pre: Type, sym: Symbol): Symbol = {
>      if (phase.flatClasses) {
>        sym
>      } else if (sym.isModuleClass) { //<---- 3920
>        adaptToNewRun(pre, sym.sourceModule).moduleClass //<-- 3921
> where I assume sym.sourceModule is null
>      } else if ((pre eq NoPrefix) || (pre eq NoType) || sym.isPackageClass) {
>        sym
>      } else {
>
> I've tried to reproduce within eclipse, but I've not yet been
> successful.  The verbose presentation logs just showed the file I was
> loading and the npe.
>
> If it makes sense to just return sym if sym.sourceModule is null, I'll
> also give that a try tomorrow.
>
That would be an obvious thing to try, yes. Thanks for helping to
clear that one up!

Unfortunately it just causes an npe elsewhere :(  the code hasn't been changed for a while so..

I am going down the stack trace adding more logging info to find the actual type used.

On Dec 6, 2011 4:53 AM, "martin odersky" <martin.odersky@epfl.ch> wrote:

On Mon, Dec 5, 2011 at 2:45 PM, Chris Twiner <chris.twiner@gmail.com> wrote:
> Sure thing:
>
>  object adaptToNewRunMap extends TypeMap {
>    private def adaptToNewRun(pre: Type, sym: Symbol): Symbol = {
>      if (phase.flatClasses) {
>        sym
>      } else if (sym.isModuleClass) { //<---- 3920
>        adaptToNewRun(pre, sym.sourceModule).moduleClass //<-- 3921
> where I assume sym.sourceModule is null
>      } else if ((pre eq NoPrefix) || (pre eq NoType) || sym.isPackageClass) {
>        sym
>      } else {
>
> I've tried to reproduce within eclipse, but I've not yet been
> successful.  The verbose presentation logs just showed the file I was
> loading and the npe.
>
> If it makes sense to just return sym if sym.sourceModule is null, I'll
> also give that a try tomorrow.
>
That would be an obvious thing to try, yes. Thanks for helping to
clear that one up!

Found a reproducible case (see attached two project files) after
adding suitable printlns. I can't seem to reproduce within a single
project, it seems that it must be a pot luck thing which of the two
Symbols it picks up.

So the problem is in Scales I have:

package object xml {
type Attributes = ListSet[Attribute]
}

AND

package xml

object Attributes {
...
}

So the type for one is indeed a module, but the `type` is not. It
seems there is a race between the setting of the two. Both symbols
must be referenced in the RaceyUser compilation unit to trigger the
problem.

In Ensime simply re-running the full type check seems to sometimes
trigger the NPE, and sometimes not. In Eclipse, with the attached
test projects, it seems a comment out one of the calls in RaceyUser
and save will disable the strangeness, closing and re-opening the file
re-introduces this - but both with or without the presentation
compiler npe's the compiler does correctly compile it and it is
runnable.

Aside from the *cough* bad idea *cough* of having a type and object
share the name - does anything like this ring bells or should I open a
new ticket?

On Tue, Dec 6, 2011 at 9:59 AM, Chris Twiner wrote:
> Unfortunately it just causes an npe elsewhere :(  the code hasn't been
> changed for a while so..
>
> I am going down the stack trace adding more logging info to find the actual
> type used.
>
> On Dec 6, 2011 4:53 AM, "martin odersky" wrote:
>>
>> On Mon, Dec 5, 2011 at 2:45 PM, Chris Twiner
>> wrote:
>> > Sure thing:
>> >
>> >  object adaptToNewRunMap extends TypeMap {
>> >    private def adaptToNewRun(pre: Type, sym: Symbol): Symbol = {
>> >      if (phase.flatClasses) {
>> >        sym
>> >      } else if (sym.isModuleClass) { //<---- 3920
>> >        adaptToNewRun(pre, sym.sourceModule).moduleClass //<-- 3921
>> > where I assume sym.sourceModule is null
>> >      } else if ((pre eq NoPrefix) || (pre eq NoType) ||
>> > sym.isPackageClass) {
>> >        sym
>> >      } else {
>> >

On Tue, Dec 6, 2011 at 8:51 PM, Chris Twiner <chris.twiner@gmail.com> wrote:

Aside from the *cough* bad idea *cough* of having a type and object
share the name - does anything like this ring bells or should I open a
new ticket?

Sounds like a similar situation to SI-5031. For that one, I worked around it by defining the same-named object in the package object, rather than in a separate file.
-jason
[1] https://issues.scala-lang.org/browse/SI-5031

lol, I had looked at that but didn't see the link due to the
difference in stack traces.

But I just tried it out, fixes it as well for the eclipse test case.

else if (sym.isModuleClass) compiles(sym.sourceModule)

the same problem, just shown in a different way.

On Tue, Dec 6, 2011 at 9:36 PM, Jason Zaugg wrote:
> On Tue, Dec 6, 2011 at 8:51 PM, Chris Twiner wrote:
>>
>> Aside from the *cough* bad idea *cough* of having a type and object
>> share the name - does anything like this ring bells or should I open a
>> new ticket?
>
> Sounds like a similar situation to SI-5031. For that one, I worked around it
> by defining the same-named object in the package object, rather than in a
> separate file.
> -jason
> [1] https://issues.scala-lang.org/browse/SI-5031