This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Compiler plugin unit-testing: how to build/get instances of Type?

11 replies
Olivier Pernet
Joined: 2010-03-13,
User offline. Last seen 42 years 45 weeks ago.

Hi everyone,

I'm working on my Scala compiler plugin, and would like to be able to
unit-test bits of it in isolation.
For this reason, I need to be able to construct a few simple Type
instances for things like String, Object, and maybe Int or Boolean.

Unfortunately, if I try to get them by calling eg.
definitions.StringClass.tpe, I get an AssertionError in the
constructor of TypeHistory, here:
assert(validFrom != NoPeriod)

I've also tried to build the type manually by
typeRef(definitions.RootClass.tpe, definitions.StringClass, Nil)

or even
definitions.StringClass.typeConstructor

but they all fail on the same assertion.

I think this is because I'm trying to use things from Global before
any phase has been run, and so types are not computed yet.
How could I initialize those basic types in my unit test? For now the
test just has an instance of Global.

Cheers,
Olivier Pernet

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Fri, Aug 27, 2010 at 06:27:48PM +0100, Olivier Pernet wrote:
> Unfortunately, if I try to get them by calling eg.
> definitions.StringClass.tpe, I get an AssertionError in the
> constructor of TypeHistory, here

Yes, if you look at the source to Global you will see that the
definitions object is initialized on the first run.

scala> new scala.tools.nsc.Global(new scala.tools.nsc.Settings)
res0: scala.tools.nsc.Global = scala.tools.nsc.Global@63f5e4b6

// like you say
scala> res0.definitions.StringClass.tpe
java.lang.AssertionError: assertion failed
at scala.Predef$.assert(Predef.scala:77)

// create a compiler run
scala> new res0.Run
res2: res0.Run = scala.tools.nsc.Global$Run@d00078c

// now init has been run
scala> res0.definitions.StringClass.tpe
res3: res0.Type = java.lang.String

Olivier Pernet
Joined: 2010-03-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Compiler plugin unit-testing: how to build/get instances of

Ok, this seems to work fine in the REPL, but when I try to create the
Run in my unit test, I get the following error:

scala.tools.nsc.MissingRequirementError: object scala not found.
at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:513)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37)
at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:38)
at scala.tools.nsc.symtab.Definitions$definitions$.UnitClass(Definitions.scala:83)
at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:785)
at scala.tools.nsc.Global$Run.(Global.scala:597)

What do you think could cause this?

Olivier

On Fri, Aug 27, 2010 at 21:03, Paul Phillips wrote:
> new res0.Run

Olivier Pernet
Joined: 2010-03-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Compiler plugin unit-testing: how to build/get instances of

I just got it to work:

val settings = new Settings
val scalaVersion = "2.8.0"
settings.classpath.tryToSet(List(
"project/boot/scala-"+scalaVersion+"/lib/scala-compiler.jar" +
":project/boot/scala-"+scalaVersion+"/lib/scala-library.jar"))
val global = new Global(settings)
new global.Run

This is because on the REPL, the Scala library jar is already on the
classpath, but wasn't in my unit test environment.

Olivier

On Sat, Aug 28, 2010 at 00:39, Olivier Pernet wrote:
> Ok, this seems to work fine in the REPL, but when I try to create the
> Run in my unit test, I get the following error:
>
> scala.tools.nsc.MissingRequirementError: object scala not found.
>        at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:513)
>        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37)
>        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:38)
>        at scala.tools.nsc.symtab.Definitions$definitions$.UnitClass(Definitions.scala:83)
>        at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:785)
>        at scala.tools.nsc.Global$Run.(Global.scala:597)
>
> What do you think could cause this?
>
> Olivier
>
>
> On Fri, Aug 27, 2010 at 21:03, Paul Phillips wrote:
>> new res0.Run
>

Olivier Pernet
Joined: 2010-03-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Compiler plugin unit-testing: how to build/get instances of

Would there be a more lightweight way to initialize just the
definitions? Creating a Run slows down my test quite a bit.

Only calling definitions.init crashes on the same assertion as described before.

If I copy the code from the Run constructor:
val phase1 = syntaxAnalyzer.newPhase(NoPhase)
phase = phase1
definitions.init

then it works in the REPL, but in the test I get:
java.lang.NullPointerException
at scala.tools.nsc.symtab.Types$class.scala$tools$nsc$symtab$Types$$unique(Types.scala:2722)
at scala.tools.nsc.symtab.Types$ThisType$.apply(Types.scala:1068)
at scala.tools.nsc.symtab.Symbols$ClassSymbol.thisType(Symbols.scala:1978)
at scala.tools.nsc.symtab.Symbols$TypeSymbol.tpe(Symbols.scala:1821)
at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:781)

Could I make this work? Would it make any speed difference compared to
just new Run?
I just need to be able to get some simple types.

Olivier

On Sat, Aug 28, 2010 at 00:47, Olivier Pernet wrote:
> I just got it to work:
>
> val settings = new Settings
> val scalaVersion = "2.8.0"
> settings.classpath.tryToSet(List(
>            "project/boot/scala-"+scalaVersion+"/lib/scala-compiler.jar" +
>            ":project/boot/scala-"+scalaVersion+"/lib/scala-library.jar"))
> val global = new Global(settings)
> new global.Run
>
> This is because on the REPL, the Scala library jar is already on the
> classpath, but wasn't in my unit test environment.
>
> Olivier
>
>
> On Sat, Aug 28, 2010 at 00:39, Olivier Pernet wrote:
>> Ok, this seems to work fine in the REPL, but when I try to create the
>> Run in my unit test, I get the following error:
>>
>> scala.tools.nsc.MissingRequirementError: object scala not found.
>>        at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:513)
>>        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37)
>>        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:38)
>>        at scala.tools.nsc.symtab.Definitions$definitions$.UnitClass(Definitions.scala:83)
>>        at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:785)
>>        at scala.tools.nsc.Global$Run.(Global.scala:597)
>>
>> What do you think could cause this?
>>
>> Olivier
>>
>>
>> On Fri, Aug 27, 2010 at 21:03, Paul Phillips wrote:
>>> new res0.Run
>>
>

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Sat, Aug 28, 2010 at 12:47:55AM +0100, Olivier Pernet wrote:
> val settings = new Settings
> val scalaVersion = "2.8.0"
> settings.classpath.tryToSet(List(
> "project/boot/scala-"+scalaVersion+"/lib/scala-compiler.jar" +
> ":project/boot/scala-"+scalaVersion+"/lib/scala-library.jar"))

Normally settings.usejavacp.value = true will do what you want.

On Sat, Aug 28, 2010 at 02:42:56AM +0100, Olivier Pernet wrote:
> Would there be a more lightweight way to initialize just the
> definitions? Creating a Run slows down my test quite a bit.

I believe this feature awaits your implementation. Based on your emails
thus far you know everything you need to know: "okay, let's see how it
crashes if I do this..."

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Sat, Aug 28, 2010 at 02:42:56AM +0100, Olivier Pernet wrote:
> If I copy the code from the Run constructor:
> val phase1 = syntaxAnalyzer.newPhase(NoPhase)
> phase = phase1
> definitions.init
>
> then it works in the REPL, but in the test I get:
> java.lang.NullPointerException
> at scala.tools.nsc.symtab.Types$class.scala$tools$nsc$symtab$Types$$unique(Types.scala:2722)

Sometimes if I start pulling a thread like this it goes and goes and
goes, but not this time. That was the whole thread.

- private var uniques: HashSet[AnyRef] = _
+ private var uniques: HashSet[AnyRef] = new HashSet("uniques", initialUniquesCapacity)

// init.scala
import scala.tools.nsc._

val global = new Global(new Settings)
import global._
global.phase = syntaxAnalyzer.newPhase(NoPhase)
definitions.init
println(definitions.StringClass.tpe)

% build/pack/bin/scala -i work/init.scala
Loading work/init.scala...
import scala.tools.nsc._
global: scala.tools.nsc.Global = scala.tools.nsc.Global@bcaeccf
import global._
java.lang.String

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Sat, Aug 28, 2010 at 12:05:25AM -0700, Paul Phillips wrote:
> val global = new Global(new Settings)
> import global._
> global.phase = syntaxAnalyzer.newPhase(NoPhase)
> definitions.init

Oh, and I was briefly hopeful this would be good news for repl startup
time, but I ran it 100 times each way and there was all of four
milliseconds difference between new current.Run() and calling init. So
unless you want to keep dredging I don't think this will be the solution
you've been dreaming of.

Mark Harrah
Joined: 2008-12-18,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Friday, August 27, 2010, Paul Phillips wrote:
> On Sat, Aug 28, 2010 at 12:47:55AM +0100, Olivier Pernet wrote:
> > val settings = new Settings
> > val scalaVersion = "2.8.0"
> > settings.classpath.tryToSet(List(
> > "project/boot/scala-"+scalaVersion+"/lib/scala-compiler.jar" +
> > ":project/boot/scala-"+scalaVersion+"/lib/scala-library.jar"))
>
> Normally settings.usejavacp.value = true will do what you want.

Because of the project/boot/scala, I'd guess the tests are running through sbt. This kind of thing is the motivation for http://gist.github.com/404272.

-Mark

> On Sat, Aug 28, 2010 at 02:42:56AM +0100, Olivier Pernet wrote:
> > Would there be a more lightweight way to initialize just the
> > definitions? Creating a Run slows down my test quite a bit.
>
> I believe this feature awaits your implementation. Based on your emails
> thus far you know everything you need to know: "okay, let's see how it
> crashes if I do this..."
>
>

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Sat, Aug 28, 2010 at 11:51:44AM -0400, Mark Harrah wrote:
> > Normally settings.usejavacp.value = true will do what you want.
>
> Because of the project/boot/scala, I'd guess the tests are running
> through sbt. This kind of thing is the motivation for
> http://gist.github.com/404272.

Funny, the very same situation in sbt brought me near tears yesterday,
until I finally realized I can get at the classpath from a test via the
context classloader.

val context = Thread.currentThread.getContextClassLoader.asInstanceOf[java.net.URLClassLoader]
val cpurls = context.getURLs.toList
val cpurlString = ClassPath.join(cpurls map (_.getPath) : _*)

[...]
scala.tools.nsc.Main.process(Array("-d", outdir.path, "-cp", cpurlString) ++ srcpaths)

Ha sbt, you cannot defeat me. BTW that url does not discuss the context
classloader. I do not completely understand its wily ways, but does
knowing that this works with current sbt alter your view on solutions?

Mark Harrah
Joined: 2008-12-18,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Saturday, August 28, 2010, Paul Phillips wrote:
> On Sat, Aug 28, 2010 at 11:51:44AM -0400, Mark Harrah wrote:
> > > Normally settings.usejavacp.value = true will do what you want.
> >
> > Because of the project/boot/scala, I'd guess the tests are running
> > through sbt. This kind of thing is the motivation for
> > http://gist.github.com/404272.
>
> Funny, the very same situation in sbt brought me near tears yesterday,
> until I finally realized I can get at the classpath from a test via the
> context classloader.
>
> val context = Thread.currentThread.getContextClassLoader.asInstanceOf[java.net.URLClassLoader]
> val cpurls = context.getURLs.toList
> val cpurlString = ClassPath.join(cpurls map (_.getPath) : _*)
>
> [...]
> scala.tools.nsc.Main.process(Array("-d", outdir.path, "-cp", cpurlString) ++ srcpaths)

I'd be surprised if this worked if you are compiling classes that reference the compiler. I'm pretty sure that the context class loader won't contain the Scala jars, since those are in a parent. Now, if you have disabled filterScalaJars, those jars might be there. This wouldn't be the usual case, though.

> Ha sbt, you cannot defeat me. BTW that url does not discuss the context
> classloader. I do not completely understand its wily ways, but does
> knowing that this works with current sbt alter your view on solutions?

Not really. It makes assumptions about the context class loader that might change.

Anyway, the patch to scalac part of the proposal ensures you don't even have to do the above. It should just work with sbt. Right now, you can do it manually, though. See the getClasspath("app" ... and getClasspath("scala" ... statements.

-Mark

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Compiler plugin unit-testing: how to build/get instances of

On Sat, Aug 28, 2010 at 01:10:16PM -0400, Mark Harrah wrote:
> Now, if you have disabled filterScalaJars, those jars might be there.
> This wouldn't be the usual case, though.

You have correctly identified mr. no-filter. Oh well.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland