caling scala.tools.nsc.Main from a TestCase throws scala.tools.nsc.MissingRequirementError: object scala not found.

On Wed, Nov 10, 2010 at 9:12 PM, Thomas Pawlitzki wrote:
> scala.tools.nsc.MissingRequirementError: object scala not found.

This mostly means, that the compiler is missing the Scala library jar
in its compile classpath. The compiler has some logic to figure out
whats the correct classpath to use. I can't say for sure right now why
it works in the JUnit + Eclipse case but in general you have to
provide it with the correct classpath manually (that's what the
startup scripts are doing normally), especially when run from a
somewhat managed environment like maven.

I had the same problem and I solved it this way:

https://github.com/jrudolph/bytecode/blob/master/bytecode/src/test/scala...

The gist of it is, that I try to recursively scan the ClassLoader (if
they are URLClassLoaders) for their URLs and additionally add
dependencies declared in MANIFEST.MF and supply this stuff to the
compiler manually as its compilation classpath.

You can take the code and adapt it to your needs. It would be cool, if
we could extract a general solution which I could then include in my
scala compiler plugin template on github.

Using

-Ylog-classpath

with scalac will give you a wealth of diagnostics about how the
compiler assembles the final classpath and about what the final
resulting classpath is.

Johannes Rudolph-2 wrote:
>
> I had the same problem and I solved it this way:
>
> https://github.com/jrudolph/bytecode/blob/master/bytecode/src/test/scala...
>
>

Sorry, but I can not really see how this (extending Settings) helps me with
calling the compiler.
I see that you do something with the classloader but can not figure out how
this helps me.

Nevertheless I added also some of your code concerning logging of
classloading to my test. See below ...

Johannes Rudolph-2 wrote:
>
> Using
>
> -Ylog-classpath
>
> with scalac will give you a wealth of diagnostics about how the
> compiler assembles the final classpath and about what the final
> resulting classpath is.
>

Thanks for the hint, this is a useful option.
I tried it and the following result is from a run using Eclipse
Classpath built from (-Ylog-classpath = true -d = .)
Defaults: object Defaults {
scalaHome =
javaBootClassPath =
/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-library.jar
/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-dbc.jar
/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-swing.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/sunrsasign.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/classes
scalaLibDirFound = None
scalaLibFound =
scalaBootClassPath =
scalaPluginPath = misc/scala-devel/plugins
}
After java boot/extdirs classpath has 1 entries:
directory classpath: .
[Classpath =
/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-library.jar:/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-dbc.jar:/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-swing.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar:.]

So the scala-library and scala-compiler jars ar used from the Scala
Enviroment added in the eclipse project.

When using maven I get the following output:

Classpath built from (-Ylog-classpath = true -d = .)
Defaults: object Defaults {
scalaHome =
javaBootClassPath =
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/sunrsasign.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/classes
scalaLibDirFound = None
scalaLibFound =
scalaBootClassPath =
scalaPluginPath = misc/scala-devel/plugins
}
After java boot/extdirs classpath has 1 entries:
directory classpath: .
[Classpath =
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar:.]

But Maven says that it has set up the classpath as follows
mvn -X clean test
...
[DEBUG] Test Classpath :
[DEBUG] /home/thomas/tmp/scala-test/target/test-classes
[DEBUG] /home/thomas/tmp/scala-test/target/classes
[DEBUG]
/home/thomas/.m2/repository/org/scala-lang/scala-library/2.8.0/scala-library-2.8.0.jar
[DEBUG]
/home/thomas/.m2/repository/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.8.0.jar
[DEBUG] /home/thomas/.m2/repository/junit/junit/4.5/junit-4.5.jar

as expected and configured in the pom.xml

Also caling the dependency-plugin with build-classpath results in the
following
/home/thomas/.m2/repository/junit/junit/4.5/junit-4.5.jar:/home/thomas/.m2/repository/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.8.0.jar:/home/thomas/.m2/repository/org/scala-lang/scala-library/2.8.0/scala-library-2.8.0.jar

This means that maven set up the classpath correctly

Also I added the follwing code to my test to see what information the
classloader has:
val cl = ClassLoader.getSystemClassLoader()
def printClassLoaders(cl: ClassLoader) {
if (cl != null) {
val urls =
if (cl.isInstanceOf[java.net.URLClassLoader])
cl.asInstanceOf[java.net.URLClassLoader].getURLs.mkString(", ")
else ""

println(" -- " + cl.toString + ": " + urls);
printClassLoaders(cl.getParent);
}
}
printClassLoaders(cl)

This produces the following:

Try to tweak surefire :

* http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loa...
* http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

try from command line, then update your pom.xml, eg :

mvn test -Dsurefire.useManifestOnlyJar=false

/davidB

On Thu, Nov 11, 2010 at 21:05, Thomas Pawlitzki wrote:
>
>
> Johannes Rudolph-2 wrote:
>>
>> I had the same problem and I solved it this way:
>>
>> https://github.com/jrudolph/bytecode/blob/master/bytecode/src/test/scala...
>>
>>
>
> Sorry, but I can not really see how this (extending Settings) helps me with
> calling the compiler.
> I see that you do something with the classloader but can not figure out how
> this helps me.
>
> Nevertheless I added also some of your code concerning logging of
> classloading to my test. See below ...
>
>
> Johannes Rudolph-2 wrote:
>>
>> Using
>>
>> -Ylog-classpath
>>
>> with scalac will give you a wealth of diagnostics about how the
>> compiler assembles the final classpath and about what the final
>> resulting classpath is.
>>
>
> Thanks for the hint, this is a useful option.
> I tried it and the following result is from a run using Eclipse
> Classpath built from (-Ylog-classpath = true -d = .)
> Defaults: object Defaults {
>  scalaHome            =
>  javaBootClassPath    =
> /home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-library.jar
> /home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-dbc.jar
> /home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-swing.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/sunrsasign.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/classes
>  scalaLibDirFound     = None
>  scalaLibFound        =
>  scalaBootClassPath   =
>  scalaPluginPath      = misc/scala-devel/plugins
> }
> After java boot/extdirs classpath has 1 entries:
>  directory classpath: .
> [Classpath =
> /home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-library.jar:/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-dbc.jar:/home/thomas/java/eclipse-galileo/configuration/org.eclipse.osgi/bundles/281/1/.cp/lib/scala-swing.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar:.]
>
> So the scala-library and scala-compiler jars ar used from the Scala
> Enviroment added in the eclipse project.
>
> When using maven I get the following output:
>
> Classpath built from (-Ylog-classpath = true -d = .)
> Defaults: object Defaults {
>  scalaHome            =
>  javaBootClassPath    =
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/sunrsasign.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/classes
>  scalaLibDirFound     = None
>  scalaLibFound        =
>  scalaBootClassPath   =
>  scalaPluginPath      = misc/scala-devel/plugins
> }
> After java boot/extdirs classpath has 1 entries:
>  directory classpath: .
> [Classpath =
> /usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar:.]
>
> But Maven says that it has set up the classpath as follows
> mvn -X clean test
> ...
> [DEBUG] Test Classpath :
> [DEBUG]   /home/thomas/tmp/scala-test/target/test-classes
> [DEBUG]   /home/thomas/tmp/scala-test/target/classes
> [DEBUG]
> /home/thomas/.m2/repository/org/scala-lang/scala-library/2.8.0/scala-library-2.8.0.jar
> [DEBUG]
> /home/thomas/.m2/repository/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.8.0.jar
> [DEBUG]   /home/thomas/.m2/repository/junit/junit/4.5/junit-4.5.jar
>
> as expected and configured in the pom.xml
>
> Also caling the dependency-plugin with build-classpath results in the
> following
> /home/thomas/.m2/repository/junit/junit/4.5/junit-4.5.jar:/home/thomas/.m2/repository/org/scala-lang/scala-compiler/2.8.0/scala-compiler-2.8.0.jar:/home/thomas/.m2/repository/org/scala-lang/scala-library/2.8.0/scala-library-2.8.0.jar
>
> This means that maven set up the classpath correctly
>
> Also I added the follwing code to my test to see what information the
> classloader has:
>    val cl = ClassLoader.getSystemClassLoader()
>    def printClassLoaders(cl: ClassLoader) {
>      if (cl != null) {
>        val urls =
>          if (cl.isInstanceOf[java.net.URLClassLoader])
>            cl.asInstanceOf[java.net.URLClassLoader].getURLs.mkString(", ")
>          else ""
>
>        println("  --  " + cl.toString + ": " + urls);
>        printClassLoaders(cl.getParent);
>      }
>    }
>    printClassLoaders(cl)
>
> This produces the following:
>  --  sun.misc.Launcher$AppClassLoader@7d772e:
> file:/tmp/surefirebooter4667252193779210205.jar
>  --  sun.misc.Launcher$ExtClassLoader@11b86e7:
> file:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar,
> file:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar,
> file:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar,
> file:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar
>
> And here I can not find anything of scala-library.
>
> So what is going wrong?
> Cheers
> Thomas
>
>
>
>
> --
> View this message in context: http://scala-programming-language.1934581.n4.nabble.com/caling-scala-too...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

Hey David,

David Bernard-3 wrote:
>
> Try to tweak surefire :
>
> *
> http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loa...
> * http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html
>
> try from command line, then update your pom.xml, eg :
>
> mvn test -Dsurefire.useManifestOnlyJar=false
>

Thanks for this hint.
I tried this (as -D argument and as configuration of the surfire-plugin
within the pom) but it do not work.

Running maven with -X tells me also that this paramter was accepted:
[DEBUG] (f) useManifestOnlyJar = false

Any more ideas?

Cheers
Thomas

try also combinaison with :
useSystemClassLoader => -Dsurefire.useSystemClassLoader=false

But IMHO, the issue come from that scala-library is not into
bootclasspath (or in the args (classpath) for your internal
callCompiler).
try to add scala-libary to surefire bootclasspath, may be via
"argLine" configuration

-Xbootclasspath/p:${settings.localRepository}/org/scala-lang/scala-library/${scala.version}/scala-library-${scala-version}.jar

ref :
* http://docs.codehaus.org/display/MAVENUSER/MavenPropertiesGuide
* http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

/davidB

On Fri, Nov 12, 2010 at 20:23, Thomas Pawlitzki wrote:
>
> Hey David,
>
>
> David Bernard-3 wrote:
>>
>> Try to tweak surefire :
>>
>> *
>> http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loa...
>> * http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html
>>
>> try from command line, then update your pom.xml, eg :
>>
>>     mvn test -Dsurefire.useManifestOnlyJar=false
>>
>
> Thanks for this hint.
> I tried this (as -D argument and as configuration of the surfire-plugin
> within the pom) but it do not work.
>
> Running maven with -X tells me also that this paramter was accepted:
> [DEBUG]   (f) useManifestOnlyJar = false
>
> Any more ideas?
>
> Cheers
> Thomas
> --
> View this message in context: http://scala-programming-language.1934581.n4.nabble.com/caling-scala-too...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

David Bernard-3 wrote:
>
> try also combinaison with :
> useSystemClassLoader => -Dsurefire.useSystemClassLoader=false
>
Already did. No success.

But IMHO, the issue come from that scala-library is not into
bootclasspath (or in the args (classpath) for your internal
callCompiler).
try to add scala-libary to surefire bootclasspath, may be via
"argLine" configuration

-Xbootclasspath/p:${settings.localRepository}/org/scala-lang/scala-library/${scala.version}/scala-library-${scala-version}.jar

Tried it also
---------------------
Calling compiler
-Ylog-classpath
-Xshow-phases
---------------------
Classpath built from (-Ylog-classpath = true -Xshow-phases = true -d = .)
Defaults: object Defaults {
scalaHome =
javaBootClassPath =
/home/thomas/.m2/repository/org/scala-lang/scala-library/2.8.1.RC4/scala-library-null.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/sunrsasign.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/classes
scalaLibDirFound = None
scalaLibFound =
scalaBootClassPath =
scalaPluginPath = misc/scala-devel/plugins
}
After java boot/extdirs classpath has 1 entries:
directory classpath: .
[Classpath =
/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/resources.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/rt.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jsse.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/jce.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/charsets.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-sun-1.6.0.16/jre/lib/ext/dnsns.jar:.]
scala.tools.nsc.MissingRequirementError: object scala not found.
at
scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:517)
at
scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37

Again, no success :-(

David give me a hint, that the arg line I use was not correct.

When using the correct one

-Xbootclasspath/p:${settings.localRepository}/org/scala-lang/scala-library/${scala.version}/scala-library-${scala.version}.jar

it works.

Thanks for your help, David.

Cheers
Thomas

On Thu, Nov 11, 2010 at 9:05 PM, Thomas Pawlitzki wrote:
> Johannes Rudolph-2 wrote:
> Sorry, but I can not really see how this (extending Settings) helps me with
> calling the compiler.
> I see that you do something with the classloader but can not figure out how
> this helps me.

Sorry, I've only now read your responses. The code in this snippet
adds classpath entries to the compiler classpath which are defined
inside a manifest, which is exactly what surefire does usually to
specify its classpath (and the Scala compiler doesn't pick these up,
as Java's runtime environment does). Anyway, it's good to have an even
easier alternative.

After including:
val run = new compiler.Run
I get an compiler error
Error in Scala compiler: object scala not found.

This problem just happens with the eclipse compiler using the Xplugin
Option, not when compiling with sbt or with eclipse run config
scala.tools.nsc.Main -Xprint:a_plugins::AnnotationsPI
-Xplugin:./../PlugIns\target\scala_2.8.1\plugins_2.8.1-1.0.jar -cp
./../PlugIns\target\scala_2.8.1\plugins_2.8.1-1.0.jar -d
target/scala_2.8.1/classes src/main/scala/annotations/Run.scala

Are compiler instances related?
How can I solve this?

thanks
stefan