classpath proposal

On Mon, Mar 22, 2010 at 9:20 PM, Paul Phillips wrote:
> Here it is! I realize it's very annoying to be sent a pile of text and
> told you must comment immediately, but I think it greatly behooves us to
> have this ironed out before we stamp another release, so the sooner I
> establish a plausible consensus the less I hold things up.
>
>  http://www.improving.org/scala/cp.html
>
> I have already gone over this with mharrah and jsuereth (ostensibly
> speaking for sbt and maven) so I don't think it neglects much.  The
> people on the To: line are those who I thought might have the most
> specific opinions about classpaths, but that's just supposition.  If
> YOU, dear reader, have comments, then please comment.

From a users perspective, this looks all very reasonable. Especially,
I like that the classpath of the running compiler is not automatically
used any more. That made never sense to me... (it was useful, however,
to recompile selected parts of the compiler without having to specify
any classpath, but I've always seen that as a hack anyway).

I agree with Johannes; this is much cleaner and way less confusing.

Donna

On Tue, Mar 23, 2010 at 8:26 AM, Johannes Rudolph
wrote:
> On Mon, Mar 22, 2010 at 9:20 PM, Paul Phillips wrote:
>> Here it is! I realize it's very annoying to be sent a pile of text and
>> told you must comment immediately, but I think it greatly behooves us to
>> have this ironed out before we stamp another release, so the sooner I
>> establish a plausible consensus the less I hold things up.
>>
>>  http://www.improving.org/scala/cp.html
>>
>> I have already gone over this with mharrah and jsuereth (ostensibly
>> speaking for sbt and maven) so I don't think it neglects much.  The
>> people on the To: line are those who I thought might have the most
>> specific opinions about classpaths, but that's just supposition.  If
>> YOU, dear reader, have comments, then please comment.
>
> From a users perspective, this looks all very reasonable. Especially,
> I like that the classpath of the running compiler is not automatically
> used any more. That made never sense to me... (it was useful, however,
> to recompile selected parts of the compiler without having to specify
> any classpath, but I've always seen that as a hack anyway).
>
> --
> Johannes
>
> -----------------------------------------------
> Johannes Rudolph
> http://virtual-void.net
>

On Mon, Mar 22, 2010 at 9:20 PM, Paul Phillips <paulp@improving.org> wrote:

Here it is! I realize it's very annoying to be sent a pile of text and
told you must comment immediately, but I think it greatly behooves us to
have this ironed out before we stamp another release, so the sooner I
establish a plausible consensus the less I hold things up.

 http://www.improving.org/scala/cp.html

I have already gone over this with mharrah and jsuereth (ostensibly
speaking for sbt and maven) so I don't think it neglects much.  The
people on the To: line are those who I thought might have the most
specific opinions about classpaths, but that's just supposition.  If
YOU, dear reader, have comments, then please comment.

It's great work, I'm glad someone took the plunge. I think everything is fine, but there are a few points I would like to clarify (probably it's just me not understanding the fine print).
I assume wherever you mention Scala, you mean the Scala compiler, in its forms (scalac, scaladoc, interpreter). In this context, the distinction between the user classpath and the boot classpath is simply for convenience, am I right? All the different flavors of the classpath end up in one flat list of jars/directories, in the ordered specified by your document. 
These classpaths are constructed by the compiler, and it won't make a difference whether it's run using 'java ... scala.tools.nsc.Main' or the 'scalac' launcher? 
What do you mean by 'java classpath'? Is it the classpath passed to 'java' or the contents of 'CLASSPATH'? I'm strongly against having it with the first meaning (it complicates matters too much, and the same can be achieved by explicitly passing it as part of the classpath), but I can see people wanting to use the environment variable. Maybe that can be the default value of '-cp'?
iulian

--
Paul Phillips      | Those who can make you believe absurdities
Analgesic          | can make you commit atrocities.
Empiricist         |     -- Voltaire
slap pi uphill!    |----------* http://www.improving.org/paulp/ *----------

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

On Wed, Mar 24, 2010 at 12:21:23PM +0100, Iulian Dragos wrote:
> I assume wherever you mention Scala, you mean the Scala compiler, in
> its forms (scalac, scaladoc, interpreter). In this context, the
> distinction between the user classpath and the boot classpath is
> simply for convenience, am I right? All the different flavors of the
> classpath end up in one flat list of jars/directories, in the ordered
> specified by your document.

Right. In fact it's not at all clear that we need three classpaths on
the scala side as well as the java side, but it's potentially handy
because of the different expansion semantics for extdirs and boot, and
the different priorities of boot and user.

Oh, something I forgot to mention in that document is that you can trace
a lot of classpath activity with -Ylog-classpath. It's a tad chatty
right now but I'll calm it down. I have resolved many a mystery with
it, especially when I have to go indirectly through ant:

ant -Dscalac.args="-Ylog-classpath"

was an occasional lifesaver.

> These classpaths are constructed by the compiler, and it won't make a
> difference whether it's run using 'java ... scala.tools.nsc.Main' or
> the 'scalac' launcher?

Correct, except of course that the scalac launcher sets scala.home, from
which the main jars are drawn. So running from java you can emulate
that outcome by passing -Dscala.home=... or you can go your own way.

> What do you mean by 'java classpath'? Is it the classpath passed to
> 'java' or the contents of 'CLASSPATH'?

Actually those are the same thing. From the standpoint of scala what it
means is "the contents of system property java.class.path", but java
populates that before scala is on the scene, and it will use -classpath
if it is given and CLASSPATH otherwise.

Environment variables are pretty close to pure evil and scala pays zero
attention to CLASSPATH (and it's been that way for quite a while, so I
know nobody is suffering too badly without it.)

I would like to urge once more that -cp defaults to CLASSPATH not ".".
Defaulting to "." is just plain dumb, IMO. Also, when in doubt we should do the same thing as javac. To keep backwards compatibility and complete compatibility with javac we can keep "." as a default if CLASSPATH is not defined. But even that is bad in my opinion. The problem with "." is that, depending where you are it gives you different set of files on the classpath everytime you compile!

Right now I can't work anymore since the proposal is implemented. My workflow is that everything gets compiled to /tmp/classes, and /tmp/classes
is always on my classpath, which is always taken by the compiler into account. That way, no more getting the wrong version because the compiler
wrote it somewhere but you picked it up somehwere else.

Until now, I essentially never used scalac but always my own scripts that called nsc.Main directly. That's how I could circumvent the stupid classpath default.

Now that Paul has filled this hole I am out of luck. On Unix I redefined by
local nsc alias to mean

javac scala.tools.nsc.Main -cp $CLASSPATH -d /tmp/classes

But on Windows I am out of luck because I do not know how to such in an environment variable to the command line. Neither % not $ prefixes seem to work.

In any case, can we please change that default? Or does anyone see a good reason why we should not?

Thanks

 -- Martin

-cp "%CLASSPATH%"

-jason

On Thu, Mar 25, 2010 at 6:13 PM, martin odersky wrote:
>
> But on Windows I am out of luck because I do not know how to such in an
> environment variable to the command line. Neither % not $ prefixes seem to
> work.

On Thu, Mar 25, 2010 at 6:20 PM, Jason Zaugg <jzaugg@gmail.com> wrote:

Try:

-cp "%CLASSPATH%"

Thanks, that will unblock me.

-- Martin
 

On Thu, Mar 25, 2010 at 06:13:53PM +0100, martin odersky wrote:
> I would like to urge once more that -cp defaults to CLASSPATH not ".".
> Defaulting to "." is just plain dumb, IMO. Also, when in doubt we
> should do the same thing as javac. To keep backwards compatibility and
> complete compatibility with javac we can keep "." as a default if
> CLASSPATH is not defined. But even that is bad in my opinion. The
> problem with "." is that, depending where you are it gives you
> different set of files on the classpath everytime you compile!

You only get "." on your classpath (then and now) if you supply no
classpath whatsoever. From that standpoint it is identical to javac.
The difference is that javac treats the existence of environment
variable CLASSPATH as supplying a classpath, whereas at present scala
does not, nor do I think it should.

I'm sure I can propose one or more simple tweaks which let you go back
to ignoring "." without altering that default. I think automatically
using the contents of CLASSPATH is far more dangerous and harder to
control than is putting "." on when none is supplied.

So: against both ideas.

> In any case, can we please change that default? Or does anyone see a
> good reason why we should not?

Here is the biggest reason why not, and to me it's huge, because I grit
my teeth imagining the influx of newbies saying "why doesn't this work."
I think we would break a potentially huge amount of stuff. At the
least, it should wait until after 2.8. For 99.9% of the world, nothing
in the classpath changes altered anything. Changing these semantics
would affect the other 99.9%.

// a.scala
class A
// b.scala
class B extends A

% scalac a.scala
% scalac b.scala
B.scala:1: error: not found: type A
class B extends A
^

On Thu, Mar 25, 2010 at 7:34 PM, Paul Phillips <paulp@improving.org> wrote:

On Thu, Mar 25, 2010 at 06:13:53PM +0100, martin odersky wrote:
> I would like to urge once more that -cp defaults to CLASSPATH not ".".
> Defaulting to "." is just plain dumb, IMO. Also, when in doubt we
> should do the same thing as javac. To keep backwards compatibility and
> complete compatibility with javac we can keep "." as a default if
> CLASSPATH is not defined. But even that is bad in my opinion. The
> problem with "." is that, depending where you are it gives you
> different set of files on the classpath everytime you compile!

You only get "." on your classpath (then and now) if you supply no
classpath whatsoever.  From that standpoint it is identical to javac.
The difference is that javac treats the existence of environment
variable CLASSPATH as supplying a classpath, whereas at present scala
does not, nor do I think it should.

I'm sure I can propose one or more simple tweaks which let you go back
to ignoring "." without altering that default.  I think automatically
using the contents of CLASSPATH is far more dangerous and harder to
control than is putting "." on when none is supplied.

So: against both ideas.

> In any case, can we please change that default? Or does anyone see a
> good reason why we should not?

Here is the biggest reason why not, and to me it's huge, because I grit
my teeth imagining the influx of newbies saying "why doesn't this work."
I think we would break a potentially huge amount of stuff.  At the
least, it should wait until after 2.8.  For 99.9% of the world, nothing
in the classpath changes altered anything.  Changing these semantics
would affect the other 99.9%.

// a.scala
class A
// b.scala
class B extends A

% scalac a.scala
% scalac b.scala
B.scala:1: error: not found: type A
class B extends A
               ^

There are people who work with classpaths and there are people who don't. If you do not work with classpaths you get "." so the behavior would be the same. If you work with classpaths you'd never do a scenario like the one above. You'd _always_ redefine scalac's output directory to be on your classpath. So I don't think it would upset things.
Also, we are changing things anyway. Anybody who uses nsc.Main directly in the build scripts (such as me) will see changed behavior. For those people it would be doubly annoying to see a change for the worse now, and then yet another change sometimes later. So I do not think backwards compatibility is an overwhelming argument. And I do not see a technical argument for the "." default. It really is the worst possible option, IMO.

Cheers

 -- Martin

OK, so let's see if I've got it straight. You want for the scala
classpath:

if -classpath/-cp is given it's that
if not, CLASSPATH is used
if that doesn't exist, only then do we get "."

Yes? Maybe I can live with this but for my own sanity working in some
sort of scala.always.ignore.env.variables setting may be necessary. And
it's on your head if the world explodes.

I think the switch to having the build process draw ANT_OPTS from the
environment was one of my least favorite changes ever. Pointy stick,
always there to do some poking.

On Thu, Mar 25, 2010 at 8:20 PM, Paul Phillips <paulp@improving.org> wrote:

OK, so let's see if I've got it straight.  You want for the scala
classpath:

 if -classpath/-cp is given it's that
 if not, CLASSPATH is used
 if that doesn't exist, only then do we get "."

Yes?

Yes, exactly. Pretty please!
 

Maybe I can live with this but for my own sanity working in some
sort of scala.always.ignore.env.variables setting may be necessary.  And
it's on your head if the world explodes.

OK. I take it on my head. Instead of the setting, just always add "-cp ." That re-establish the old behavior.

I think the switch to having the build process draw ANT_OPTS from the
environment was one of my least favorite changes ever.  Pointy stick,
always there to do some poking.

 I don't know. I could not live without ANT_OPTS. Or rather, instead of the environment valriable I'd be forced to use an alias with my ant settings. I don't see much of a difference between the two.

Cheers

 -- Martin