java conversions

On Sat, May 15, 2010 at 6:14 AM, Paul Phillips <paulp@improving.org> wrote:

An ArrayBuffer is a Buffer and an IndexedSeq, so r21934 creates some
unpleasantness.

I meant to have a look at this today, because that seems to be the problem with the Eclipse nightly build failures. The underlying problem is that the current JavaConversions design is explicit that Seq should not be convertible to javal.util.List. Instead there used to be a conversion with just Buffer, and now there is also one with IndexedSeq.

I'd like to do away with this, and make all Seqs convert into java.util.List.
The current ScalaDoc explanation says:

*   Note that no conversion is provided from
*   <code>scala.collection.immutable.List</code>
 *    to <code>java.util.List</code>. Instead it is convertible to an immutable
 *    <code>java.util.Collection</code> which provides size and interaction
 *    capabilities, but not access by index as would be provided by
 *    <code>java.util.List</code>.<br/>
 *    This is intentional: in combination the implementation of
 *    <code>scala.collection.immutable.List</code> and the typical usage
 *    patterns of <code>java.util.List</code> would perform extremely poorly.

I think that argument does not hold water. First, LinkedList is an implementation of java.util.List, so nowhere does java.util.List guarantee fast random access.
Second, ListBuffer is a Buffer, so the restriction does not prevent common implementations with linear cost indexing to be converted to java.util.List.

So, let's just replace IndexedSeq by Seq, and we will have no more problems. Miles, what do you think?

Cheers

 -- Martin

scala> import collection.JavaConversions._
import collection.JavaConversions._

scala> import collection.mutable.ArrayBuffer
import collection.mutable.ArrayBuffer

scala> var x: java.util.List[_] = ArrayBuffer(5)
<console>:9: error: ambiguous reference to overloaded definition,
both method asList in object JavaConversions of type [A](b: scala.collection.mutable.IndexedSeq[A])java.util.List[A]
and  method asList in object JavaConversions of type [A](b: scala.collection.mutable.Buffer[A])java.util.List[A]
match argument types (scala.collection.mutable.ArrayBuffer[Int]) and expected result type java.util.List[_]
      var x: java.util.List[_] = ArrayBuffer(5)
                                            ^

I guess things could be worse, although there is a rosy glow of
circuitousness in the air when in order to have one type "implicitly"
converted to a second type, we need specify a third, irrelevant type.

scala> var x: java.util.List[_] = (ArrayBuffer(5): IndexedSeq[Int])
x: java.util.List[_] = [5]

--
Paul Phillips      | All men are frauds.  The only difference between
Protagonist        | them is that some admit it.  I myself deny it.
Empiricist         |     -- H. L. Mencken
pull his pi pal!   |----------* http://www.improving.org/paulp/ *----------

On Sat, May 15, 2010 at 10:49 AM, martin odersky wrote:
> I think that argument does not hold water. First, LinkedList is an
> implementation of java.util.List, so nowhere does java.util.List guarantee
> fast random access.

True, but I'd be prepared to bet that most uses of java.util.List in
real code assume O(1) random access ... java.util.LinkedList is used
very infrequently relative to ArrayList and Arrays.asList.

> Second, ListBuffer is a Buffer, so the restriction does not prevent common
> implementations with linear cost indexing to be converted to java.util.List.
>
> So, let's just replace IndexedSeq by Seq, and we will have no more problems.
> Miles, what do you think?

The original design was based around an intentional restriction to
mutable types only, and round tripping of object identity through a
sequence of Scala => Java => Scala or Java => Scala => Java
conversions.

If we're dropping *both* of those then I really think it would be
worth reconsidering the entire library. Obviously it's late in the day
for 2.8, but it might be better to do that now than leave ourselves
with an unfortunate legacy.

Cheers,

Miles

On Sat, May 15, 2010 at 12:14 PM, Miles Sabin <miles@milessabin.com> wrote:

On Sat, May 15, 2010 at 10:49 AM, martin odersky <martin.odersky@epfl.ch> wrote:
> I think that argument does not hold water. First, LinkedList is an
> implementation of java.util.List, so nowhere does java.util.List guarantee
> fast random access.

True, but I'd be prepared to bet that most uses of java.util.List in
real code assume O(1) random access ... java.util.LinkedList is used
very infrequently relative to ArrayList and Arrays.asList.

> Second, ListBuffer is a Buffer, so the restriction does not prevent common
> implementations with linear cost indexing to be converted to java.util.List.
>
> So, let's just replace IndexedSeq by Seq, and we will have no more problems.
> Miles, what do you think?

The original design was based around an intentional restriction to
mutable types only, and round tripping of object identity through a
sequence of Scala => Java => Scala  or Java => Scala => Java
conversions.

If we're dropping *both* of those then I really think it would be
worth reconsidering the entire library. Obviously it's late in the day
for 2.8, but it might be better to do that now than leave ourselves
with an unfortunate legacy.

I am happy to discuss but I need something more specific than that as argument.

We discussed the round-tripping already. The List thing is a possible design, but I think we should not be in the business of erecting barriers against potential misuse which also throw out valid uses. If we do conversions we should try to get out of the way of programmers as much as possible, and not try to second guess their intentions.

I now understand better why people say that Clojure has better interop than Scala. I was always wondering about that argument because in principle it should make no difference whether one achieves this with implicits or with subtyping as Clojure does. But if we intentionally cripple conversions from the most common Scala types to the most common Java interfaces the pro Clojure argument makes more sense.

Cheers

 -- Martin

On Sat, May 15, 2010 at 11:31 AM, martin odersky wrote:
>> If we're dropping *both* of those then I really think it would be
>> worth reconsidering the entire library. Obviously it's late in the day
>> for 2.8, but it might be better to do that now than leave ourselves
>> with an unfortunate legacy.
>>
> I am happy to discuss but I need something more specific than that as
> argument.
>
> We discussed the round-tripping already. The List thing is a possible
> design, but I think we should not be in the business of erecting barriers
> against potential misuse which also throw out valid uses. If we do
> conversions we should try to get out of the way of programmers as much as
> possible, and not try to second guess their intentions.

That wasn't really intended as an argument, more as an observation.

I wrote JavaConversions with those constraints in place, and removing
them makes the shape of the library feel pretty unmotivated and ad
hoc.

I have absolutely no objections if you want to go that way, but if you
do, then I think it would be worth considering starting from a
different place.

Jorge has a fairly well developed Java collection interop library ...
why not encourage him to submit that as a replacement for
JavaConversions? FTR, I've suggested that to Jorge on more than one
occasion.

Cheers,

Miles

On Sat, May 15, 2010 at 12:47 PM, Miles Sabin <miles@milessabin.com> wrote:

On Sat, May 15, 2010 at 11:31 AM, martin odersky <martin.odersky@epfl.ch> wrote:
>> If we're dropping *both* of those then I really think it would be
>> worth reconsidering the entire library. Obviously it's late in the day
>> for 2.8, but it might be better to do that now than leave ourselves
>> with an unfortunate legacy.
>>
> I am happy to discuss but I need something more specific than that as
> argument.
>
> We discussed the round-tripping already. The List thing is a possible
> design, but I think we should not be in the business of erecting barriers
> against potential misuse which also throw out valid uses. If we do
> conversions we should try to get out of the way of programmers as much as
> possible, and not try to second guess their intentions.

That wasn't really intended as an argument, more as an observation.

I wrote JavaConversions with those constraints in place, and removing
them makes the shape of the library feel pretty unmotivated and ad
hoc.

I have absolutely no objections if you want to go that way, but if you
do, then I think it would be worth considering starting from a
different place.

Jorge has a fairly well developed Java collection interop library ...
why not encourage him to submit that as a replacement for
JavaConversions? FTR, I've suggested that to Jorge on more than one
occasion.

Jorge's design is a bit different because conversions are explicit, using asJava/asScala methods.
It's too late to change into that (and I am not sure it would be better). What I propose is take
the basic JavaConversions architecture but essentially support all the conversions that Jorge supports.

I had a look what the remaining differences are:

  java.lang.Comparable => scala.math.Ordered   // other direction works by subtyping
  java.util.Comparator => scala.math.Ordering      // other direction works by subtyping

Paul, how about adding these two implicit conversions in the Ordering/Ordered companion objects?

  scala.collection.mutable.Map <=> java.util.Dictionary

Miles, any reason not to support this? Right now there's a conversion from java.util.Properties to
scala's Map. Why not generalize to all dictionaries?

Cheers

 -- Martin




Miles

--
Miles Sabin
tel: +44 7813 944 528
gtalk: miles@milessabin.com
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

On Sat, May 15, 2010 at 12:19 PM, martin odersky wrote:
>   scala.collection.mutable.Map <=> java.util.Dictionary
>
> Miles, any reason not to support this? Right now there's a conversion from
> java.util.Properties to
> scala's Map. Why not generalize to all dictionaries?

From java.util.Dictionary's javadoc,

NOTE: This class is obsolete. New implementations should implement
the Map interface, rather than extending this class.

java.util.Properties is widely used enough to be worth special casing.

Cheers,

Miles

On Sat, May 15, 2010 at 1:31 PM, Miles Sabin <miles@milessabin.com> wrote:

On Sat, May 15, 2010 at 12:19 PM, martin odersky <martin.odersky@epfl.ch> wrote:
>   scala.collection.mutable.Map <=> java.util.Dictionary
>
> Miles, any reason not to support this? Right now there's a conversion from
> java.util.Properties to
> scala's Map. Why not generalize to all dictionaries?

From java.util.Dictionary's javadoc,

 NOTE: This class is obsolete. New implementations should implement
the Map interface, rather than extending this class.

Yes, but what if one needs interop with old implementations?  Maybe there are no important implementations left except Properties, but I do not want to be the judge of that.

My position would be again to go for maximum flexibility.

Cheers

 -- Martin

I'm about to board a flight, but I think I can make my library work
with both approaches (explicit and implicit conversions), depending on
the imported object. This could in theory make it API-compatible with
what's in trunk.

More info when I land.

--j

On Sat, May 15, 2010 at 6:19 AM, martin odersky wrote:

On Sat, May 15, 2010 at 12:47 PM, Miles Sabin wrote:

On Sat, May 15, 2010 at 11:31 AM, martin odersky wrote:
>> If we're dropping *both* of those then I really think it would be
>> worth reconsidering the entire library. Obviously it's late in the day
>> for 2.8, but it might be better to do that now than leave ourselves
>> with an unfortunate legacy.
>>
> I am happy to discuss but I need something more specific than that as
> argument.
>
> We discussed the round-tripping already. The List thing is a possible
> design, but I think we should not be in the business of erecting barriers
> against potential misuse which also throw out valid uses. If we do
> conversions we should try to get out of the way of programmers as much as
> possible, and not try to second guess their intentions.

That wasn't really intended as an argument, more as an observation.

I wrote JavaConversions with those constraints in place, and removing
them makes the shape of the library feel pretty unmotivated and ad
hoc.

I have absolutely no objections if you want to go that way, but if you
do, then I think it would be worth considering starting from a
different place.

Jorge has a fairly well developed Java collection interop library ...
why not encourage him to submit that as a replacement for
JavaConversions? FTR, I've suggested that to Jorge on more than one
occasion.

Jorge's design is a bit different because conversions are explicit,
using asJava/asScala methods.
It's too late to change into that (and I am not sure it would be
better). What I propose is take
the basic JavaConversions architecture but essentially support all the
conversions that Jorge supports.

I had a look what the remaining differences are:

  java.lang.Comparable => scala.math.Ordered   // other direction
works by subtyping
  java.util.Comparator => scala.math.Ordering      // other direction
works by subtyping

Paul, how about adding these two implicit conversions in the
Ordering/Ordered companion objects?

  scala.collection.mutable.Map <=> java.util.Dictionary

Miles, any reason not to support this? Right now there's a conversion
from java.util.Properties to
scala's Map. Why not generalize to all dictionaries?

Cheers

On Sun, May 16, 2010 at 1:11 AM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:

I'm about to board a flight, but I think I can make my library work
with both approaches (explicit and implicit conversions), depending on
the imported object. This could in theory make it API-compatible with
what's in trunk.

That would be an interesting thing to do. Another idea is to always have asJava/asScala
mappers and to have in addition implicits that can be imported from JavaConversions, as is the case now.

Cheers

 -- Martin

On Sun, May 16, 2010 at 9:12 AM, martin odersky wrote:
> On Sun, May 16, 2010 at 1:11 AM, Jorge Ortiz wrote:
>> I'm about to board a flight, but I think I can make my library work
>> with both approaches (explicit and implicit conversions), depending on
>> the imported object. This could in theory make it API-compatible with
>> what's in trunk.
>>
> That would be an interesting thing to do. Another idea is to always have
> asJava/asScala mappers and to have in addition implicits that can be imported
> from JavaConversions, as is the case now.

I think that that would be the right way to go.

Cheers,

Miles

+1
That would catch all my use cases


On 16 May 2010 09:22, Miles Sabin <miles@milessabin.com> wrote:

On Sun, May 16, 2010 at 9:12 AM, martin odersky <martin.odersky@epfl.ch> wrote:
> On Sun, May 16, 2010 at 1:11 AM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:
>> I'm about to board a flight, but I think I can make my library work
>> with both approaches (explicit and implicit conversions), depending on
>> the imported object. This could in theory make it API-compatible with
>> what's in trunk.
>>
> That would be an interesting thing to do. Another idea is to always have
> asJava/asScala mappers and to have in addition implicits that can be imported
> from JavaConversions, as is the case now.

I think that that would be the right way to go.

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
gtalk: miles@milessabin.com
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

--
Kevin Wright

mail/google talk: kev.lee.wright@googlemail.com
wave: kev.lee.wright@googlewave.com
skype: kev.lee.wright
twitter: @thecoda

>>>>> "martin" == martin odersky writes:

martin> That would be an interesting thing to do. Another idea is to
martin> always have asJava/asScala mappers and to have in addition
martin> implicits that can be imported from JavaConversions, as is the
martin> case now.

+1

On Sat, May 15, 2010 at 01:19:05PM +0200, martin odersky wrote:
> I had a look what the remaining differences are:
>
> java.lang.Comparable => scala.math.Ordered // other direction works by
> subtyping
> java.util.Comparator => scala.math.Ordering // other direction works
> by subtyping
>
> Paul, how about adding these two implicit conversions in the
> Ordering/Ordered companion objects?

Trying to catch up with stuff, I found I never replied to this. Each
time I find getting these ordered conversions to be very tricky, but I
think I have it. Do you want me to check it in or hang onto it?

Side note. Someone might think these two formulations are equivalent:

def by[T, S: Ordering](f: T => S): Ordering[T] =
fromLessThan((x, y) => implicitly[Ordering[S]].lt(f(x), f(y)))

def by[T, S](f: T => S)(implicit ord: Ordering[S]): Ordering[T] =
fromLessThan((x, y) => ord.lt(f(x), f(y)))

The former is the one currently in trunk; the latter is how I had to
rewrite it to get it to compile. The reason is this:

implicit def comparatorToOrdering[A](implicit cmp: Comparator[A]): Ordering[A]

But Ordering extends Comparator, so in addition to upconverting our
Comparators this catches all the Orderings which are already orderings.
Since it's in the low priority trait it's not normally in the way, but
"by" being also on Ordering cohabitates with it.

I don't think it's a problem, just mentioning it.