you say tomato, I say incomparable

On Fri, Jun 12, 2009 at 3:11 PM, Paul Phillips wrote:
> Since I'm sure nobody is tired of equality yet, here are a couple more
> ways to bypass scala's attempts to modify the boxed primitive equality
> semantics.  It's definitely a hornet's nest trying to infer much from a
> static type unless it's a final class.
>
>
> Ironically, declaring "Comparable" parameters renders them incomparable:
>
> scala> def boop(x1: Comparable[_], x2: Comparable[_]) = x1 == x2
> boop: (x1: java.lang.Comparable[_],x2: java.lang.Comparable[_])Boolean
>
> scala> boop(new java.lang.Integer(5), new java.lang.Long(5))
> res1: Boolean = false
>
>
> Or less ironically, Serializable:
>
> scala> def boop(x1: java.io.Serializable, x2: java.io.Serializable) = x1 == x2
> boop: (x1: java.io.Serializable,x2: java.io.Serializable)Boolean
>
> scala> boop(new java.lang.Integer(5), new java.lang.Long(5))
> res2: Boolean = false
>
Oops. I think that needs to be fixed. We need to test for numeric
types also for all
interfaces inherited by them. So the same logic that applies now for
Any and AnyRef
values must also apply to Comparable and Serializable values.

Cheers

On Fri, Jun 12, 2009 at 06:56:51PM +0200, martin odersky wrote:
> Oops. I think that needs to be fixed. We need to test for numeric
> types also for all interfaces inherited by them. So the same logic
> that applies now for Any and AnyRef values must also apply to
> Comparable and Serializable values.

This indeed is what I've already done. When all is said and done there
is quite a pile of logic, but among the ELEVEN (!) numeric(ish) classes
of interest, equality appears to do what one might naively expect.

"Eleven, what eleven?" I heard someone say. They are:

Byte, Short, Int, Long, Character(*), Float, Double,
scala.BigDecimal, scala.BigInt,
java.math.BigDecimal, java.math.BigInteger

(*) I would totally be in favor of excluding Character, however it was
already in the mix and I just work here:
scala> (new java.lang.Character(55.toChar)) == 55
res0: Boolean = true

One might be tempted to punt on the unwrapped Big* classes, but I find
it impossible to justify going to the lengths necessary for the first
nine and leaving a situation where BigInt(0).bigInteger != 0.

This test passes now.

val zeros: Map[String, AnyRef] = Map(
"scala.BigDecimal" -> BigDecimal(0),
"scala.BigInt" -> BigInt(0),
"java.BigDecimal" -> new java.math.BigDecimal(0),
"java.BigInteger" -> new java.math.BigInteger("0"),
"Double" -> new java.lang.Double(0.0d),
"Float" -> new java.lang.Float(0.0f),
"Long" -> new java.lang.Long(0L),
"Integer" -> new java.lang.Integer(0),
"Short" -> new java.lang.Short(0.toShort),
"Byte" -> new java.lang.Byte(0.toByte),
"Char" -> new java.lang.Character(0.toChar)
)

for ((n1, z1) <- zeros ; (n2, z2) <- zeros) {
assert(z1 == z2 && z2 == z1, n1 + " not equal to " + n2)
}

I presume/hope/pray we draw a line in the sand here, and all other
present and future subclasses of Number are utterly on their own.

Hi Paul,

I don't really care for Java's BigInt and BigDecimal. The reason we
want to be correct for java.lang.Character/Integer/Long and friends is
that these are the boxed versions of Scala types. So in fact, it might
beven be slightly better to have no special magic for Java's BigInt
and BigDecimal. On the other hand, it would be really nice of future
implemenetaions of Number form the Scala side could be makde to behave
correctly.

What do you think?

It's important for BigDecimal to have its own equality: only really makes sense of the scales are
the same, too. And any floating point equality definition which doesn't contains a delta is
profoundly dubious.

~~ Robert.

Paul Phillips wrote:
> On Fri, Jun 12, 2009 at 06:56:51PM +0200, martin odersky wrote:
>> Oops. I think that needs to be fixed. We need to test for numeric
>> types also for all interfaces inherited by them. So the same logic
>> that applies now for Any and AnyRef values must also apply to
>> Comparable and Serializable values.
>
> This indeed is what I've already done. When all is said and done there
> is quite a pile of logic, but among the ELEVEN (!) numeric(ish) classes
> of interest, equality appears to do what one might naively expect.
>
> "Eleven, what eleven?" I heard someone say. They are:
>
> Byte, Short, Int, Long, Character(*), Float, Double,
> scala.BigDecimal, scala.BigInt,
> java.math.BigDecimal, java.math.BigInteger
>
> (*) I would totally be in favor of excluding Character, however it was
> already in the mix and I just work here:
> scala> (new java.lang.Character(55.toChar)) == 55
> res0: Boolean = true
>
> One might be tempted to punt on the unwrapped Big* classes, but I find
> it impossible to justify going to the lengths necessary for the first
> nine and leaving a situation where BigInt(0).bigInteger != 0.
>
> This test passes now.
>
> val zeros: Map[String, AnyRef] = Map(
> "scala.BigDecimal" -> BigDecimal(0),
> "scala.BigInt" -> BigInt(0),
> "java.BigDecimal" -> new java.math.BigDecimal(0),
> "java.BigInteger" -> new java.math.BigInteger("0"),
> "Double" -> new java.lang.Double(0.0d),
> "Float" -> new java.lang.Float(0.0f),
> "Long" -> new java.lang.Long(0L),
> "Integer" -> new java.lang.Integer(0),
> "Short" -> new java.lang.Short(0.toShort),
> "Byte" -> new java.lang.Byte(0.toByte),
> "Char" -> new java.lang.Character(0.toChar)
> )
>
> for ((n1, z1) <- zeros ; (n2, z2) <- zeros) {
> assert(z1 == z2 && z2 == z1, n1 + " not equal to " + n2)
> }
>
> I presume/hope/pray we draw a line in the sand here, and all other
> present and future subclasses of Number are utterly on their own.
>

On Fri, Jun 12, 2009 at 07:48:50PM +0200, martin odersky wrote:
> I don't really care for Java's BigInt and BigDecimal. The reason we
> want to be correct for java.lang.Character/Integer/Long and friends is
> that these are the boxed versions of Scala types. So in fact, it might
> beven be slightly better to have no special magic for Java's BigInt
> and BigDecimal.

The problem is that in order to have any hope of symmetry and
transitivity holding, everything which is ever acknowledged as equal to
the same value in another class must always be equal to the same value
in all of the other classes. In order to not have blatant internal
contradictions in numeric equality, everyone who ever works together
must always work together.

Before my most recent commit here was BigInt's equality method:

override def equals (that: Any): Boolean = that match {
case that: BigInt => this equals that
case that: java.lang.Double => this.bigInteger.doubleValue == that.doubleValue
case that: java.lang.Float => this.bigInteger.floatValue == that.floatValue
case that: java.lang.Number => this equals BigInt(that.longValue)
case that: java.lang.Character => this equals BigInt(that.charValue.asInstanceOf[Int])
case _ => false
}

Now ignoring that this is super broken because it pulls a
java.math.BigInteger through longValue and then back to BigInt, this is
clearly trying to be equal to other java.lang.Numbers such as
java.math.BigInteger, as well as all the primitives. Same for BigDecimal:

override def equals(that: Any): Boolean = that match {
case that: BigDecimal => this equals that
case that: java.lang.Double => this.bigDecimal.doubleValue == that.doubleValue
case that: java.lang.Float => this.bigDecimal.floatValue == that.floatValue
case that: java.lang.Number => this equals BigDecimal(that.longValue)
case that: java.lang.Character => this equals BigDecimal(that.charValue.asInstanceOf[Int])
case _ => false
}

(Both these methods look quite a bit different now as I took what I read
as the intent and tried to make it correct.)

But we cannot do anything like what these are doing and then not have
the wrapper types take equivalent measures. Otherwise

scala.BigInt(0) == java.math.BigInteger(0)
scala.BigInt(0) == java.lang.Integer(0)
java.math.BigInteger(0) != java.lang.Integer(0)

Now if we want to say scala.BigDecimal(0) != java.math.BigDecimal(0) and
so on, we can do that. But they have to be fully excluded.

As I believe I have already achieved something like the maximum possible
internal consistency among the 11 aforementioned types, I am in favor of
keeping the unwrapped type logic in there.

> On the other hand, it would be really nice of future implemenetaions
> of Number form the Scala side could be makde to behave correctly.

This is impossible in a general sense because the "Number" interface
only provides potentially lossy methods. If you have two Numbers you
can sometimes tell that they're NOT equal, but you can't ever be sure
they're equal because who knows what it's returning for longValue() or
doubleValue(), the real value or some overflow or rounding error?

scala> def cmp(x: Number, y: Number) = x.longValue == y.longValue
cmp: (x: java.lang.Number,y: java.lang.Number)Boolean

scala> cmp(BigInt("1337681181807494279"), BigInt("394853894578934759834759"))
res1: Boolean = true

scala> def cmp(x: Number, y: Number) = x.doubleValue == y.doubleValue
cmp: (x: java.lang.Number,y: java.lang.Number)Boolean

scala> cmp(Math.MAX_LONG, Math.MAX_LONG - 500)
res3: Boolean = true

But it'd be possible to just special case those future scala Number
types like BigInt and BigDecimal already have to be.

> The problem is that in order to have any hope of symmetry and
> transitivity holding, everything which is ever acknowledged as equal to
> the same value in another class must always be equal to the same value
> in all of the other classes. In order to not have blatant internal
> contradictions in numeric equality, everyone who ever works together
> must always work together.
>
...which is why inheritance destroys the capability of implementing an on-object #equals. The whole
thing is totally broke, and it was a mistake for Java to create Object#equals in the first place.

Commented about this on my blog a few times:
* http://enfranchisedmind.com/blog/posts/a-java-gotcha/
* http://snipr.com/jztrd

~~ Robert Fischer, Smokejumper IT Consulting.
Enfranchised Mind Blog http://EnfranchisedMind.com/blog

Check out my book, "Grails Persistence with GORM and GSQL"!
http://www.smokejumperit.com/redirect.html

Thanks for the explanations. I think it's a toss up to either fully
include or exclude Java's BigInt and BigDecimal. If we can include
them without a cost in performance, I'm OK with it. I see your point
about lossy comparisons. But there might be a way around it. For
instance, we could try a double dispatch scheme: If the other type is
a subtype of ScalaNumber, and we do not have it in our list of
comparisons, forward to its equals method. Seomthing like that might
work.

Cheers

I don't suppose there's any chance we could consider simplifying the
whole thing and not trying to force different numeric types to be
equal at all?

I know it's a breaking change, but equality between different numeric
types is almost never what you want, destroys transitivity of
equality, is evidently extremely subtle to get right and (as we've
seen by the recent compiler results) can really hurt performance. It
would be nice if we could go back to the cleaner approach of only
allowing things of the same type to be equal.

On Fri, Jun 12, 2009 at 10:19:35PM +0100, David MacIver wrote:
> I don't suppose there's any chance we could consider simplifying the
> whole thing and not trying to force different numeric types to be
> equal at all?

That's fine, if they're incomparable.

If (0 == 0L) is a type error, fine.

If they're equal, fine.

But if it's silently false, that's simply unacceptable -- even if it
wouldn't break a metric ton of code.

And as long as we are using java's equals and java's boxes, a type error
isn't going to fly. By process of elimination, we have to do something
like what we're doing.

So this all goes in circles. I'm in full agreement that one has to hold
one's nose to look too closely at the situation, and as I've said before
I'd rather use some other operator which isn't shackled to java. But at
that point you're solving some other problem, not the problem we have.