Parameterized types, maps and Manifests

On Wed, Jun 16, 2010 at 03:35:16PM -0700, richard emberson wrote:
> I'd like to put some parameterized types into a map and then when I
> pull them out again, have them "know" what type there were originally.

You haven't specified your requirements very precisely. There are
numerous ways to implement a heterogeneous map. But if a particular
key is always associated with values of a particular type, the most
straightforward way would be to record that type in the type of the
key:

final class Key[T]

class Map {
val hm = new HashMap[Any, Any]

def put[T](key : Key[T], value : T) {
this.hm.put(key, value)
}

def get[T](key : Key[T]) : T =
this.get(key).asInstanceOf[T]
}

This is safe because keys are always compared by identity, and each
key is associated with a unique type.

This approach may or may not work for you. You had better explain in
more detail what you are trying to accompish.

Lauri

Lauri Alanko schrieb:

20100617000230 [dot] GE3128 [at] svm-18 [dot] cs [dot] helsinki [dot] fi" type="cite">

On Wed, Jun 16, 2010 at 03:35:16PM -0700, richard emberson wrote:
  

I'd like to put some parameterized types into a map and then when I
pull them out again, have them "know" what type there were originally.
    

You haven't specified your requirements very precisely. There are
numerous ways to implement a heterogeneous map. But if a particular
key is always associated with values of a particular type, the most
straightforward way would be to record that type in the type of the
key:

final class Key[T]

class Map {
  val hm = new HashMap[Any, Any]
  
  def put[T](key : Key[T], value : T) {
    this.hm.put(key, value)
  }
  
  def get[T](key : Key[T]) : T =
    this.get(key).asInstanceOf[T]
}

This is safe because keys are always compared by identity, and each
key is associated with a unique type.
  

I have a question to this: shouldn't the type parameter of Key be erased? How comes that it is still passed to the get method to cast the value?

The return type is T, so you have to cast to convince the compiler
that the type of the return object matches the specified return type.

On Thu, Jun 17, 2010 at 1:37 PM, Daniel Degrandi
wrote:
> Lauri Alanko schrieb:
>
> On Wed, Jun 16, 2010 at 03:35:16PM -0700, richard emberson wrote:
>
>
> I'd like to put some parameterized types into a map and then when I
> pull them out again, have them "know" what type there were originally.
>
>
> You haven't specified your requirements very precisely. There are
> numerous ways to implement a heterogeneous map. But if a particular
> key is always associated with values of a particular type, the most
> straightforward way would be to record that type in the type of the
> key:
>
> final class Key[T]
>
> class Map {
> val hm = new HashMap[Any, Any]
>
> def put[T](key : Key[T], value : T) {
> this.hm.put(key, value)
> }
>
> def get[T](key : Key[T]) : T =
> this.get(key).asInstanceOf[T]
> }
>
> This is safe because keys are always compared by identity, and each
> key is associated with a unique type.
>
>
> I have a question to this: shouldn't the type parameter of Key be erased?
> How comes that it is still passed to the get method to cast the value?
>

As far as I understand things:

The type parameter is erased at runtime.
But it is "passed" to the method at compile time, where T is known.

-
Andreas

Am 17.06.2010 um 19:37 schrieb Daniel Degrandi:
> Lauri Alanko schrieb:
>> final class Key[T]
>>
>> class Map {
>> val hm = new HashMap[Any, Any]
>>
>> def put[T](key : Key[T], value : T) {
>> this.hm.put(key, value)
>> }
>>
>> def get[T](key : Key[T]) : T =
>> this.get(key).asInstanceOf[T]
>> }
>>
> I have a question to this: shouldn't the type parameter of Key be erased? How comes that it is still passed to the get method to cast the value?

Andreas Flierl schrieb:

BDA82DA1-DC92-4E3B-AF76-D6B480254DC0 [at] flierl [dot] eu" type="cite">

As far as I understand things:

The type parameter is erased at runtime.
But it is "passed" to the method at compile time, where T is known.
  

I think this is one of the many thing I'l have to put and memorize into the category "that's the way it is", without deeply understanding it.
Especially because:

this works (and throws a ClassCastException if the two don't match):

    def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]

this never works because of type erasure:

    def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]

I'm sure there is a rational for this.. or maybe not.... :-(

Dan

BDA82DA1-DC92-4E3B-AF76-D6B480254DC0 [at] flierl [dot] eu" type="cite">

-
Andreas

Am 17.06.2010 um 19:37 schrieb Daniel Degrandi:
  

Lauri Alanko schrieb:
    

final class Key[T]

class Map {
  val hm = new HashMap[Any, Any]
  
  def put[T](key : Key[T], value : T) {
    this.hm.put(key, value)
  }
  
  def get[T](key : Key[T]) : T =
    this.get(key).asInstanceOf[T]
}
 
      

I have a question to this: shouldn't the type parameter of Key be erased? How comes that it is still passed to the get method to cast the value?
    

i don't see a difference

> I think this is one of the many thing I'l have to put and memorize
> into the category "that's the way it is", without deeply understanding it.
> Especially because:
>
> this works (and throws a ClassCastException if the two don't match):
>
> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>
> this never works because of type erasure:
>
> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>
> I'm sure there is a rational for this.. or maybe not.... :-(
>
> Dan

On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
> I think this is one of the many thing I'l have to put and memorize
> into the category "that's the way it is", without deeply understanding
> it. Especially because:
>
> this works (and throws a ClassCastException if the two don't match):
>
> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>
> this never works because of type erasure:
>
> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]

Your finding this surprising is an indication that your model of what is
going on is inaccurate. I don't know if you need to deeply understand
it, but if you're going to keep programming on the jvm you probably
don't want to throw up your hands about erasure.

The reason the first one works is that every time cast is called, some
type is determined for T. At compile time. Nothing is running yet. So
if you pass something scala knows is a Key[String] as the first
argument, it will cast the second to a String.

The reason the second one doesn't is that you are asking for information
which does not exist, because there is no T. That value could be
anything, and so could type parameter T, but an instance check has to
compile to something concrete. There's no there there.

The more interesting part of erasure isn't even touched on in the above,
and that is in x2 below: it does not issue a CCE, because the JVM
doesn't see any difference between a List[Int] and a List[String].
Until you try to get something out of it, anyway.

object Test {
trait Key[T]
def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]

def f = {
val x1 = cast(new Key[String] {}, "abc")
println(x1)

val x2 = cast(new Key[List[Int]] {}, List("abc"))
println(x2)

val x3 = cast(new Key[Float] {}, "abc")
println(x3)
}

def main(args: Array[String]): Unit = f
}

// output
abc
List(abc)
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Float

The bytecode knows all, sees all:

8: ldc #34; //String abc
10: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
13: checkcast #38; //class java/lang/String

56: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
59: checkcast #67; //class scala/collection/immutable/List

78: ldc #34; //String abc
80: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
83: invokestatic #76; //Method scala/runtime/BoxesRunTime.unboxToFloat:(Ljava/lang/Object;)F

now i see it. and i'm puzzled.

HamsterofDeath schrieb:
> i don't see a difference
>
>
>> I think this is one of the many thing I'l have to put and memorize
>> into the category "that's the way it is", without deeply understanding it.
>> Especially because:
>>
>> this works (and throws a ClassCastException if the two don't match):
>>
>> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>>
>> this never works because of type erasure:
>>
>> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>>
>> I'm sure there is a rational for this.. or maybe not.... :-(
>>
>> Dan
>>
>
>

Paul Phillips schrieb:
> On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
>
>> I think this is one of the many thing I'l have to put and memorize
>> into the category "that's the way it is", without deeply understanding
>> it. Especially because:
>>
>> this works (and throws a ClassCastException if the two don't match):
>>
>> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>>
>> this never works because of type erasure:
>>
>> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>>
>
> Your finding this surprising is an indication that your model of what is
> going on is inaccurate. I don't know if you need to deeply understand
> it, but if you're going to keep programming on the jvm you probably
> don't want to throw up your hands about erasure.
>
> The reason the first one works is that every time cast is called, some
> type is determined for T. At compile time. Nothing is running yet. So
> if you pass something scala knows is a Key[String] as the first
> argument, it will cast the second to a String.
>
> The reason the second one doesn't is that you are asking for information
> which does not exist, because there is no T. That value could be
> anything, and so could type parameter T, but an instance check has to
> compile to something concrete. There's no there there.
>
>
where does the information about T come from in the first example, any
why doesn't this source apply to the second case? i don't see the
difference.

You might me surprised that this works:

object Works {
class Key[T]

def f = {
val key = new Key[Int]
val value = "abc"

println(cast(key, value)) // here's the only difference
}

def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
}

whereas this doesn't work:

object DoesNotWork {
class Key[T]

def f = {
val key = new Key[Int]
val value = "abc"

val x = cast(key, value) // here's the only difference
println(x)
}

def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
}

asInstanceOf can be checked at compile time as the type is known to
the compiler
but isInstanceOf cannot be known as the type T is only determined at
runtime (polymorphism anyone?)
but at runtime the Type is erase. So how to tell what value is an instance of?

-Stefan

2010/6/18 HamsterofDeath :
> Paul Phillips schrieb:
>> On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
>>
>>> I think this is one of the many thing I'l have to put and memorize
>>> into the category "that's the way it is", without deeply understanding
>>> it. Especially because:
>>>
>>> this works (and throws a ClassCastException if the two don't match):
>>>
>>>    def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>>>
>>> this never works because of type erasure:
>>>
>>>    def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>>>
>>
>> Your finding this surprising is an indication that your model of what is
>> going on is inaccurate.  I don't know if you need to deeply understand
>> it, but if you're going to keep programming on the jvm you probably
>> don't want to throw up your hands about erasure.
>>
>> The reason the first one works is that every time cast is called, some
>> type is determined for T.  At compile time.  Nothing is running yet.  So
>> if you pass something scala knows is a Key[String] as the first
>> argument, it will cast the second to a String.
>>
>> The reason the second one doesn't is that you are asking for information
>> which does not exist, because there is no T.  That value could be
>> anything, and so could type parameter T, but an instance check has to
>> compile to something concrete.  There's no there there.
>>
>>
> where does the information about T come from in the first example, any
> why doesn't this source apply to the second case? i don't see the
> difference.
>

On Fri, Jun 18, 2010 at 09:53:21AM +0200, Andreas Flierl wrote:
> You might me surprised that this works:

There are subtleties in this example which are tangential to the erasure
question. In this case it's the use of "Any", which doesn't exist on
the jvm and requires scala to box and unbox everything. The CCE takes
place when scala unboxes the return value of cast, but when you call
println with the cast expression as an argument it doesn't bother
unboxing it since it's about to box it up again anyway.

Paul Phillips schrieb:
> Your finding this surprising is an indication that your model of what is
> going on is inaccurate. I don't know if you need to deeply understand
> it, but if you're going to keep programming on the jvm you probably
> don't want to throw up your hands about erasure.
>
I am pretty sure my model of what is going on is _VERY_ inaccurate ;-)

> The reason the first one works is that every time cast is called, some
> type is determined for T. At compile time. Nothing is running yet. So
> if you pass something scala knows is a Key[String] as the first
> argument, it will cast the second to a String.
>
> The reason the second one doesn't is that you are asking for information
> which does not exist, because there is no T. That value could be
> anything, and so could type parameter T, but an instance check has to
> compile to something concrete. There's no there there.
>
> The more interesting part of erasure isn't even touched on in the above,
> and that is in x2 below: it does not issue a CCE, because the JVM
> doesn't see any difference between a List[Int] and a List[String].
> Until you try to get something out of it, anyway.
>
> object Test {
> trait Key[T]
> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>
> def f = {
> val x1 = cast(new Key[String] {}, "abc")
> println(x1)
>
> val x2 = cast(new Key[List[Int]] {}, List("abc"))
> println(x2)
>
> val x3 = cast(new Key[Float] {}, "abc")
> println(x3)
> }
>
> def main(args: Array[String]): Unit = f
> }
>
> // output
> abc
> List(abc)
> java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Float
>

Well, I know that no type whatsoever is ever hold at runtime. No issue
here. I can see how a cast can be done at compile time, since the
compiler has all the information at hand. But why can't a type check be
made at compile time:

object Test {
trait Key[T]
def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]

def f = {
val x1 = cast(new Key[String] {}, "abc")
println(x1)

val x2 = compare(new Key[Int] {}, "abc")
println(x2)
}
def main(args: Array[String]): Unit = f
}

Every information is there. If I can see that I'm comparing an Int with
a String, then the compiler can see it. So, I know that type checks are
made at runtime, but beyond the "that's how it is" fact, is there a
strict requirement for this?

Dan

> The bytecode knows all, sees all:
>
> 8: ldc #34; //String abc
> 10: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
> 13: checkcast #38; //class java/lang/String
>
> 56: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
> 59: checkcast #67; //class scala/collection/immutable/List
>
> 78: ldc #34; //String abc
> 80: invokevirtual #36; //Method cast:(LTest$Key;Ljava/lang/Object;)Ljava/lang/Object;
> 83: invokestatic #76; //Method scala/runtime/BoxesRunTime.unboxToFloat:(Ljava/lang/Object;)F
>
>

this doesn't make sense. if the compiler knows what T is at compile
time, then both should work because the compiler can just replace T with
the actual type. if the compiler doesn't know, both can't work.

Stefan Langer schrieb:
> asInstanceOf can be checked at compile time as the type is known to
> the compiler
> but isInstanceOf cannot be known as the type T is only determined at
> runtime (polymorphism anyone?)
> but at runtime the Type is erase. So how to tell what value is an instance of?
>
> -Stefan
>
> 2010/6/18 HamsterofDeath :
>
>> Paul Phillips schrieb:
>>
>>> On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
>>>
>>>
>>>> I think this is one of the many thing I'l have to put and memorize
>>>> into the category "that's the way it is", without deeply understanding
>>>> it. Especially because:
>>>>
>>>> this works (and throws a ClassCastException if the two don't match):
>>>>
>>>> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>>>>
>>>> this never works because of type erasure:
>>>>
>>>> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>>>>
>>>>
>>> Your finding this surprising is an indication that your model of what is
>>> going on is inaccurate. I don't know if you need to deeply understand
>>> it, but if you're going to keep programming on the jvm you probably
>>> don't want to throw up your hands about erasure.
>>>
>>> The reason the first one works is that every time cast is called, some
>>> type is determined for T. At compile time. Nothing is running yet. So
>>> if you pass something scala knows is a Key[String] as the first
>>> argument, it will cast the second to a String.
>>>
>>> The reason the second one doesn't is that you are asking for information
>>> which does not exist, because there is no T. That value could be
>>> anything, and so could type parameter T, but an instance check has to
>>> compile to something concrete. There's no there there.
>>>
>>>
>>>
>> where does the information about T come from in the first example, any
>> why doesn't this source apply to the second case? i don't see the
>> difference.
>>
>>
>
>

On Fri, Jun 18, 2010 at 10:13:03AM +0200, Daniel Degrandi wrote:
> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>
> Every information is there. If I can see that I'm comparing an Int
> with a String, then the compiler can see it.

No, it can't. "Static typing" means something specific. In the compare
function above, T can represent any type. And value could be of any
type. What bytecode do you imagine it generating for that method which
will include checks involving Ints and Strings?

> So, I know that type checks are made at runtime, but beyond the
> "that's how it is" fact, is there a strict requirement for this?

An "instance check" is an inherently runtime activity. If you want to
do something based on types at compile time, you don't have to dance
around it. Call it whatever type it is. It's like asking your mom what
your own name is. You're supposed to already know it.

On Fri, Jun 18, 2010 at 10:10 AM, HamsterofDeath wrote:
> this doesn't make sense. if the compiler knows what T is at compile
> time, then both should work because the compiler can just replace T with
> the actual type. if the compiler doesn't know, both can't work.

Everything that Paul said is right. Let's add another bit which might
help you. Look again at the example:

On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
> this works (and throws a ClassCastException if the two don't match):
>
> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]

The asInstanceOf does in fact nothing at runtime here. It's just there
to calm down the compiler. We know all types are erased so every thing
is Object here behind the scenes. That's the way all boils down to, if
you are writing generic methods on a runtime with erasure like the
JVM.

*But*: at the call-site the compiler can infer the necessary bits and
checks its assertions by inserting a cast there. (That's the same
behaviour as with Java generics, of course.)

> this never works because of type erasure:
>
> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
As before, a generic method has no additional type information at
runtime => can't work.

so both times, T is not available and the cast is "added from the
outside, and "added from the outside" does not work for inInstanceOf
since it's inside the method?

Johannes Rudolph schrieb:
> On Fri, Jun 18, 2010 at 10:10 AM, HamsterofDeath wrote:
>
>> this doesn't make sense. if the compiler knows what T is at compile
>> time, then both should work because the compiler can just replace T with
>> the actual type. if the compiler doesn't know, both can't work.
>>
>
> Everything that Paul said is right. Let's add another bit which might
> help you. Look again at the example:
>
> On Fri, Jun 18, 2010 at 08:16:31AM +0200, Daniel Degrandi wrote:
>
>> this works (and throws a ClassCastException if the two don't match):
>>
>> def cast[T](k: Key[T], value: Any) = value.asInstanceOf[T]
>>
>
> The asInstanceOf does in fact nothing at runtime here. It's just there
> to calm down the compiler. We know all types are erased so every thing
> is Object here behind the scenes. That's the way all boils down to, if
> you are writing generic methods on a runtime with erasure like the
> JVM.
>
> *But*: at the call-site the compiler can infer the necessary bits and
> checks its assertions by inserting a cast there. (That's the same
> behaviour as with Java generics, of course.)
>
>
>> this never works because of type erasure:
>>
>> def compare[T](k: Key[T], value: Any) = value.isInstanceOf[T]
>>
> As before, a generic method has no additional type information at
> runtime => can't work.
>
>
>

On Fri, Jun 18, 2010 at 10:58 AM, HamsterofDeath wrote:
> so both times, T is not available and the cast is "added from the
> outside, and "added from the outside" does not work for inInstanceOf
> since it's inside the method?

Yes. But one should note, that the cast in the bytecodes at the
call-site is independent of the implementation (it has to be, of
course). It is just a usual component of working around erasure. The
asInstanceOf in the implementation is after erasure just a no-op. Java
emits a warning for this case not sure about scala.

Quite a subtlety, that these two independent aspects of erasure - in
this case - add up to do what one would expect.

Paul Phillips schrieb:
> An "instance check" is an inherently runtime activity. If you want to
> do something based on types at compile time, you don't have to dance
> around it. Call it whatever type it is. It's like asking your mom what
> your own name is. You're supposed to already know it.
>
>

Johannes Rudolph schrieb:
> The asInstanceOf does in fact nothing at runtime here. It's just there
> to calm down the compiler. We know all types are erased so every thing
> is Object here behind the scenes. That's the way all boils down to, if
> you are writing generic methods on a runtime with erasure like the
> JVM.

Thank you. These explanations made it quite clear for me. And helps me
to improve the way I might design things.

This list and its contributors are great!

Dan

On Thu, Jun 17, 2010 at 07:37:31PM +0200, Daniel Degrandi wrote:
> > def get[T](key : Key[T]) : T =
> > this.get(key).asInstanceOf[T]

> I have a question to this: shouldn't the type parameter of Key be
> erased? How comes that it is still passed to the get method to cast
> the value?

That's a good observation.

It shouldn't come as a surprise that you can use a type parameter as a
type argument within the body of a polymorphic method. That is very
commonplace, and both the parameter and the argument are ultimately
erased.

However, asInstanceOf is of course a bit special, as it is _not_
always a (parametrically) polymorphic method: sometimes the run-time
behavior of the operation depends on the type argument: the argument
type is used for a run-time instanceof check to make sure that the
cast is legal. You are right that type erasure prevents this when the
argument to asInstanceOf contains type variables. However, instead of
making the operation illegal in such a situation, the run-time check
is simply elided (or more precisely, only the _erasure_ of the type
argument is used for the run-time check).

The result is that asInstanceOf behaves in a very unexpected fashion:

scala> val x : Any = "foo".asInstanceOf[Int]
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer

scala> def cast[T](a : Any) = a.asInstanceOf[T]
cast: [T](a: Any)T

scala> val x : Any = cast[Int]("foo")
x: Any = foo

This is admittedly confusing. I have previously argued that the
different uses of .asInstanceOf ought to be expressed in different
ways to clarify the situation:

http://article.gmane.org/gmane.comp.lang.scala.user/11362

In particular, I don't think .asInstanceOf should be used for any
run-time checks, simply because the very syntax of the operation
suggests that it is parametric, and that the type argument gets
erased.

Lauri