unpickler question

On Sun, Nov 15, 2009 at 5:31 PM, Paul Phillips wrote:
> While optimizing for the optimizer I noticed this in the UnPickler:
>
>        case PACKAGEtree =>
>          val symbol = readSymbolRef()
>          val pid = readTreeRef().asInstanceOf[RefTree]
>          val stats = until(end, readTreeRef)
>          PackageDef(pid, stats) setType tpe
>
> Notice the symbol is discarded.  In all other cases when a symbol is
> read it is applied to the tree with setSymbol.  This is a bug, right?
> And if it's not, it needs a comment as to why it's the only tree who
> doesn't want its unpickled symbol.
>
It's probably a bug, but I let Lex have the last word on this.
Pickling is actually pretty easy, I find. What's upsetting you most?

Cheers

On Sun, Nov 15, 2009 at 05:45:00PM +0100, martin odersky wrote:
> It's probably a bug, but I let Lex have the last word on this.
> Pickling is actually pretty easy, I find. What's upsetting you most?

For one thing there is no verifiable correspondence between the pickler
and unpickler. The bugs which materialize seem to be very mysterious;
or at least I struggle to pinpoint the issue, and they often remain open
a long time (cf #373737.) In most areas of the compiler it seems like there
is one person who is basically authoritative about how it's supposed to
work and (more importantly) exactly what the current code actually does,
but if such a person exists for the pickler I have not figured out who
it is.

My blundering attempts to tackle pickler issues have resulted in such
great moments as "scala 2.7.6" so I'm a little shy in the vicinity.
Here is one I worked around to solve the immediate issue but whose
ticket I left open:

http://lampsvn.epfl.ch/trac/scala/ticket/2323

Below is the patch. I can't believe this is the right fix, but it did
address the reported problem and it hasn't (obviously) broken anything
else in the two months it's been in there, and gilles indicated he
didn't have a better plan at present: but the recurring dream is that
all development comes to a halt for days over some completely mysterious
issue which is eventually tracked back to this patch, and I have to
change my identity and go off the grid.

- val clazz = recycle(owner.newRefinementClass(NoPosition))
+ // having $anonfun as owner causes the pickler to break upon unpickling; see ticket #2323
+ val nonAnonOwner = (owner.ownerChain dropWhile (_.isAnonymousFunction)).headOption getOrElse NoSymbol
+ val clazz = recycle(nonAnonOwner.newRefinementClass(NoPosition))

In a nutshell, what upsets me (and I should clarify I don't actually get
upset: I only invoke emotions rhetorically in an attempt to mimic the
way the hoo-mans use them) is that I know how frustrating these bugs are
to come across for a user, but even when I can fix them I don't know how
to convince myself I'm fixing them correctly. And if we have our eye on
achieving binary compatibility someday, this region is likely to rise in
prominence and complexity.