the mayor of xmltown

Fix it now.  I'm all for API breakage if we can get this issue fixed.

On Mon, Nov 30, 2009 at 8:54 PM, Paul Phillips <paulp@improving.org> wrote:

We're so close to having consistent equality, it seems like a shame to
ship 2.8 with a whole segment of the standard library which violates not
only the equals/hashCode contract, but our own collections equality
contracts.  Unfortunately this looks inevitable unless I can get the OK
to smash up XML a bit.

Here is the irresolvable issue (which has been nothing but pain from the
word go) which cannot be worked around: most every class in the XML lib
descends from Seq[Node].  I'm sure this seemed like a good idea at the
time, but when a Node IS a Seq[Node], there is no hope.  Here is an
actual inheritance line:

 Seq[Node]
 NodeSeq
 Node
 SpecialNode
 Atom
 Text

Now the XML lib seems to depend on things flipflopping between Seq[Node]
and NodeSeq as the phase of the moon warrants, but it has other
expectations - let's look at NodeSeq's equals method:

 override def equals(x: Any): Boolean = x match {
   case z:Node      => (length == 1) && z == apply(0)
   case z:Seq[_]    => sameElements(z)
   case z:String    => text == z
   case _           => false
 }

Ha ha, right.

Hypothetically speaking, let's say we removed all the nutty
irreconcilable equals methods from the XML classes.  At that point
nothing would work at all, because the first time you tried to compare
two nodes you'd see this:

scala> new Atom(42) == new Atom(42)
java.lang.StackOverflowError
       at scala.xml.Node.theSeq(Node.scala:147)
       at scala.xml.NodeSeq.iterator(NodeSeq.scala:54)
       at scala.collection.IterableLike$class.sameElements(IterableLike.scala:325)
       at scala.xml.NodeSeq.sameElements(NodeSeq.scala:46)
       ...

This makes perfect sense, because of course two freaking Atoms are
sequences which should have their individual elements compared.  What
are those elements? Well let's see, there's just this one Atom -- but
hey that's a sequence... the infinite regression is avoided right now
through a delicate and totally inconsistent set of overrides.

I could fix all this, I think.  I might even be able to keep 2.7 code
working via implicits, just different ones.  It's not what I'm looking
to do at this point, but neither is shipping the XML lib in its current
form.  But I don't want to touch another line of XML unless I can first
rip Seq out of the object hierarchy and send it out of XMLtown on a
rail.  Or, maybe I should just chmod 0000 scala/xml and filter all the
bug reports to /dev/null, so I can stop being tempted.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Vivid              | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

+1. That's the right time to do so.
 ----------------------------------------------
Eric TORREBORRE
T +61 411 707 402
E etorreborre@yahoo.com
B http://etorreborre.blogspot.com
P http://specs.googlecode.com
----------------------------------------------

De : David Pollak <feeder.of.the.bears@gmail.com>
À : Paul Phillips <paulp@improving.org>
Cc : scala-internals@listes.epfl.ch
Envoyé le : Mar 1 Décembre 2009, 16 h 04 min 35 s
Objet : Re: [scala-internals] the mayor of xmltown

Fix it now.  I'm all for API breakage if we can get this issue fixed.

On Mon, Nov 30, 2009 at 8:54 PM, Paul Phillips <paulp [at] improving [dot] org" target="_blank" href="mailto:paulp@improving.org" rel="nofollow">paulp@improving.org> wrote:

We're so close to having consistent equality, it seems like a shame to
ship 2.8 with a whole segment of the standard library which violates not
only the equals/hashCode contract, but our own collections equality
contracts.  Unfortunately this looks inevitable unless I can get the OK
to smash up XML a bit.

Here is the irresolvable issue (which has been nothing but pain from the
word go) which cannot be worked around: most every class in the XML lib
descends from Seq[Node].  I'm sure this seemed like a good idea at the
time, but when a Node IS a Seq[Node], there is no hope.  Here is an
actual inheritance line:

 Seq[Node]
 NodeSeq
 Node
 SpecialNode
 Atom
 Text

Now the XML lib seems to depend on things flipflopping between Seq[Node]
and NodeSeq as the phase of the moon warrants, but it has other
expectations - let's look at NodeSeq's equals method:

 override def equals(x: Any): Boolean = x match {
   case z:Node      => (length == 1) && z == apply(0)
   case z:Seq[_]    => sameElements(z)
   case z:String    => text == z
   case _           => false
 }

Ha ha, right.

Hypothetically speaking, let's say we removed all the nutty
irreconcilable equals methods from the XML classes.  At that point
nothing would work at all, because the first time you tried to compare
two nodes you'd see this:

scala> new Atom(42) == new Atom(42)
java.lang.StackOverflowError
       at scala.xml.Node.theSeq(Node.scala:147)
       at scala.xml.NodeSeq.iterator(NodeSeq.scala:54)
       at scala.collection.IterableLike$class.sameElements(IterableLike.scala:325)
       at scala.xml.NodeSeq.sameElements(NodeSeq.scala:46)
       ...

This makes perfect sense, because of course two freaking Atoms are
sequences which should have their individual elements compared.  What
are those elements? Well let's see, there's just this one Atom -- but
hey that's a sequence... the infinite regression is avoided right now
through a delicate and totally inconsistent set of overrides.

I could fix all this, I think.  I might even be able to keep 2.7 code
working via implicits, just different ones.  It's not what I'm looking
to do at this point, but neither is shipping the XML lib in its current
form.  But I don't want to touch another line of XML unless I can first
rip Seq out of the object hierarchy and send it out of XMLtown on a
rail.  Or, maybe I should just chmod 0000 scala/xml and filter all the
bug reports to /dev/null, so I can stop being tempted.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Vivid              | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

+1, it's now or never.

On Tue, Dec 1, 2009 at 10:28 AM, Eric Torreborre <etorreborre@yahoo.com> wrote:

+1. That's the right time to do so.
 ----------------------------------------------
Eric TORREBORRE
T +61 411 707 402
E etorreborre@yahoo.com
B http://etorreborre.blogspot.com
P http://specs.googlecode.com
----------------------------------------------

De : David Pollak <feeder.of.the.bears@gmail.com>
À : Paul Phillips <paulp@improving.org>
Cc : scala-internals@listes.epfl.ch
Envoyé le : Mar 1 Décembre 2009, 16 h 04 min 35 s
Objet : Re: [scala-internals] the mayor of xmltown

Fix it now.  I'm all for API breakage if we can get this issue fixed.

On Mon, Nov 30, 2009 at 8:54 PM, Paul Phillips <paulp@improving.org> wrote:

We're so close to having consistent equality, it seems like a shame to
ship 2.8 with a whole segment of the standard library which violates not
only the equals/hashCode contract, but our own collections equality
contracts.  Unfortunately this looks inevitable unless I can get the OK
to smash up XML a bit.

Here is the irresolvable issue (which has been nothing but pain from the
word go) which cannot be worked around: most every class in the XML lib
descends from Seq[Node].  I'm sure this seemed like a good idea at the
time, but when a Node IS a Seq[Node], there is no hope.  Here is an
actual inheritance line:

 Seq[Node]
 NodeSeq
 Node
 SpecialNode
 Atom
 Text

Now the XML lib seems to depend on things flipflopping between Seq[Node]
and NodeSeq as the phase of the moon warrants, but it has other
expectations - let's look at NodeSeq's equals method:

 override def equals(x: Any): Boolean = x match {
   case z:Node      => (length == 1) && z == apply(0)
   case z:Seq[_]    => sameElements(z)
   case z:String    => text == z
   case _           => false
 }

Ha ha, right.

Hypothetically speaking, let's say we removed all the nutty
irreconcilable equals methods from the XML classes.  At that point
nothing would work at all, because the first time you tried to compare
two nodes you'd see this:

scala> new Atom(42) == new Atom(42)
java.lang.StackOverflowError
       at scala.xml.Node.theSeq(Node.scala:147)
       at scala.xml.NodeSeq.iterator(NodeSeq.scala:54)
       at scala.collection.IterableLike$class.sameElements(IterableLike.scala:325)
       at scala.xml.NodeSeq.sameElements(NodeSeq.scala:46)
       ...

This makes perfect sense, because of course two freaking Atoms are
sequences which should have their individual elements compared.  What
are those elements? Well let's see, there's just this one Atom -- but
hey that's a sequence... the infinite regression is avoided right now
through a delicate and totally inconsistent set of overrides.

I could fix all this, I think.  I might even be able to keep 2.7 code
working via implicits, just different ones.  It's not what I'm looking
to do at this point, but neither is shipping the XML lib in its current
form.  But I don't want to touch another line of XML unless I can first
rip Seq out of the object hierarchy and send it out of XMLtown on a
rail.  Or, maybe I should just chmod 0000 scala/xml and filter all the
bug reports to /dev/null, so I can stop being tempted.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Vivid              | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

--
Viktor Klang
| "A complex system that works is invariably
| found to have evolved from a simple system
| that worked." - John Gall

Blog: klangism.blogspot.com
Twttr: twitter.com/viktorklang
Code: github.com/viktorklang

Le 01/12/2009 10:28, Eric Torreborre a écrit :
> +1. That's the right time to do so.

For what it worths... +1

Paul Phillips wrote:
> Here is the irresolvable issue (which has been nothing but pain from the
> word go) which cannot be worked around: most every class in the XML lib
> descends from Seq[Node]. I'm sure this seemed like a good idea at the
> time, but when a Node IS a Seq[Node], there is no hope. Here is an
>
Another unintuitive side-effect of this scheme:

1. root \ "foo" flatMap { e => e \ "bar" }
2. root \ "foo" flatMap { e => e \ "@bar" }
3. root \ "foo" \ "bar"
4. root \ "foo" \ "@bar"

1 & 2 work naturally (\ only on NodeSeq). 3 only works because there's a
kludge in \ that makes it work. 4 fails (by not finding what you expect
it should find).

Short version: +1

-sz

On Tue, Dec 1, 2009 at 4:54 AM, Paul Phillips wrote:
> We're so close to having consistent equality, it seems like a shame to
> ship 2.8 with a whole segment of the standard library which violates not
> only the equals/hashCode contract, but our own collections equality
> contracts.  Unfortunately this looks inevitable unless I can get the OK
> to smash up XML a bit.

Thumbs up from me ...

Cheers,

Miles

+1

On Mon, Nov 30, 2009 at 11:54 PM, Paul Phillips <paulp@improving.org> wrote:

We're so close to having consistent equality, it seems like a shame to
ship 2.8 with a whole segment of the standard library which violates not
only the equals/hashCode contract, but our own collections equality
contracts.  Unfortunately this looks inevitable unless I can get the OK
to smash up XML a bit.

Here is the irresolvable issue (which has been nothing but pain from the
word go) which cannot be worked around: most every class in the XML lib
descends from Seq[Node].  I'm sure this seemed like a good idea at the
time, but when a Node IS a Seq[Node], there is no hope.  Here is an
actual inheritance line:

 Seq[Node]
 NodeSeq
 Node
 SpecialNode
 Atom
 Text

Now the XML lib seems to depend on things flipflopping between Seq[Node]
and NodeSeq as the phase of the moon warrants, but it has other
expectations - let's look at NodeSeq's equals method:

 override def equals(x: Any): Boolean = x match {
   case z:Node      => (length == 1) && z == apply(0)
   case z:Seq[_]    => sameElements(z)
   case z:String    => text == z
   case _           => false
 }

Ha ha, right.

Hypothetically speaking, let's say we removed all the nutty
irreconcilable equals methods from the XML classes.  At that point
nothing would work at all, because the first time you tried to compare
two nodes you'd see this:

scala> new Atom(42) == new Atom(42)
java.lang.StackOverflowError
       at scala.xml.Node.theSeq(Node.scala:147)
       at scala.xml.NodeSeq.iterator(NodeSeq.scala:54)
       at scala.collection.IterableLike$class.sameElements(IterableLike.scala:325)
       at scala.xml.NodeSeq.sameElements(NodeSeq.scala:46)
       ...

This makes perfect sense, because of course two freaking Atoms are
sequences which should have their individual elements compared.  What
are those elements? Well let's see, there's just this one Atom -- but
hey that's a sequence... the infinite regression is avoided right now
through a delicate and totally inconsistent set of overrides.

I could fix all this, I think.  I might even be able to keep 2.7 code
working via implicits, just different ones.  It's not what I'm looking
to do at this point, but neither is shipping the XML lib in its current
form.  But I don't want to touch another line of XML unless I can first
rip Seq out of the object hierarchy and send it out of XMLtown on a
rail.  Or, maybe I should just chmod 0000 scala/xml and filter all the
bug reports to /dev/null, so I can stop being tempted.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Vivid              | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

--
http://erikengbrecht.blogspot.com/

Wow,
In the entire history of Scala's libraries, this could very well be the least controversial change *ever*

On Tue, Dec 1, 2009 at 11:55 AM, Erik Engbrecht <erik.engbrecht@gmail.com> wrote:

+1

On Mon, Nov 30, 2009 at 11:54 PM, Paul Phillips <paulp@improving.org> wrote:

We're so close to having consistent equality, it seems like a shame to
ship 2.8 with a whole segment of the standard library which violates not
only the equals/hashCode contract, but our own collections equality
contracts.  Unfortunately this looks inevitable unless I can get the OK
to smash up XML a bit.

Here is the irresolvable issue (which has been nothing but pain from the
word go) which cannot be worked around: most every class in the XML lib
descends from Seq[Node].  I'm sure this seemed like a good idea at the
time, but when a Node IS a Seq[Node], there is no hope.  Here is an
actual inheritance line:

 Seq[Node]
 NodeSeq
 Node
 SpecialNode
 Atom
 Text

Now the XML lib seems to depend on things flipflopping between Seq[Node]
and NodeSeq as the phase of the moon warrants, but it has other
expectations - let's look at NodeSeq's equals method:

 override def equals(x: Any): Boolean = x match {
   case z:Node      => (length == 1) && z == apply(0)
   case z:Seq[_]    => sameElements(z)
   case z:String    => text == z
   case _           => false
 }

Ha ha, right.

Hypothetically speaking, let's say we removed all the nutty
irreconcilable equals methods from the XML classes.  At that point
nothing would work at all, because the first time you tried to compare
two nodes you'd see this:

scala> new Atom(42) == new Atom(42)
java.lang.StackOverflowError
       at scala.xml.Node.theSeq(Node.scala:147)
       at scala.xml.NodeSeq.iterator(NodeSeq.scala:54)
       at scala.collection.IterableLike$class.sameElements(IterableLike.scala:325)
       at scala.xml.NodeSeq.sameElements(NodeSeq.scala:46)
       ...

This makes perfect sense, because of course two freaking Atoms are
sequences which should have their individual elements compared.  What
are those elements? Well let's see, there's just this one Atom -- but
hey that's a sequence... the infinite regression is avoided right now
through a delicate and totally inconsistent set of overrides.

I could fix all this, I think.  I might even be able to keep 2.7 code
working via implicits, just different ones.  It's not what I'm looking
to do at this point, but neither is shipping the XML lib in its current
form.  But I don't want to touch another line of XML unless I can first
rip Seq out of the object hierarchy and send it out of XMLtown on a
rail.  Or, maybe I should just chmod 0000 scala/xml and filter all the
bug reports to /dev/null, so I can stop being tempted.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Vivid              | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

--
http://erikengbrecht.blogspot.com/

Hi Paul,

I am not an expert on XML, but it seems to me that the idea that Nodes
_are_ NodeSeqs is inherent in the definition of XML. So I'd be really
scared changing that relationship because I would have no idea what
would break. Another question is what the relationship between NodeSeq
and Node[Seq] should be. Here we have considerably more wiggle room it
seems, and the current subtyping relationship might not be the final
word. Just like the previous (faked) subtyping relationship between
arrays and Seq was not the final word either. I have cc'ed Burak,
maybe he can comment.

The community should decide whether a drastic change that late in the
game is desirable, or whether we should wait until after the first 2.8
beta. So far people seem to want change now.

Cheers

Why not also ask those at scala.lang.xml? Like, someone with expertise in
the XML data model?

Miguel
http://www.sts.tu-harburg.de/people/mi.garcia

On Tue, Dec 1, 2009 at 12:06 PM, martin odersky wrote:
> I am not an expert on XML, but it seems to me that the idea that Nodes
> _are_ NodeSeqs is inherent in the definition of XML.

Not so:

* The XML Recommendation [1] doesn't use the term "Node" at all.

* In the W3C DOM[2], Nodes have a "childNodes" attribute, but they
aren't themselves sequences of Nodes.

* In the XML Infoset[3] information items can have a "children"
property, but aren't them selves sequences of such.

* The XPath model is derived from the XML Infoset [4].

[1] http://www.w3.org/TR/REC-xml/
[2] http://www.w3.org/TR/2000/WD-DOM-Level-1-20000929/level-one-core.html#ID...
[3] http://www.w3.org/TR/xml-infoset/#infoitem
[4] http://www.w3.org/TR/xpath#infoset

Cheers,

Miles

On Tue, Dec 1, 2009 at 1:30 PM, Miles Sabin wrote:
> On Tue, Dec 1, 2009 at 12:06 PM, martin odersky wrote:
>> I am not an expert on XML, but it seems to me that the idea that Nodes
>> _are_ NodeSeqs is inherent in the definition of XML.
>
> Not so:
>
> * The XML Recommendation [1] doesn't use the term "Node" at all.
>
> * In the W3C DOM[2], Nodes have a "childNodes" attribute, but they
> aren't themselves sequences of Nodes.
>
> * In the XML Infoset[3] information items can have a "children"
> property, but aren't them selves sequences of such.
>
> * The XPath model is derived from the XML Infoset [4].
>
> [1] http://www.w3.org/TR/REC-xml/
> [2] http://www.w3.org/TR/2000/WD-DOM-Level-1-20000929/level-one-core.html#ID...
> [3] http://www.w3.org/TR/xml-infoset/#infoitem
> [4] http://www.w3.org/TR/xpath#infoset
>
> Cheers,
>
The point I was referring to is that XML does automatic sequence
flattening: There is no such thing as a Seq[Seq[Node]], it's taken to
be equivalent to a Seq[Node]. If you look at formal treatments of XML
such as Xduce you will see that they very much subscribe to this rule,
i.e. X** is the same as X* in Xduce. An easy way to achieve this
required collapse of types is to make Node a subtype of Seq[Node].
There might be other ways to achieve the same effect, of course.

Cheers

On Tue, Dec 1, 2009 at 1:13 PM, martin odersky wrote:
> The point I was referring to is that XML does automatic sequence
> flattening: There is no such thing as a Seq[Seq[Node]], it's taken to
> be equivalent to a Seq[Node].

I can't comment on Xduce, but the concept of "automatic sequence
flattening" is completely foreign to XML and any of it's data models
or schema languages. It just doesn't have sequences of sequences, so
X** doesn't make any sense.

Which isn't to say that a good Scala API for XML must not generalize
to supporting sequences of sequences, but conversely, there's nothing
inherent in XML which requires it to support them either.

Cheers,

Miles

On Tue, Dec 1, 2009 at 1:18 PM, Miles Sabin wrote:
> On Tue, Dec 1, 2009 at 1:13 PM, martin odersky wrote:
>> The point I was referring to is that XML does automatic sequence
>> flattening: There is no such thing as a Seq[Seq[Node]], it's taken to
>> be equivalent to a Seq[Node].
>
> I can't comment on Xduce, but the concept of "automatic sequence
> flattening" is completely foreign to XML and any of it's data models
> or schema languages. It just doesn't have sequences of sequences, so
> X** doesn't make any sense.

To qualify that a little, given a schema language we might have A*B*C*
which is, in a sense, a sequence of sequences. But given that the
schema is in play, we can't legitimately flatten it. And if we take
away the schema we just have a simple sequence.

Cheers,

Miles

On Tue, Dec 1, 2009 at 2:18 PM, Miles Sabin wrote:
> On Tue, Dec 1, 2009 at 1:13 PM, martin odersky wrote:
>> The point I was referring to is that XML does automatic sequence
>> flattening: There is no such thing as a Seq[Seq[Node]], it's taken to
>> be equivalent to a Seq[Node].
>
> I can't comment on Xduce, but the concept of "automatic sequence
> flattening" is completely foreign to XML and any of it's data models
> or schema languages. It just doesn't have sequences of sequences, so
> X** doesn't make any sense.
>
Yes, I agree. What I meant is the * arises out of regex expressions.
X* (zero or more copies of an X) is of course a central concept in
DTDs and XML schema and Relax NG. And there it holds that X** is the
same as X* and that X is a subtype of X*.

The question is what's the internal representation of an X*?

Cheers

On Tue, Dec 1, 2009 at 4:06 AM, martin odersky <martin.odersky@epfl.ch> wrote:

Hi Paul,

I am not an expert on XML, but it seems to me that the idea that Nodes
_are_ NodeSeqs is inherent in the definition of XML. So I'd be really
scared changing that relationship because I would have no idea what
would break. Another question is what the relationship between NodeSeq
and Node[Seq] should be. Here we have considerably more wiggle room it
seems, and the current subtyping relationship might not be the final
word. Just like the previous (faked) subtyping relationship between
arrays and Seq was not the final word either. I have cc'ed Burak,
maybe he can comment.

Martin,

When I first started using Scala's XML libraries, the Node/NodeSeq duality struck me as odd.  There are times when I really need a Node... a single one... a single element to operate on.  The fact that this single thing is also a collection (not contains a collection, but itself a collection) is not, even after 3 years, intuitive.  I still have to walk through the process of thinking about what I'm planning to operate on each time I try to operate on a Node/NodeSeq.

While I'm not an expert in the theory of XML nor am I a big user of SOAP and other complex structures built on XML, I can say that I have significant experience building and traversing complex XML (XHTML and REST-style web services) and I would strongly prefer removing the inheritance duality of Node/NodeSeq.

Thanks,

David


 

The community should decide whether a drastic change that late in the
game is desirable, or whether we should wait until after the first 2.8
beta. So far people seem to want change now.

Cheers

martin odersky wrote:
> Yes, I agree. What I meant is the * arises out of regex expressions.
> X* (zero or more copies of an X) is of course a central concept in
> DTDs and XML schema and Relax NG. And there it holds that X** is the
> same as X* and that X is a subtype of X*.
>

AFAICT (and please correct me if I'm wrong -- I have been surprised a
few times by scala.xml), this is different from what Scala does at the
moment. If you see X as a subtype of X*, then an instance of X would be
the same as a sequence containing only that instance (i.o.w. Node.toSeq
would return this, not this :: Nil), whereas a scala.xml.Node is a
sequence of its children. IMHO, seeing a Node as a NodeSeq should rather
be done by an implicit conversion, if at all. The required flattening
when performing queries is done automatically when binding over the
NodeSeq monad.

-sz

On Tue, Dec 01, 2009 at 01:25:54PM +0100, Miguel Garcia wrote:
> Why not also ask those at scala.lang.xml? Like, someone with expertise
> in the XML data model?

Of course, I only used -internals because I wasn't sure the people who
would need to get behind such a change read it, and wasn't that
confident it ever took off as a place to discuss XML. I forwarded my
original message to the list and solicited input.