[sacala-users] Scala and mutable XML

Though I never got as far as writing it, I've played with ideas like this for
some time. My Java day job involves lots of mutable XML docs, so I'd be
interested in seeing, using and helping with anything you produce.

What sort of methods / DSL will you have for mutating the XML?

Silvio Bierman wrote:
>
> Hello all,
>
> I have a user case (a complex hierarchical editor of abstractions on XML
> data) where I need a mutable representation of XML, a la Java DOM.
> However, I want to handle/pass XML mainly in the scala.xml immutable way.
> I am in the process of implementing a DOM-like library with immutable
> Scala XML as the underlying representation. I gives each mutable DOM node
> a peer that corresponds to a Scala XML node. When a mutation happens
> somewhere in the document all peers in the path from that location up
> until the DOM document root will be replaced with different peers, of
> course. (Will only work when the XML is a proper tree)
>
> This gives a nice advantage over plain DOM: the root peer node values in
> time represent maximally overlapping immutable versions of the document
> during the editing phase. Thus the XML editor can very easily and
> efficiently implement undo/redo features.
>
> I was wondering if such an implementation of DOM over Scala XML already
> exists. If not, does it sound worthwhile to extend what I have now and
> make it a full implementation of DOM?
>
> Gr. Silvio
>
>

Marcus Downing wrote:
>
> Though I never got as far as writing it, I've played with ideas like this
> for some time. My Java day job involves lots of mutable XML docs, so I'd
> be interested in seeing, using and helping with anything you produce.
>

I have been shoving around some code and left it broken last night. Next
week I will repair the damage I did and send you the code.

Marcus Downing wrote:
>
> What sort of methods / DSL will you have for mutating the XML?
>

Something like this:

abstract class MutableNode extends scala.xml.Node
{
def peer : scala.xml.Node
def parent : MutableNode

def appendChild(child : MutableNode)
def insertBefore(child : MutableNode,before : MutableNode)
def removeChild(child : MutableNode)
def replaceChild(replace : MutableNode,with : MutableNode)
def setNodeValue(value : String)

//several convenience methods
}

class MutableElem(private var _parent : MutableNode,private var _peer :
scala.xml.Elem) extends MutableNode
{
//delegate all scala.xml.node methods (except child) to _peer
def child : Seq[MutableNode] = ...
def peer = _peer
def parent = _parent

def setParam(name : String,value : Seq[Node])
def setParam(name : String,value : String) //convenience method
}

The mutation primitives replace the peer node, work up in the mutable
hierarchy to replace the parent peer nodes and arrange the parent/child
hierarchy.

Currently each mutation does this creating a new version of the underlying
immutable document. For efficiency reasons one could imagine some laziness
here (trigger mutations when accessing dirty peers) but I have not yet given
that any attention. It would require more value state wrapping, of course,
so I am not yet convinced this would buy us anything.

Gr. Silvio

Now that I thought about the laziness just enough to write this down I
realize it would be quite easy (and cheap) to implement the upward
propagation of peers as a lazy process. I will put that in as well.

I was going to go into detail on how I thought the laziness would work, but
you appear to be doing it fine without me. I'll avoid disturbing your
thoughts until they're more fully developed.

Silvio Bierman wrote:
>
> Now that I thought about the laziness just enough to write this down I
> realize it would be quite easy (and cheap) to implement the upward
> propagation of peers as a lazy process. I will put that in as well.
>

Silvio Bierman wrote:
> I have a user case (a complex hierarchical editor of abstractions on XML
> data) where I need a mutable representation of XML, a la Java DOM. However,
> I want to handle/pass XML mainly in the scala.xml immutable way.
> I am in the process of implementing a DOM-like library with immutable Scala
> XML as the underlying representation. I gives each mutable DOM node a peer
> that corresponds to a Scala XML node. When a mutation happens somewhere in
> the document all peers in the path from that location up until the DOM
> document root will be replaced with different peers, of course. (Will only
> work when the XML is a proper tree)
>
How about a Zipper instead of a mutable representation? The original use
case for the Zipper was a structured editor.

Your post inspired me to finally write
http://szeiger.de/blog/2009/12/27/a-zipper-for-scala-xml/ after having
the implementation lying around for a week :)

-sz

On Sun, Dec 27, 2009 at 1:57 PM, Stefan Zeiger wrote:
> Silvio Bierman wrote:
>> I have a user case (a complex hierarchical editor of abstractions on XML
>> data) where I need a mutable representation of XML, a la Java DOM. However,
>> I want to handle/pass XML mainly in the scala.xml immutable way.
>> I am in the process of implementing a DOM-like library with immutable Scala
>> XML as the underlying representation. I gives each mutable DOM node a peer
>> that corresponds to a Scala XML node. When a mutation happens somewhere in
>> the document all peers in the path from that location up until the DOM
>> document root will be replaced with different peers, of course.  (Will only
>> work when the XML is a proper tree)
>>
> How about a Zipper instead of a mutable representation? The original use
> case for the Zipper was a structured editor.
>
> Your post inspired me to finally write
> http://szeiger.de/blog/2009/12/27/a-zipper-for-scala-xml/ after having
> the implementation lying around for a week :)
>

Funnily enough, I've been writing a - start from scratch - xml rep,
fixing all the things I don't like about scala xml, and undoubtedly
introducing a tonne of things others wouldn't like :-). A zipper
based approach (working on an xml tree) is a very sensible way to do
it, let the data structure do the house keeping.

For those interested the code doesn't provide a nice modification api
yet (just modify in place and removeAndZipUp, I'm aiming on modifying
directly from xpaths) but the queries look like:

path \\* Elements.localName("DontRedeclare")

or

path \ "NoNamespace"

(for default "" namespace).

or

prefixed \@ ("urn:prefix"::"attr")

for qualified. (Comparisoms are always qname based, no localname only
unless Elements.localName is used).

path \\*(_ === "prefixed text") for text comparisoms etc.

The only real issue is different Paths for comparisoms of duplicates
etc ( | or \@*\^ ) \^ is parent axis in my xpaths. So I've also
added a position (stack[Int] for nodes) that I'll use for modifiying
(i.e. moveFrom(position1, position2), that calls zipUp where
appropriate).

As for integeration with scala xml - I'm planning an implicit to
convert fool typers and then re-write the tree to use my xml directly
but thats much further down the line.

Hello Stefan,

Although the Zipper idea seems to be an interesting concept it does not
provide an aspect of a mutable DOM that is very convenient for compound XML
editors: non-persistence!

The type of XML editor I am interested in maps nodes in the document to some
type of "node-editor" that creates the abstraction over that node and
possibly a number of child nodes. The NodeEditor holds on to the node and
when called into view presents the visual abstraction over the node allowing
the user to modify it within the defined boundaries for that node. It also
exposes which child nodes are not part of its own abstraction but may be
handled by other NodeEditor instances that may be visualized as
"child-editors" of this Node editor. Thus, the global editor interface
exposes the XML document as a tree hierarchy of NodeEditor instances. The
mapping from XML nodes to TreeOf[NodeEditor] is created by a factory that
maps XML nodes to some descendant of NodeEditor and the way each individual
NodeEditor instance exposes visible child nodes.

What is very convenient in this construction is that underlying node editors
can change their parts of the document without having to interfere with the
ancestor NodeEditors. A DOM representation with a mutable representation of
nodes has this property.

Unless I misunderstand the Zipper idea the NodeLoc is not mutable. Changing
the document at a leaf level will not make the modifications available at
higher levels unless we maneuver through the ancestor editors and change
their NodeLoc values. This gives the NodeLoc little advantage (in this
sense) over the plain Node.

I can understand the advantage of NodeLoc for XPath purposes but XPath plays
no role in my XML editor use case.

In short: a mutabe/non-persistent Node wrapper enables mutiple independent
viewers of (parts of) a document to share it and see changes to the entire
document made by other viewers. If I understand correctly NodeLoc does not
have that property.

I am very much aware of the drawbacks of mutable DOM and have suffered the
consequences with Java/DOM in the past. We are now rewriting everything with
Scala/scala.xml.Node. Only inside a subcomponent like an XML editor (we have
several types of these each handling different types of complex user
definitions represented as XML) would I like a
DOM-like/mutable/non-persistent abstraction.

The Zipper looks interesting and in some ways resembles what I do internally
to create new versions of sub-documents. It seems like an excellent way to
get full XPath for Scala XML.

If I am misunderstanding the concept and the Zipper would match my use case
after all I would gladly stand corrected. No use in re-inventing the wheel
if good/better wheels are available.

Gr. Silvio

You don't misunderstand the Zipper which is in Scalaz, but perhaps the
suggestion was made to have you reconsider using a mutable DOM.

Tony Morris
http://tmorris.net/

On Dec 28, 2009, at 12:35 AM, Silvio Bierman wrote:

>
> Hello Stefan,
>
> Although the Zipper idea seems to be an interesting concept it does
> not
> provide an aspect of a mutable DOM that is very convenient for
> compound XML
> editors: non-persistence!
>
> The type of XML editor I am interested in maps nodes in the document
> to some
> type of "node-editor" that creates the abstraction over that node and
> possibly a number of child nodes. The NodeEditor holds on to the
> node and
> when called into view presents the visual abstraction over the node
> allowing
> the user to modify it within the defined boundaries for that node.
> It also
> exposes which child nodes are not part of its own abstraction but
> may be
> handled by other NodeEditor instances that may be visualized as
> "child-editors" of this Node editor. Thus, the global editor interface
> exposes the XML document as a tree hierarchy of NodeEditor
> instances. The
> mapping from XML nodes to TreeOf[NodeEditor] is created by a factory
> that
> maps XML nodes to some descendant of NodeEditor and the way each
> individual
> NodeEditor instance exposes visible child nodes.
>
> What is very convenient in this construction is that underlying node
> editors
> can change their parts of the document without having to interfere
> with the
> ancestor NodeEditors. A DOM representation with a mutable
> representation of
> nodes has this property.
>
> Unless I misunderstand the Zipper idea the NodeLoc is not mutable.
> Changing
> the document at a leaf level will not make the modifications
> available at
> higher levels unless we maneuver through the ancestor editors and
> change
> their NodeLoc values. This gives the NodeLoc little advantage (in this
> sense) over the plain Node.
>
> I can understand the advantage of NodeLoc for XPath purposes but
> XPath plays
> no role in my XML editor use case.
>
> In short: a mutabe/non-persistent Node wrapper enables mutiple
> independent
> viewers of (parts of) a document to share it and see changes to the
> entire
> document made by other viewers. If I understand correctly NodeLoc
> does not
> have that property.
>
> I am very much aware of the drawbacks of mutable DOM and have
> suffered the
> consequences with Java/DOM in the past. We are now rewriting
> everything with
> Scala/scala.xml.Node. Only inside a subcomponent like an XML editor
> (we have
> several types of these each handling different types of complex user
> definitions represented as XML) would I like a
> DOM-like/mutable/non-persistent abstraction.
>
> The Zipper looks interesting and in some ways resembles what I do
> internally
> to create new versions of sub-documents. It seems like an excellent
> way to
> get full XPath for Scala XML.
>
> If I am misunderstanding the concept and the Zipper would match my
> use case
> after all I would gladly stand corrected. No use in re-inventing the
> wheel
> if good/better wheels are available.
>
> Gr. Silvio
>

Tony Morris-4 wrote:
>
> You don't misunderstand the Zipper which is in Scalaz, but perhaps the
> suggestion was made to have you reconsider using a mutable DOM.
>

Well, I have reconsidered it many times. Coming from a functional
programming background I strongly prefer handling XML the immutable Scala
way. But I have tried several approaches and in this use case the
immutability is a real pain. The XML documents are extremely large and
complex consisting of 100+ abstractions per document type. Making the
fragment editors explicitly cooperate as they would have to with an
immutable representation is not workable.

That is why I decided my best approach would be a mutable DOM wrapped around
the immutable representation, only to be used inside an XML editor and its
fragment editors. That way the mutable DOM is sort of like a cooperative
view on the document.

Gr. Silvio

I am using HXT right now to parse a 60GB XML file into "an immutable
DOM". Coincidence?

Of course, Scala is impure and strict but I'm sure you can work around
that (I do). If not, I wish you all the best on that mutable DOM thing :)

Silvio Bierman wrote:
> Tony Morris-4 wrote:
>
>> You don't misunderstand the Zipper which is in Scalaz, but perhaps the
>> suggestion was made to have you reconsider using a mutable DOM.
>>
>>
>
> Well, I have reconsidered it many times. Coming from a functional
> programming background I strongly prefer handling XML the immutable Scala
> way. But I have tried several approaches and in this use case the
> immutability is a real pain. The XML documents are extremely large and
> complex consisting of 100+ abstractions per document type. Making the
> fragment editors explicitly cooperate as they would have to with an
> immutable representation is not workable.
>
> That is why I decided my best approach would be a mutable DOM wrapped around
> the immutable representation, only to be used inside an XML editor and its
> fragment editors. That way the mutable DOM is sort of like a cooperative
> view on the document.
>
> Gr. Silvio
>
>

Tony Morris-4 wrote:
>
> I am using HXT right now to parse a 60GB XML file into "an immutable
> DOM". Coincidence?
>
> Of course, Scala is impure and strict but I'm sure you can work around
> that (I do). If not, I wish you all the best on that mutable DOM thing :)
>

Thanks for that.

Understand be correctly: I love Scala as it is. After my FP period in an
academic environment reality pushed me towards C, then C++ and in the late
90s Java. So I have had my share of imperative/OO programming and have
learned to appreciate its advantages. That is exactly why I was drawn to
Scala: it gives me back many of the things I loved in FP without having to
sacrifice the things I like about imperative programming.

I got this mutable DOM thing working quite nicely. It's not too bad and as I
said, given some added generic abstractions and some different class naming
it could have easily posed as a cooperative XML sharing framework which
would make it go down much more easily with immutability fetishists :-)

But for my modest application I am happy to leave it as it is. On hindsight,
extending it into a full DOM implementation seems like major overkill to me
so I may leave that to someone much more ambitious.

Gr. Silvio

Maybe you should swap notes with Anthony Coates and look at some of the XML stuff currently going on in scala-incubator...

2009/12/28 Silvio Bierman <sbierman@jambo-software.com>

Tony Morris-4 wrote:
>
> I am using HXT right now to parse a 60GB XML file into "an immutable
> DOM". Coincidence?
>
> Of course, Scala is impure and strict but I'm sure you can work around
> that (I do). If not, I wish you all the best on that mutable DOM thing :)
>

Thanks for that.

Understand be correctly: I love Scala as it is. After my FP period in an
academic environment reality pushed me towards C, then C++ and in the late
90s Java. So I have had my share of imperative/OO programming and have
learned to appreciate its advantages. That is exactly why I was drawn to
Scala: it gives me back many of the things I loved in FP without having to
sacrifice the things I like about imperative programming.

I got this mutable DOM thing working quite nicely. It's not too bad and as I
said, given some added generic abstractions and some different class naming
it could have easily posed as a cooperative XML sharing framework which
would make it go down much more easily with immutability fetishists :-)

But for my modest application I am happy to leave it as it is. On hindsight,
extending it into a full DOM implementation seems like major overkill to me
so I may leave that to someone much more ambitious.

Gr. Silvio

--
View this message in context: http://old.nabble.com/-sacala-users--Scala-and-mutable-XML-tp26905465p26942024.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Kevin Wright

mail/google talk: kev.lee.wright@googlemail.com
wave: kev.lee.wright@googlewave.com
skype: kev.lee.wright
twitter: @thecoda