Typed XML

On Sun, Nov 13, 2011 at 12:38 AM, Jordi Salvat i Alabart <jordi.salvat.i.alabart@gmail.com> wrote:

I've been working on an idea to decouple the compiler from the scala.xml classes and allow for type safety in XML literals and patterns (with a level of type-based XML validation as bonus), while maintaining backward compatibility.

The implementation is far from complete or mature -- probably 30% done -- but sufficient to know the idea is feasible.

I would appreciate input on the value of the idea and help in refining and developing it further.

If you can spare 10 or 15 minutes reading https://github.com/jsalvata/scala/wiki/Typed-XML , I will appreciate all and any comments, suggestions, ideas, and contributions.

You might want to reference the String interpolation SIP [1], which mentions:
"In the long term, this technique might open up the possibility to eliminate native support for XML literals in Scala. But this is not part of the current SIP"
-jason
[1] http://docs.scala-lang.org/sips/pending/string-interpolation.html 

On Sat, Nov 12, 2011 at 21:50, Jason Zaugg wrote:
> On Sun, Nov 13, 2011 at 12:38 AM, Jordi Salvat i Alabart
> wrote:
>>
>> I've been working on an idea to decouple the compiler from the scala.xml
>> classes and allow for type safety in XML literals and patterns (with a level
>> of type-based XML validation as bonus), while maintaining backward
>> compatibility.
>>
>> The implementation is far from complete or mature -- probably 30% done --
>> but sufficient to know the idea is feasible.
>>
>> I would appreciate input on the value of the idea and help in refining and
>> developing it further.
>>
>> If you can spare 10 or 15 minutes reading
>> https://github.com/jsalvata/scala/wiki/Typed-XML , I will appreciate all and
>> any comments, suggestions, ideas, and contributions.
>
> You might want to reference the String interpolation SIP [1], which
> mentions:
> "In the long term, this technique might open up the possibility to eliminate
> native support for XML literals in Scala. But this is not part of the
> current SIP"
> -jason
> [1] http://docs.scala-lang.org/sips/pending/string-interpolation.html

Mind you, that long term will be possible in conjunction with the
macros for Scala project. Alone, it is not enough to check stuff at
compile time.

Hi,

well, looks like there is a whole lot going on in this space recently.

From my perspective it looks like you're more or less interested in Type Providers for XML which read Schema definitions. You should probably speak to the SIQ/SLINK/... people at TypeSafe. They more or less need the same, just that their data definition sink is a database instead of some Schema file.

Bye,

Simon

Macros might provide a backwards-compatible solution if the in-language XML became part of the AST, rather than being desugared within the parser phase.

In-language XML based on string interpolation would never be backwards-compatible. Is that considered a viable roadmap, given the existing Lift & al. codebase?

--
Salut,

Jordi.

2011/11/13 Daniel Sobral <dcsobral@gmail.com>

On Sat, Nov 12, 2011 at 21:50, Jason Zaugg <jzaugg@gmail.com> wrote:
> On Sun, Nov 13, 2011 at 12:38 AM, Jordi Salvat i Alabart
> <jordi.salvat.i.alabart@gmail.com> wrote:
>>
>> I've been working on an idea to decouple the compiler from the scala.xml
>> classes and allow for type safety in XML literals and patterns (with a level
>> of type-based XML validation as bonus), while maintaining backward
>> compatibility.
>>
>> The implementation is far from complete or mature -- probably 30% done --
>> but sufficient to know the idea is feasible.
>>
>> I would appreciate input on the value of the idea and help in refining and
>> developing it further.
>>
>> If you can spare 10 or 15 minutes reading
>> https://github.com/jsalvata/scala/wiki/Typed-XML , I will appreciate all and
>> any comments, suggestions, ideas, and contributions.
>
> You might want to reference the String interpolation SIP [1], which
> mentions:
> "In the long term, this technique might open up the possibility to eliminate
> native support for XML literals in Scala. But this is not part of the
> current SIP"
> -jason
> [1] http://docs.scala-lang.org/sips/pending/string-interpolation.html

Mind you, that long term will be possible in conjunction with the
macros for Scala project. Alone, it is not enough to check stuff at
compile time.

--
Daniel C. Sobral

I travel to the future all the time.

On Sun, Nov 13, 2011 at 1:14 PM, Jordi Salvat i Alabart <jordi.salvat.i.alabart@gmail.com> wrote:

Macros might provide a backwards-compatible solution if the in-language XML became part of the AST, rather than being desugared within the parser phase.

In-language XML based on string interpolation would never be backwards-compatible. Is that considered a viable roadmap, given the existing Lift & al. codebase?

Based on previous threads on the topic, XML literals won't be going anywhere soon. But given a viable alternative (both syntactically and within the library), a long deprecation cycle, and maybe some tooling to help migrate, they could be eventually removed.
This is just my impression, and doesn't reflect an official a road map.
-jason

hiya,

I'd really recommend (from Declaring-and-using-unmarshallers - URIs),
if you want to improve things, not to use prefixes and strings only,
Scala's type system can be used to handle QNames directly (see Scales
Xml's use of QName and AttributeQName). It took DOM way too long to
add QNames into the mix directly, we shouldn't keep repeating the same
mistakes.

I'm really not sure how using URIs can lead to being error prone.

I like the idea of pushing validation into the compilation stage and
can see how parts of documents with snippets etc could work but I'm
not seeing how this works with composability. Say you have a small
xml part that is always looking the same way but has different parent
elements, you can't validate them without their last used - about to
serialize context.

For Scales, QNames and structure (outside of streams) are compiler
friendly (qnames can throw on the first definition though - even there
I was tempted to push for Either). The XPath like syntax is to the
extent that inputs can be verified by the compiler, but I'd like to
nail that further down (e.g. don't allow element or attribute filters
after using a text predicate).

I'm not sure if there is a better sweet spot that can really help the
developer produce valid xml, but I'm happy someone is looking for it
:-)

cheers,
Chris

On Sun, Nov 13, 2011 at 12:38 AM, Jordi Salvat i Alabart
wrote:
> Hi,
>
> I've been working on an idea to decouple the compiler from the scala.xml
> classes and allow for type safety in XML literals and patterns (with a level
> of type-based XML validation as bonus), while maintaining backward
> compatibility.
>
> The implementation is far from complete or mature -- probably 30% done --
> but sufficient to know the idea is feasible.
>
> I would appreciate input on the value of the idea and help in refining and
> developing it further.
>
> If you can spare 10 or 15 minutes reading
> https://github.com/jsalvata/scala/wiki/Typed-XML , I will appreciate all and
> any comments, suggestions, ideas, and contributions.
>
> --
> Salut,
>
> Jordi.
>
>

Not sure there is a direct relation...

Where can I find out more about type providers, and about the plans and status of the work on them? Do the SIQ/SLINK people have a mailing list?

--
Thanks,

Jordi.

2011/11/13 Simon Ochsenreither <simon.ochsenreither@googlemail.com>

Hi,

well, looks like there is a whole lot going on in this space recently.

From my perspective it looks like you're more or less interested in Type Providers for XML which read Schema definitions. You should probably speak to the SIQ/SLINK/... people at TypeSafe. They more or less need the same, just that their data definition sink is a database instead of some Schema file.

Bye,

Simon

Hi.

2011/11/13 Chris Twiner <chris.twiner@gmail.com>

hiya,

I'd really recommend (from Declaring-and-using-unmarshallers - URIs),
if you want to improve things, not to use prefixes and strings only,
Scala's type system can be used to handle QNames directly (see Scales
Xml's use of QName and AttributeQName).  It took DOM way too long to
add QNames into the mix directly, we shouldn't keep repeating the same
mistakes.

I had doubts here. I've added a note in the wiki to give further thought to the issue.

 

I'm really not sure how using URIs can lead to being error prone.

You're right: not if they are validated. It might add some verbosity, but even that can be alleviated if a mechanism is provided to define prefix mappings with a scope larger than a single XML literal -- maybe the scala Processing Instruction can be extended for that purpose.

I like the idea of pushing validation into the compilation stage and
can see how parts of documents with snippets etc could work but I'm
not seeing how this works with composability.  Say you have a small
xml part that is always looking the same way but has different parent
elements, you can't validate them without their last used - about to
serialize context.

Each "unmarshaller" can choose whether to support snippets. If it does, it will need to ensure that each snippet receives an appropriate type, and that parent elements only accept children of the right type. E.g. an <li> would be of type "ListItem", and <ol> and <ul> would only accept children of this type.

I am sure there will be XML languages/schema definitions where this will not be enough... if you can provide examples where you anticipate problems, that would help validating the concept and discovering its limits.

For Scales, QNames and structure (outside of streams) are compiler
friendly (qnames can throw on the first definition though - even there
I was tempted to push for Either).  The XPath like syntax is to the
extent that inputs can be verified by the compiler, but I'd like to
nail that further down (e.g. don't allow element or attribute filters
after using a text predicate).

I'm not sure if there is a better sweet spot that can really help the
developer produce valid xml, but I'm happy someone is looking for it
:-)

Let's see if this leads anywhere. Thanks for your support.
 

cheers,
Chris

On Sun, Nov 13, 2011 at 12:38 AM, Jordi Salvat i Alabart
<jordi.salvat.i.alabart@gmail.com> wrote:
> Hi,
>
> I've been working on an idea to decouple the compiler from the scala.xml
> classes and allow for type safety in XML literals and patterns (with a level
> of type-based XML validation as bonus), while maintaining backward
> compatibility.
>
> The implementation is far from complete or mature -- probably 30% done --
> but sufficient to know the idea is feasible.
>
> I would appreciate input on the value of the idea and help in refining and
> developing it further.
>
> If you can spare 10 or 15 minutes reading
> https://github.com/jsalvata/scala/wiki/Typed-XML , I will appreciate all and
> any comments, suggestions, ideas, and contributions.
>
> --
> Salut,
>
> Jordi.
>
>

On 2011-11-13 13:37, Jordi Salvat i Alabart wrote:
> Not sure there is a direct relation...
>
> Where can I find out more about type providers, and about the plans
> and status of the work on them? Do the SIQ/SLINK people have a mailing
> list?

We have no public resources about SLICK itself so far but Eugene has a
site about his work on macros at http://scalamacros.org/ where you can
read up on type macros / type providers. They could be used to implement
XML binding transparently at compile-time. For XML literals you might
not want to generate types for binding but validate a generic DOM
against a schema instead. Assuming an XML syntax based on Martin's
alternative string interpolation SIP + macros, this could also be done.
If you're looking into modifying the compiler or writing compiler
plug-ins, this kind of use of type providers and macros does not really
offer anything new -- it just makes it a lot easier to use.

Cheers,
Stefan (Lead developer for SLICK at Typesafe)

Hi Stefan,

is there any estimated time of arrival? Or plans for further information about the project? F#3 is already shipping with a developer preview (Type Providers) and both Ceylon and Kotlin (Scala wannabes) want to release some beta at the end of the year, that's why I ask.

The fact that so many “experimental” stuff is involved (the stuff around SIQ, Ferry, scala-virtualized, scalamacros, ...) makes me wonder if this is pure research stuff or some to-be-shipped product with a massive feature/scope creep.

I really don't want to complain, but sometimes timing is crucial ...

Thanks and bye!


Simon

On 2011-11-16 17:31, Simon Ochsenreither wrote:
> is there any estimated time of arrival? Or plans for further
> information about the project? F#3 is already shipping with a
> developer preview (Type Providers) and both Ceylon and Kotlin (Scala
> wannabes) want to release some beta at the end of the year, that's why
> I ask.

Well, we already have ScalaQuery as a production-ready short-term
solution. I can't promise any concrete date for a first release of SLICK
at this time (but we do have a solid project plan with important
milestones internally).

> The fact that so many “experimental” stuff is involved (the stuff
> around SIQ, Ferry, scala-virtualized, scalamacros, ...) makes me
> wonder if this is pure research stuff or some to-be-shipped product
> with a massive feature/scope creep.

The scope is indeed large but a lot of the research has already be done
and we are working on a shippable product. SIQ and Ferry are done (but
further research for translation strategies is ongoing), LINQ has
already demonstrated the code-lifting approach, Martin has been working
on reification/code-lifting in Scala for several months now (check it
out in trunk). scala-virtualized has come along nicely but it is not in
focus for SLICK right now. Macros really are new but except for type
providers they are not essential. Even without them, you can still use
Code.lift to capture an AST and do the processing at run-time (like LINQ
does).

> I really don't want to complain, but sometimes timing is crucial ...

The early bird may get the worm but the second mouse gets the cheese ;-)

-sz

Hi Stefan,

sounds absolutely great! Thanks for the information!

> Macros really are new but except for type providers they are not essential.

Yes, I just think not having to write all that boilerplate database table description code would be a killer feature against the whole JPA/Hibernate/JEE consortium.

Thank you very much for all the hard work!

Great to see such a progress!

Thanks and bye,


Simon

Thanks for the pointer, Stefan. I'll research that path.


2011/11/16 Stefan Zeiger <szeiger@novocode.com>

On 2011-11-13 13:37, Jordi Salvat i Alabart wrote:

Not sure there is a direct relation...

Where can I find out more about type providers, and about the plans and status of the work on them? Do the SIQ/SLINK people have a mailing list?

We have no public resources about SLICK itself so far but Eugene has a site about his work on macros at http://scalamacros.org/ where you can read up on type macros / type providers. They could be used to implement XML binding transparently at compile-time. For XML literals you might not want to generate types for binding but validate a generic DOM against a schema instead. Assuming an XML syntax based on Martin's alternative string interpolation SIP + macros, this could also be done. If you're looking into modifying the compiler or writing compiler plug-ins, this kind of use of type providers and macros does not really offer anything new -- it just makes it a lot easier to use.

Cheers,
Stefan (Lead developer for SLICK at Typesafe)