Modularisation of Scala

On Wed, Sep 2, 2009 at 11:37 AM, Alex Blewitt wrote:
> So far, the discussion has just started; Heiko has been OSGi-fying the
> current Scala modules [4], Josh Suereth has been looking at Mavenizing the
> Scala build process and modularizing Scala along with it [5], and Roman has
> taken what Heiko has done and extracted out the Actors, Concurrent, Testing
> and XML into separate modules [6].

From my point of view, the main requirements that I have for Eclipse
IDE and compiler development within Eclipse are,

1. The one SVN module/Eclipse project <=> one OSGi bundle
correspondance that we currently have remains.

2. Explicit OSGi manifest files can be made available trivially under
/META-INF/MANIFEST.MF. Currently 'trivially' means
"checked out of SVN along with the rest of the project".

So long as those two conditions are met, I really don't much care how
things are arranged.

That said, I have a mild preference against adding additional tools to
the build, given that OSGi manifest files are so simple to create by
hand (and then when checked into SVN are straightforwardly versioned
as simple text). But that's not a deal breaker so long as any tools
proposed will fit into EPFLs nightly build process (ie. they will need
to run standalone and/or be bundlable with all their dependencies).

Those are my constraints. On the positive side, I would like to see
the scala library sources being repackaged as one or more independent
top-level SVN modules/Eclipse projects (each with it's own OSGi
manifest). The current scala-library module/project in SVN fakes this
layout to some extent, but it would be beneficial to have this done
properly (ie. the sources moved as well).

The advantage of a reorganization of this sort is that it would make
it a great deal easier to use Eclipse for development of the standard
library: library sources would then appear as first class source
components of first class source projects rather than as uninterpreted
binary components of the compiler project. The current arrangement
means that, paradoxically, it's easy to interactively develop the
compiler itself, but not the standard library.

The downside of a reorganization is that it affects the main scala SVN
module, and that would, at least briefly, interfere with the way
people are working with it currently ... the affected parties being
mainly EPFL people, paupl and me.

Cheers,

Miles

Miles,
Thanx for your positive attitude towards repackaging.
As far as I understand, you need OSGi to be the module system of choice, right? No Simple Module System or Jigsaw, but OSGi because of Eclipse. Well, from my point of view, that's not a bad choice ;-)
You mentioned, that OSGi manifests are easy to create. I disagree. Of course it is a pice of cake to create the file and add some of the simple headers. But in order to have an OSGi manifest suitable for the real world, the packages used by exported packages (uses directive) have to be calculated automatically based upon the actual code. This is not possible applying the manifest first approach (writing the manifest yourself or via the Eclipse PDE editor). Hence IMHO it is necessary to introduce BND or the Maven Bundle Plug-in.
Cheers,
Heiko
2009/9/2 Miles Sabin <miles@milessabin.com>

From my point of view, the main requirements that I have for Eclipse
IDE and compiler development within Eclipse are,

1. The one SVN module/Eclipse project <=> one OSGi bundle
correspondance that we currently have remains.

2. Explicit OSGi manifest files can be made available trivially under
<Eclipse project>/META-INF/MANIFEST.MF. Currently 'trivially' means
"checked out of SVN along with the rest of the project".

So long as those two conditions are met, I really don't much care how
things are arranged.

That said, I have a mild preference against adding additional tools to
the build, given that OSGi manifest files are so simple to create by
hand (and then when checked into SVN are straightforwardly versioned
as simple text). But that's not a deal breaker so long as any tools
proposed will fit into EPFLs nightly build process (ie. they will need
to run standalone and/or be bundlable with all their dependencies).

Those are my constraints. On the positive side, I would like to see
the scala library sources being repackaged as one or more independent
top-level SVN modules/Eclipse projects (each with it's own OSGi
manifest). The current scala-library module/project in SVN fakes this
layout to some extent, but it would be beneficial to have this done
properly (ie. the sources moved as well).

The advantage of a reorganization of this sort is that it would make
it a great deal easier to use Eclipse for development of the standard
library: library sources would then appear as first class source
components of first class source projects rather than as uninterpreted
binary components of the compiler project. The current arrangement
means that, paradoxically, it's easy to interactively develop the
compiler itself, but not the standard library.

The downside of a reorganization is that it affects the main scala SVN
module, and that would, at least briefly, interfere with the way
people are working with it currently ... the affected parties being
mainly EPFL people, paupl and me.

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

--

My blog: heikoseeberger.name
Follow me: twitter.com/hseeberger
OSGi on Scala: www.scalamodules.org
Lift, the simply functional web framework: liftweb.net

On Wed, Sep 2, 2009 at 2:30 PM, Heiko
Seeberger wrote:
> As far as I understand, you need OSGi to be the module system of choice,
> right? No Simple Module System or Jigsaw, but OSGi because of Eclipse. Well,
> from my point of view, that's not a bad choice ;-)

Yup.

> You mentioned, that OSGi manifests are easy to create. I disagree. Of course
> it is a pice of cake to create the file and add some of the simple headers.
> But in order to have an OSGi manifest suitable for the real world, the
> packages used by exported packages (uses directive) have to be calculated
> automatically based upon the actual code. This is not possible applying the
> manifest first approach (writing the manifest yourself or via the Eclipse
> PDE editor). Hence IMHO it is necessary to introduce BND or the Maven Bundle
> Plug-in.

Again, no problem, so long as my constraints are met and the tool fits
into the EPFL nightly build. I suspect that BND would be the lightest
weight option here. Presumably there's no particular issue with having
BND generate /META-INF/MANIFEST.MF for use at
development time?

Cheers,

Miles

2009/9/2 Miles Sabin <miles@milessabin.com>

Again, no problem, so long as my constraints are met and the tool fits
into the EPFL nightly build. I suspect that BND would be the lightest
weight option here. Presumably there's no particular issue with having
BND generate <Eclipse project>/META-INF/MANIFEST.MF for use at
development time?

This is possible with the Maven Bundle Plug-in, hence it should also somehow be possible with BND.Peter?
Cheers,
Heiko 

2009/9/2 Heiko Seeberger :
> 2009/9/2 Miles Sabin
>>
>> Again, no problem, so long as my constraints are met and the tool fits
>> into the EPFL nightly build. I suspect that BND would be the lightest
>> weight option here. Presumably there's no particular issue with having
>> BND generate /META-INF/MANIFEST.MF for use at
>> development time?
>
> This is possible with the Maven Bundle Plug-in, hence it should also somehow
> be possible with BND.
> Peter?
> Cheers,
> Heiko

Normally you can replace your JAR command and custom manifest file
with bnd since bnd provides an Ant task:

So why do you need the manifest.mf file? Does the build chain require
it somehow?

On Wed, Sep 2, 2009 at 3:37 AM, Alex Blewitt <alex.blewitt@gmail.com> wrote:

Following on from my previous blog post on the subject [1] a few people expressed an interest in being able to help drive this forwards. In the spirit of open and inclusion discussion, we should use an open mailing list to discuss ideas, and so that they're publicly visible after the fact.

It has been suggested (by Miles and Heiko) that we use the scala-internals mailing list to bring this debate to the table. It's a public list, but you need to be subscribed to send messages to the list [2]. By all accounts, this is a relatively low-volume list (in the order of 5 messages per day [3]) so hopefully this won't dissuade people from joining in in order to contribute. If it is, I can be a gateway and forward messages to the list on your behalf.

If the existing membership of scala-internals object or disagree that this is the best place to have the conversation, please let me know and I'll set up a separate Google Groups to discuss this on.

So far, the discussion has just started; Heiko has been OSGi-fying the current Scala modules [4], Josh Suereth has been looking at Mavenizing the Scala build process and modularizing Scala along with it [5], and Roman has taken what Heiko has done and extracted out the Actors, Concurrent, Testing and XML into separate modules [6].

I'm all for splitting Scala's libraries into modules.  One question... doesn't the XML stuff need to stay in the core library as the compiler emits code that depends on this library?
 

I'll wait a day for people to sign up on the list (instructions at [2]) and for any existing members of the scala-internals to object to this use of the mailing list, and then start the ball rolling tomorrow with a document designed to describe the end goals of modularisation.

Alex

[1] http://alblue.blogspot.com/2009/08/modularity-for-scala.html
[2] http://listes.epfl.ch/doc_en.cgi?liste=scala-internals
[3] http://dir.gmane.org/gmane.comp.lang.scala.internals
[4] http://wiki.github.com/hseeberger/scala-lang-osgi
[5] http://wiki.github.com/jsuereth/scala-jigsaw
[6] http://bitbucket.org/romanroe/scala-lang-osgi-modules/

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp

I was thinking about that the whole time, too. But I'll grant that, if no XML literals are used, no library calls will result.   Anyway, I'd much prefer a framework which enabled arbitrary complex literals to be configurable. Having XML a step farther from the main library is a good thing in this respect.

On Wed, Sep 2, 2009 at 1:48 PM, David Pollak <feeder.of.the.bears@gmail.com> wrote:

On Wed, Sep 2, 2009 at 3:37 AM, Alex Blewitt <alex.blewitt@gmail.com> wrote:

Following on from my previous blog post on the subject [1] a few people expressed an interest in being able to help drive this forwards. In the spirit of open and inclusion discussion, we should use an open mailing list to discuss ideas, and so that they're publicly visible after the fact.

It has been suggested (by Miles and Heiko) that we use the scala-internals mailing list to bring this debate to the table. It's a public list, but you need to be subscribed to send messages to the list [2]. By all accounts, this is a relatively low-volume list (in the order of 5 messages per day [3]) so hopefully this won't dissuade people from joining in in order to contribute. If it is, I can be a gateway and forward messages to the list on your behalf.

If the existing membership of scala-internals object or disagree that this is the best place to have the conversation, please let me know and I'll set up a separate Google Groups to discuss this on.

So far, the discussion has just started; Heiko has been OSGi-fying the current Scala modules [4], Josh Suereth has been looking at Mavenizing the Scala build process and modularizing Scala along with it [5], and Roman has taken what Heiko has done and extracted out the Actors, Concurrent, Testing and XML into separate modules [6].

I'm all for splitting Scala's libraries into modules.  One question... doesn't the XML stuff need to stay in the core library as the compiler emits code that depends on this library?
 

I'll wait a day for people to sign up on the list (instructions at [2]) and for any existing members of the scala-internals to object to this use of the mailing list, and then start the ball rolling tomorrow with a document designed to describe the end goals of modularisation.

Alex

[1] http://alblue.blogspot.com/2009/08/modularity-for-scala.html
[2] http://listes.epfl.ch/doc_en.cgi?liste=scala-internals
[3] http://dir.gmane.org/gmane.comp.lang.scala.internals
[4] http://wiki.github.com/hseeberger/scala-lang-osgi
[5] http://wiki.github.com/jsuereth/scala-jigsaw
[6] http://bitbucket.org/romanroe/scala-lang-osgi-modules/

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

Right...I suspect XML is always a compile time dependency but not necessarily a runtime dependency.

On Wed, Sep 2, 2009 at 12:55 PM, Daniel Sobral <dcsobral@gmail.com> wrote:

I was thinking about that the whole time, too. But I'll grant that, if no XML literals are used, no library calls will result.   Anyway, I'd much prefer a framework which enabled arbitrary complex literals to be configurable. Having XML a step farther from the main library is a good thing in this respect.

On Wed, Sep 2, 2009 at 1:48 PM, David Pollak <feeder.of.the.bears@gmail.com> wrote:

On Wed, Sep 2, 2009 at 3:37 AM, Alex Blewitt <alex.blewitt@gmail.com> wrote:

Following on from my previous blog post on the subject [1] a few people expressed an interest in being able to help drive this forwards. In the spirit of open and inclusion discussion, we should use an open mailing list to discuss ideas, and so that they're publicly visible after the fact.

It has been suggested (by Miles and Heiko) that we use the scala-internals mailing list to bring this debate to the table. It's a public list, but you need to be subscribed to send messages to the list [2]. By all accounts, this is a relatively low-volume list (in the order of 5 messages per day [3]) so hopefully this won't dissuade people from joining in in order to contribute. If it is, I can be a gateway and forward messages to the list on your behalf.

If the existing membership of scala-internals object or disagree that this is the best place to have the conversation, please let me know and I'll set up a separate Google Groups to discuss this on.

So far, the discussion has just started; Heiko has been OSGi-fying the current Scala modules [4], Josh Suereth has been looking at Mavenizing the Scala build process and modularizing Scala along with it [5], and Roman has taken what Heiko has done and extracted out the Actors, Concurrent, Testing and XML into separate modules [6].

I'm all for splitting Scala's libraries into modules.  One question... doesn't the XML stuff need to stay in the core library as the compiler emits code that depends on this library?
 

I'll wait a day for people to sign up on the list (instructions at [2]) and for any existing members of the scala-internals to object to this use of the mailing list, and then start the ball rolling tomorrow with a document designed to describe the end goals of modularisation.

Alex

[1] http://alblue.blogspot.com/2009/08/modularity-for-scala.html
[2] http://listes.epfl.ch/doc_en.cgi?liste=scala-internals
[3] http://dir.gmane.org/gmane.comp.lang.scala.internals
[4] http://wiki.github.com/hseeberger/scala-lang-osgi
[5] http://wiki.github.com/jsuereth/scala-jigsaw
[6] http://bitbucket.org/romanroe/scala-lang-osgi-modules/

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

--
http://erikengbrecht.blogspot.com/

On Wed, Sep 2, 2009 at 3:24 PM, Roman Roelofsen wrote:
> So why do you need the manifest.mf file? Does the build chain require
> it somehow?

I did the current official OSGi-fication of the Scala compiler and
standard library to support Scala IDE development using the Eclipse
PDE which requires literal manifest files. I want to see the official
OSGi-fication enhanced, nevertheless it has to continue to serve it's
original purpose, and that means that means that literal manifest
files have to be readily available.

But from what Peter Kriens tells me this shouldn't be an issue with
bnd ... it can take a literal manifest file as input and munge it
appropriately.

Cheers,

Miles

> I did the current official OSGi-fication of the Scala compiler and
> standard library to support Scala IDE development using the Eclipse
> PDE which requires literal manifest files.

Yes, but using bnd might remove the need to use the PDE tool chain. I
never used PDE that much so I am not sure what features beyond the
manifest creation people would use.

> ...
> But from what Peter Kriens tells me this shouldn't be an issue with
> bnd ... it can take a literal manifest file as input and munge it
> appropriately.

Yes and no. Bnd is quite powerful and I am sure that we can tell it to
only create a MANIFEST.MF file wherever we want. But there are some
semantic differences when it comes to header interpretation and you have
to explicitly mark places where bnd should extend the configuration (or
leave it completely open). So using an existing MANIFEST.MF won't give
you anything (except some validations I guess).

The build process that I currently can imagine with bnd and pde would be
something like this:

1) Create a bnd configuration file
2) Use bnd to create a MANIFEST.MF file. This file should be considered
to be read-only
3) Utilize the PDE tools somehow based on the MANIFEST.MF
4) Use any jar command to zip the classes and manifest

What AFAIK will *not* work:

1) Create the MANIFEST.MF file with PDE
2) Use bnd to enhance the manifest and to create the jar file

Maybe I miss some of the Scala & Eclipse PDE use cases but at least I
can say that I am a full-time Eclipse and BND user and I never felt the
need to use PDE.

Cheers,

Roman

On Wed, Sep 2, 2009 at 7:04 PM, Roman
Roelofsen wrote:
>> I did the current official OSGi-fication of the Scala compiler and
>> standard library to support Scala IDE development using the Eclipse
>> PDE which requires literal manifest files.
>
> Yes, but using bnd might remove the need to use the PDE tool chain. I
> never used PDE that much so I am not sure what features beyond the
> manifest creation people would use.

The manifest editor is probably the only bit of the PDE that I _don't_ use!

The PDE is a framework for developing, debugging and testing OSGi
based applications. In particular, in a self-hosted context, Eclipse
treats a project which has the same structure as an unpacked OSGi
bundle as if it were an actual executable OSGi bundle at runtime.

This is essential for Scala IDE development, and won't be going away.
I'm open to suggestions which support this scenario using (other)
tools, however.

> The build process that I currently can imagine with bnd and pde would be
> something like this:
>
> 1) Create a bnd configuration file
> 2) Use bnd to create a MANIFEST.MF file. This file should be considered
> to be read-only
> 3) Utilize the PDE tools somehow based on the MANIFEST.MF
> 4) Use any jar command to zip the classes and manifest

OK, this sounds like it could work for me, assuming that the read-only
manifest file is (re-)created periodically at /META-INF/MANIFEST.MF.

Peter seems to think that there are some issues with doing it this way
tho' ... I'd be keen for him to explain them here if possible.

> What AFAIK will *not* work:
>
> 1) Create the MANIFEST.MF file with PDE
> 2) Use bnd to enhance the manifest and to create the jar file

Could you expand on that?

The status quo is that the manifest files are created and maintained
by hand (in effect as text files). Presumably these could be enhanced
by bnd?

> Maybe I miss some of the Scala & Eclipse PDE use cases but at least I
> can say that I am a full-time Eclipse and BND user and I never felt the
> need to use PDE.

You don't do self-hosted development?

Cheers,

Miles

> The manifest editor is probably the only bit of the PDE that I _don't_ use!
>
> The PDE is a framework for developing, debugging and testing OSGi
> based applications. In particular, in a self-hosted context, Eclipse
> treats a project which has the same structure as an unpacked OSGi
> bundle as if it were an actual executable OSGi bundle at runtime.
>
> This is essential for Scala IDE development, and won't be going away.

Hey, we got our first requirement ;-)

> I'm open to suggestions which support this scenario using (other)
> tools, however.

Since I never used PDE that much I hope that someone else can show up
with some ideas ;-)

>> The build process that I currently can imagine with bnd and pde would be
>> something like this:
>>
>> 1) Create a bnd configuration file
>> 2) Use bnd to create a MANIFEST.MF file. This file should be considered
>> to be read-only
>> 3) Utilize the PDE tools somehow based on the MANIFEST.MF
>> 4) Use any jar command to zip the classes and manifest
>
> OK, this sounds like it could work for me, assuming that the read-only
> manifest file is (re-)created periodically at root>/META-INF/MANIFEST.MF.
>
> Peter seems to think that there are some issues with doing it this way
> tho' ... I'd be keen for him to explain them here if possible.

I just had a chat with Peter and it seems that the BND & PDE integration
is tighter than I thought. The above process is mostly ok, except that
2) can have read-write support and be created by PDE.

>> What AFAIK will *not* work:
>>
>> 1) Create the MANIFEST.MF file with PDE
>> 2) Use bnd to enhance the manifest and to create the jar file
>
> Could you expand on that?

As Peter told me, this seems to work. However, only under these contraints:

1) PDE generates e.g. "Import-Package: scala". BND throws away that
statement and generates it by itself, adds version numbers, etc

2) If you want a combination, you need to write "Import-Package:
*,scala". The "*" tells bnd where it can expand. My fear is that PDE
will stumble on "*"

In pratice, 1) should work perfectly well. AFAIK BND will even transform
PDE's Require-Bundle bias to proper Import-Package statements.

> The status quo is that the manifest files are created and maintained
> by hand (in effect as text files). Presumably these could be enhanced
> by bnd?

It seems that the combination works well. You can decide to put as many
information in the MANIFEST for PDE as you want. After this, BND will
apply some of its own semantics to generate the bundles.

>> Maybe I miss some of the Scala & Eclipse PDE use cases but at least I
>> can say that I am a full-time Eclipse and BND user and I never felt the
>> need to use PDE.
>
> You don't do self-hosted development?

Nope. I build with Maven or Ant, collect/assemble the bundles and start
the system with Pax Runner, ModuleFusion or Apache Karaf. For debugging
I use the Java remote debugger concept. I guess I should point out that
I never write Eclipse Plugins/RCP apps. If I would, I probably would use
PDE. I am more a enterprise OSGi/server runtime guy.

Cheers,

Roman

On Wed, Sep 2, 2009 at 8:27 PM, Roman
Roelofsen wrote:

All sounds good ...

>> The status quo is that the manifest files are created and maintained
>> by hand (in effect as text files). Presumably these could be enhanced
>> by bnd?
>
> It seems that the combination works well. You can decide to put as many
> information in the MANIFEST for PDE as you want. After this, BND will
> apply some of its own semantics to generate the bundles.

OK, I think that if we take something like this as the starting point
for the mechanism then we should be in good shape.

>>> Maybe I miss some of the Scala & Eclipse PDE use cases but at least I
>>> can say that I am a full-time Eclipse and BND user and I never felt the
>>> need to use PDE.
>>
>> You don't do self-hosted development?
>
> Nope. I build with Maven or Ant, collect/assemble the bundles and start
> the system with Pax Runner, ModuleFusion or Apache Karaf. For debugging
> I use the Java remote debugger concept. I guess I should point out that
> I never write Eclipse Plugins/RCP apps. If I would, I probably would use
> PDE. I am more a enterprise OSGi/server runtime guy.

I've used the Eclipse PDE for this kind of thing too (a large
server-side OSGi/Equinox based application) and it worked extremely
well: eliminating bundle .jar creation from the edit/compile/test
cycle was a huge win ... I can't recommend it highly enough.

Cheers,

Miles

Hi there,   One idea that may or may not be of interest is support for scala [1] in sigil [2]   Sigil is a build system that uses a simple properties file very similar to BND and uses BND under the hood to do the bundle generation. Sigil itself is built using sigil if you want to take a look around...the files to look for are:   **/sigil.properties **/ivy.xml **/build.xml bldcommon/ivysettings.xml bldcommon/sigil-defaults.properties bldcommon/sigil-repos.properties
I'm actually currently away from base at the moment and the internet connection is lousy at my hotel but I'll try to keep a watch on this thread in case anyone is interested.   Regards,   Dave   [1] http://issues.apache.org/jira/browse/FELIX-1409 [2] http://felix.apache.org/site/apache-felix-sigil.html   On Wed, Sep 2, 2009 at 8:40 PM, Miles Sabin <miles@milessabin.com> wrote:

On Wed, Sep 2, 2009 at 8:27 PM, Roman
Roelofsen<roman.roelofsen@googlemail.com> wrote:
<snip/>

All sounds good ...

>> The status quo is that the manifest files are created and maintained
>> by hand (in effect as text files). Presumably these could be enhanced
>> by bnd?
>
> It seems that the combination works well. You can decide to put as many
> information in the MANIFEST for PDE as you want. After this, BND will
> apply some of its own semantics to generate the bundles.

OK, I think that if we take something like this as the starting point
for the mechanism then we should be in good shape.

>>> Maybe I miss some of the Scala & Eclipse PDE use cases but at least I
>>> can say that I am a full-time Eclipse and BND user and I never felt the
>>> need to use PDE.
>>
>> You don't do self-hosted development?
>
> Nope. I build with Maven or Ant, collect/assemble the bundles and start
> the system with Pax Runner, ModuleFusion or Apache Karaf. For debugging
> I use the Java remote debugger concept. I guess I should point out that
> I never write Eclipse Plugins/RCP apps. If I would, I probably would use
> PDE. I am more a enterprise OSGi/server runtime guy.

I've used the Eclipse PDE for this kind of thing too (a large
server-side OSGi/Equinox based application) and it worked extremely
well: eliminating bundle .jar creation from the edit/compile/test
cycle was a huge win ... I can't recommend it highly enough.

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

On 2 Sep 2009, at 20:40, Miles Sabin wrote:

> On Wed, Sep 2, 2009 at 8:27 PM, Roman
> Roelofsen wrote:
>
>
> All sounds good ...
>
>>> The status quo is that the manifest files are created and maintained
>>> by hand (in effect as text files). Presumably these could be
>>> enhanced
>>> by bnd?
>>
>> It seems that the combination works well. You can decide to put as
>> many
>> information in the MANIFEST for PDE as you want. After this, BND will
>> apply some of its own semantics to generate the bundles.
>
> OK, I think that if we take something like this as the starting point
> for the mechanism then we should be in good shape.

FWIW I've used this kind of approach for other bundles (and despite
the name, PDE isn't just about Eclipse Plugins). The ability to self-
host, particularly during debugging, is an excellent toolkit.

At this point, it's worth throwing another item into the mix; PAX
Construct. This is based on Maven, and utilises PAX-Runner et al to
get off the ground (so easy to test with felix/equinox from the
command line).

http://www.ops4j.org/projects/pax/construct/

This uses BND under the covers to generate the manifest, and when the
install is generated, you end up with a new version of the MANIFEST.MF
in the right place for Eclipse PDE development. (You need to get it to
not generate a new timestamp each time, otherwise the source code
control system goes nuts, but that's easy enough to do).

There's a whole chapter of this at Sonatype's docs on integrating OSGi
with Maven:

http://www.sonatype.com/books/mcookbook/reference/osgi.html

In fact, you can already read about using Scala as a source language
with an OSGi module:

http://www.turmoildrivendevelopment.com/2009/05/osgi-with-scala-java-gro...

I've found this to be an excellent way of building OSGi modules
headlessly with Maven, and given that it uses an OSGi/Maven Scala
library to get off the ground and build it, seems to fit naturally
with the modularisation proposal.

(What didn't work particularly well was the Scala IDE, which is -
currently - based around JDT rather than PDE. Although I could follow
the above tutorial to get a Scala bundle built and off the ground, I
had to resort to editing the Scala code in a text editor.)

I've posted the draft requirements of the modularisation to the Scala
internals list (http://article.gmane.org/gmane.comp.lang.scala.internals/1104
) and not explicitly cc'd those on this mail because it would be
better to move this conversation onto the list. However, I wanted to
leave this pointer in case there are some on the CC list that haven't
yet subscribed ...

In addition, the links to the pax-construct modules here are offered
as a possible compromise between having bundles which are capable of
being self-hosted in PDE whilst at the same time using a generated BND
processed manifest, which seems to be the desired outcome of these two.

I also offered as a discussion point whether it is desirable for Scala
projects to be based on PDE or not, or at least make that as an
option. If we make the Scala IDE capable of using modules created as
in Brian's blog entry, then we might have a simple way of building
future Scala+Java+OSGi bundles.

Anyway ... let's debate the requirements (on the separate thread) and
then, once we've got that defined, we can investigate what tools or
build chains we can use.

Alex