Problem with bumping version number

I'll fix that.

To do so, I'll have to introduce MSIL into the Sabbus build process.
I'll move MSIL files from the MSIL module to the Scala module in SVN.

> OK, So I tried to bump up the major version number in my branch, but
> the quick.comp build fails with the messages shown below.
>
> So it seems the build process is not version clean. Who can help?
>
> Thanks
>

On Wed, May 6, 2009 at 1:11 PM, Gilles Dubochet wrote:
> I'll fix that.
>
> To do so, I'll have to introduce MSIL into the Sabbus build process. I'll
> move MSIL files from the MSIL module to the Scala module in SVN.

Weren't we just talking about using a branch to deal with MSIL vs.
non-MSIL issues? In which case Sabbus would branch along with all the
rest ...

Cheers,

Miles

A temporary workaround is altering the version number check in the
unpickler (in checkVersion()) so that it temporarily accepts both version
numbers, then using that generated compiler to recompile msil.jar, then
revert the checking code to the original. Adding the msil.jar generation to
sabbus is a more practical long-term solution, though.
Toni

Gilles Dubochet wrote:
> I'll fix that.
>
> To do so, I'll have to introduce MSIL into the Sabbus build process.
> I'll move MSIL files from the MSIL module to the Scala module in SVN.
>
>> OK, So I tried to bump up the major version number in my branch, but
>> the quick.comp build fails with the messages shown below.
>>
>> So it seems the build process is not version clean. Who can help?
>>
>> Thanks
>>

Miles Sabin wrote:
> Weren't we just talking about using a branch to deal with MSIL vs.
> non-MSIL issues? In which case Sabbus would branch along with all the
> rest ...
>

Actually, I fail to see how generating a separate MSIL branch for the whole
codebase is a good idea at all. First of all, the code would sneakily
diverge as people over time introduce platform-specific changes to just one
of the two. Second, even if we assume that changes are only made to trunk,
and propagated to a hypothetical msil branch, there is no indication of
where the msil-specific details are just by looking at trunk: any change
there can affect things in an msil-incompatible way, which makes keeping
the two versions aligned even more problematic.

To that, you have to compound the release process: shall we have a 2.8
branch, a 2.9 branch, a 2.8-msil branch, and a 2.9-msil branch? And what if
one more target is added? All that seems totally nonsensical to me.

I would propose instead a different procedure. Right now, there are a bunch
of source files that are crudely overwritten with mostly identical, but
slightly different, msil-specific ones. I would propose that the
platform-specific parts of those files are isolated, and called from an
abstract, platform-independent core codebase. The platform-specific
implementation parts can just be included from a target-specific directory
during compilation. For example:

Right now:

directory pkg/ File A.scala, contains platform-dependant m()

directory msil/pkg/ File A.scala, almost identical, has different m()

When compiling for Java, pkg is used. When compiling for MSIL, the file is
overwritten with msil/pkg/A.scala.

A better approach would be to separate more finely platform-independent and
platform-dependent portions, for example:

directory pkg/ Aabstract.scala, m() calls its abstract method m1()

directory java/pkg/A.scala, extends Aabstract, implements m1()
directory msil/pkg/A.scala, extends Aabstract, implements m1()

The subdirs java/* and msil/* contain only platform-specific code. One of
them is compiled with the rest, depending on the target platform, in a
manner similar to what is happening now with msil. "A" still end up in the
same package.

Advantages:

- platform-dependent parts are clearly visible from the general code
- it becomes a lot easier to add further target platforms, if need be
- no drift with separate codebases
- much easier release process

Just my 2c :)
Toni

On Wed, May 06, 2009 at 02:45:04PM +0200, Antonio Cunei wrote:
> I would propose instead a different procedure. Right now, there are a
> bunch of source files that are crudely overwritten with mostly
> identical, but slightly different, msil-specific ones. I would propose
> that the platform-specific parts of those files are isolated, and
> called from an abstract, platform-independent core codebase. The
> platform-specific implementation parts can just be included from a
> target-specific directory during compilation.

Yes, this is what I had in mind when I wrote:

While on the subject, could we perhaps investigate improving the
architecture architecture? Specifically "if (!forMSIL)" seems like
something that could in most cases be dealt with more nicely by
abstracting out the parts which differ and having msil and java each
mix their own trait into Global. I would be glad to do the work.

I am still happy to do the work, but as it's no small amount of work I'm
not inclined to undertake it unless everyone is in approximate agreement
about what the end result ought to look like.

There must be much to be learned from the great many open source
projects out there which support many more than two platforms. (And I'd
suppose there are even some which support these specific two platforms?)

I agree that branching is a suboptimal solution, but on the other hand
as someone only interested in the java side I can certainly see the
appeal. It's as if I had a neighbor who throws really loud parties once
in a while; I could see the appeal of his being sent to prison for a
crime he didn't commit, because hey, no loud parties, but that doesn't
mean I'd plant a murder weapon in his house. Um, assuming I had one for
some reason.

From the dept of inadequate analogies,

Gilles Dubochet wrote:
>> I'll fix that.
>> To do so, I'll have to introduce MSIL into the Sabbus build process.
>> I'll move MSIL files from the MSIL module to the Scala module in SVN.
>>

Gilles,

I tried to change the version number after your latest changes and to
rebuild starr, but checkVersion() still chokes on the same files.

I can do the msil.jar conversion manually in trunk anytime, but be aware
that Sabbus doesn't seem to be doing the right thing yet (unless more magic
is hidden somewhere).

Toni

>>> error: error while loading AssemblyBuilder, Scala signature
>>> AssemblyBuilder has wrong version
>>> [scalacfork] expected: 5.1
>>> [scalacfork] found: 4.1
>>> [scalacfork] error: error while loading ModuleBuilder, Scala signature
>>> ModuleBuilder has wrong version
>>> [scalacfork] expected: 5.1
>>> [scalacfork] found: 4.1
>>> [scalacfork] error: error while loading ILGenerator, Scala signature
>>> ILGenerator has wrong version
>>> [scalacfork] expected: 5.1
>>> [scalacfork] found: 4.1

> I tried to change the version number after your latest changes and
> to rebuild starr, but checkVersion() still chokes on the same files.
>
> I can do the msil.jar conversion manually in trunk anytime, but be
> aware that Sabbus doesn't seem to be doing the right thing yet
> (unless more magic is hidden somewhere).

I should have mentioned it in the commit log: if you need to rebuild
MSIL as part of the Sabbus build process, you must add the "newlibs"
target before any other ant target. For example, you can build using
"ant newlibs build" or "ant newlibs test". I used an explicit flag so
that the normal case, where MSIL needn't be rebuilt, is as fast as
before.

Once the compiler generating incompatible class files is ready, you
can create a new STARR compiler using "ant newlibs newstarr", which
will replace the STARR standard library and compiler, as well as
"msil.jar".

Cheers,
Gilles.

Gilles Dubochet wrote:
> I should have mentioned it in the commit log: if you need to rebuild
> MSIL as part of the Sabbus build process, you must add the "newlibs"
> target before any other ant target. For example, you can build using
> "ant newlibs build" or "ant newlibs test". I used an explicit flag so
> that the normal case, where MSIL needn't be rebuilt, is as fast as before.
>

Uhm. That doesn't seem to work as expected. Also, ant all.clean does not
seem to remove build/libs. Can you please try it again?

My log is the following:

$ svn co https://lampsvn.epfl.ch/svn-repos/scala/scala/trunk -r17657
[...]
A trunk/README
A trunk/build.xml
U trunk
Checked out revision 17657.
$ cd trunk
$ nano -w src/compiler/scala/tools/nsc/symtab/classfile/PickleFormat.scala
$ fgrep MajorVersion
src/compiler/scala/tools/nsc/symtab/classfile/PickleFormat.scala
val MajorVersion = 5
$ ant newlibs test
Buildfile: build.xml

newlibs:

init:
[echo] Build number is '2.8.0.r17657-b20090507121630'
[echo] Built 7 May 2009, 12:16:30 from revision 17657 with Java
HotSpot(TM) Server VM 1.5.0_16

locker.start:

[...]
quick.comp:
[mkdir] Created dir:
/localhome/cunei/test/trunk/build/quick/classes/compiler
[scalacfork] Compiling 215 files to
/localhome/cunei/test/trunk/build/quick/classes/compiler
[scalacfork] error: error while loading AssemblyBuilder, Scala signature
AssemblyBuilder has wrong version
[scalacfork] expected: 5.1
[scalacfork] Compiling 95 files to
/localhome/cunei/test/trunk/build/quick/classes/library
[propertyfile] Creating new property file:
/localhome/cunei/test/trunk/build/quick/classes/library/library.properties
[stopwatch] [quick.lib.timer: 1:52.644 sec]

[...]
libs.msilpack:
[jar] Building jar: /localhome/cunei/test/trunk/build/libs/msil.jar

libs.done:

quick.libs:

quick.pre-comp:

quick.comp:
[mkdir] Created dir:
/localhome/cunei/test/trunk/build/quick/classes/compiler
[scalacfork] Compiling 215 files to
/localhome/cunei/test/trunk/build/quick/classes/compiler
[scalacfork] error: error while loading AssemblyBuilder, Scala signature
AssemblyBuilder has wrong version
[scalacfork] expected: 5.1
[scalacfork] found: 4.1
...etc...