Usage of Scala in JavaEE environments

Leo,

my experience is that the quality of async I/0 support is indeed web container
specific

I implemented a simple feature that warns a user when his session is
about to expire (using Lift, and Lift uses (his own kind of (?)) Actors)

Anyway,
the app worked fine with Jetty and Tomcat, but it had issues with Glassfish

Luc

On Fri, Sep 24, 2010 at 6:13 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi,

I'm a Scala newbie. I'd like to ask a question that was probably asked a few
times already, but I couldn't find any elaborative answer by searching Scala
mailing lists.

I'd like to deploy my Scala code in JavaEE environment. I'm interested in
deployment on both HTTP servlets and EJB containers.

Specifically, I'm wondering if it is possible to use such features of Scala
like actors and continuations in the mentioned environments. As far as I
understand, these features require some sort of Scala-specific runtime
support (e.g. in form of a dedicated runtime support library). More over,
actors probably need ability to use Java multi-threading (Executors,etc)
which may be problematic in EJB containers, where it is forbidden by the EJB
specification. But even in the servlet containers, there is probably some
tweaking required to use application server threads instead of raw Java
threads...

Note: I'm aware of the fact that there are Scala specific Web frameworks,
but I have to use a JavaEE enviroment because it is a customer requirement
who has a lot of legacy code written for JavaEE. It should be possible to
reuse, interact with this code.

The goal I'm trying to achieve is:
 I'd like to be able to implement a system for asynchronous requests
processing, similar in spirit to Jetty continuations, but a bit more
generic, i.e. supporting more protocols (not only HTTP) and able to
suspend/resume mutiple times during request processing. My idea was to use
Scala actors and continuations for e.g. non-blocking, asynchronous IO: Each
request gets an instance of a light-weight actor for processing it. All
actors are scheduled on a dedicated thread pool with a small number of
threads. Each request processing actor is a pretty much straight-line code,
which eventually invokes NIO, async IO operations.  In such a case, I'd just
by means of Scala's continuations support suspend current flow of execution
inside actor and then resume it later once a notification about IO operation
compeletion has arrived.
 Does this approach makes sense? Are there any specific limitations related
to the current support for actors and continuations that I should be aware
of?

 Thanks for any help,
  Leo Romanoff
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2553702.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
   __~O
  -\ <,
(*)/ (*)

reality goes far beyond imagination

Luc,

Thanks for your reply.

But the focus of my question was not that much on the async IO as such, but
about the ability to use Scala's continuations and actors (btw, which ones
are allowed there - event-based or thread-based?) inside JavaEE environments
and possible limitations or shortcomings when doing that. May be there are
also some other Scala features that should be used with care in such
environments.

Overall, I'm wondering about the options for doing integration of Scala code
with the rest of JavaEE. For example, can async JMS comminication channel be
combined with a Scala actor that would get the messages and process them?
What about integration with EJB containers or with XML Web Services? To put
it simply: Where one can use Scala instead of Java WITHOUT ANY MAJOR
RESTRICTIONS in Scala's feature usage (e.g. not only purely syntactic, but
also runtime features like actors and continuations are still available) for
development of JavaEE components and where it is not so advisable? With
limitations of its feature set, I guess Scala can be used anywhere, where
Java can be used. But then the restricted Scala subset may be almost equal
to Java (modulo Scala's syntactic sugar). In this case, the benefit of
writing such components in Scala becomes arguable.

From what I've read about Scala so far, I got the impression that to use
Scala runtime features, one needs to use special Scala-based containers or
app. servers, which are not JavaEE based or JavaEE compliant. I.e. in this
sense, Scala exists in its own world and integration with JavaEE is not
quite easy yet. But may be I misunderstand the current situation and
integration with JavaEE is much easier than I think?

Another topic I tried to scratch was about the idea of implementing
non-blocking IO waiting and async request processing on the server-side by
means of actors and using Scala's (delimited) continuations. I've seen some
examples of such things implemented e.g. by means of Javaflow continuations
framework, but doing it using a language with native continuations support
is probably much easier, cleaner, efficient and more elegant. As a teaser:
Can something like Jetty continuations be implemented in a natural way by
means of Scala continuations?

-Leo

Luc Duponcheel wrote:
>
> Leo,
>
> my experience is that the quality of async I/0 support is indeed web
> container specific
>
> I implemented a simple feature that warns a user when his session is
> about to expire (using Lift, and Lift uses (his own kind of (?)) Actors)
>
> Anyway, the app worked fine with Jetty and Tomcat, but it had issues with
> Glassfish
>
> Luc
>
>

Leo,

The short answer is "it'll just work".

The longer answer is http://blog.lostlake.org/index.php?/archives/73-For-all-you-know,-its-just-another-Java-library.html

The even longer answer is the Actors are "just" a library (just is in quotes because the library is a complex and powerful piece of work, but to the JVM, it's a bunch of valid byte-code that calls a bunch of method in the Java standard library) on top of JVM threads, so as far as your contain knows, you're just creating a few threads and calling some methods.

Continuations are the same (except they don't even need separate threads).  Continuations result in a bunch of classes with particular interfaces (FunctionXX). 

Scala does no runtime byte-code rewriting or other magic.  Scala results in legal bytecode that executes on and 1.5+ JVM and interoperates with any Java library.

On Fri, Sep 24, 2010 at 12:58 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

Luc,

Thanks for your reply.

But the focus of my question was not that much on the async IO as such, but
about the ability to use Scala's continuations and actors (btw, which ones
are allowed there - event-based or thread-based?) inside JavaEE environments
and possible limitations or shortcomings when doing that. May be there are
also some other Scala features that should be used with care in such
environments.

Overall, I'm wondering about the options for doing integration of Scala code
with the rest of JavaEE. For example, can async JMS comminication channel be
combined with a Scala actor that would get the messages and process them?
What about integration with EJB containers or with XML Web Services? To put
it simply: Where one can use Scala instead of Java WITHOUT ANY MAJOR
RESTRICTIONS in Scala's feature usage (e.g. not only purely syntactic, but
also runtime features like actors and continuations are still available) for
development of JavaEE components and where it is not so advisable? With
limitations of its feature set, I guess Scala can be used anywhere, where
Java can be used. But then the restricted Scala subset may be almost equal
to Java (modulo Scala's syntactic sugar). In this case, the benefit of
writing such components in Scala becomes arguable.

From what I've read about Scala so far, I got the impression that to use
Scala runtime features, one needs to use special Scala-based containers or
app. servers, which are not JavaEE based or JavaEE compliant. I.e. in this
sense, Scala exists in its own world and integration with JavaEE is not
quite easy yet. But may be I misunderstand the current situation and
integration with JavaEE is much easier than I think?

Another topic I tried to scratch was about the idea of implementing
non-blocking IO waiting and async request processing on the server-side by
means of actors and using Scala's (delimited) continuations. I've seen some
examples of such things implemented e.g. by means of Javaflow continuations
framework, but doing it using a language with native continuations support
is probably much easier, cleaner, efficient and more elegant. As a teaser:
Can something like Jetty continuations be implemented in a natural way by
means of Scala continuations?

-Leo


Luc Duponcheel wrote:
>
> Leo,
>
> my experience is that the quality of async I/0 support is indeed web
> container specific
>
> I implemented a simple feature that warns a user when his session is
> about to expire (using Lift, and Lift uses (his own kind of (?)) Actors)
>
> Anyway, the app worked fine with Jetty and Tomcat, but it had issues with
> Glassfish
>
> Luc
>
>
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2584162.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

Yes and No.

Yes, scala makes a great J2EE programming environment. It has access to the
entire J2EE and Java APIs and libraries. Deployment wise, it's YAJ,
yet-another-jar.

I believe Lift is servlet API-friendly, so that takes care of your web
layer.

No, you should not use actors in J2EE. That's what MDBs are for.

No, you should not use continuations in J2EE, that's what BPEL and similar
are for.

The E for Enterprise means visibility, management, administration...hence
BPEL and MDBs rather than continuations and actors.

Cheers,
Razie

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Hey - here's a good opportunity for an open project: actor DSL mapped to
MDB/JMS to simplify that message-based programming...

I mean async a ! m and the sync bridge a !? m...although this now has state
which needs backed up...interesting, eh?

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

On Sat, Sep 25, 2010 at 11:05 PM, Razvan Cojocaru-2 <pub@razie.com> wrote:

Hey - here's a good opportunity for an open project: actor DSL mapped to
MDB/JMS to simplify that message-based programming...

I mean async a ! m and the sync bridge a !? m...although this now has state
which needs backed up...interesting, eh?

One link to rule them all: http://doc.akkasource.org/camel
 

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys:  http://scripster.razie.com Scripster  and
http://github.com/razie/gremlins Gremlins
Follow me:  http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter ,  http://github.com/razie GitHub .

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2713877.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

NICE!

I had forgotten about camel ... will take a look at how it evolved.

How can I bind an actor to be the body of an MDB? Would that be a special
scheduler? I still want container-specific handling of failures, poison
messages, transactions etc...

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

David,

Thanks for your explanations.

dpp wrote:
>
> Leo,
>
> The short answer is "it'll just work".
>
> The longer answer is
> http://blog.lostlake.org/index.php?/archives/73-For-all-you-know,-its-ju...
>
> The even longer answer is the Actors are "just" a library (just is in
> quotes
> because the library is a complex and powerful piece of work, but to the
> JVM,
> it's a bunch of valid byte-code that calls a bunch of method in the Java
> standard library) on top of JVM threads, so as far as your contain knows,
> you're just creating a few threads and calling some methods.
>

I see. I understand now that for a JVM and Servlet container it is just a
bunch of java classes creating/using threads.

But what about usage of Actors inside an EJB container, e.g. if an EJB would
call it? AFAIK, according to the EJB spec, EJB containers should not permit
explicit thread-management (e.g. thread creation) by user-code as it may
lead to conflicts.

dpp wrote:
>
> Continuations are the same (except they don't even need separate threads).
> Continuations result in a bunch of classes with particular interfaces
> (FunctionXX).
>
> Scala does no runtime byte-code rewriting or other magic. Scala results
> in
> legal bytecode that executes on and 1.5+ JVM and interoperates with any
> Java
> library.
>

OK. I wrongly assumed that Scala continuations would do some sort of a
dynamic invocation stack capturing at run-time, as Javaflow or Kilim
frameworks do. If this would be the case, then I would expect problems as
JavaEE containers have no clue about continuations and capturing deep
invocation stacks including container code invocations does not make too
much sense. Even if such deep stack snapshot would be taken, it most likely
could not be resumed later, as JavaEE container is not designed with this in
mind. But probably this is not required for Scala continuations, as all
transformations are done at compile time and depth of stacks to be captured
is limited (that's why thay are called delimited continuations, right?).

I'm still wondering about implementing something like Jetty continuations,
but in a protocol independent way. I'd like to generalize/extend what Jetty
does for handling of HTTP protocol requests to cover other protocols (I
guess it would eventually go beyond standard JavaEE containers). The goal
is to be able to escape the thread-per-request approach to achieve better
scalability. And this of course also requires the ability to avoid any
blocking waiting for results of long IO operations, because it would occupy
a worker thread without any reason, preventing it from processing other
requests in the mean time. My idea was to suspend/resume the actor
processing the request, similar to Jetty continuations. It would be also
nice to do it transparently for a developer, so that the code still looks
like a usual sequential code, but in reality suspends/resumes on long async
IO operations. And I thought that Scala continuations could be used to
implement this. So that one could write in the actor body something like:

exec_pre_io_code();
// the following invocation would suspend the current actor. Later, when a
notification about IO operation
// completion is received, current actor will be resumed and would continue
with exec_after_io_code()
exec_long_io_operation(io_operation_params);
exec_after_io_code();

It feels like something very similar to CPS transformation done by Scala
compiler's continuations plugin is required here. Or?

So, is it possible to implement something like this using Scala
continuations? Is it the right way to go to achieve the desired result? May
be there are better ways to solve it?

Viktor Klang wrote:
>
> On Sat, Sep 25, 2010 at 11:05 PM, Razvan Cojocaru-2 wrote:
>
>>
>> Hey - here's a good opportunity for an open project: actor DSL mapped to
>> MDB/JMS to simplify that message-based programming...
>>
>> I mean async a ! m and the sync bridge a !? m...although this now has
>> state
>> which needs backed up...interesting, eh?
>>
> One link to rule them all: http://doc.akkasource.org/camel
>

Very cool!!! From the usability and ease of programming points of view, it
looks really impressive. I'll try to play with it and see how good it
performs.

-Leo

Razvan Cojocaru-2 wrote:
>
> No, you should not use actors in J2EE. That's what MDBs are for.
>

I'm not quite convinced. On the performance side, I think MDBs are too heavy
weight and do not scale under load as good as Actors.
And then, using MDBs explicitly leads to a rather tight coupling to JMS. I'd
rather prefer a higher abstraction which hides the concrete protocol used
for communication. Eventually, I'd like to use another protocol later (e.g.
Comet, RMI, web services, REST, etc). In this sense, Akka Camel link was
quite interesting, because it hides concrete protocols behind actor's
facade.

Razvan Cojocaru-2 wrote:
>
> No, you should not use continuations in J2EE, that's what BPEL and similar
> are for.
>

Yes, BPEL is a more declarative and transparent way to define workflows. But
it is mostly limited to XML web serviced domain (and there are many more
domains, where you want to define workflows), has rather limited access to
the rest of Java platform (if you only use BPEL standard, without any vendor
specific extensions). And, according to my experience, BPEL peroformance is
not exactly very impressive or scalable. And I'm interested in building a
very scalable system. It should be able to handle e.g. 50000 or 100000
simultaneous requests with minimal latency and using a minmal number of
nodes.

BTW, any links to highly scalable workflow engines running on JVM that can
coupe easily with such number of requests? I'd be very interested to know
more about them.

Thanks,
-Leo

Why is a thread a different kind of resource than say a place in a
queue waiting for a reply, to go to so much trouble? That's its
meaning: capture execution state and store it in an execution queue
until a core gets bored...why build different abstractions which are
not scalable or fail-overable? Containers tie sessions and invocations
to...yeah, threads!

If you need state to span threads, I would use a workflow of some
kind, especially the kinds that persist state transitionally and can
recover it on failure or migration...

On 9/25/10, Leo Romanoff wrote:
>
> David,
>
> Thanks for your explanations.
>
>
> dpp wrote:
>>
>> Leo,
>>
>> The short answer is "it'll just work".
>>
>> The longer answer is
>> http://blog.lostlake.org/index.php?/archives/73-For-all-you-know,-its-ju...
>>
>> The even longer answer is the Actors are "just" a library (just is in
>> quotes
>> because the library is a complex and powerful piece of work, but to the
>> JVM,
>> it's a bunch of valid byte-code that calls a bunch of method in the Java
>> standard library) on top of JVM threads, so as far as your contain knows,
>> you're just creating a few threads and calling some methods.
>>
>
> I see. I understand now that for a JVM and Servlet container it is just a
> bunch of java classes creating/using threads.
>
> But what about usage of Actors inside an EJB container, e.g. if an EJB would
> call it? AFAIK, according to the EJB spec, EJB containers should not permit
> explicit thread-management (e.g. thread creation) by user-code as it may
> lead to conflicts.
>
>
> dpp wrote:
>>
>> Continuations are the same (except they don't even need separate threads).
>> Continuations result in a bunch of classes with particular interfaces
>> (FunctionXX).
>>
>> Scala does no runtime byte-code rewriting or other magic. Scala results
>> in
>> legal bytecode that executes on and 1.5+ JVM and interoperates with any
>> Java
>> library.
>>
>
> OK. I wrongly assumed that Scala continuations would do some sort of a
> dynamic invocation stack capturing at run-time, as Javaflow or Kilim
> frameworks do. If this would be the case, then I would expect problems as
> JavaEE containers have no clue about continuations and capturing deep
> invocation stacks including container code invocations does not make too
> much sense. Even if such deep stack snapshot would be taken, it most likely
> could not be resumed later, as JavaEE container is not designed with this in
> mind. But probably this is not required for Scala continuations, as all
> transformations are done at compile time and depth of stacks to be captured
> is limited (that's why thay are called delimited continuations, right?).
>
> I'm still wondering about implementing something like Jetty continuations,
> but in a protocol independent way. I'd like to generalize/extend what Jetty
> does for handling of HTTP protocol requests to cover other protocols (I
> guess it would eventually go beyond standard JavaEE containers). The goal
> is to be able to escape the thread-per-request approach to achieve better
> scalability. And this of course also requires the ability to avoid any
> blocking waiting for results of long IO operations, because it would occupy
> a worker thread without any reason, preventing it from processing other
> requests in the mean time. My idea was to suspend/resume the actor
> processing the request, similar to Jetty continuations. It would be also
> nice to do it transparently for a developer, so that the code still looks
> like a usual sequential code, but in reality suspends/resumes on long async
> IO operations. And I thought that Scala continuations could be used to
> implement this. So that one could write in the actor body something like:
>
> exec_pre_io_code();
> // the following invocation would suspend the current actor. Later, when a
> notification about IO operation
> // completion is received, current actor will be resumed and would continue
> with exec_after_io_code()
> exec_long_io_operation(io_operation_params);
> exec_after_io_code();
>
> It feels like something very similar to CPS transformation done by Scala
> compiler's continuations plugin is required here. Or?
>
> So, is it possible to implement something like this using Scala
> continuations? Is it the right way to go to achieve the desired result? May
> be there are better ways to solve it?
>
>
>
> --
> View this message in context:
> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

We have one, in-house...very efficient, nicely fail-overable and
load-balanceable, does JMs and EJB natively. It's heavy on the DB
though...I could inquire about licensing. Active BPEL was nice too,
last time I looked at it.

100k simultaneous requests? Assuming you should keep that many ports
open to client and that this is not google style (you implied
downstream ports to DB via I/O)...nice system to design! You're
looking at a reasonably sized cluster. See how much you can negociate
the fail-over-able aspect per request type. That would buy you lots of
resources.

By keeping the ports open I meant that you actually want 100k sockets
open simultaneousky rather that accept say 1000 per node and let TCP
queue the rest.

Cheers

On 9/25/10, Leo Romanoff wrote:
>
>
> Razvan Cojocaru-2 wrote:
>>
>> No, you should not use actors in J2EE. That's what MDBs are for.
>>
>
> I'm not quite convinced. On the performance side, I think MDBs are too heavy
> weight and do not scale under load as good as Actors.
> And then, using MDBs explicitly leads to a rather tight coupling to JMS. I'd
> rather prefer a higher abstraction which hides the concrete protocol used
> for communication. Eventually, I'd like to use another protocol later (e.g.
> Comet, RMI, web services, REST, etc). In this sense, Akka Camel link was
> quite interesting, because it hides concrete protocols behind actor's
> facade.
>
>
>
> Razvan Cojocaru-2 wrote:
>>
>> No, you should not use continuations in J2EE, that's what BPEL and similar
>> are for.
>>
>
> Yes, BPEL is a more declarative and transparent way to define workflows. But
> it is mostly limited to XML web serviced domain (and there are many more
> domains, where you want to define workflows), has rather limited access to
> the rest of Java platform (if you only use BPEL standard, without any vendor
> specific extensions). And, according to my experience, BPEL peroformance is
> not exactly very impressive or scalable. And I'm interested in building a
> very scalable system. It should be able to handle e.g. 50000 or 100000
> simultaneous requests with minimal latency and using a minmal number of
> nodes.
>
> BTW, any links to highly scalable workflow engines running on JVM that can
> coupe easily with such number of requests? I'd be very interested to know
> more about them.
>
> Thanks,
> -Leo
>
> --
> View this message in context:
> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

Razvan Cojocaru-2 wrote:
>
> Why is a thread a different kind of resource than say a place in a
> queue waiting for a reply, to go to so much trouble? That's its
> meaning: capture execution state and store it in an execution queue
> until a core gets bored...why build different abstractions which are
> not scalable or fail-overable? Containers tie sessions and invocations
> to...yeah, threads!
>

IMHO, one-thread-per-session does not scale when it comes to the number of
simultaneous requests that can be processed. Plus, in situations, where
thread waits for results of IO operations or external service invocations
most of the time, threads just block without doing anything useful and CPU
power is wasted. It may even turn out that other tasks cannot be processed
in the meantime, as all threads are already occupied and are waiting.

Razvan Cojocaru-2 wrote:
>
> If you need state to span threads, I would use a workflow of some
> kind, especially the kinds that persist state transitionally and can
> recover it on failure or migration...
>

You make a point, when you talk about using a workflow with persistent
state.
This is to some extent what I'm trying to implement :-) Basically, there is
no existing workflow engine that satisfies my needs, therefore I'd like to
create one. If there are good reasons, I'd even give up on JavaEE
compliance, when implementing it.

So, some of my questions could be understood in the sense:
- Is Scala a good tool for building such a workflow engine system which is
able to persist state transitionally, does not really block OS/Java threads
on synchronous operations and uses NIO instead? Specifically, can usage of
actors and continuations make a possible workflow engine implementation
cleaner and simpler??? I'm under the impression that it should be the case,
as I've seen nice sketches implemented by means of F#'s asynchronous
workflows (and they are based on the transformation by compiler doing a
rewriting of code into CPS form) see e.g.
http://www.slideshare.net/mdlm/understanding-f-workflows or
http://blogs.msdn.com/b/dsyme/archive/2007/10/11/introducing-f-asynchron...).

It would be cool, just to suspend and persist the current state (i.e. a
continuation of a workflow) at any point (e.g. on waiting for a result of a
long lasting IO operation or invocation of an external service), and later
just resume it once results are ready. If this would be possible, then a
mapping from an abstract workflow like BPEL script to Scala could be rather
staight forward. And the workflow steps would look rather sequential even
though it is not the case under the hood (in reality, NIO and the like will
be used).

BTW, is it possible in Scala to save the state of the current actor by
taking a continuation and persisting/serializing it to a filesystem or DB,
so that it can be fetched from there and continued (much) later? Would it be
possible to do it on a different instance of the JVM or only on the same
instance?

Without continuations and CPS rewriting supported by the programming
language, most workflow engines are implemented simply by doing/modelling
these transformations by hand. I've done it this way as well and it even
works, but it is really painful, very time consuming and error-prone. It
was OK to do it just at the couple of places (where I had to split my Java
functions into beforeIO and afterIO parts at the place of long IO calls and
implemented the state persistence between these two parts as well as IO
completion notification to trigger the afterIO part, thus modelling closures
and continuations by hand). But doing this at a dozen of places would
certainly be a nightmare. You really need a compiler/tooling/language
support for such exercises. That's why Scala compiler's continuations plugin
or Kilim bytecode rewriter seem to be so promising IMHO.

-Leo

Now you're talking :)

As it happens, there is such a workflow in scala :) checkout my little pet
project http://blog.razie.com/2010/09/scala-workflow-engine-and-dsl.html

We can add persistence and transactionality easily - actually that's what I
was going to do next. Workflows are much easier and lighter to persist than
continuations, IMHO, because their state (if history is not needed, which
continuations cannot capture anyways) can be captured in one word (per
parallel branch)...more or less... so you're looking at updating one word in
a transactional database - the protocol takes more traffic :).

And yes, it uses a thread pool, so workflows don't block a thread. It was
using actors until last week when I had to kill long running activities and
couldn't figure out how to...kill an actor... ethically.... :(

About the threads though, the point is that CPU is not wasted, as long as at
least a few threads actually do process stuff, the other hundreds can wait
for whatever. We customarily use 200-300 theads per JVM in our deployments,
to keep web users happy, running on clusters of 64 core T2k.

If you're interested in the gremlins workflow thing, fork it and let me
know... I'll look at the F# workflows, didn't know about them...just found
out about F# recently...hmm talk about competing platforms threatening
scala. I actually have my own theory about the very negative influence
Oracle has - it basically owns the entire J2EE stack now, from hardware, db,
app server, language everything. A lot of our customers, long time unix
shops are now quite twitchy...those that license Oracle products may know
why...

Hmm...scala may prove the wrong bet in the long run...but time will tell.
For now I love it and there's plenty of JVM stuff around, so...well, sorry
for the rant.

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

> Now you're talking :)
> As it happens, there is such a workflow in scala :)

:-)

> checkout my little pet project
> http://blog.razie.com/2010/09/scala-workflow-engine-and-dsl.html

I'll have a look at it.

> We can add persistence and transactionality easily - actually that's what
> I was going to do next.

Cool!

> Workflows are much easier and lighter to persist than continuations, IMHO,
> because their state (if
> history is not needed, which continuations cannot capture anyways) can be
> captured in one word (per
> parallel branch)...more or less... so you're looking at updating one word
> in a transactional database -
> the protocol takes more traffic :).

In pinciple I agree. But I'm not so sure about the history. I'd like to
store it for workflow debugging purposes at least. But this is another
issue. No need to discuss it at this moment.
Also, your "one word" approach assumes static workflow, i.e. (the topology
of) all paths and steps are known in advance. Im also looking into more
dynamic workflows. They may require more information to be saved eventually.

> And yes, it uses a thread pool, so workflows don't block a thread. It was
> using actors until last week
> when I had to kill long running activities and couldn't figure out how
> to...kill an actor... ethically.... :(

Interesting! Wouldn't this work by sending some sort of a special message to
this actor? Can you clarify a bit the exact problem?

> About the threads though, the point is that CPU is not wasted, as long as
> at least a few threads
> actually do process stuff, the other hundreds can wait for whatever. We
> customarily use 200-300
> theads per JVM in our deployments, to keep web users happy, running on
> clusters of 64 core T2k.

I see your point. I haven't looked into clustering that much yet. I was
thinking mostly of getting max of one node. Obviously, at some point the
node will be overloaded and you need to start clustering.

> If you're interested in the gremlins workflow thing, fork it and let me
> know...

Thanks! I'll have a look at it.

> I'll look at the F#
> workflows, didn't know about them...just found out about F# recently...hmm
> talk about competing
> platforms threatening scala. I actually have my own theory about the very
> negative influence Oracle
> has - it basically owns the entire J2EE stack now, from hardware, db, app
> server, language everything.
>A lot of our customers, long time unix shops are now quite twitchy...those
that license Oracle products
> may know why...
> Hmm...scala may prove the wrong bet in the long run...but time will tell.
> For now I love it and there's
> plenty of JVM stuff around, so...well, sorry for the rant.

I know exactly what you mean and share your feelings! ;-) Still I think in
the enterprise & telco world, J2EE will stay for a while. There is so much
legacy in Java that you cannot just replace or migrate it easily. But for
the desktops and web applications, who knows if Scala/java are the right bet
...

Slightly off-topic, but related: There is quite some progress in the
.Net/CLR world during last few years. I was surprised to see it. F#, Axum
and some other projects are quite interesting. FP influence seems to grow
inside Microsoft. C# evolves also much faster than Java. Let's see where it
will end up in a few years.

-Leo

>Also, your "one word" approach assumes static workflow, i.e. (the topology
of) all paths and steps are known in advance. Im
>also looking into more dynamic workflows. They may require more information
to be saved eventually.

Opa. Normally, in a workflow, you pre-define the paths and at runtime just
choose one branch or another. Are you thinking to modify the definition on
the fly? You could just choose sub-workflows instead...you'd rather define
them on the fly? Interesting...haven't seen that approach before.

Actually, in the distributed part, the remote workflow structures are
unknown to the parent and that could give you the dynamic behaviour - once I
get there :)

> And yes, it uses a thread pool, so workflows don't block a thread. It was
> using actors until last week
> when I had to kill long running activities and couldn't figure out how
> to...kill an actor... ethically.... :(

>Interesting! Wouldn't this work by sending some sort of a special message
to this actor? Can you clarify a bit the exact
>problem?

Learning about "kill -9" was one of the best moments of my life...I hate
slow processors, I.O and stuff...I'm not genius, but smart enough to work
around slowness in real-time and an ill-timed popup can ruin my day...I love
things that die the moment you ask them to, which is not the case for
instance with Eclipse's build threads, or I.E. tabs sometimes, eh?

Unnatural death is a normal state of affairs on the internet. This little
workflow of mine will have it as a built-in assumption.

Mental picture: every active gremlin rides the bomb and yells YYYYEEE-HAAA,
Slim Pickens style :)

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

http://en.wikipedia.org/wiki/File:Slim-pickens_riding-the-bomb_enh-lores...

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

I'm sorry - I have to reconsider my opinion on Microsoft vs Oracle... even
backed by Microsoft, F# is much to ugly to survive! It's worse than I would
envision Basic# to have been.

Who designs this junk? why not just make up Haskell# ? it makes more sense!

F# asynchronous "workflows" ? seriously?

Ride the bomb, gremlins!!!

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

> I'm sorry - I have to reconsider my opinion on Microsoft vs Oracle... even
backed by Microsoft, F# is
> much to ugly to survive! It's worse than I would envision Basic# to have
> been
> Who designs this junk?

F# is just a .Net version of the OCaml
(http://caml.inria.fr/ocaml/index.en.html), which in turn is one of the best
implementations of ML-based languages. OCaml is known to produce very
efficient native code, almost achieving C speed. And OCaml is first of all a
functional language (not lazy by default, as Haskell), but with OO elements.
And in F#, Microsoft introduced a few new elements for better integration
with the rest of .Net platform.

Whatever F# is, if you look at the amount of explicit type annotations, it
is very small. It really infers types much better as it is functions and
uses Milner's type system. So, in this sense at least, it (and any proper FP
language like Haskell) wins hands down compared to Scala, where you have to
write types all over the place to help compiler. But of course, it probably
sucks on the OO side.

> why not just make up Haskell# ? it makes more sense!

There is actually Haskell for .Net platform! And a few other languages :-)
http://dotnetpowered.com/languages.aspx

> F# asynchronous "workflows" ? seriously?

Well, of course their "workflows" are not workflows in the same sense as
e.g. BPEL. They are more a way for asynchronous programming and lazy or
deferred computations. But I've seen examples how they can be used for
implementing something like BPEL rather easily.

On 27/09/2010 06:31, Leo Romanoff wrote:
> There is actually Haskell for .Net platform! And a few other languages :-)
> http://dotnetpowered.com/languages.aspx

An impressive list, I didn't know there was so much languages for this platform, some
being remixes of old languages (COBOL...), others being original.
Although I see holes, I don't see Fantom for example (ex Fan language).

And well, I suppose the list of JVM languages isn't short either.

F# workflows are just the F# equivalent of Haskell's do notation, or
Scala's foreach magic on monads. Don Syme even remarks in expert f#
that the reason they're called workflows is because he talked to Simon
Peyton-Jones and they agreed that a large source of the confusion
surrounding monads is that people get caught up in the terminology.

If you spend a little time with f#, you quickly come to appreciate the
expressiveness and beauty of the language. Sure, it looks ugly to
begin with, but that's because it's much less c-like than, for
instance, scala.

On Mon, Sep 27, 2010 at 9:12 AM, Philippe Lhoste wrote:
> On 27/09/2010 06:31, Leo Romanoff wrote:
>>
>> There is actually Haskell for .Net platform! And a few other languages :-)
>> http://dotnetpowered.com/languages.aspx
>
> An impressive list, I didn't know there was so much languages for this
> platform, some being remixes of old languages (COBOL...), others being
> original.
> Although I see holes, I don't see Fantom for example (ex Fan language).
>
> And well, I suppose the list of JVM languages isn't short either.
>
> --
> Philippe Lhoste

Its possile. I went through a few tutorials and they made up too many
non-sensical keywords sprinkled all over the place, borrowing from too
many languages. I went through the historyand did go through ML and
OCAML and those are much nicer _and regular_ in comparison.

"member x.Value" ? Seriously?

Anyways, messing with the semantics or meaning that most people attach
to certain words, like "workflows" is not a brilliant move in my mind.
its something else (hitching a ride, marketing...nothing
professional). I read that reasoning as well and they could have
called it "execution path" or "execution strategy" or whatever else.

It's possible they had other restrictions in the Microsoft universe
but the outcome is a verbose, non regular language.

On 9/27/10, Alec Zorab wrote:
> F# workflows are just the F# equivalent of Haskell's do notation, or
> Scala's foreach magic on monads. Don Syme even remarks in expert f#
> that the reason they're called workflows is because he talked to Simon
> Peyton-Jones and they agreed that a large source of the confusion
> surrounding monads is that people get caught up in the terminology.
>
> If you spend a little time with f#, you quickly come to appreciate the
> expressiveness and beauty of the language. Sure, it looks ugly to
> begin with, but that's because it's much less c-like than, for
> instance, scala.
>
> On Mon, Sep 27, 2010 at 9:12 AM, Philippe Lhoste wrote:
>> On 27/09/2010 06:31, Leo Romanoff wrote:
>>>
>>> There is actually Haskell for .Net platform! And a few other languages
>>> :-)
>>> http://dotnetpowered.com/languages.aspx
>>
>> An impressive list, I didn't know there was so much languages for this
>> platform, some being remixes of old languages (COBOL...), others being
>> original.
>> Although I see holes, I don't see Fantom for example (ex Fan language).
>>
>> And well, I suppose the list of JVM languages isn't short either.
>>
>> --
>> Philippe Lhoste

They also call them "Computation Expressions" which is as good a name
as any other if you're trying to convey meaning to someone who doesn't
know what a monad is.

On Mon, Sep 27, 2010 at 1:00 PM, Razvan (Pub) Cojocaru wrote:
> Its possile. I went through a few tutorials and they made up too many
> non-sensical keywords sprinkled all over the place, borrowing from too
> many languages. I went through the historyand did go through ML and
> OCAML and those are much nicer _and regular_ in comparison.
>
> "member x.Value" ? Seriously?
>
> Anyways, messing with the semantics or meaning that most people attach
> to certain words, like "workflows" is not a brilliant move in my mind.
> its something else (hitching a ride, marketing...nothing
> professional). I read that reasoning as well and they could have
> called it "execution path" or "execution strategy" or whatever else.
>
> It's possible they had other restrictions in the Microsoft universe
> but the outcome is a verbose, non regular language.
>
>
> On 9/27/10, Alec Zorab wrote:
>> F# workflows are just the F# equivalent of Haskell's do notation, or
>> Scala's foreach magic on monads. Don Syme even remarks in expert f#
>> that the reason they're called workflows is because he talked to Simon
>> Peyton-Jones and they agreed that a large source of the confusion
>> surrounding monads is that people get caught up in the terminology.
>>
>> If you spend a little time with f#, you quickly come to appreciate the
>> expressiveness and beauty of the language. Sure, it looks ugly to
>> begin with, but that's because it's much less c-like than, for
>> instance, scala.
>>
>> On Mon, Sep 27, 2010 at 9:12 AM, Philippe Lhoste wrote:
>>> On 27/09/2010 06:31, Leo Romanoff wrote:
>>>>
>>>> There is actually Haskell for .Net platform! And a few other languages
>>>> :-)
>>>> http://dotnetpowered.com/languages.aspx
>>>
>>> An impressive list, I didn't know there was so much languages for this
>>> platform, some being remixes of old languages (COBOL...), others being
>>> original.
>>> Although I see holes, I don't see Fantom for example (ex Fan language).
>>>
>>> And well, I suppose the list of JVM languages isn't short either.
>>>
>>> --
>>> Philippe Lhoste
>>> --  (near) Paris -- France
>>> --  http://Phi.Lho.free.fr
>>> --  --  --  --  --  --  --  --  --  --  --  --  --  --
>>>
>>>
>>
>
> --
> Sent from my mobile device
>

Well, I have a new found respect for scala and Martin's team now, after
looking at all these other languages over the weekend.

I mean: val, var, def, if-else, match-case and generic scope {} ... that's
more than half the basic language and can be explained in what, 5 minutes
and 3 slides? To anyone with any background!

Put a "lazy" in front of val and get a "workflow" as well as a cache for the
results. Put an "object" and get an automated singleton with factory and
everything... add () instead of [] and [] instead of <> and that's like 2/3
of the language now: 3 more slides :)

Cheers,
Razie

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Well, aside from the fact that I dislike BPEL(*),  I don't see how it can replace continuations. In fact, it is even against general recommendations for BPEL to use it where continuations are used -- at the method's flow control level.

(*) I think XML is the wrong solution to almost everything, though not necessarily a _bad_ solution. I open an exception where it comes to code -- code should never, ever, be written in XML.

On Sat, Sep 25, 2010 at 18:02, Razvan Cojocaru-2 <pub@razie.com> wrote:

Yes and No.

Yes, scala makes a great J2EE programming environment. It has access to the
entire J2EE and Java APIs and libraries. Deployment wise, it's YAJ,
yet-another-jar.

I believe Lift is servlet API-friendly, so that takes care of your web
layer.

No, you should not use actors in J2EE. That's what MDBs are for.

No, you should not use continuations in J2EE, that's what BPEL and similar
are for.

The E for Enterprise means visibility, management, administration...hence
BPEL and MDBs rather than continuations and actors.

Cheers,
Razie

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys:  http://scripster.razie.com Scripster  and
http://github.com/razie/gremlins Gremlins
Follow me:  http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter ,  http://github.com/razie GitHub .

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2713873.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Daniel C. Sobral

I travel to the future all the time.

Daniel - same here, I don't like XML and especially BPEL XMLs - I meant that
as a general direction. In fact, that's why I started that gremlins project
at http://github.com/razie/gremlins - it looks very nice as a scala internal
(or external) DSL...with a choice of shapes to choose from...

A "workflow" is different from code (like F#'s "workflow" or continuations)
because it generally decouples the concept of "activity" from their
underlying code or implementation and can save/suspend/resume state
in-between.

One difference between workflow persisted state and actors or continuations,
that I was talking to Greg about is, for instance, upgrading or patching a
large system in production, with messages in queues or continuations in
progress: you cannot version the "code" and have different versions running.
Well, not easily and not visibly anyways.

While, because in a serious workflow, the state is decoupled from the actual
implementation or code, you can. You can upgrade the code with workflows "in
progress"...as long as the changes are obviously not incompatible, like
adding more required input arguments to the next activity...

It's always a question of how late a binding your performance requirements
allow and your flexibility requires...and I think we live interesting times
because of scala DSL's...

Cheers,
Razie

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Sorry, I mentioned actors and continuations...actors are actually more
flexible and could persist the messages in between, much like a workflow
would.

The trouble that I have with actors is just visibility into the "template"
or "process" and actual state...otherwise they're a cool cheap message
processing framework.

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Razvan,

Just to make it clear:
I didn't mean that I've seen BPEL-like workflows mapped directly to F# code
that "asynchronous workflows".
Instead. I meant that I've seen examples of BPEL-like engines built using
this technology internally. These engines would read a workflow description
formulated using a special syntax, then build internal representation/model
of such a workflow and then execute it step by step. This workflow execution
process is implemented mainly by means of "asynchronous workflows". It
allowed them to easily handle long lasting IO operations avoiding blocking
waiting, etc.
BTW, I'm not sure, if persistence of a workflow state and fault tolerance
could be easily implemented using this approach.

-Leo

Razvan Cojocaru-2 wrote:
>
> Its possile. I went through a few tutorials and they made up too many
> non-sensical keywords sprinkled all over the place, borrowing from too
> many languages. I went through the historyand did go through ML and
> OCAML and those are much nicer _and regular_ in comparison.
>
> "member x.Value" ? Seriously?
>
> Anyways, messing with the semantics or meaning that most people attach
> to certain words, like "workflows" is not a brilliant move in my mind.
> its something else (hitching a ride, marketing...nothing
> professional). I read that reasoning as well and they could have
> called it "execution path" or "execution strategy" or whatever else.
>
> It's possible they had other restrictions in the Microsoft universe
> but the outcome is a verbose, non regular language.
>
>
> On 9/27/10, Alec Zorab wrote:
>> F# workflows are just the F# equivalent of Haskell's do notation, or
>> Scala's foreach magic on monads. Don Syme even remarks in expert f#
>> that the reason they're called workflows is because he talked to Simon
>> Peyton-Jones and they agreed that a large source of the confusion
>> surrounding monads is that people get caught up in the terminology.
>>
>> If you spend a little time with f#, you quickly come to appreciate the
>> expressiveness and beauty of the language. Sure, it looks ugly to
>> begin with, but that's because it's much less c-like than, for
>> instance, scala.
>>
>> On Mon, Sep 27, 2010 at 9:12 AM, Philippe Lhoste wrote:
>>> On 27/09/2010 06:31, Leo Romanoff wrote:
>>>>
>>>> There is actually Haskell for .Net platform! And a few other languages
>>>> :-)
>>>> http://dotnetpowered.com/languages.aspx
>>>
>>> An impressive list, I didn't know there was so much languages for this
>>> platform, some being remixes of old languages (COBOL...), others being
>>> original.
>>> Although I see holes, I don't see Fantom for example (ex Fan language).
>>>
>>> And well, I suppose the list of JVM languages isn't short either.
>>>
>>> --
>>> Philippe Lhoste
>>> --  (near) Paris -- France
>>> --  http://Phi.Lho.free.fr
>>> --  --  --  --  --  --  --  --  --  --  --  --  --  --
>>>
>>>
>>
>

Well - sorry for hijacking this scala-user thread for what turned into a
workflow discussion - but given the mix of actors and stuff it may be
interesting.

one of the things to look at is mapping a workflow to actors. That's easy
and would basically give the actor framework a nice DSL to express overall
logic: "this sends that because that waits for this...and then visit that
other thing". Encapsulating the state in messages, which can be persisted
makes sense. The gremlins engine worked that way without the encapsulation
part, since I was lazy and didn't decouple the state...

So v(c) (c ? P | c ! Q) as a DSL for two actors P and Q. Then, using some
internal actor API to relate the messages pending back to a process instance
ID and rebuild a visual representation of the state that way...

So that's why I'm now playing with the two main components: the DSL and the
all-encompassing graph model...my engine is a joke right now (it does work,
though!).

Otherwise, you're right. just using the "F# async workflows" behind the
scenes, you can't persist state... from what I've read so far anyways. You
need a place to stop and capture state between steps and then worry about
persisting that state efficiently...

Sorry - let's continue the discussion on a different thread, if is there's
any interest. I created a separate forum for the gremlins project:
http://gremlins.968335.n3.nabble.com/

Cheers

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .