Fwd: Re: Usage of Scala in JavaEE environments

David,

First of all, thanks a lot for your reply.

>> Leo Romanoff wrote:
>> OK. I wrongly assumed that Scala continuations would do some sort of a
>> dynamic invocation stack capturing at run-time, as Javaflow or Kilim
>> frameworks do. If this would be the case, then I would expect problems as
>> JavaEE containers have no clue about continuations and capturing deep
>> invocation stacks including container code invocations does not make too
>> much sense. Even if such deep stack snapshot would be taken, it most
>> likely
>> could not be resumed later, as JavaEE container is not designed with this
>> in mind. But probably this is not required for Scala continuations, as
>> all
>> transformations are done at compile time and depth of stacks to be
>> captured
>> is limited (that's why thay are called delimited continuations, right?).

> dpp wrote:
>Scala's delimited continuations (delimited being the key word) are done at
>compile time, so all the accessible state is known by the compiler and
>captured by the compiler into the continuation object (just like functions
>close over scope.) So, the byte-codes are legal, there's no byte-code
>re-writing at run-time and they should "just work" in every execution
>environment.

OK. I understand.

Just out of couriosity: Are there any attempts at the library or language
runtime level to also support dynamic continuations based on capturing
current call stack and suspending and later restoring of the state, as it is
done by e.g. Javaflow library?

I'm wondering, because this would allow to work with almost ANY 3rd party
library invoked at the current point, even if it is was not rewritten in the
CPS style by Scala compiler. More over, it would remove the limitation that
function call leading to a continuation should be sort of the tail call in
the caller stackframe, i.e. you with it you could "cut" the control flow of
all the callers on the call stack "in the middle" when suspending and later
restore everything. While being more expensive at runtime, it gives also a
lot of flexibility.

It is also interesting, because Python and some other languages use
continuations to implement very cheap and efficient lightweight
multithreading. What they roughly do may look in Java like follows:
- They start on a thread pool something like Runnables/Callables supplied
by the user. But they don't start it directly. Instead they start it using a
small dedicated Runnable wrapper around this user-supplied
Runnable/Callable.
- The wrapper starts the real Runnable via a dedicated Continuations
support function of the Continuations libray. The result of such a call will
be either empty continuation, i.e. there were no Continuation.suspend()
calls in user-code and execution of Runnable just finished. Or a caught
continuation object will be returned, containing the captured stack frames
and their state (all stack frames upto this wrapper stack frame are
captured. In this sense, it delimits the bottom of stack for a
continuation). In this case, the wrapper does some bookkeeping and exits,
thus releasing a thread for other tasks!
- Later, when user decides to proceed with the continuation, she just calls
Continuation.resume(continuation_object_captured_before). This would submit
the same wrapper to the thread pool. Once it gets scheduled for execution,
it would rebuild the stack frames (by calling the same methods in the same
order and restoring the control flow) and their state and thus resume from
the place where it was stopped, i.e. right after Continuation.suspend()
call in the user-code.

I hope this explanation was half-way understandible. For a code example, see
e.g.
http://commons.apache.org/sandbox/javaflow/apidocs/org/apache/commons/ja...
or its implementation.

>> I'm still wondering about implementing something like Jetty
>> continuations,
>> but in a protocol independent way. I'd like to generalize/extend what
>> Jetty
>> does for handling of HTTP protocol requests to cover other protocols (I
>> guess it would eventually go beyond standard JavaEE containers). The
>> goal
>> is to be able to escape the thread-per-request approach to achieve better
>> scalability.

>Scala's delimited continuations are not a magic "freeze the thread at any
>point and resume it later" mechanism. The continuation my only be resumed
>at points where it is marked. From a syntactic standpoint, it's cleaner
>than using a for comprehension, but in Stambecco (a distributed actor
system
>I've got as a side project), you can do:

>for {
> future_a <- invoke_bit_of_work_1
> future_b <- invoke_bit_of_work_2
> future_c <- invoke_bit_of_work_3
> a <- future_a
> b <- future_b
> c <- future_c
>} {
> // do stuff with a, b, and c
>}

>The for comprehension returns immediately, but as a, b, and c become
>available, the computation is continued on worker threads.

Sorry, David, but I'm not skilled in Scala for comprehensions, futures and
lazy evaluation. Could you explain this example a bit more, or write it down
using real Scala code instead of pseudocode? This would really help me. At
the moment, I don't quite understand if the process of waiting for a,b and c
to become available occupies the current thread? Is it done using
Future.get(), is so, it would block current thread, or?

> And this of course also requires the ability to avoid any
> blocking waiting for results of long IO operations, because it would
> occupy
> a worker thread without any reason, preventing it from processing other
> requests in the mean time. My idea was to suspend/resume the actor
> processing the request, similar to Jetty continuations.

>Please don't mix Actors and continuations. At least all the actor
>implementations I'm aware of have some very precise scheduling systems as
>well as other constraints. The ability to resume a continuation on a work
>thread as part of a larger Actor message handing event would not be
possible
>in any of the Scala Actor implementations.

OK. This is a good point to take into account.

> It would be also
> nice to do it transparently for a developer, so that the code still looks
> like a usual sequential code, but in reality suspends/resumes on long
> async
> IO operations. And I thought that Scala continuations could be used to
> implement this. So that one could write in the actor body something like:
>
> exec_pre_io_code();
> // the following invocation would suspend the current actor. Later, when a
> notification about IO operation
> // completion is received, current actor will be resumed and would
> continue
> with exec_after_io_code()
> exec_long_io_operation(io_operation_params);
> exec_after_io_code();
>
> It feels like something very similar to CPS transformation done by Scala
> compiler's continuations plugin is required here. Or?
>

>If you can live with doing he stuff in a for comprehension (as above), you
>can do it today without continuations or actors.

See my explanation above.

>
>> So, is it possible to implement something like this using Scala
>> continuations? Is it the right way to go to achieve the desired result?
>> May
>> be there are better ways to solve it?
>>

>There are lots of ways to skin the cat you're looking at in Scala.
I'm very interested in learning more of them! ;-)

Thanks,
Leo

On Mon, Sep 27, 2010 at 10:12 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

David,

First of all, thanks a lot for your reply.

>> Leo Romanoff wrote:
>> OK. I wrongly assumed that Scala continuations would do some sort of a
>> dynamic invocation stack  capturing at run-time, as Javaflow or Kilim
>> frameworks do. If this would be the case, then I would expect problems as
>> JavaEE containers have no clue about continuations and capturing deep
>> invocation stacks including container code invocations does not make too
>> much sense. Even if such deep stack snapshot would be taken, it most
>> likely
>> could not be resumed later, as JavaEE container is not designed with this
>> in mind.  But probably this is not required for Scala continuations, as
>> all
>> transformations are done at compile time and depth of stacks to be
>> captured
>> is limited (that's why thay are called delimited continuations, right?).


> dpp wrote:
>Scala's delimited continuations (delimited being the key word) are done at
>compile time, so all the accessible state is known by the compiler and
>captured by the compiler into the continuation object (just like functions
>close over scope.)  So, the byte-codes are legal, there's no byte-code
>re-writing at run-time and they should "just work" in every execution
>environment.

OK. I understand.

Just out of couriosity: Are there any attempts at the library or language
runtime level to also support dynamic continuations based on capturing
current call stack and suspending and later restoring of the state, as it is
done by e.g. Javaflow library?

I'm wondering, because this would allow to work with almost ANY 3rd party
library invoked at the current point, even if it is was not rewritten in the
CPS style by Scala compiler. More over, it would remove the limitation that
function call leading to a continuation should be sort of the tail call in
the caller stackframe, i.e. you with it you could "cut" the control flow of
all the callers on the call stack "in the middle" when suspending and later
restore everything. While being more expensive at runtime, it gives also a
lot of flexibility.

The delimited continuations are much, much cheaper (they consume far less space and time to create the continuation) and they are much more friendly.  There are Java-based projects the do full continuations and Tiark has some data on the performance of each in some paper (I don't have a link.)  

It is also interesting, because Python and some other languages use
continuations to implement very cheap and efficient lightweight
multithreading. What they roughly do may look in Java like follows:
 - They start on a thread pool something like Runnables/Callables supplied
by the user. But they don't start it directly. Instead they start it using a
small dedicated Runnable wrapper around this user-supplied
Runnable/Callable.
 - The wrapper starts the real Runnable via a dedicated Continuations
support function of the Continuations libray. The result of such a call will
be either empty continuation, i.e. there were no Continuation.suspend()
calls in user-code and execution of Runnable just finished. Or a caught
continuation object will be returned, containing the captured stack frames
and their state (all stack frames upto this wrapper stack frame are
captured. In this sense, it delimits the bottom of stack for a
continuation). In this case, the wrapper does some bookkeeping and exits,
thus releasing a thread for other tasks!
- Later,  when user decides to proceed with the continuation, she just calls
Continuation.resume(continuation_object_captured_before). This would submit
the same wrapper to the thread pool. Once it gets scheduled for execution,
it would rebuild the stack frames (by calling the same methods in the same
order and restoring the control flow) and their state and thus resume from
the place where it was stopped, i.e. right after  Continuation.suspend()
call in the user-code.

I hope this explanation was half-way understandible. For a code example, see
e.g.
http://commons.apache.org/sandbox/javaflow/apidocs/org/apache/commons/javaflow/Continuation.html
or its implementation.

Yeah... I'm not a fan of that kind of coding.  There's just too much room for pain by snapshotting the whole stack when all you really want is a small part of the stack.  

>> I'm still wondering about implementing something like Jetty
>> continuations,
>> but in a protocol independent way. I'd like to generalize/extend what
>> Jetty
>> does for handling of HTTP protocol requests to cover other protocols (I
>> guess it would eventually go beyond standard JavaEE containers).  The
>> goal
>> is to be able to escape the thread-per-request approach to achieve better
>> scalability.


>Scala's delimited continuations are not a magic "freeze the thread at any
>point and resume it later" mechanism.  The continuation my only be resumed
>at points where it is marked.  From a syntactic standpoint, it's cleaner
>than using a for comprehension, but in Stambecco (a distributed actor
system
>I've got as a side project), you can do:

>for {
>  future_a <- invoke_bit_of_work_1
> future_b <- invoke_bit_of_work_2
>  future_c <- invoke_bit_of_work_3
>  a <- future_a
>  b <- future_b
>  c <- future_c
>} {
>  // do stuff with a, b, and c
>}

>The for comprehension returns immediately, but as a, b, and c become
>available, the computation is continued on worker threads.

Sorry, David,  but I'm not skilled in Scala for comprehensions, futures and
lazy evaluation. Could you explain this example a bit more, or write it down
using real Scala code instead of pseudocode?

This is real code (you have to implement invoke_bit_of_work_1, invoke_bit_of_work_2 & invoke_bit_of_work_3), but it gives you the idea of what is possible with Scala's for comprehension (well, monads).  

This would really help me. At
the moment, I don't quite understand if the process of waiting for a,b and c
to become available occupies the current thread? Is it done using
Future.get(), is so, it would block current thread, or?

Because the for comprehension returns Unit, it returns immediately (or perhaps just after the invoke messages have been sent).  When the foreach method is invoked on future_a, if future_a is satisfied, the thread continues.  If future_a is not satisfied, the function passed to foreach is noted in future_a and when future_a is satisfied, work is enqueued on a threadpool to continue the calculation.  I do this in Stambecco http://stambecco.org  

> And this of course also requires the ability to avoid any
> blocking waiting for results of long IO operations, because it would
> occupy
> a worker thread without any reason, preventing it from processing other
> requests in the mean time. My idea was to suspend/resume the actor
> processing the request,  similar to Jetty continuations.


>Please don't mix Actors and continuations.  At least all the actor
>implementations I'm aware of have some very precise scheduling systems as
>well as other constraints.  The ability to resume a continuation on a work
>thread as part of a larger Actor message handing event would not be
possible
>in any of the Scala Actor implementations.

OK. This is a good point to take into account.

> It would be also
> nice to do it transparently for a developer, so that the code still looks
> like a usual sequential code, but in reality suspends/resumes on long
> async
> IO operations. And I thought that Scala continuations could be used to
> implement this. So that one could write in the actor body something like:
>
> exec_pre_io_code();
> // the following invocation would suspend the current actor. Later, when a
> notification about IO operation
> // completion is received, current actor will be resumed and would
> continue
> with exec_after_io_code()
> exec_long_io_operation(io_operation_params);
> exec_after_io_code();
>
> It feels like something very similar to CPS transformation done by Scala
> compiler's continuations plugin is required here. Or?
>

>If you can live with doing he stuff in a for comprehension (as above), you
>can do it today without continuations or actors.

See my explanation above.

>
>> So, is it possible to implement something like this using Scala
>> continuations? Is it the right way to go to achieve the desired result?
>> May
>> be there are better ways to solve it?
>>

>There are lots of ways to skin the cat you're looking at in Scala.
I'm very interested in learning more of them! ;-)

Thanks,
 Leo
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2716613.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

So Gents,

I've implemented "scala workflows", see the example at:
http://github.com/razie/gremlins/blob/master/src/test/scala/razie/wfs/Sc...

If you want to write simple flow descriptions, here's an example (note the
par, seq and later). The workflows is defined by executing the def. This
creates the workflow of "activities" which will be in invoked at run time on
a thread pool.

I will look at implementing the monadic thing as well, with the bind (let
bang in F#) and we have what they call the async monad...

If you don't like threads, then play god and tell the gremlins to live in an
engine with actors instead:

override def setUp() = { razie.wf.Gremlins.liveInside (new Engine with
Threads) }
override def setUp() = { razie.wf.Gremlins.liveInside (new Engine with
Actors) }

I'm interested how would you rate this style, harder or easier than using 1.
threads, 2. fork/join, 3. actors ?


def fapp(app: String) (in: Any): Any = {
val x = in.toString + "-" + app;
println ("------------- woohoo " + x);
x
}

def wsp2 =
seq {
par {
seq {
par {
seq {
fapp("a") _
}
seq {
fapp("b") _
}
}
sort[String] (_ < _)
later { case x : List[String] => x mkString "," }
}
seq {
par {
seq {
fapp("a") _
}
seq {
fapp("b") _
}
}
sort[String] (_ < _)
later {
case l: List[String] => l mkString ","
}
}
}
foldLeft[String] ("folded:") (_ + "," + _)
}

def testwsp2 = expect ("folded:,1-a,1-b,1-a,1-b") { wsp2 run 1 }

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Ok Gents - last hijacking of this thread - I promise.

I just checked in the letbang, oblivious to the fact that I already had the
seq/par monad:

def wfa1 = seq {
val a = let! seq { _ + "-a" }
later { case _ => a.get + "-b" }
}

def testwfa1 = expect ("1-a-b") { prun (wfa1, 1) }

By God, I love scala !!! :)))

cheers,
Razie

-----
Razvan Cojocaru,
Work: http://www.sigma-systems.com
Playground: http://wiki.homecloud.ca
Latest cool toys: http://scripster.razie.com Scripster and
http://github.com/razie/gremlins Gremlins
Follow me: http://feeds.razie.com/RazvanTech RSS Feed ,
http://twitter.com/razie Twitter , http://github.com/razie GitHub .

Hi Razvan,

Razvan Cojocaru-2 wrote:
>
> Ok Gents - last hijacking of this thread - I promise.
>

I'd really suggest to move the detailed discussion about workflows into a
dedicated thread on this mailing list or on the dedicated Nabble discussion
forum that you created before. Otherwise we are drifting more and more from
the original subject of this thread.

Razvan Cojocaru-2 wrote:
>
> I just checked in the letbang, oblivious to the fact that I already had
> the seq/par monad:
>
>
> def wfa1 = seq {
> val a = let! seq { _ + "-a" }
> later { case _ => a.get + "-b" }
> }
>
> def testwfa1 = expect ("1-a-b") { prun (wfa1, 1) }
>
>
Regarding your latest experiments with workflows: it looks really nice!

BTW, do you support besides par and seq elements, also conditional branching
(e.g. if, switch) inside workflows? I think this would be nice.

It is also not quite clear for me, how the asynchronous invocations of
services problem is solved inside this framework, e.g. what happens to the
worker thread when one of the workflow steps invokes an external service XML
Web Service? Is it still occupied be this step and waits for result of the
invocation? Or do you have a more asynchronous solution?

-Leo

By God, I love scala !!! :)))

cheers,
Razie

Hi David,

Thanks for your explanations and your patience with me!

> > dpp wrote:
> >Scala's delimited continuations (delimited being the key word) are done
> at
> >compile time, so all the accessible state is known by the compiler and
> >captured by the compiler into the continuation object (just like
> functions
> >close over scope.) So, the byte-codes are legal, there's no byte-code
> >re-writing at run-time and they should "just work" in every execution
> >environment.
>
>> OK. I understand.
>>
>> Just out of couriosity: Are there any attempts at the library or language
>> runtime level to also support dynamic continuations based on capturing
>> current call stack and suspending and later restoring of the state, as it
>> is done by e.g. Javaflow library?

>> I'm wondering, because this would allow to work with almost ANY 3rd party
>> library invoked at the current point, even if it is was not rewritten in
>> the CPS style by Scala compiler. More over, it would remove the
>> limitation that
>> function call leading to a continuation should be sort of the tail call
>> in
>> the caller stackframe, i.e. you with it you could "cut" the control flow
>> of
>> all the callers on the call stack "in the middle" when suspending and
>> later
>> restore everything. While being more expensive at runtime, it gives also
>> a
>> lot of flexibility.

>The delimited continuations are much, much cheaper (they consume far less
>space and time to create the continuation) and they are much more friendly.

No doubts about that.

For a new project where I defined the architecture and build it from
scratch, I'd most likely use Scala's delimited continuations.

But I'm trying to figure out also if there is a possibility to support other
kinds of continuations in Scala.
The pragmatic reason for this question is the fact that there are a lot of
legacy systems and 3rd party libraries written in Java. Rewriting all of
them to use Scala continuations is just out of scope and probably
impossible. Therefore, having an implementation of continuations that may be
combined with almost ANY existing Java library is very important!

> There are Java-based projects the do full continuations

Yes. I mentioned them before. This is Kilim and Javaflow, for example.

> and Tiark has some data on the performance of each in some paper (I don't
> have a link.)

Yes, I found it:
http://lamp.epfl.ch/~rompf/continuations-icfp09.pdf

>> It is also interesting, because Python and some other languages use
>> continuations to implement very cheap and efficient lightweight
>> multithreading. What they roughly do may look in Java like follows:
>> - They start on a thread pool something like Runnables/Callables
>> supplied
>> by the user. But they don't start it directly. Instead they start it
>> using
>> a small dedicated Runnable wrapper around this user-supplied
>> Runnable/Callable.
>> - The wrapper starts the real Runnable via a dedicated Continuations
>> support function of the Continuations libray. The result of such a call
>> will be either empty continuation, i.e. there were no
>> Continuation.suspend()
>> calls in user-code and execution of Runnable just finished. Or a caught
>> continuation object will be returned, containing the captured stack
>> frames
>> and their state (all stack frames upto this wrapper stack frame are
>> captured. In this sense, it delimits the bottom of stack for a
>> continuation). In this case, the wrapper does some bookkeeping and exits,
>> thus releasing a thread for other tasks!
>> - Later, when user decides to proceed with the continuation, she just
>> calls
>> Continuation.resume(continuation_object_captured_before). This would
>> submit
>> the same wrapper to the thread pool. Once it gets scheduled for
>> execution,
>> it would rebuild the stack frames (by calling the same methods in the
>> same
>> order and restoring the control flow) and their state and thus resume
>> from
>> the place where it was stopped, i.e. right after Continuation.suspend()
>> call in the user-code.
>>
>> I hope this explanation was half-way understandible. For a code example,
>> see e.g.
>> http://commons.apache.org/sandbox/javaflow/apidocs/org/apache/commons/ja...
>> or its implementation.

>Yeah... I'm not a fan of that kind of coding. There's just too much room
>for pain by snapshotting the whole stack when all you really want is a
small
>part of the stack.

There are ways to limit it. You can tell it to snapshot just upto a certain
method X on the call-stack. This would serve as a delimiter. But of course,
it is not as efficient as CPS transformation and leads to more overhead.

>> >Scala's delimited continuations are not a magic "freeze the thread at
>> any
>> >point and resume it later" mechanism. The continuation my only be
>> resumed
>> >at points where it is marked. From a syntactic standpoint, it's cleaner
>> >than using a for comprehension, but in Stambecco (a distributed actor
>> system
>> >I've got as a side project), you can do:
>>
>> >for {
>> > future_a <- invoke_bit_of_work_1
>> > future_b <- invoke_bit_of_work_2
>> > future_c <- invoke_bit_of_work_3
>> > a <- future_a
>> > b <- future_b
>> > c <- future_c
>> >} {
>> > // do stuff with a, b, and c
>> >}
>>
>> >The for comprehension returns immediately, but as a, b, and c become
>> >available, the computation is continued on worker threads.
>>
>> Sorry, David, but I'm not skilled in Scala for comprehensions, futures
>> and
>> lazy evaluation. Could you explain this example a bit more, or write it
>> down
>> using real Scala code instead of pseudocode?

>This is real code (you have to
>implement invoke_bit_of_work_1, invoke_bit_of_work_2
>& invoke_bit_of_work_3), but it gives you the idea of what is possible with
>Scala's for comprehension (well, monads).

First of all, your example works with Stambecco workers/actors (which I
assume) or is it a standard Scala with standard library?

>> This would really help me. At
>> the moment, I don't quite understand if the process of waiting for a,b
>> and
>> c to become available occupies the current thread? Is it done using
>> Future.get(), is so, it would block current thread, or?

> Because the for comprehension returns Unit, it returns immediately (or
> perhaps just after the invoke messages have been sent).

I understand. Futures do not wait for completion. You just submit a task to
a threadpool and immediately get a future object back.

BTW, is it correct to say that your foreach approach requires this foreach
to be the last statement on the current function and thread, i.e. it is a
sort of "tailcall"? Otherwise the code after the whole "foreach {}" would be
executed right after you got your future objects back, or? And the body of
foreach would be executed only once features are done and ready, i.e.
eventually much later. If my understanding is correct, it may result in a
very weired flow of execution, as from looking at the code you'd assume a
sequential execution (first foreach block, then anything after it).

> When the foreach method is invoked on future_a, if future_a is satisfied,
> the thread
> continues.

Yes. That's easy.

> If future_a is not satisfied, the function passed to foreach is
> noted in future_a

You mean that the Unit representing a closure of the body (or whatever comes
after a<-future_a) is stored as sort of a continuation callback associated
with the future_a? So that later, when future_a is satisfied this callback
can be invoked?

> and when future_a is satisfied,

But how do you check that future_a is satisfied? See my comments below.

> work is enqueued on a threadpool to continue the calculation. I do this
> in Stambecco
> http://stambecco.org

I looked at the code, hopefully at the right place. Do you mean this class?
(BTW all classes are dated back in 2008. Is it correct?)

https://www.assembla.com/code/stambecco/git/nodes/core/src/main/scala/or...

I mean this method:

/**
* Return a continuation that allows you to execute code when the
* answer becomes available, without blocking the current thread.
*/
def continue: Responder[Box[T]] = {
var done = false
var funcList: List[() => Unit] = Nil
val lockObj = new Object
var resp: Box[T] = Empty

askFunc((f: Box[T]) => {
val lst = lockObj.synchronized{if (!done) {resp = f; done = true;
funcList} else Nil}
lst.foreach(f => LAScheduler.execute(f))
} , this)

new Responder[Box[T]] {
def respond(f: Box[T] => Unit) = {
val b: () => Unit = lockObj.synchronized {
val tf = () => f(resp)
if (done) tf
else {
funcList ::= tf
() => {}
}
}

b.apply()
}
}
}
}

If I understand your code correctly, you try to check if future is done,
i.e. computed. And if not, you put it into the list of futures to check next
time for their completion. Then you submit this "check futures" function for
execution on the threadpool. You repeat this, until the future is satisfied
and then you invoke the callback, i.e. the body of the foreach. Please
correct me, if I'm wrong.

But then, it means that you do polling/busy waiting on futures, actively
checking for their completion. This busy waiting can keep CPU active even if
nothing happens. Moreover, I'm not sure about scalability of this approach.
What if you have 50000 such foreach loops executed at the same time by
different actors? I have the impression that you'll get a huge set of
scheduled "check futures" tasks that would eat all of you CPU time.

Another thing which I don't quite get is: What if the computation enclosed
into the future is something like an invocation of an external XML Web
Service which may take a lot of time. I guess the thread which really
performs this invocation (and this is most likely not the thread that
executed a function containing the mentioned foreach construct, but a thread
from a dedicated thread pool) would do a blocking wait for the results, or?
If so, we just shifted the problem from the original thread to another
thread.

And the problem that I see is: If you need to perform simultaneously 50000
or 10000 such invocations, you cannot do it if threads are being blocked
somewhere. JVM just does not support that many threads. The only solution is
IMHO to become REALLY asynchronous, i.e. NIO and no scheduled threadpool
task can do a blocking wait! Approach that you described does not seem to
solve it, if I understand it correctly. Therefore we are back to the
original questions:

- Can this be somehow achieved by means of Scala, be it actors,
continuations or anything else? And I'm really looking for a solution,where
I do not do most of the job by hands, i.e splitting of functions into parts,
building closures, managing callbacks, etc. I want my language, its runtime
and library to do it for me.

- I feel that it can be done with continuations, but I don't know how to do
it with Scala delimited continuations. They seem to be not powerful enough
for me. But I may be wrong here.

While I understand your point about what can be done today with Scala for
comprehensions and current continuations, I'm still interested in the
potential possibilities to achieve what I sketched above and in previous
mails using Scala and without rewriting all of the (3rd party) code. May be
this is not possible today, may be someone has implemented similar systems
already. Any feedback is appreciated.

It would be also interesting to get Tiark's opinion about all those
continuation related questions. Any ideas about using Javaflow or Kilim like
approaches for Scala (I've read that people were trying to use it for Swarm,
http://www.scala-lang.org/node/3485)? What about the JVM support for
continuations? (see
http://wiki.jvmlangsummit.com/images/2/2b/JVMLanguageSummit_Stadler_Cont...
and
http://weblogs.java.net/blog/forax/archive/2009/11/19/holy-crap-jvm-has-...)

Thanks,
-Leo

> > It would be also
> > nice to do it transparently for a developer, so that the code still
> looks
> > like a usual sequential code, but in reality suspends/resumes on long
> > async
> > IO operations. And I thought that Scala continuations could be used to
> > implement this. So that one could write in the actor body something
> like:
> >
> > exec_pre_io_code();
> > // the following invocation would suspend the current actor. Later, when
> a
> > notification about IO operation
> > // completion is received, current actor will be resumed and would
> > continue
> > with exec_after_io_code()
> > exec_long_io_operation(io_operation_params);
> > exec_after_io_code();
> >
> > It feels like something very similar to CPS transformation done by Scala
> > compiler's continuations plugin is required here. Or?
> >
>
> >If you can live with doing he stuff in a for comprehension (as above),
> you
> >can do it today without continuations or actors.

On Wed, Sep 29, 2010 at 9:35 AM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi David,

Thanks for your explanations and your patience with me!

> > dpp wrote:
> >Scala's delimited continuations (delimited being the key word) are done
> at
> >compile time, so all the accessible state is known by the compiler and
> >captured by the compiler into the continuation object (just like
> functions
> >close over scope.)  So, the byte-codes are legal, there's no byte-code
> >re-writing at run-time and they should "just work" in every execution
> >environment.
>
>> OK. I understand.
>>
>> Just out of couriosity: Are there any attempts at the library or language
>> runtime level to also support dynamic continuations based on capturing
>> current call stack and suspending and later restoring of the state, as it
>> is done by e.g. Javaflow library?

>> I'm wondering, because this would allow to work with almost ANY 3rd party
>> library invoked at the current point, even if it is was not rewritten in
>> the CPS style by Scala compiler. More over, it would remove the
>> limitation that
>> function call leading to a continuation should be sort of the tail call
>> in
>> the caller stackframe, i.e. you with it you could "cut" the control flow
>> of
>> all the callers on the call stack "in the middle" when suspending and
>> later
>> restore everything. While being more expensive at runtime, it gives also
>> a
>> lot of flexibility.


>The delimited continuations are much, much cheaper (they consume far less
>space and time to create the continuation) and they are much more friendly.

No doubts about that.

For a new project where I defined the architecture and build it from
scratch, I'd most likely use Scala's delimited continuations.

But I'm trying to figure out also if there is a possibility to support other
kinds of continuations in Scala.

Scala compiles to legal JVM bytecode, so for any existing system that does continuations on the JVM, it should "just work" with Scala.  In the event that it doesn't, most library authors are pretty good about fixing their libraries.
 

>> resumed
>> >at points where it is marked.  From a syntactic standpoint, it's cleaner
>> >than using a for comprehension, but in Stambecco (a distributed actor
>> system
>> >I've got as a side project), you can do:
>>
>> >for {
>> >  future_a <- invoke_bit_of_work_1
>> > future_b <- invoke_bit_of_work_2
>> >  future_c <- invoke_bit_of_work_3
>> >  a <- future_a
>> >  b <- future_b
>> >  c <- future_c
>> >} {
>> >  // do stuff with a, b, and c
>> >}
>>
>> >The for comprehension returns immediately, but as a, b, and c become
>> >available, the computation is continued on worker threads.
>>
>> Sorry, David,  but I'm not skilled in Scala for comprehensions, futures
>> and
>> lazy evaluation. Could you explain this example a bit more, or write it
>> down
>> using real Scala code instead of pseudocode?

>This is real code (you have to
>implement invoke_bit_of_work_1, invoke_bit_of_work_2
>& invoke_bit_of_work_3), but it gives you the idea of what is possible with
>Scala's for comprehension (well, monads).

>> >point and resume it later" mechanism.  The continuation my only be
First of all, your example works with Stambecco workers/actors (which I
assume) or is it a standard Scala with standard library?

These examples are based on what can be done with Scala.  Parts of Stambecco are inspired by the Scala standard library.  But the reason for the example is that if the above for-comprehension style is acceptable to you, then it's a matter of a few thousand lines of code to get you what you want.  Those lines exist in Stambecco.  Similar lines exist in Akka and ScalaZ... and if there's ever consensus on how to do it, those lines could make their way into the standard Scala libraries.
 

>> This would really help me. At
>> the moment, I don't quite understand if the process of waiting for a,b
>> and
>> c to become available occupies the current thread? Is it done using
>> Future.get(), is so, it would block current thread, or?


> Because the for comprehension returns Unit, it returns immediately (or
> perhaps just after the invoke messages have been sent).

I understand. Futures do not wait for completion. You just submit a task to
a threadpool and immediately get a future object back.

BTW, is it correct to say that your foreach approach requires this foreach
to be the last statement on the current function and thread, i.e. it is a
sort of "tailcall"? Otherwise the code after the whole "foreach {}" would be
executed right after you got your future objects back, or? And the body of
foreach would be executed only once features are done and ready, i.e.
eventually much later. If my understanding is correct, it may result in a
very weired flow of execution, as from looking at the code you'd assume a
sequential execution (first foreach block, then anything after it).

> When the foreach method is invoked on future_a, if future_a is satisfied,
> the thread
> continues.

Yes. That's easy.

> If future_a is not satisfied, the function passed to foreach is
> noted in future_a

You mean that the Unit representing a closure of the body (or whatever comes
after a<-future_a) is stored as sort of a continuation callback associated
with the future_a? So that later, when future_a is satisfied this callback
can be invoked?

> and  when future_a is satisfied,

But how do you check that future_a is satisfied? See my comments below.

> work is enqueued on a threadpool to continue the calculation.  I do this
> in Stambecco
> http://stambecco.org

I looked at the code, hopefully at the right place. Do you mean this class?
(BTW all classes are dated back in 2008. Is it correct?)

https://www.assembla.com/code/stambecco/git/nodes/core/src/main/scala/org/stambecco/internal/Saddle.scala?rev=c46d8e8af7a747648fd6842b12dc0c1bdf6858a3

I mean this method:

 /**
  * Return a continuation that allows you to execute code when the
  * answer becomes available, without blocking the current thread.
  */
 def continue: Responder[Box[T]] = {
   var done = false
   var funcList: List[() => Unit] = Nil
   val lockObj = new Object
   var resp: Box[T] = Empty

   askFunc((f: Box[T]) => {
       val lst = lockObj.synchronized{if (!done) {resp = f; done = true;
funcList} else Nil}
       lst.foreach(f => LAScheduler.execute(f))
     } , this)

   new Responder[Box[T]] {
     def respond(f: Box[T] => Unit) = {
       val b: () => Unit = lockObj.synchronized {
         val tf = () => f(resp)
         if (done) tf
         else {
           funcList ::= tf
           () => {}
         }
       }

       b.apply()
     }
   }
 }
}

If I understand your code correctly, you try to check if future is done,
i.e. computed. And if not, you put it into the list of futures to check next
time for their completion. Then you submit this "check futures" function for
execution on the threadpool. You repeat this, until the future is satisfied
and then you invoke the callback, i.e. the body of the foreach.  Please
correct me, if I'm wrong.

But then, it means that you do polling/busy waiting on futures, actively
checking for their completion. This busy waiting can keep CPU active even if
nothing happens. Moreover,  I'm not sure about scalability of this approach.
What if you have 50000 such foreach loops executed at the same time by
different actors? I have the impression that you'll get a huge set of
scheduled "check futures" tasks that would eat all of you CPU time.

Another thing which I don't quite get is: What if the computation enclosed
into the future is something like an invocation of an external XML Web
Service which may take a lot of time. I guess the thread which really
performs this invocation (and this is most likely not the thread that
executed a function containing the mentioned foreach construct, but a thread
from a dedicated thread pool) would do a blocking wait for the results, or?
If so, we just shifted the problem from the original thread to another
thread.

And the problem that I see is: If you need to perform simultaneously 50000
or 10000 such invocations, you cannot do it if threads are being blocked
somewhere. JVM just does not support that many threads. The only solution is
IMHO to become REALLY asynchronous, i.e. NIO and no scheduled threadpool
task can do a blocking wait! Approach that you described does not seem to
solve it, if I understand it correctly. Therefore we are back to the
original questions:

 - Can this be somehow achieved by means of Scala, be it actors,
continuations or anything else? And I'm really looking for a solution,where
I do not do most of the job by hands, i.e splitting of functions into parts,
building closures, managing callbacks, etc. I want my language, its runtime
and library to do it for me.

 - I feel that it can be done with continuations, but I don't know how to do
it with Scala delimited continuations. They seem to be not powerful enough
for me. But I may be wrong here.

While I understand your point about what can be done today with Scala for
comprehensions and current continuations, I'm still interested in the
potential possibilities to achieve what I sketched above and in previous
mails using Scala and without rewriting all of the (3rd party) code. May be
this is not possible today, may be someone has implemented similar systems
already. Any feedback is appreciated.

It would be also interesting to get Tiark's opinion about all those
continuation related questions. Any ideas about using Javaflow or Kilim like
approaches for Scala (I've read that people were trying to use it for Swarm,
http://www.scala-lang.org/node/3485)? What about the JVM support for
continuations? (see
http://wiki.jvmlangsummit.com/images/2/2b/JVMLanguageSummit_Stadler_Continuations.pdf
and
http://weblogs.java.net/blog/forax/archive/2009/11/19/holy-crap-jvm-has-coroutinecontinuationfiber-etc)

Thanks,
 -Leo

> > It would be also
> > nice to do it transparently for a developer, so that the code still
> looks
> > like a usual sequential code, but in reality suspends/resumes on long
> > async
> > IO operations. And I thought that Scala continuations could be used to
> > implement this. So that one could write in the actor body something
> like:
> >
> > exec_pre_io_code();
> > // the following invocation would suspend the current actor. Later, when
> a
> > notification about IO operation
> > // completion is received, current actor will be resumed and would
> > continue
> > with exec_after_io_code()
> > exec_long_io_operation(io_operation_params);
> > exec_after_io_code();
> >
> > It feels like something very similar to CPS transformation done by Scala
> > compiler's continuations plugin is required here. Or?
> >
>
> >If you can live with doing he stuff in a for comprehension (as above),
> you
> >can do it today without continuations or actors.


--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2719351.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

I hit send too early... more to come.

On Wed, Sep 29, 2010 at 10:11 AM, David Pollak <feeder.of.the.bears@gmail.com> wrote:

On Wed, Sep 29, 2010 at 9:35 AM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi David,

Thanks for your explanations and your patience with me!

> > dpp wrote:
> >Scala's delimited continuations (delimited being the key word) are done
> at
> >compile time, so all the accessible state is known by the compiler and
> >captured by the compiler into the continuation object (just like
> functions
> >close over scope.)  So, the byte-codes are legal, there's no byte-code
> >re-writing at run-time and they should "just work" in every execution
> >environment.
>
>> OK. I understand.
>>
>> Just out of couriosity: Are there any attempts at the library or language
>> runtime level to also support dynamic continuations based on capturing
>> current call stack and suspending and later restoring of the state, as it
>> is done by e.g. Javaflow library?

>> I'm wondering, because this would allow to work with almost ANY 3rd party
>> library invoked at the current point, even if it is was not rewritten in
>> the CPS style by Scala compiler. More over, it would remove the
>> limitation that
>> function call leading to a continuation should be sort of the tail call
>> in
>> the caller stackframe, i.e. you with it you could "cut" the control flow
>> of
>> all the callers on the call stack "in the middle" when suspending and
>> later
>> restore everything. While being more expensive at runtime, it gives also
>> a
>> lot of flexibility.


>The delimited continuations are much, much cheaper (they consume far less
>space and time to create the continuation) and they are much more friendly.

No doubts about that.

For a new project where I defined the architecture and build it from
scratch, I'd most likely use Scala's delimited continuations.

But I'm trying to figure out also if there is a possibility to support other
kinds of continuations in Scala.

Scala compiles to legal JVM bytecode, so for any existing system that does continuations on the JVM, it should "just work" with Scala.  In the event that it doesn't, most library authors are pretty good about fixing their libraries.
 

>> resumed
>> >at points where it is marked.  From a syntactic standpoint, it's cleaner
>> >than using a for comprehension, but in Stambecco (a distributed actor
>> system
>> >I've got as a side project), you can do:
>>
>> >for {
>> >  future_a <- invoke_bit_of_work_1
>> > future_b <- invoke_bit_of_work_2
>> >  future_c <- invoke_bit_of_work_3
>> >  a <- future_a
>> >  b <- future_b
>> >  c <- future_c
>> >} {
>> >  // do stuff with a, b, and c
>> >}
>>
>> >The for comprehension returns immediately, but as a, b, and c become
>> >available, the computation is continued on worker threads.
>>
>> Sorry, David,  but I'm not skilled in Scala for comprehensions, futures
>> and
>> lazy evaluation. Could you explain this example a bit more, or write it
>> down
>> using real Scala code instead of pseudocode?

>This is real code (you have to
>implement invoke_bit_of_work_1, invoke_bit_of_work_2
>& invoke_bit_of_work_3), but it gives you the idea of what is possible with
>Scala's for comprehension (well, monads).

>> >point and resume it later" mechanism.  The continuation my only be
First of all, your example works with Stambecco workers/actors (which I
assume) or is it a standard Scala with standard library?

These examples are based on what can be done with Scala.  Parts of Stambecco are inspired by the Scala standard library.  But the reason for the example is that if the above for-comprehension style is acceptable to you, then it's a matter of a few thousand lines of code to get you what you want.  Those lines exist in Stambecco.  Similar lines exist in Akka and ScalaZ... and if there's ever consensus on how to do it, those lines could make their way into the standard Scala libraries.
 

>> This would really help me. At
>> the moment, I don't quite understand if the process of waiting for a,b
>> and
>> c to become available occupies the current thread? Is it done using
>> Future.get(), is so, it would block current thread, or?


> Because the for comprehension returns Unit, it returns immediately (or
> perhaps just after the invoke messages have been sent).

I understand. Futures do not wait for completion. You just submit a task to
a threadpool and immediately get a future object back.

BTW, is it correct to say that your foreach approach requires this foreach
to be the last statement on the current function and thread, i.e. it is a
sort of "tailcall"? Otherwise the code after the whole "foreach {}" would be
executed right after you got your future objects back, or? And the body of
foreach would be executed only once features are done and ready, i.e.
eventually much later. If my understanding is correct, it may result in a
very weired flow of execution, as from looking at the code you'd assume a
sequential execution (first foreach block, then anything after it).

> When the foreach method is invoked on future_a, if future_a is satisfied,
> the thread
> continues.

Yes. That's easy.

> If future_a is not satisfied, the function passed to foreach is
> noted in future_a

You mean that the Unit representing a closure of the body (or whatever comes
after a<-future_a) is stored as sort of a continuation callback associated
with the future_a? So that later, when future_a is satisfied this callback
can be invoked?

> and  when future_a is satisfied,

But how do you check that future_a is satisfied? See my comments below.

> work is enqueued on a threadpool to continue the calculation.  I do this
> in Stambecco
> http://stambecco.org

I looked at the code, hopefully at the right place. Do you mean this class?
(BTW all classes are dated back in 2008. Is it correct?)

https://www.assembla.com/code/stambecco/git/nodes/core/src/main/scala/org/stambecco/internal/Saddle.scala?rev=c46d8e8af7a747648fd6842b12dc0c1bdf6858a3

I mean this method:

 /**
  * Return a continuation that allows you to execute code when the
  * answer becomes available, without blocking the current thread.
  */
 def continue: Responder[Box[T]] = {
   var done = false
   var funcList: List[() => Unit] = Nil
   val lockObj = new Object
   var resp: Box[T] = Empty

   askFunc((f: Box[T]) => {
       val lst = lockObj.synchronized{if (!done) {resp = f; done = true;
funcList} else Nil}
       lst.foreach(f => LAScheduler.execute(f))
     } , this)

   new Responder[Box[T]] {
     def respond(f: Box[T] => Unit) = {
       val b: () => Unit = lockObj.synchronized {
         val tf = () => f(resp)
         if (done) tf
         else {
           funcList ::= tf
           () => {}
         }
       }

       b.apply()
     }
   }
 }
}

If I understand your code correctly, you try to check if future is done,
i.e. computed. And if not, you put it into the list of futures to check next
time for their completion. Then you submit this "check futures" function for
execution on the threadpool. You repeat this, until the future is satisfied
and then you invoke the callback, i.e. the body of the foreach.  Please
correct me, if I'm wrong.

But then, it means that you do polling/busy waiting on futures, actively
checking for their completion. This busy waiting can keep CPU active even if
nothing happens. Moreover,  I'm not sure about scalability of this approach.
What if you have 50000 such foreach loops executed at the same time by
different actors? I have the impression that you'll get a huge set of
scheduled "check futures" tasks that would eat all of you CPU time.

Another thing which I don't quite get is: What if the computation enclosed
into the future is something like an invocation of an external XML Web
Service which may take a lot of time. I guess the thread which really
performs this invocation (and this is most likely not the thread that
executed a function containing the mentioned foreach construct, but a thread
from a dedicated thread pool) would do a blocking wait for the results, or?
If so, we just shifted the problem from the original thread to another
thread.

And the problem that I see is: If you need to perform simultaneously 50000
or 10000 such invocations, you cannot do it if threads are being blocked
somewhere. JVM just does not support that many threads. The only solution is
IMHO to become REALLY asynchronous, i.e. NIO and no scheduled threadpool
task can do a blocking wait! Approach that you described does not seem to
solve it, if I understand it correctly. Therefore we are back to the
original questions:

 - Can this be somehow achieved by means of Scala, be it actors,
continuations or anything else? And I'm really looking for a solution,where
I do not do most of the job by hands, i.e splitting of functions into parts,
building closures, managing callbacks, etc. I want my language, its runtime
and library to do it for me.

 - I feel that it can be done with continuations, but I don't know how to do
it with Scala delimited continuations. They seem to be not powerful enough
for me. But I may be wrong here.

While I understand your point about what can be done today with Scala for
comprehensions and current continuations, I'm still interested in the
potential possibilities to achieve what I sketched above and in previous
mails using Scala and without rewriting all of the (3rd party) code. May be
this is not possible today, may be someone has implemented similar systems
already. Any feedback is appreciated.

It would be also interesting to get Tiark's opinion about all those
continuation related questions. Any ideas about using Javaflow or Kilim like
approaches for Scala (I've read that people were trying to use it for Swarm,
http://www.scala-lang.org/node/3485)? What about the JVM support for
continuations? (see
http://wiki.jvmlangsummit.com/images/2/2b/JVMLanguageSummit_Stadler_Continuations.pdf
and
http://weblogs.java.net/blog/forax/archive/2009/11/19/holy-crap-jvm-has-coroutinecontinuationfiber-etc)

Thanks,
 -Leo

> > It would be also
> > nice to do it transparently for a developer, so that the code still
> looks
> > like a usual sequential code, but in reality suspends/resumes on long
> > async
> > IO operations. And I thought that Scala continuations could be used to
> > implement this. So that one could write in the actor body something
> like:
> >
> > exec_pre_io_code();
> > // the following invocation would suspend the current actor. Later, when
> a
> > notification about IO operation
> > // completion is received, current actor will be resumed and would
> > continue
> > with exec_after_io_code()
> > exec_long_io_operation(io_operation_params);
> > exec_after_io_code();
> >
> > It feels like something very similar to CPS transformation done by Scala
> > compiler's continuations plugin is required here. Or?
> >
>
> >If you can live with doing he stuff in a for comprehension (as above),
> you
> >can do it today without continuations or actors.


--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2719351.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

After all those continuations discussions, let's come back to the original
questions about Scala use in JavaEE environments.

dpp wrote:
>
>
>>> But what about usage of Actors inside an EJB container, e.g. if an EJB
>>> would call it? AFAIK, according to the EJB spec, EJB containers should
>>> not permit
>>> explicit thread-management (e.g. thread creation) by user-code as it may
>>> lead to conflicts.
>
>
>> If there's a way to transfer an EJB across a thread (e.g., to a worker
>> thread that does background processing), the very same mechanism would be
>> used in Actors.
>

Well, there is no direct access to threads or java.concurrent inside EJB
components. There is no such thing like Thread at all. EJBs are "actors" in
the JavaEE world. And the EJB specification forbids direct manipulations
with threads by means of Threads API ;-(
This is a serious limitation. Some vendors provide certain kinds of
workarounds. For example, BEA and IBM provide a special WorkManager API.
Glassfish has something similar. You can get a WorkManager, which is
something like an Executor factory. But they are not standardized and not
compatible with each other. These APIs are modeled after j.u.concurrent, but
are not 100% the same thing. Nevertheless, supporting these major app
servers is very important. May be something can be done about it? May be
Scala actor's support and library can be extended to become configurable so
that ThreadPools factories and the like can be provided by an application
server as well?

Another thing, I'd like to understand is about publishing and discovery of
Scala actors in JavaEE environments. Most JavaEE components (EJBs, MDBs, JMS
queues, etc) published themselves in JNDI directory, so that other JavaEE
applications running on the same or other application servers can find them.
I think, it would be nice to have the same for Scala actors, so that other
JavaEE apps can discover them and send messages to them. So, here is a
question:
- Is it possible to publish an Actor in the JNDI?
- Is it possible to invoke an Actor that runs inside a different JavaEE
application, i.e. it is part of another EAR/WAR file and was loaded in a
different classloader, as it is always the case in the JavaEE world. Each
app runs in the same JVM, but in its own "sandbox".

These questions covered just the most obvious things. I guess there are more
issues of this kind. Please add them, if I haven't mentioned them yet.

I think, many Java professionals would really like to be able to use Scala
for JavaEE development instead of plain old Java. Scala is much nicer and
more powerful as a language. To be able to do it, it is important to play
the game according to JavaEE rules, to be compliant with it and to have a
seamless integration of Scala components into JavaEE world as first-class
citizens. If this kind of support is not in place yet, what are the plans of
Scala developers or promoters? Is anyone working on it? Is it on the
roadmap?

* appeal *
I know that implementing this kind of features may seem not very sexy for
Scala language developers. It is not so interesting as type systems,
functional features, etc. But having a clear picture and even better a good
support for JavaEE may be crucial for adoption of Scala in the enterprise
world. And this adoption by the enterprise development segment may become
the most important factor for Scala for moving from a "research/academic
language league" to the mainstream language. Enterprises and telcos are also
the places where the real money are. So, having them on board is very useful
from many different perspectives.
* End of appeal *

Thanks,
Leo

On Wed, Sep 29, 2010 at 9:35 AM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi David,

Thanks for your explanations and your patience with me!

> > dpp wrote:
> >Scala's delimited continuations (delimited being the key word) are done
> at
> >compile time, so all the accessible state is known by the compiler and
> >captured by the compiler into the continuation object (just like
> functions
> >close over scope.)  So, the byte-codes are legal, there's no byte-code
> >re-writing at run-time and they should "just work" in every execution
> >environment.
>
>> OK. I understand.
>>
>> Just out of couriosity: Are there any attempts at the library or language
>> runtime level to also support dynamic continuations based on capturing
>> current call stack and suspending and later restoring of the state, as it
>> is done by e.g. Javaflow library?

>> I'm wondering, because this would allow to work with almost ANY 3rd party
>> library invoked at the current point, even if it is was not rewritten in
>> the CPS style by Scala compiler. More over, it would remove the
>> limitation that
>> function call leading to a continuation should be sort of the tail call
>> in
>> the caller stackframe, i.e. you with it you could "cut" the control flow
>> of
>> all the callers on the call stack "in the middle" when suspending and
>> later
>> restore everything. While being more expensive at runtime, it gives also
>> a
>> lot of flexibility.


>The delimited continuations are much, much cheaper (they consume far less
>space and time to create the continuation) and they are much more friendly.

No doubts about that.

For a new project where I defined the architecture and build it from
scratch, I'd most likely use Scala's delimited continuations.

But I'm trying to figure out also if there is a possibility to support other
kinds of continuations in Scala.
The pragmatic reason for this question is the fact that there are a lot of
legacy systems and 3rd party libraries written in Java. Rewriting all of
them to use Scala continuations is just out of scope and probably
impossible. Therefore, having an implementation of continuations that may be
combined with almost ANY existing Java library is very important!

> There are Java-based projects the do full continuations

Yes. I mentioned them before. This is Kilim and Javaflow, for example.

> and Tiark has some data on the performance of each in some paper (I don't
> have a link.)

Yes, I found it:
http://lamp.epfl.ch/~rompf/continuations-icfp09.pdf


>> It is also interesting, because Python and some other languages use
>> continuations to implement very cheap and efficient lightweight
>> multithreading. What they roughly do may look in Java like follows:
>>  - They start on a thread pool something like Runnables/Callables
>> supplied
>> by the user. But they don't start it directly. Instead they start it
>> using
>> a small dedicated Runnable wrapper around this user-supplied
>> Runnable/Callable.
>>  - The wrapper starts the real Runnable via a dedicated Continuations
>> support function of the Continuations libray. The result of such a call
>> will be either empty continuation, i.e. there were no
>> Continuation.suspend()
>> calls in user-code and execution of Runnable just finished. Or a caught
>> continuation object will be returned, containing the captured stack
>> frames
>> and their state (all stack frames upto this wrapper stack frame are
>> captured. In this sense, it delimits the bottom of stack for a
>> continuation). In this case, the wrapper does some bookkeeping and exits,
>> thus releasing a thread for other tasks!
>> - Later,  when user decides to proceed with the continuation, she just
>> calls
>> Continuation.resume(continuation_object_captured_before). This would
>> submit
>> the same wrapper to the thread pool. Once it gets scheduled for
>> execution,
>> it would rebuild the stack frames (by calling the same methods in the
>> same
>> order and restoring the control flow) and their state and thus resume
>> from
>> the place where it was stopped, i.e. right after  Continuation.suspend()
>> call in the user-code.
>>
>> I hope this explanation was half-way understandible. For a code example,
>> see e.g.
>> http://commons.apache.org/sandbox/javaflow/apidocs/org/apache/commons/javaflow/Continuation.html
>> or its implementation.


>Yeah... I'm not a fan of that kind of coding.  There's just too much room
>for pain by snapshotting the whole stack when all you really want is a
small
>part of the stack.

There are ways to limit it. You can tell it to snapshot just upto a certain
method X on the call-stack. This would serve as a delimiter. But of course,
it is not as efficient as CPS transformation and leads to more overhead.


>> >Scala's delimited continuations are not a magic "freeze the thread at
>> any
>> >point and resume it later" mechanism.  The continuation my only be
>> resumed
>> >at points where it is marked.  From a syntactic standpoint, it's cleaner
>> >than using a for comprehension, but in Stambecco (a distributed actor
>> system
>> >I've got as a side project), you can do:
>>
>> >for {
>> >  future_a <- invoke_bit_of_work_1
>> > future_b <- invoke_bit_of_work_2
>> >  future_c <- invoke_bit_of_work_3
>> >  a <- future_a
>> >  b <- future_b
>> >  c <- future_c
>> >} {
>> >  // do stuff with a, b, and c
>> >}
>>
>> >The for comprehension returns immediately, but as a, b, and c become
>> >available, the computation is continued on worker threads.
>>
>> Sorry, David,  but I'm not skilled in Scala for comprehensions, futures
>> and
>> lazy evaluation. Could you explain this example a bit more, or write it
>> down
>> using real Scala code instead of pseudocode?

>This is real code (you have to
>implement invoke_bit_of_work_1, invoke_bit_of_work_2
>& invoke_bit_of_work_3), but it gives you the idea of what is possible with
>Scala's for comprehension (well, monads).

First of all, your example works with Stambecco workers/actors (which I
assume) or is it a standard Scala with standard library?

>> This would really help me. At
>> the moment, I don't quite understand if the process of waiting for a,b
>> and
>> c to become available occupies the current thread? Is it done using
>> Future.get(), is so, it would block current thread, or?


> Because the for comprehension returns Unit, it returns immediately (or
> perhaps just after the invoke messages have been sent).

I understand. Futures do not wait for completion. You just submit a task to
a threadpool and immediately get a future object back.

BTW, is it correct to say that your foreach approach requires this foreach
to be the last statement on the current function and thread, i.e. it is a
sort of "tailcall"?

So, foreach(f: A => Unit) takes a function that takes an A and returns a Unit.  So, you're returning no meaningful value to the caller (like Java's void).  The following for comprehension:

for {
  a <- calcA
  b <- calcB
  c <- calcC
  } {doSomething(a,b,c)}

gets turned into:

calcA.foreach(a => calcB.foreach(b => calcC.foreach(c => doSomething(a,b,c))))
 

Otherwise the code after the whole "foreach {}" would be
executed right after you got your future objects back, or? And the body of
foreach would be executed only once features are done and ready, i.e.
eventually much later. If my understanding is correct, it may result in a
very weired flow of execution, as from looking at the code you'd assume a
sequential execution (first foreach block, then anything after it).

The current thread in the above example returns almost immediately, but the computation is continued on another thread when the computations are satisfied.
 

> When the foreach method is invoked on future_a, if future_a is satisfied,
> the thread
> continues.

Yes. That's easy.

> If future_a is not satisfied, the function passed to foreach is
> noted in future_a

You mean that the Unit representing a closure of the body (or whatever comes
after a<-future_a) is stored as sort of a continuation callback associated
with the future_a? So that later, when future_a is satisfied this callback
can be invoked?

In the above example, we're passing functions to the foreach method.  Functions can be thought of as callbacks, but they are, well, instances of objects that implement the FunctionXX trait and have an apply() method.  So, you can apply (call) them now, later, lots of times, once, never, put them in a hash table and apply them lots of times.
 

> and  when future_a is satisfied,

But how do you check that future_a is satisfied? See my comments below.

> work is enqueued on a threadpool to continue the calculation.  I do this
> in Stambecco
> http://stambecco.org

I looked at the code, hopefully at the right place. Do you mean this class?
(BTW all classes are dated back in 2008. Is it correct?)

I don't see any 2008 dates in Stambecco.  Most of the Stambecco code was written in late 2009 and early 2010.
 

https://www.assembla.com/code/stambecco/git/nodes/core/src/main/scala/org/stambecco/internal/Saddle.scala?rev=c46d8e8af7a747648fd6842b12dc0c1bdf6858a3

I mean this method:

 /**
  * Return a continuation that allows you to execute code when the
  * answer becomes available, without blocking the current thread.
  */
 def continue: Responder[Box[T]] = {
   var done = false
   var funcList: List[() => Unit] = Nil
   val lockObj = new Object
   var resp: Box[T] = Empty

   askFunc((f: Box[T]) => {
       val lst = lockObj.synchronized{if (!done) {resp = f; done = true;
funcList} else Nil}
       lst.foreach(f => LAScheduler.execute(f))
     } , this)

   new Responder[Box[T]] {
     def respond(f: Box[T] => Unit) = {
       val b: () => Unit = lockObj.synchronized {
         val tf = () => f(resp)
         if (done) tf
         else {
           funcList ::= tf
           () => {}
         }
       }

       b.apply()
     }
   }
 }
}

If I understand your code correctly, you try to check if future is done,
i.e. computed. And if not, you put it into the list of futures to check next
time for their completion.

If the future is satisfied, then execute the function.  If the future is not satisfied, add the function to the list of functions to be executed when the future is satisfied.
 

Then you submit this "check futures" function for
execution on the threadpool.

Nope.  The only thing that's submitted to the threadpool is the list of functions to be executed once the future is satisfied.
 

You repeat this, until the future is satisfied
and then you invoke the callback, i.e. the body of the foreach.  Please
correct me, if I'm wrong.

But then, it means that you do polling/busy waiting on futures, actively
checking for their completion.

Nope.  The only computation that happens on a future is when one of two things happens: (1) the future is satisfied or (2) the future is applied (the foreach method is called on the Responder).  So there's no "polling", there's only reaction to an external call.

In the for-comprehension example above, there will be 1 function added (respond will be called once) and there will be one call to satisfy the future (okay, there may be more, but subsequent calls will be ignored).  So, you've got 2 different method invocations, each method invocation is O(1).  If neither of the methods are invoked (there's no attempt to call foreach, or there's no attempt to satisfy the future), you'll have the future sitting around in memory until there are no references to it and then it will be GCed like any other JVM object.
 

This busy waiting can keep CPU active even if
nothing happens. Moreover,  I'm not sure about scalability of this approach.
What if you have 50000 such foreach loops executed at the same time by
different actors?

This is *NOT* actor-based code.  Mixing threading models with Scala Actor threading is dangerous and bad.

This is an implementation of continuation passing style of coding.
 

I have the impression that you'll get a huge set of
scheduled "check futures" tasks that would eat all of you CPU time.

You'd be wrong about that.  You'd have 50,000 object hanging around consuming memory until they were garbage collected.  If the future is satisfied, then the computation is continued and if the future is never satisfied and the reference to it goes away, then it's no different than creating a String.
 

Another thing which I don't quite get is: What if the computation enclosed
into the future is something like an invocation of an external XML Web
Service which may take a lot of time. I guess the thread which really
performs this invocation (and this is most likely not the thread that
executed a function containing the mentioned foreach construct, but a thread
from a dedicated thread pool) would do a blocking wait for the results, or?
If so, we just shifted the problem from the original thread to another
thread.

Lift's long polling uses a similar mechanism.  If your Lift app is running in a container that supports continuations, Lift detects this and uses the continuations to suspend the current thread.  When there's a change in one of the Comet components, Lift resumes the computation of the response and delivers it to the container.
 

And the problem that I see is: If you need to perform simultaneously 50000
or 10000 such invocations, you cannot do it if threads are being blocked
somewhere.

That's why you use thread pooling.
 

JVM just does not support that many threads. The only solution is
IMHO to become REALLY asynchronous, i.e. NIO and no scheduled threadpool
task can do a blocking wait! Approach that you described does not seem to
solve it, if I understand it correctly. Therefore we are back to the
original questions:

 - Can this be somehow achieved by means of Scala, be it actors,
continuations or anything else? And I'm really looking for a solution,where
I do not do most of the job by hands, i.e splitting of functions into parts,
building closures, managing callbacks, etc. I want my language, its runtime
and library to do it for me.

It's called an operating system.  If you want seamless scheduling with arbitrary suspension and resumption of a call stack where scheduling is based on waiting for external IO and other events, that's want the operating system does.  If you want to give hints, there are mechanisms for converting those hints to execution scheduling, then I've outlined how it can be done above.
 

 - I feel that it can be done with continuations, but I don't know how to do
it with Scala delimited continuations. They seem to be not powerful enough
for me. But I may be wrong here.

While I understand your point about what can be done today with Scala for
comprehensions and current continuations, I'm still interested in the
potential possibilities to achieve what I sketched above and in previous
mails using Scala and without rewriting all of the (3rd party) code. May be
this is not possible today, may be someone has implemented similar systems
already. Any feedback is appreciated.

If you've got a real-world description of a real project that has these requirements, that's cool.  I don't know of any systems that are expected to handle 10,000 threads of computation simultaneously on a single box.
 

It would be also interesting to get Tiark's opinion about all those
continuation related questions. Any ideas about using Javaflow or Kilim like
approaches for Scala (I've read that people were trying to use it for Swarm,
http://www.scala-lang.org/node/3485)? What about the JVM support for
continuations? (see
http://wiki.jvmlangsummit.com/images/2/2b/JVMLanguageSummit_Stadler_Continuations.pdf
and
http://weblogs.java.net/blog/forax/archive/2009/11/19/holy-crap-jvm-has-coroutinecontinuationfiber-etc)

Thanks,
 -Leo

> > It would be also
> > nice to do it transparently for a developer, so that the code still
> looks
> > like a usual sequential code, but in reality suspends/resumes on long
> > async
> > IO operations. And I thought that Scala continuations could be used to
> > implement this. So that one could write in the actor body something
> like:
> >
> > exec_pre_io_code();
> > // the following invocation would suspend the current actor. Later, when
> a
> > notification about IO operation
> > // completion is received, current actor will be resumed and would
> > continue
> > with exec_after_io_code()
> > exec_long_io_operation(io_operation_params);
> > exec_after_io_code();
> >
> > It feels like something very similar to CPS transformation done by Scala
> > compiler's continuations plugin is required here. Or?
> >
>
> >If you can live with doing he stuff in a for comprehension (as above),
> you
> >can do it today without continuations or actors.


--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2719351.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Blog: http://goodstuff.im
Surf the harmonics

Thanks a lot, David!

Now I finally understand your foreach-based solution!

>> Another thing which I don't quite get is: What if the computation
>> enclosed
>> into the future is something like an invocation of an external XML Web
>> Service which may take a lot of time. I guess the thread which really
>> performs this invocation (and this is most likely not the thread that
>> executed a function containing the mentioned foreach construct, but a
>> thread
>> from a dedicated thread pool) would do a blocking wait for the results,
>> or?
>> If so, we just shifted the problem from the original thread to another
>> thread.

> Lift's long polling uses a similar mechanism. If your Lift app is running
> in a container that supports continuations, Lift detects this and uses the
> continuations to suspend the current thread. When there's a change in one
> of the Comet components, Lift resumes the computation of the response and
> delivers it to the container.

I understand. My problem is that I'm sort of trying to implement a
container. So, I cannot count on something else that already supports
continuations like Jetty. I need to become a container that can do it ;-)

> JVM just does not support that many threads. The only solution is
> IMHO to become REALLY asynchronous, i.e. NIO and no scheduled threadpool
> task can do a blocking wait! Approach that you described does not seem to
> solve it, if I understand it correctly. Therefore we are back to the
> original questions:
>
> - Can this be somehow achieved by means of Scala, be it actors,
> continuations or anything else? And I'm really looking for a
> solution,where
> I do not do most of the job by hands, i.e splitting of functions into
> parts,
> building closures, managing callbacks, etc. I want my language, its
> runtime
> and library to do it for me.
>

> It's called an operating system. If you want seamless scheduling with
> arbitrary suspension and resumption of a call stack where scheduling is
> based on waiting for external IO and other events, that's want the
> operating
> system does.

I'm not so sure about it. I'd say it is something that e.g. an application
server may do.
Moreover, Python folks can do it as well using C-based continuations for
ultra-fast cooperative light-threads implementation. But it is easier for
them, as you have full control over your stack in C.

> If you want to give hints, there are mechanisms for converting those hints
> to execution scheduling, then > I've outlined how it can be done above.

Thanks a lot for that!

>> - I feel that it can be done with continuations, but I don't know how to
>> do
>> it with Scala delimited continuations. They seem to be not powerful
>> enough
>> for me. But I may be wrong here.
>>
>> While I understand your point about what can be done today with Scala for
>> comprehensions and current continuations, I'm still interested in the
>> potential possibilities to achieve what I sketched above and in previous
>> mails using Scala and without rewriting all of the (3rd party) code. May
>> be
>> this is not possible today, may be someone has implemented similar
>> systems
>> already. Any feedback is appreciated.

> If you've got a real-world description of a real project that has these
> requirements, that's cool. I don't know of any systems that are expected
> to
> handle 10,000 threads of computation simultaneously on a single box.

That's easy. Think of a workflow engine. Imagine that each client may
trigger a new instance of the workflow. And each workflow consists mostly of
invocations of different external enterprise services. So, most of the time
it does not "compute", instead it just invokes external services and waits
for results, then it aggregates them and eventually returns to the client.
Or just triggers another workflow.

If you ask how realistically is it to get 10000 simultaneous clients, I can
also give you a hint. Think about telecom communication services. Each big
mobile/cellular operator usually has a few million users. And usually a few
thousands of them have ongoing conversations simultaneously. On the
Christmas eve this number may get even much higher ;-) Now imagine that each
attempt to setup a phone call triggers a workflow.

Of course, you may argue about how many users you want to handle on a single
box. But if you have a few millions of users at the same time, you may need
a lot of boxes. So, density plays a big role in such cases.

Remark: I have built a similar system in JavaEE and it even works. It
handles even 20000 requests. But the implementation is not so nice. I'm 100%
sure it can be done better and cleaner in Scala, but I need to find out how.
That's one of the reasons for my questions.

Thanks,
Leo

Any feedback on:

- usage of Actors inside EJB containers and possible multithreading issues?
- ability to publish (via JNDI?), discover and invoke Actors deployed as
part of different JavaEE apps running on the same or different nodes?

Thanks,
-Leo

Leo Romanoff wrote:
>
> After all those continuations discussions, let's come back to the original
> questions about Scala use in JavaEE environments.
>
>
> dpp wrote:
>>
>>
>>>> But what about usage of Actors inside an EJB container, e.g. if an EJB
>>>> would call it? AFAIK, according to the EJB spec, EJB containers should
>>>> not permit
>>>> explicit thread-management (e.g. thread creation) by user-code as it
>>>> may
>>>> lead to conflicts.
>>
>>
>>> If there's a way to transfer an EJB across a thread (e.g., to a worker
>>> thread that does background processing), the very same mechanism would
>>> be
>>> used in Actors.
>>
>
> Well, there is no direct access to threads or java.concurrent inside EJB
> components. There is no such thing like Thread at all. EJBs are "actors"
> in the JavaEE world. And the EJB specification forbids direct
> manipulations with threads by means of Threads API ;-(
> This is a serious limitation. Some vendors provide certain kinds of
> workarounds. For example, BEA and IBM provide a special WorkManager API.
> Glassfish has something similar. You can get a WorkManager, which is
> something like an Executor factory. But they are not standardized and not
> compatible with each other. These APIs are modeled after j.u.concurrent,
> but are not 100% the same thing. Nevertheless, supporting these major app
> servers is very important. May be something can be done about it? May be
> Scala actor's support and library can be extended to become configurable
> so that ThreadPools factories and the like can be provided by an
> application server as well?
>
> Another thing, I'd like to understand is about publishing and discovery of
> Scala actors in JavaEE environments. Most JavaEE components (EJBs, MDBs,
> JMS queues, etc) published themselves in JNDI directory, so that other
> JavaEE applications running on the same or other application servers can
> find them. I think, it would be nice to have the same for Scala actors, so
> that other JavaEE apps can discover them and send messages to them. So,
> here is a question:
> - Is it possible to publish an Actor in the JNDI?
> - Is it possible to invoke an Actor that runs inside a different JavaEE
> application, i.e. it is part of another EAR/WAR file and was loaded in a
> different classloader, as it is always the case in the JavaEE world. Each
> app runs in the same JVM, but in its own "sandbox".
>
> These questions covered just the most obvious things. I guess there are
> more issues of this kind. Please add them, if I haven't mentioned them
> yet.
>
> I think, many Java professionals would really like to be able to use Scala
> for JavaEE development instead of plain old Java. Scala is much nicer and
> more powerful as a language. To be able to do it, it is important to play
> the game according to JavaEE rules, to be compliant with it and to have a
> seamless integration of Scala components into JavaEE world as first-class
> citizens. If this kind of support is not in place yet, what are the plans
> of Scala developers or promoters? Is anyone working on it? Is it on the
> roadmap?
>
> * appeal *
> I know that implementing this kind of features may seem not very sexy for
> Scala language developers. It is not so interesting as type systems,
> functional features, etc. But having a clear picture and even better a
> good support for JavaEE may be crucial for adoption of Scala in the
> enterprise world. And this adoption by the enterprise development segment
> may become the most important factor for Scala for moving from a
> "research/academic language league" to the mainstream language.
> Enterprises and telcos are also the places where the real money are. So,
> having them on board is very useful from many different perspectives.
> * End of appeal *
>
> Thanks,
> Leo
>
>
>
>

I think my original reply stands.

You should not use actors or threads in a container, as it will easily
mangle contexts, transactions and whatnot.

Having said that, if you have sufficient experience and can carefully tippy
toe your way around that problem, you could...well, use them.

As for the distributed part, I think both akka and Stambecco have support
for remote actors, probably not via JNDI though.

Cheers,
Razie

-----Original Message-----
From: Leo Romanoff [mailto:romixlev@yahoo.com]
Sent: Friday, October 01, 2010 1:12 PM
To: scala-user@listes.epfl.ch
Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments

Any feedback on:

- usage of Actors inside EJB containers and possible multithreading issues?
- ability to publish (via JNDI?), discover and invoke Actors deployed as
part of different JavaEE apps running on the same or different nodes?

Thanks,
-Leo

Leo Romanoff wrote:
>
> After all those continuations discussions, let's come back to the
> original questions about Scala use in JavaEE environments.
>
>
> dpp wrote:
>>
>>
>>>> But what about usage of Actors inside an EJB container, e.g. if an
>>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>>>> should not permit explicit thread-management (e.g. thread creation)
>>>> by user-code as it may lead to conflicts.
>>
>>
>>> If there's a way to transfer an EJB across a thread (e.g., to a
>>> worker thread that does background processing), the very same
>>> mechanism would be used in Actors.
>>
>
> Well, there is no direct access to threads or java.concurrent inside
> EJB components. There is no such thing like Thread at all. EJBs are
"actors"
> in the JavaEE world. And the EJB specification forbids direct
> manipulations with threads by means of Threads API ;-( This is a
> serious limitation. Some vendors provide certain kinds of workarounds.
> For example, BEA and IBM provide a special WorkManager API.
> Glassfish has something similar. You can get a WorkManager, which is
> something like an Executor factory. But they are not standardized and
> not compatible with each other. These APIs are modeled after
> j.u.concurrent, but are not 100% the same thing. Nevertheless,
> supporting these major app servers is very important. May be something
> can be done about it? May be Scala actor's support and library can be
> extended to become configurable so that ThreadPools factories and the
> like can be provided by an application server as well?
>
> Another thing, I'd like to understand is about publishing and
> discovery of Scala actors in JavaEE environments. Most JavaEE
> components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
> directory, so that other JavaEE applications running on the same or
> other application servers can find them. I think, it would be nice to
> have the same for Scala actors, so that other JavaEE apps can discover
> them and send messages to them. So, here is a question:
> - Is it possible to publish an Actor in the JNDI?
> - Is it possible to invoke an Actor that runs inside a different
> JavaEE application, i.e. it is part of another EAR/WAR file and was
> loaded in a different classloader, as it is always the case in the
> JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>
> These questions covered just the most obvious things. I guess there
> are more issues of this kind. Please add them, if I haven't mentioned
> them yet.
>
> I think, many Java professionals would really like to be able to use
> Scala for JavaEE development instead of plain old Java. Scala is much
> nicer and more powerful as a language. To be able to do it, it is
> important to play the game according to JavaEE rules, to be compliant
> with it and to have a seamless integration of Scala components into
> JavaEE world as first-class citizens. If this kind of support is not
> in place yet, what are the plans of Scala developers or promoters? Is
> anyone working on it? Is it on the roadmap?
>
> * appeal *
> I know that implementing this kind of features may seem not very sexy
> for Scala language developers. It is not so interesting as type
> systems, functional features, etc. But having a clear picture and
> even better a good support for JavaEE may be crucial for adoption of
> Scala in the enterprise world. And this adoption by the enterprise
> development segment may become the most important factor for Scala for
> moving from a "research/academic language league" to the mainstream
language.
> Enterprises and telcos are also the places where the real money are.
> So, having them on board is very useful from many different perspectives.
> * End of appeal *
>
> Thanks,
> Leo
>
>
>
>

--
View this message in context:
http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
vaEE-environments-tp2553702p2936337.html
Sent from the Scala - User mailing list archive at Nabble.com.

On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru <pub@razie.com> wrote:

I think my original reply stands.

You should not use actors or threads in a container, as it will easily
mangle contexts, transactions and whatnot.

I disagree.
It totally depends what you're running in that container.
 

Having said that, if you have sufficient experience and can carefully tippy
toe your way around that problem, you could...well, use them.

As for the distributed part, I think both akka and Stambecco have support
for remote actors, probably not via JNDI though.

Cheers,
Razie

-----Original Message-----
From: Leo Romanoff [mailto:romixlev@yahoo.com]
Sent: Friday, October 01, 2010 1:12 PM
To: scala-user@listes.epfl.ch
Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments


Any feedback on:

- usage of Actors inside EJB containers and possible multithreading issues?
- ability to publish (via JNDI?), discover and invoke Actors deployed as
part of different JavaEE apps running on the same or different nodes?

Thanks,
 -Leo


Leo Romanoff wrote:
>
> After all those continuations discussions, let's come back to the
> original questions about Scala use in JavaEE environments.
>
>
> dpp wrote:
>>
>>
>>>> But what about usage of Actors inside an EJB container, e.g. if an
>>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>>>> should not permit explicit thread-management (e.g. thread creation)
>>>> by user-code as it may lead to conflicts.
>>
>>
>>> If there's a way to transfer an EJB across a thread (e.g., to a
>>> worker thread that does background processing), the very same
>>> mechanism would be used in Actors.
>>
>
> Well, there is no direct access to threads or java.concurrent inside
> EJB components. There is no such thing like Thread at all. EJBs are
"actors"
> in the JavaEE world. And the EJB specification forbids direct
> manipulations with threads by means of Threads API ;-( This is a
> serious limitation. Some vendors provide certain kinds of workarounds.
> For example, BEA and IBM provide a special WorkManager API.
> Glassfish has something similar. You can get a WorkManager, which is
> something like an Executor factory. But they are not standardized and
> not compatible with each other. These APIs are modeled after
> j.u.concurrent, but are not 100% the same thing. Nevertheless,
> supporting these major app servers is very important. May be something
> can be done about it? May be Scala actor's support and library can be
> extended to become configurable so that ThreadPools factories and the
> like can be provided by an application server as well?
>
> Another thing, I'd like to understand is about publishing and
> discovery of Scala actors in JavaEE environments. Most JavaEE
> components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
> directory, so that other JavaEE applications running on the same or
> other application servers can find them. I think, it would be nice to
> have the same for Scala actors, so that other JavaEE apps can discover
> them and send messages to them. So, here is a question:
>  - Is it possible to publish an Actor in the JNDI?
>  - Is it possible to invoke an Actor that runs inside a different
> JavaEE application, i.e. it is part of another EAR/WAR file and was
> loaded in a different classloader, as it is always the case in the
> JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>
> These questions covered just the most obvious things. I guess there
> are more issues of this kind. Please add them, if I haven't mentioned
> them yet.
>
> I think, many Java professionals would really like to be able to use
> Scala for JavaEE development instead of plain old Java. Scala is much
> nicer and more powerful as a language. To be able to do it, it is
> important to play the game according to JavaEE rules, to be compliant
> with it and to have a seamless integration of Scala components into
> JavaEE world as first-class citizens. If this kind of support is not
> in place yet, what are the plans of Scala developers or promoters? Is
> anyone working on it? Is it on the roadmap?
>
> * appeal *
> I know that implementing this kind of features may seem not very sexy
> for Scala language developers. It is not so interesting as type
> systems, functional features, etc.  But having a clear picture and
> even better a good support for JavaEE may be crucial for adoption of
> Scala in the enterprise world. And this adoption by the enterprise
> development segment may become the most important factor for Scala for
> moving from a "research/academic language league" to the mainstream
language.
> Enterprises and telcos are also the places where the real money are.
> So, having them on board is very useful from many different perspectives.
> * End of appeal *
>
> Thanks,
>   Leo
>
>
>
>

--
View this message in context:
http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-Ja
vaEE-environments-tp2553702p2936337.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

Indeed - I assumed you'd run a J2EE container because you're using
"heavy" J2EE services like XA transactions, authentication contexts,
fail-over etc.

On 10/1/10, Viktor Klang wrote:
> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru wrote:
>
>> I think my original reply stands.
>>
>> You should not use actors or threads in a container, as it will easily
>> mangle contexts, transactions and whatnot.
>>
>
> I disagree.
> It totally depends what you're running in that container.
>
>
>>
>> Having said that, if you have sufficient experience and can carefully
>> tippy
>> toe your way around that problem, you could...well, use them.
>>
>> As for the distributed part, I think both akka and Stambecco have support
>> for remote actors, probably not via JNDI though.
>>
>> Cheers,
>> Razie
>>
>> -----Original Message-----
>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>> Sent: Friday, October 01, 2010 1:12 PM
>> To: scala-user@listes.epfl.ch
>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments
>>
>>
>> Any feedback on:
>>
>> - usage of Actors inside EJB containers and possible multithreading
>> issues?
>> - ability to publish (via JNDI?), discover and invoke Actors deployed as
>> part of different JavaEE apps running on the same or different nodes?
>>
>> Thanks,
>> -Leo
>>
>>
>> Leo Romanoff wrote:
>> >
>> > After all those continuations discussions, let's come back to the
>> > original questions about Scala use in JavaEE environments.
>> >
>> >
>> > dpp wrote:
>> >>
>> >>
>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>> >>>> should not permit explicit thread-management (e.g. thread creation)
>> >>>> by user-code as it may lead to conflicts.
>> >>
>> >>
>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>> >>> worker thread that does background processing), the very same
>> >>> mechanism would be used in Actors.
>> >>
>> >
>> > Well, there is no direct access to threads or java.concurrent inside
>> > EJB components. There is no such thing like Thread at all. EJBs are
>> "actors"
>> > in the JavaEE world. And the EJB specification forbids direct
>> > manipulations with threads by means of Threads API ;-( This is a
>> > serious limitation. Some vendors provide certain kinds of workarounds.
>> > For example, BEA and IBM provide a special WorkManager API.
>> > Glassfish has something similar. You can get a WorkManager, which is
>> > something like an Executor factory. But they are not standardized and
>> > not compatible with each other. These APIs are modeled after
>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>> > supporting these major app servers is very important. May be something
>> > can be done about it? May be Scala actor's support and library can be
>> > extended to become configurable so that ThreadPools factories and the
>> > like can be provided by an application server as well?
>> >
>> > Another thing, I'd like to understand is about publishing and
>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>> > directory, so that other JavaEE applications running on the same or
>> > other application servers can find them. I think, it would be nice to
>> > have the same for Scala actors, so that other JavaEE apps can discover
>> > them and send messages to them. So, here is a question:
>> > - Is it possible to publish an Actor in the JNDI?
>> > - Is it possible to invoke an Actor that runs inside a different
>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>> > loaded in a different classloader, as it is always the case in the
>> > JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>> >
>> > These questions covered just the most obvious things. I guess there
>> > are more issues of this kind. Please add them, if I haven't mentioned
>> > them yet.
>> >
>> > I think, many Java professionals would really like to be able to use
>> > Scala for JavaEE development instead of plain old Java. Scala is much
>> > nicer and more powerful as a language. To be able to do it, it is
>> > important to play the game according to JavaEE rules, to be compliant
>> > with it and to have a seamless integration of Scala components into
>> > JavaEE world as first-class citizens. If this kind of support is not
>> > in place yet, what are the plans of Scala developers or promoters? Is
>> > anyone working on it? Is it on the roadmap?
>> >
>> > * appeal *
>> > I know that implementing this kind of features may seem not very sexy
>> > for Scala language developers. It is not so interesting as type
>> > systems, functional features, etc. But having a clear picture and
>> > even better a good support for JavaEE may be crucial for adoption of
>> > Scala in the enterprise world. And this adoption by the enterprise
>> > development segment may become the most important factor for Scala for
>> > moving from a "research/academic language league" to the mainstream
>> language.
>> > Enterprises and telcos are also the places where the real money are.
>> > So, having them on board is very useful from many different
>> > perspectives.
>> > * End of appeal *
>> >
>> > Thanks,
>> > Leo
>> >
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>>
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>> vaEE-environments-tp2553702p2936337.html
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Viktor Klang,
> Code Connoisseur
> Work: www.akkasource.com
> Code: github.com/viktorklang
> Follow: twitter.com/viktorklang
> Read: klangism.tumblr.com
>

Razvan and Viktor - thanks for your comments!

But what I get to hear still sounds very fuzzy for me. See below.

>> Having said that, if you have sufficient experience and can carefully
>> tippy
>> toe your way around that problem, you could...well, use them.

This sounds like a "try to do it yourself - and may be you are lucky"
exercise

>> As for the distributed part, I think both akka and Stambecco have support
>> for remote actors, probably not via JNDI though.

OK. This means that this mechanism is not quite JavaEE compliant.

What I'm trying to figure out in this whole thread is some sort of best
practices or direction for an (average) JavaEE developer for starting using
Scala and its libraries and frameworks in her daily work. Often, in big
JavaEE shops (e.g. enterprises and telcos), you have to use a huge existing
JavaEE-based codebase. Rewriting everything the Scala-way or to become
Web-based is often out of question. Therefore, the natural basic
requirements for using Scala in this context are:

- that Scala code should be able to at least interwork (bind, invoke,
consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).

- In a perfect world, it should be possible to implement JavaEE components
(servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
standards for packaging and deployment. Is it possible now? If not, is it
envisioned?

- It would be useful to expose Scala specific components (e.g. actors) in a
JavaEE friendly way, so that they can be consumed by other JavaEE
components. What about application packaging in this case?

- What is a guideline for Scala components development? Does it make sense
to strive for integration with JavaEE? Or is it better to use Scala-specific
frameworks? Should Scala be used as Java++ for JavaEE components development
(i.e. only new and better language constructs, but e.g. no actors, as they
depend too much on multithreading implementation and ability to explicitly
control it). Or does it make sense to use actors and the like to implement
JavaEE components?

I think having clear statements (somewhere on major scala-related sites?)
answering all these questions would be a great help for many JavaEE
developers planning to switch to Scala.

At the moment this kind of information is lacking. I've spent quite some
time to find out any information about what is possible and what's not. It
seems to be a "black art" at the moment. I could collect some picies here
and there (e.g. how to develop @WebService annotated Web Services, how to
use actors with MDBs, how to use actors with HTTP servlets by means of
Lift), but there is no clear picture explained anywhere.

My current understanding based on what I've found is:

- It is always possible to use Scala as Java++, i.e. as mainly a syntactic
improvement with better modularity, type ineference and functional features.
Any constructs, concepts like actors, which require some sort of run-time
support are useful only in those cases, where explicit control over
multithreading is allowed (e.g. HTTP servlets). If this control is not
possible (e.g. EJB containers), actors cannot be used so easily without
further tweaks.

- Overall, there are Scala frameworks provide alternative (compared to
JavaEE) ways for persistance, distribution, discovery of components and
methods for their deployment, e.g. Akka. While these methods are eventually
even better than JavaEE, they are not compliant with it in many cases. In
this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
from a JavaEE perspective. I don't know if this is an intended positioning
of such components or is it due to the fact that Scala integration with
JavaEE was not too much in focus till now.

- At the moment, most Scala apps (except for HTTP based Lift apps?) are
mostly standalone components or need a Scala specific container for their
deployment (Akka core?) as opposed to deployment on JavaEE application
servers.

- It is not clear if Scala components can easily access existing remote and
local EJBs. It is not clear, if existing EJBs can easily access Scala
components (e.g. actors) out of the box.

With all that in mind, I'd say that the state of Scala<->JavaEE integration
is RATHER POOR and not clearly explained anywhere with all its pros and
cons. There seems to be also a rather low interest in making this kind of
integration easier. This is probably because currently major adopters of
Scala are using it rather for a "from scratch" apps development, e.g. for
Web apps development (e.g. Lift-based) or for something like financial
services (e.g. Akka). In such cases, integration with legacy JavaEE systems
is probably not so important.

Please correct me, if this analysis is very wrong. I'd be happy to see that
integration with/into JavaEE apps is much better and much easier.

Thanks,
-Leo

Razvan Cojocaru-2 wrote:
>
> Indeed - I assumed you'd run a J2EE container because you're using
> "heavy" J2EE services like XA transactions, authentication contexts,
> fail-over etc.
>
>
> On 10/1/10, Viktor Klang wrote:
>> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru wrote:
>>
>>> I think my original reply stands.
>>>
>>> You should not use actors or threads in a container, as it will easily
>>> mangle contexts, transactions and whatnot.
>>>
>>
>> I disagree.
>> It totally depends what you're running in that container.
>>
>>
>>>
>>> Having said that, if you have sufficient experience and can carefully
>>> tippy
>>> toe your way around that problem, you could...well, use them.
>>>
>>> As for the distributed part, I think both akka and Stambecco have
>>> support
>>> for remote actors, probably not via JNDI though.
>>>
>>> Cheers,
>>> Razie
>>>
>>> -----Original Message-----
>>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>>> Sent: Friday, October 01, 2010 1:12 PM
>>> To: scala-user@listes.epfl.ch
>>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments
>>>
>>>
>>> Any feedback on:
>>>
>>> - usage of Actors inside EJB containers and possible multithreading
>>> issues?
>>> - ability to publish (via JNDI?), discover and invoke Actors deployed as
>>> part of different JavaEE apps running on the same or different nodes?
>>>
>>> Thanks,
>>> -Leo
>>>
>>>
>>> Leo Romanoff wrote:
>>> >
>>> > After all those continuations discussions, let's come back to the
>>> > original questions about Scala use in JavaEE environments.
>>> >
>>> >
>>> > dpp wrote:
>>> >>
>>> >>
>>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>>> >>>> should not permit explicit thread-management (e.g. thread creation)
>>> >>>> by user-code as it may lead to conflicts.
>>> >>
>>> >>
>>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>>> >>> worker thread that does background processing), the very same
>>> >>> mechanism would be used in Actors.
>>> >>
>>> >
>>> > Well, there is no direct access to threads or java.concurrent inside
>>> > EJB components. There is no such thing like Thread at all. EJBs are
>>> "actors"
>>> > in the JavaEE world. And the EJB specification forbids direct
>>> > manipulations with threads by means of Threads API ;-( This is a
>>> > serious limitation. Some vendors provide certain kinds of workarounds.
>>> > For example, BEA and IBM provide a special WorkManager API.
>>> > Glassfish has something similar. You can get a WorkManager, which is
>>> > something like an Executor factory. But they are not standardized and
>>> > not compatible with each other. These APIs are modeled after
>>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>>> > supporting these major app servers is very important. May be something
>>> > can be done about it? May be Scala actor's support and library can be
>>> > extended to become configurable so that ThreadPools factories and the
>>> > like can be provided by an application server as well?
>>> >
>>> > Another thing, I'd like to understand is about publishing and
>>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>>> > directory, so that other JavaEE applications running on the same or
>>> > other application servers can find them. I think, it would be nice to
>>> > have the same for Scala actors, so that other JavaEE apps can discover
>>> > them and send messages to them. So, here is a question:
>>> > - Is it possible to publish an Actor in the JNDI?
>>> > - Is it possible to invoke an Actor that runs inside a different
>>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>>> > loaded in a different classloader, as it is always the case in the
>>> > JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>>> >
>>> > These questions covered just the most obvious things. I guess there
>>> > are more issues of this kind. Please add them, if I haven't mentioned
>>> > them yet.
>>> >
>>> > I think, many Java professionals would really like to be able to use
>>> > Scala for JavaEE development instead of plain old Java. Scala is much
>>> > nicer and more powerful as a language. To be able to do it, it is
>>> > important to play the game according to JavaEE rules, to be compliant
>>> > with it and to have a seamless integration of Scala components into
>>> > JavaEE world as first-class citizens. If this kind of support is not
>>> > in place yet, what are the plans of Scala developers or promoters? Is
>>> > anyone working on it? Is it on the roadmap?
>>> >
>>> > * appeal *
>>> > I know that implementing this kind of features may seem not very sexy
>>> > for Scala language developers. It is not so interesting as type
>>> > systems, functional features, etc. But having a clear picture and
>>> > even better a good support for JavaEE may be crucial for adoption of
>>> > Scala in the enterprise world. And this adoption by the enterprise
>>> > development segment may become the most important factor for Scala for
>>> > moving from a "research/academic language league" to the mainstream
>>> language.
>>> > Enterprises and telcos are also the places where the real money are.
>>> > So, having them on board is very useful from many different
>>> > perspectives.
>>> > * End of appeal *
>>> >
>>> > Thanks,
>>> > Leo
>>> >
>>> >
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>>
>>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>>> vaEE-environments-tp2553702p2936337.html
>>> Sent from the Scala - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>> --
>> Viktor Klang,
>> Code Connoisseur
>> Work: www.akkasource.com
>> Code: github.com/viktorklang
>> Follow: twitter.com/viktorklang
>> Read: klangism.tumblr.com
>>
>

No comments? No reflections?

Is it because the analysis is correct and there is nothing to add?
Or because this thread got too long? ;-)

-Leo

Leo Romanoff wrote:
>
> Razvan and Viktor - thanks for your comments!
>
> But what I get to hear still sounds very fuzzy for me. See below.
>
>>> Having said that, if you have sufficient experience and can carefully
>>> tippy
>>> toe your way around that problem, you could...well, use them.
>
> This sounds like a "try to do it yourself - and may be you are lucky"
> exercise
>
>>> As for the distributed part, I think both akka and Stambecco have
>>> support
>>> for remote actors, probably not via JNDI though.
>
> OK. This means that this mechanism is not quite JavaEE compliant.
>
>
> What I'm trying to figure out in this whole thread is some sort of best
> practices or direction for an (average) JavaEE developer for starting
> using Scala and its libraries and frameworks in her daily work. Often, in
> big JavaEE shops (e.g. enterprises and telcos), you have to use a huge
> existing JavaEE-based codebase. Rewriting everything the Scala-way or to
> become Web-based is often out of question. Therefore, the natural basic
> requirements for using Scala in this context are:
>
> - that Scala code should be able to at least interwork (bind, invoke,
> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>
> - In a perfect world, it should be possible to implement JavaEE components
> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
> standards for packaging and deployment. Is it possible now? If not, is it
> envisioned?
>
> - It would be useful to expose Scala specific components (e.g. actors) in
> a JavaEE friendly way, so that they can be consumed by other JavaEE
> components. What about application packaging in this case?
>
> - What is a guideline for Scala components development? Does it make sense
> to strive for integration with JavaEE? Or is it better to use
> Scala-specific frameworks? Should Scala be used as Java++ for JavaEE
> components development (i.e. only new and better language constructs, but
> e.g. no actors, as they depend too much on multithreading implementation
> and ability to explicitly control it). Or does it make sense to use actors
> and the like to implement JavaEE components?
>
> I think having clear statements (somewhere on major scala-related sites?)
> answering all these questions would be a great help for many JavaEE
> developers planning to switch to Scala.
>
> At the moment this kind of information is lacking. I've spent quite some
> time to find out any information about what is possible and what's not. It
> seems to be a "black art" at the moment. I could collect some picies here
> and there (e.g. how to develop @WebService annotated Web Services, how to
> use actors with MDBs, how to use actors with HTTP servlets by means of
> Lift), but there is no clear picture explained anywhere.
>
> My current understanding based on what I've found is:
>
> - It is always possible to use Scala as Java++, i.e. as mainly a
> syntactic improvement with better modularity, type ineference and
> functional features. Any constructs, concepts like actors, which require
> some sort of run-time support are useful only in those cases, where
> explicit control over multithreading is allowed (e.g. HTTP servlets). If
> this control is not possible (e.g. EJB containers), actors cannot be used
> so easily without further tweaks.
>
> - Overall, there are Scala frameworks provide alternative (compared to
> JavaEE) ways for persistance, distribution, discovery of components and
> methods for their deployment, e.g. Akka. While these methods are
> eventually even better than JavaEE, they are not compliant with it in many
> cases. In this sense, _Scala components exists in THEIR OWN (may be
> BETTER?) WORLD_, from a JavaEE perspective. I don't know if this is an
> intended positioning of such components or is it due to the fact that
> Scala integration with JavaEE was not too much in focus till now.
>
> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
> mostly standalone components or need a Scala specific container for their
> deployment (Akka core?) as opposed to deployment on JavaEE application
> servers.
>
> - It is not clear if Scala components can easily access existing remote
> and local EJBs. It is not clear, if existing EJBs can easily access Scala
> components (e.g. actors) out of the box.
>
> With all that in mind, I'd say that the state of Scala<->JavaEE
> integration is RATHER POOR and not clearly explained anywhere with all its
> pros and cons. There seems to be also a rather low interest in making this
> kind of integration easier. This is probably because currently major
> adopters of Scala are using it rather for a "from scratch" apps
> development, e.g. for Web apps development (e.g. Lift-based) or for
> something like financial services (e.g. Akka). In such cases, integration
> with legacy JavaEE systems is probably not so important.
>
> Please correct me, if this analysis is very wrong. I'd be happy to see
> that integration with/into JavaEE apps is much better and much easier.
>
> Thanks,
> -Leo
>
>
> Razvan Cojocaru-2 wrote:
>>
>> Indeed - I assumed you'd run a J2EE container because you're using
>> "heavy" J2EE services like XA transactions, authentication contexts,
>> fail-over etc.
>>
>>
>> On 10/1/10, Viktor Klang wrote:
>>> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru wrote:
>>>
>>>> I think my original reply stands.
>>>>
>>>> You should not use actors or threads in a container, as it will easily
>>>> mangle contexts, transactions and whatnot.
>>>>
>>>
>>> I disagree.
>>> It totally depends what you're running in that container.
>>>
>>>
>>>>
>>>> Having said that, if you have sufficient experience and can carefully
>>>> tippy
>>>> toe your way around that problem, you could...well, use them.
>>>>
>>>> As for the distributed part, I think both akka and Stambecco have
>>>> support
>>>> for remote actors, probably not via JNDI though.
>>>>
>>>> Cheers,
>>>> Razie
>>>>
>>>> -----Original Message-----
>>>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>>>> Sent: Friday, October 01, 2010 1:12 PM
>>>> To: scala-user@listes.epfl.ch
>>>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE
>>>> environments
>>>>
>>>>
>>>> Any feedback on:
>>>>
>>>> - usage of Actors inside EJB containers and possible multithreading
>>>> issues?
>>>> - ability to publish (via JNDI?), discover and invoke Actors deployed
>>>> as
>>>> part of different JavaEE apps running on the same or different nodes?
>>>>
>>>> Thanks,
>>>> -Leo
>>>>
>>>>
>>>> Leo Romanoff wrote:
>>>> >
>>>> > After all those continuations discussions, let's come back to the
>>>> > original questions about Scala use in JavaEE environments.
>>>> >
>>>> >
>>>> > dpp wrote:
>>>> >>
>>>> >>
>>>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>>>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB
>>>> containers
>>>> >>>> should not permit explicit thread-management (e.g. thread
>>>> creation)
>>>> >>>> by user-code as it may lead to conflicts.
>>>> >>
>>>> >>
>>>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>>>> >>> worker thread that does background processing), the very same
>>>> >>> mechanism would be used in Actors.
>>>> >>
>>>> >
>>>> > Well, there is no direct access to threads or java.concurrent inside
>>>> > EJB components. There is no such thing like Thread at all. EJBs are
>>>> "actors"
>>>> > in the JavaEE world. And the EJB specification forbids direct
>>>> > manipulations with threads by means of Threads API ;-( This is a
>>>> > serious limitation. Some vendors provide certain kinds of
>>>> workarounds.
>>>> > For example, BEA and IBM provide a special WorkManager API.
>>>> > Glassfish has something similar. You can get a WorkManager, which is
>>>> > something like an Executor factory. But they are not standardized and
>>>> > not compatible with each other. These APIs are modeled after
>>>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>>>> > supporting these major app servers is very important. May be
>>>> something
>>>> > can be done about it? May be Scala actor's support and library can be
>>>> > extended to become configurable so that ThreadPools factories and the
>>>> > like can be provided by an application server as well?
>>>> >
>>>> > Another thing, I'd like to understand is about publishing and
>>>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>>>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>>>> > directory, so that other JavaEE applications running on the same or
>>>> > other application servers can find them. I think, it would be nice to
>>>> > have the same for Scala actors, so that other JavaEE apps can
>>>> discover
>>>> > them and send messages to them. So, here is a question:
>>>> > - Is it possible to publish an Actor in the JNDI?
>>>> > - Is it possible to invoke an Actor that runs inside a different
>>>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>>>> > loaded in a different classloader, as it is always the case in the
>>>> > JavaEE world. Each app runs in the same JVM, but in its own
>>>> "sandbox".
>>>> >
>>>> > These questions covered just the most obvious things. I guess there
>>>> > are more issues of this kind. Please add them, if I haven't mentioned
>>>> > them yet.
>>>> >
>>>> > I think, many Java professionals would really like to be able to use
>>>> > Scala for JavaEE development instead of plain old Java. Scala is much
>>>> > nicer and more powerful as a language. To be able to do it, it is
>>>> > important to play the game according to JavaEE rules, to be compliant
>>>> > with it and to have a seamless integration of Scala components into
>>>> > JavaEE world as first-class citizens. If this kind of support is not
>>>> > in place yet, what are the plans of Scala developers or promoters? Is
>>>> > anyone working on it? Is it on the roadmap?
>>>> >
>>>> > * appeal *
>>>> > I know that implementing this kind of features may seem not very sexy
>>>> > for Scala language developers. It is not so interesting as type
>>>> > systems, functional features, etc. But having a clear picture and
>>>> > even better a good support for JavaEE may be crucial for adoption of
>>>> > Scala in the enterprise world. And this adoption by the enterprise
>>>> > development segment may become the most important factor for Scala
>>>> for
>>>> > moving from a "research/academic language league" to the mainstream
>>>> language.
>>>> > Enterprises and telcos are also the places where the real money are.
>>>> > So, having them on board is very useful from many different
>>>> > perspectives.
>>>> > * End of appeal *
>>>> >
>>>> > Thanks,
>>>> > Leo
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>>>> vaEE-environments-tp2553702p2936337.html
>>>> Sent from the Scala - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>> --
>>> Viktor Klang,
>>> Code Connoisseur
>>> Work: www.akkasource.com
>>> Code: github.com/viktorklang
>>> Follow: twitter.com/viktorklang
>>> Read: klangism.tumblr.com
>>>
>>

Sorry Leo, I've been meaning to have time to answer that monster reply all week, but I'm 2 days away from getting Akka 1.0-M1 out the door, so if you want a more speedy answer you'll have to distill your questions a bit.

Cheers,

On Wed, Oct 6, 2010 at 9:23 AM, Leo Romanoff <romixlev@yahoo.com> wrote:

No comments? No reflections?

Is it because the analysis is correct and there is nothing to add?
Or because this thread got too long? ;-)

-Leo


Leo Romanoff wrote:
>
> Razvan and Viktor - thanks for your comments!
>
> But what I get to hear still sounds very fuzzy for me. See below.
>
>>> Having said that, if you have sufficient experience and can carefully
>>> tippy
>>> toe your way around that problem, you could...well, use them.
>
> This sounds like a "try to do it yourself - and may be you are lucky"
> exercise
>
>>> As for the distributed part, I think both akka and Stambecco have
>>> support
>>> for remote actors, probably not via JNDI though.
>
> OK. This means that this mechanism is not quite JavaEE compliant.
>
>
> What I'm trying to figure out in this whole thread is some sort of best
> practices or direction for an (average) JavaEE developer for starting
> using Scala and its libraries and frameworks in her daily work. Often, in
> big JavaEE shops (e.g. enterprises and telcos), you have to use a huge
> existing JavaEE-based codebase. Rewriting everything the Scala-way or to
> become Web-based is often out of question. Therefore, the natural basic
> requirements for using Scala in this context are:
>
> - that Scala code should be able to at least interwork (bind, invoke,
> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>
> - In a perfect world, it should be possible to implement JavaEE components
> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
> standards for packaging and deployment. Is it possible now? If not, is it
> envisioned?
>
> - It would be useful to expose Scala specific components (e.g. actors) in
> a JavaEE friendly way, so that they can be consumed by other JavaEE
> components. What about application packaging in this case?
>
> - What is a guideline for Scala components development? Does it make sense
> to strive for integration with JavaEE? Or is it better to use
> Scala-specific frameworks? Should Scala be used as Java++ for JavaEE
> components development (i.e. only new and better language constructs, but
> e.g. no actors, as they depend too much on multithreading implementation
> and ability to explicitly control it). Or does it make sense to use actors
> and the like to implement JavaEE components?
>
> I think having clear statements (somewhere on major scala-related sites?)
> answering all these questions would be a great help for many JavaEE
> developers planning to switch to Scala.
>
> At the moment this kind of information is lacking. I've spent quite some
> time to find out any information about what is possible and what's not. It
> seems to be a "black art" at the moment. I could collect some picies here
> and there (e.g. how to develop @WebService annotated Web Services, how to
> use actors with MDBs, how to use actors with HTTP servlets by means of
> Lift), but there is no clear picture explained anywhere.
>
> My current understanding based on what I've found is:
>
>  - It is always possible to use Scala as Java++, i.e. as mainly a
> syntactic improvement with better modularity, type ineference and
> functional features. Any constructs, concepts like actors, which require
> some sort of run-time support are useful only in those cases, where
> explicit control over multithreading is allowed (e.g. HTTP servlets). If
> this control is not possible (e.g. EJB containers), actors cannot be used
> so easily without further tweaks.
>
> - Overall, there are Scala frameworks provide alternative (compared to
> JavaEE) ways for persistance, distribution, discovery of components and
> methods for their deployment, e.g. Akka. While these methods are
> eventually even better than JavaEE, they are not compliant with it in many
> cases. In this sense, _Scala components exists in THEIR OWN (may be
> BETTER?) WORLD_, from a JavaEE perspective. I don't know if this is an
> intended positioning of such components or is it due to the fact that
> Scala integration with JavaEE was not too much in focus till now.
>
> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
> mostly standalone components or need a Scala specific container for their
> deployment (Akka core?) as opposed to deployment on JavaEE application
> servers.
>
> - It is not clear if Scala components can easily access existing remote
> and local EJBs. It is not clear, if existing EJBs can easily access Scala
> components (e.g. actors) out of the box.
>
> With all that in mind, I'd say that the state of Scala<->JavaEE
> integration is RATHER POOR and not clearly explained anywhere with all its
> pros and cons. There seems to be also a rather low interest in making this
> kind of integration easier. This is probably because currently major
> adopters of Scala are using it rather for a "from scratch" apps
> development, e.g. for Web apps development (e.g. Lift-based) or for
> something like financial services (e.g. Akka). In such cases, integration
> with legacy JavaEE systems is probably not so important.
>
> Please correct me, if this analysis is very wrong. I'd be happy to see
> that integration with/into JavaEE apps is much better and much easier.
>
> Thanks,
>   -Leo
>
>
> Razvan Cojocaru-2 wrote:
>>
>> Indeed - I assumed you'd run a J2EE container because you're using
>> "heavy" J2EE services like XA transactions, authentication contexts,
>> fail-over etc.
>>
>>
>> On 10/1/10, Viktor Klang <viktor.klang@gmail.com> wrote:
>>> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru <pub@razie.com> wrote:
>>>
>>>> I think my original reply stands.
>>>>
>>>> You should not use actors or threads in a container, as it will easily
>>>> mangle contexts, transactions and whatnot.
>>>>
>>>
>>> I disagree.
>>> It totally depends what you're running in that container.
>>>
>>>
>>>>
>>>> Having said that, if you have sufficient experience and can carefully
>>>> tippy
>>>> toe your way around that problem, you could...well, use them.
>>>>
>>>> As for the distributed part, I think both akka and Stambecco have
>>>> support
>>>> for remote actors, probably not via JNDI though.
>>>>
>>>> Cheers,
>>>> Razie
>>>>
>>>> -----Original Message-----
>>>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>>>> Sent: Friday, October 01, 2010 1:12 PM
>>>> To: scala-user@listes.epfl.ch
>>>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE
>>>> environments
>>>>
>>>>
>>>> Any feedback on:
>>>>
>>>> - usage of Actors inside EJB containers and possible multithreading
>>>> issues?
>>>> - ability to publish (via JNDI?), discover and invoke Actors deployed
>>>> as
>>>> part of different JavaEE apps running on the same or different nodes?
>>>>
>>>> Thanks,
>>>>  -Leo
>>>>
>>>>
>>>> Leo Romanoff wrote:
>>>> >
>>>> > After all those continuations discussions, let's come back to the
>>>> > original questions about Scala use in JavaEE environments.
>>>> >
>>>> >
>>>> > dpp wrote:
>>>> >>
>>>> >>
>>>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>>>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB
>>>> containers
>>>> >>>> should not permit explicit thread-management (e.g. thread
>>>> creation)
>>>> >>>> by user-code as it may lead to conflicts.
>>>> >>
>>>> >>
>>>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>>>> >>> worker thread that does background processing), the very same
>>>> >>> mechanism would be used in Actors.
>>>> >>
>>>> >
>>>> > Well, there is no direct access to threads or java.concurrent inside
>>>> > EJB components. There is no such thing like Thread at all. EJBs are
>>>> "actors"
>>>> > in the JavaEE world. And the EJB specification forbids direct
>>>> > manipulations with threads by means of Threads API ;-( This is a
>>>> > serious limitation. Some vendors provide certain kinds of
>>>> workarounds.
>>>> > For example, BEA and IBM provide a special WorkManager API.
>>>> > Glassfish has something similar. You can get a WorkManager, which is
>>>> > something like an Executor factory. But they are not standardized and
>>>> > not compatible with each other. These APIs are modeled after
>>>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>>>> > supporting these major app servers is very important. May be
>>>> something
>>>> > can be done about it? May be Scala actor's support and library can be
>>>> > extended to become configurable so that ThreadPools factories and the
>>>> > like can be provided by an application server as well?
>>>> >
>>>> > Another thing, I'd like to understand is about publishing and
>>>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>>>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>>>> > directory, so that other JavaEE applications running on the same or
>>>> > other application servers can find them. I think, it would be nice to
>>>> > have the same for Scala actors, so that other JavaEE apps can
>>>> discover
>>>> > them and send messages to them. So, here is a question:
>>>> >  - Is it possible to publish an Actor in the JNDI?
>>>> >  - Is it possible to invoke an Actor that runs inside a different
>>>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>>>> > loaded in a different classloader, as it is always the case in the
>>>> > JavaEE world. Each app runs in the same JVM, but in its own
>>>> "sandbox".
>>>> >
>>>> > These questions covered just the most obvious things. I guess there
>>>> > are more issues of this kind. Please add them, if I haven't mentioned
>>>> > them yet.
>>>> >
>>>> > I think, many Java professionals would really like to be able to use
>>>> > Scala for JavaEE development instead of plain old Java. Scala is much
>>>> > nicer and more powerful as a language. To be able to do it, it is
>>>> > important to play the game according to JavaEE rules, to be compliant
>>>> > with it and to have a seamless integration of Scala components into
>>>> > JavaEE world as first-class citizens. If this kind of support is not
>>>> > in place yet, what are the plans of Scala developers or promoters? Is
>>>> > anyone working on it? Is it on the roadmap?
>>>> >
>>>> > * appeal *
>>>> > I know that implementing this kind of features may seem not very sexy
>>>> > for Scala language developers. It is not so interesting as type
>>>> > systems, functional features, etc.  But having a clear picture and
>>>> > even better a good support for JavaEE may be crucial for adoption of
>>>> > Scala in the enterprise world. And this adoption by the enterprise
>>>> > development segment may become the most important factor for Scala
>>>> for
>>>> > moving from a "research/academic language league" to the mainstream
>>>> language.
>>>> > Enterprises and telcos are also the places where the real money are.
>>>> > So, having them on board is very useful from many different
>>>> > perspectives.
>>>> > * End of appeal *
>>>> >
>>>> > Thanks,
>>>> >   Leo
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-Ja
>>>> vaEE-environments-tp2553702p2936337.html<http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-Ja%0AvaEE-environments-tp2553702p2936337.html>
>>>> Sent from the Scala - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>> --
>>> Viktor Klang,
>>> Code Connoisseur
>>> Work:   www.akkasource.com
>>> Code:   github.com/viktorklang
>>> Follow: twitter.com/viktorklang
>>> Read:   klangism.tumblr.com
>>>
>>
>> --
>> Sent from my mobile device
>>
>>
>
>

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2964361.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

No problem, Victor. Important things first!

I can wait until you are ready with the release and have a bit of a spare
time to respond.

Cheers,
Leo

Viktor Klang wrote:
>
> Sorry Leo, I've been meaning to have time to answer that monster reply all
> week, but I'm 2 days away from getting Akka 1.0-M1 out the door, so if you
> want a more speedy answer you'll have to distill your questions a bit.
>
> Cheers,
>
> On Wed, Oct 6, 2010 at 9:23 AM, Leo Romanoff wrote:
>
>>
>> No comments? No reflections?
>>
>> Is it because the analysis is correct and there is nothing to add?
>> Or because this thread got too long? ;-)
>>
>> -Leo
>>
>>
>

Well, since Viktor is busy, let me take a stab at this again.

There is a little bit of confusion here. Scala is a language. Actors
are a library, much like java.util.concurrent, for concurrent
programming.
Scala is as useful for j2ee as java, since it compiles to the same
bytecode and can access the same libraries.

So, java++ like you said it.

However, keep in mind that j2ee is just one framework. So is .net and
Corba. Many applications do not require the j2ee services and can
simply be a lift+akka application, this being potentially faster as
well. If you have real-time constraints, you may simply have to do it
this way.

I would see lift as a servlet/spring replacement and akka as a
container replacement if you must look at them this way. Osgi as a
packaging/management replacement.

(State-less/ful) Beans are evil and should be replaced by servlets in
a world at rest anyways, right? :)

What modern j2ee containers have on these new frameworks is management
tools, enterprise integration etc - which will no doubt at some point
appear here as well.

Cheers,
Razvan

On 10/2/10, Leo Romanoff wrote:
>
> Razvan and Viktor - thanks for your comments!
>
> But what I get to hear still sounds very fuzzy for me. See below.
>
>>> Having said that, if you have sufficient experience and can carefully
>>> tippy
>>> toe your way around that problem, you could...well, use them.
>
> This sounds like a "try to do it yourself - and may be you are lucky"
> exercise
>
>>> As for the distributed part, I think both akka and Stambecco have support
>>> for remote actors, probably not via JNDI though.
>
> OK. This means that this mechanism is not quite JavaEE compliant.
>
>
> What I'm trying to figure out in this whole thread is some sort of best
> practices or direction for an (average) JavaEE developer for starting using
> Scala and its libraries and frameworks in her daily work. Often, in big
> JavaEE shops (e.g. enterprises and telcos), you have to use a huge existing
> JavaEE-based codebase. Rewriting everything the Scala-way or to become
> Web-based is often out of question. Therefore, the natural basic
> requirements for using Scala in this context are:
>
> - that Scala code should be able to at least interwork (bind, invoke,
> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>
> - In a perfect world, it should be possible to implement JavaEE components
> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
> standards for packaging and deployment. Is it possible now? If not, is it
> envisioned?
>
> - It would be useful to expose Scala specific components (e.g. actors) in a
> JavaEE friendly way, so that they can be consumed by other JavaEE
> components. What about application packaging in this case?
>
> - What is a guideline for Scala components development? Does it make sense
> to strive for integration with JavaEE? Or is it better to use Scala-specific
> frameworks? Should Scala be used as Java++ for JavaEE components development
> (i.e. only new and better language constructs, but e.g. no actors, as they
> depend too much on multithreading implementation and ability to explicitly
> control it). Or does it make sense to use actors and the like to implement
> JavaEE components?
>
> I think having clear statements (somewhere on major scala-related sites?)
> answering all these questions would be a great help for many JavaEE
> developers planning to switch to Scala.
>
> At the moment this kind of information is lacking. I've spent quite some
> time to find out any information about what is possible and what's not. It
> seems to be a "black art" at the moment. I could collect some picies here
> and there (e.g. how to develop @WebService annotated Web Services, how to
> use actors with MDBs, how to use actors with HTTP servlets by means of
> Lift), but there is no clear picture explained anywhere.
>
> My current understanding based on what I've found is:
>
> - It is always possible to use Scala as Java++, i.e. as mainly a syntactic
> improvement with better modularity, type ineference and functional features.
> Any constructs, concepts like actors, which require some sort of run-time
> support are useful only in those cases, where explicit control over
> multithreading is allowed (e.g. HTTP servlets). If this control is not
> possible (e.g. EJB containers), actors cannot be used so easily without
> further tweaks.
>
> - Overall, there are Scala frameworks provide alternative (compared to
> JavaEE) ways for persistance, distribution, discovery of components and
> methods for their deployment, e.g. Akka. While these methods are eventually
> even better than JavaEE, they are not compliant with it in many cases. In
> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
> from a JavaEE perspective. I don't know if this is an intended positioning
> of such components or is it due to the fact that Scala integration with
> JavaEE was not too much in focus till now.
>
> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
> mostly standalone components or need a Scala specific container for their
> deployment (Akka core?) as opposed to deployment on JavaEE application
> servers.
>
> - It is not clear if Scala components can easily access existing remote and
> local EJBs. It is not clear, if existing EJBs can easily access Scala
> components (e.g. actors) out of the box.
>
> With all that in mind, I'd say that the state of Scala<->JavaEE integration
> is RATHER POOR and not clearly explained anywhere with all its pros and
> cons. There seems to be also a rather low interest in making this kind of
> integration easier. This is probably because currently major adopters of
> Scala are using it rather for a "from scratch" apps development, e.g. for
> Web apps development (e.g. Lift-based) or for something like financial
> services (e.g. Akka). In such cases, integration with legacy JavaEE systems
> is probably not so important.
>
> Please correct me, if this analysis is very wrong. I'd be happy to see that
> integration with/into JavaEE apps is much better and much easier.
>
> Thanks,
> -Leo
>
>
> Razvan Cojocaru-2 wrote:
>>
>> Indeed - I assumed you'd run a J2EE container because you're using
>> "heavy" J2EE services like XA transactions, authentication contexts,
>> fail-over etc.
>>
>>
>> On 10/1/10, Viktor Klang wrote:
>>> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru wrote:
>>>
>>>> I think my original reply stands.
>>>>
>>>> You should not use actors or threads in a container, as it will easily
>>>> mangle contexts, transactions and whatnot.
>>>>
>>>
>>> I disagree.
>>> It totally depends what you're running in that container.
>>>
>>>
>>>>
>>>> Having said that, if you have sufficient experience and can carefully
>>>> tippy
>>>> toe your way around that problem, you could...well, use them.
>>>>
>>>> As for the distributed part, I think both akka and Stambecco have
>>>> support
>>>> for remote actors, probably not via JNDI though.
>>>>
>>>> Cheers,
>>>> Razie
>>>>
>>>> -----Original Message-----
>>>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>>>> Sent: Friday, October 01, 2010 1:12 PM
>>>> To: scala-user@listes.epfl.ch
>>>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments
>>>>
>>>>
>>>> Any feedback on:
>>>>
>>>> - usage of Actors inside EJB containers and possible multithreading
>>>> issues?
>>>> - ability to publish (via JNDI?), discover and invoke Actors deployed as
>>>> part of different JavaEE apps running on the same or different nodes?
>>>>
>>>> Thanks,
>>>> -Leo
>>>>
>>>>
>>>> Leo Romanoff wrote:
>>>> >
>>>> > After all those continuations discussions, let's come back to the
>>>> > original questions about Scala use in JavaEE environments.
>>>> >
>>>> >
>>>> > dpp wrote:
>>>> >>
>>>> >>
>>>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>>>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>>>> >>>> should not permit explicit thread-management (e.g. thread creation)
>>>> >>>> by user-code as it may lead to conflicts.
>>>> >>
>>>> >>
>>>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>>>> >>> worker thread that does background processing), the very same
>>>> >>> mechanism would be used in Actors.
>>>> >>
>>>> >
>>>> > Well, there is no direct access to threads or java.concurrent inside
>>>> > EJB components. There is no such thing like Thread at all. EJBs are
>>>> "actors"
>>>> > in the JavaEE world. And the EJB specification forbids direct
>>>> > manipulations with threads by means of Threads API ;-( This is a
>>>> > serious limitation. Some vendors provide certain kinds of workarounds.
>>>> > For example, BEA and IBM provide a special WorkManager API.
>>>> > Glassfish has something similar. You can get a WorkManager, which is
>>>> > something like an Executor factory. But they are not standardized and
>>>> > not compatible with each other. These APIs are modeled after
>>>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>>>> > supporting these major app servers is very important. May be something
>>>> > can be done about it? May be Scala actor's support and library can be
>>>> > extended to become configurable so that ThreadPools factories and the
>>>> > like can be provided by an application server as well?
>>>> >
>>>> > Another thing, I'd like to understand is about publishing and
>>>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>>>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>>>> > directory, so that other JavaEE applications running on the same or
>>>> > other application servers can find them. I think, it would be nice to
>>>> > have the same for Scala actors, so that other JavaEE apps can discover
>>>> > them and send messages to them. So, here is a question:
>>>> > - Is it possible to publish an Actor in the JNDI?
>>>> > - Is it possible to invoke an Actor that runs inside a different
>>>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>>>> > loaded in a different classloader, as it is always the case in the
>>>> > JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>>>> >
>>>> > These questions covered just the most obvious things. I guess there
>>>> > are more issues of this kind. Please add them, if I haven't mentioned
>>>> > them yet.
>>>> >
>>>> > I think, many Java professionals would really like to be able to use
>>>> > Scala for JavaEE development instead of plain old Java. Scala is much
>>>> > nicer and more powerful as a language. To be able to do it, it is
>>>> > important to play the game according to JavaEE rules, to be compliant
>>>> > with it and to have a seamless integration of Scala components into
>>>> > JavaEE world as first-class citizens. If this kind of support is not
>>>> > in place yet, what are the plans of Scala developers or promoters? Is
>>>> > anyone working on it? Is it on the roadmap?
>>>> >
>>>> > * appeal *
>>>> > I know that implementing this kind of features may seem not very sexy
>>>> > for Scala language developers. It is not so interesting as type
>>>> > systems, functional features, etc. But having a clear picture and
>>>> > even better a good support for JavaEE may be crucial for adoption of
>>>> > Scala in the enterprise world. And this adoption by the enterprise
>>>> > development segment may become the most important factor for Scala for
>>>> > moving from a "research/academic language league" to the mainstream
>>>> language.
>>>> > Enterprises and telcos are also the places where the real money are.
>>>> > So, having them on board is very useful from many different
>>>> > perspectives.
>>>> > * End of appeal *
>>>> >
>>>> > Thanks,
>>>> > Leo
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>>>> vaEE-environments-tp2553702p2936337.html
>>>> Sent from the Scala - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>> --
>>> Viktor Klang,
>>> Code Connoisseur
>>> Work: www.akkasource.com
>>> Code: github.com/viktorklang
>>> Follow: twitter.com/viktorklang
>>> Read: klangism.tumblr.com
>>>
>>
>> --
>> Sent from my mobile device
>>
>>
>
> --
> View this message in context:
> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

Leo,

I just left a shop where we mixed scala and ejb.  I'll try to draft a good response for you shortly. The short answer is you have some great ideas, but there are some gotchas to be careful with.

Specifically:

Scala and session beans = great

Scala and jndi = good (i wish there were more async apis here...)

Scala and Rmi/serialization  = a few things to be careful of.

Scala and Entity Beans = ok

Actors and Mdb = dangerous

I'll follow up with details later today.

On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:


Razvan and Viktor - thanks for your comments!

But what I get to hear still sounds very fuzzy for me. See below.

>> Having said that, if you have sufficient experience and can carefully
>> tippy
>> toe your way a...

This sounds like a "try to do it yourself - and may be you are lucky"
exercise

>> As for the distributed part, I think both akka and Stambecco have support
>> for remote actors, ...

OK. This means that this mechanism is not quite JavaEE compliant.


What I'm trying to figure out in this whole thread is some sort of best
practices or direction for an (average) JavaEE developer for starting using
Scala and its libraries and frameworks in her daily work. Often, in big
JavaEE shops (e.g. enterprises and telcos), you have to use a huge existing
JavaEE-based codebase. Rewriting everything the Scala-way or to become
Web-based is often out of question. Therefore, the natural basic
requirements for using Scala in this context are:

- that Scala code should be able to at least interwork (bind, invoke,
consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).

- In a perfect world, it should be possible to implement JavaEE components
(servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
standards for packaging and deployment. Is it possible now? If not, is it
envisioned?

- It would be useful to expose Scala specific components (e.g. actors) in a
JavaEE friendly way, so that they can be consumed by other JavaEE
components. What about application packaging in this case?

- What is a guideline for Scala components development? Does it make sense
to strive for integration with JavaEE? Or is it better to use Scala-specific
frameworks? Should Scala be used as Java++ for JavaEE components development
(i.e. only new and better language constructs, but e.g. no actors, as they
depend too much on multithreading implementation and ability to explicitly
control it). Or does it make sense to use actors and the like to implement
JavaEE components?

I think having clear statements (somewhere on major scala-related sites?)
answering all these questions would be a great help for many JavaEE
developers planning to switch to Scala.

At the moment this kind of information is lacking. I've spent quite some
time to find out any information about what is possible and what's not. It
seems to be a "black art" at the moment. I could collect some picies here
and there (e.g. how to develop @WebService annotated Web Services, how to
use actors with MDBs, how to use actors with HTTP servlets by means of
Lift), but there is no clear picture explained anywhere.

My current understanding based on what I've found is:

 - It is always possible to use Scala as Java++, i.e. as mainly a syntactic
improvement with better modularity, type ineference and functional features.
Any constructs, concepts like actors, which require some sort of run-time
support are useful only in those cases, where explicit control over
multithreading is allowed (e.g. HTTP servlets). If this control is not
possible (e.g. EJB containers), actors cannot be used so easily without
further tweaks.

- Overall, there are Scala frameworks provide alternative (compared to
JavaEE) ways for persistance, distribution, discovery of components and
methods for their deployment, e.g. Akka. While these methods are eventually
even better than JavaEE, they are not compliant with it in many cases. In
this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
from a JavaEE perspective. I don't know if this is an intended positioning
of such components or is it due to the fact that Scala integration with
JavaEE was not too much in focus till now.

- At the moment, most Scala apps (except for HTTP based Lift apps?) are
mostly standalone components or need a Scala specific container for their
deployment (Akka core?) as opposed to deployment on JavaEE application
servers.

- It is not clear if Scala components can easily access existing remote and
local EJBs. It is not clear, if existing EJBs can easily access Scala
components (e.g. actors) out of the box.

With all that in mind, I'd say that the state of Scala<->JavaEE integration
is RATHER POOR and not clearly explained anywhere with all its pros and
cons. There seems to be also a rather low interest in making this kind of
integration easier. This is probably because currently major adopters of
Scala are using it rather for a "from scratch" apps development, e.g. for
Web apps development (e.g. Lift-based) or for something like financial
services (e.g. Akka). In such cases, integration with legacy JavaEE systems
is probably not so important.

Please correct me, if this analysis is very wrong. I'd be happy to see that
integration with/into JavaEE apps is much better and much easier.

Thanks,

-Leo


Razvan Cojocaru-2 wrote:
>

> Indeed - I assumed you'd run a J2EE container because you're using
> "heavy" J2EE services like XA...

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html

Sent from the Scala - User mailing list archive at Nabble.com.

To follow up:
Remember that inside the J2EE container you should not create your own threads.   If you are creating your own threads, you begin to look for container-specific extensions and hacks and things get ugly quick.
In our usage, we had a 100% Scala application that ran load-testing on our J2EE server.   This was a simple application made up of a bunch of Actors representing physical entities.   They were driven with a discrete event model and would communicate to the j2EE server via JNDI/RMI.   As stated before, Session Beans and Entity beans were fine within Scala. This actually worked great!
We also had a Scala layer inside EJB.   We used the "loaner" pattern to control JTA Transactions within EJB.   (i.e. transaction { foo } was a lot easier to write!).   
We had a set of MDBs for our processing pipeline.  Upon reflection, I think we would have benefited from dropping j2ee and just using Akka, however hindsight is 20/20.  I will state this:
MDBs have a high overhead in usage.   They interact with transactions and your EJB container in strange ways.   Actors have a low overhead.    MDBs are parallelized by default.   Actors are conceptually single-threaded.  This is *huge*.   An MDB is not an actor, and you can't treat it like one.   You can sort of make MDBs  be actors, but it can be painful.   One of the biggest issues we had in our application was threading issues relating to MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once again, given enough time I would have written something using Akka that was far better.

So basically, my recommendation for EJB is to use it as designed when required.   You can add some nice Scala features (JNDI lookups via loaner pattern, Transactions via loaner pattern if you need to take control of them, etc.).  However if you want to use things like Actors, I would highly recommend creating another component in the system, perhaps using Akka, that interacts via JNDI + RMI.   This would allow what you create to look like another EJB, inter-operate with the RMI dependency injection, and still use advanced Scala libraries.
The reality of the situation is the EJBs are very heavyweight abstraction with high overhead.   Although the parallelism is crazy for them, it assumes each session/mdb call is complete and large.   Actors have a different feel to them and are far lighter-weight concepts.   Akka provides you access to the light-weight as well as layers to build the complexity you need.   I learned, from Spring, that this approach is far more scalable in the long run.   I'd recommend working up an Akka->EJB interface for the rest of your team ;)

- Josh
.
On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:

Leo,

I just left a shop where we mixed scala and ejb.  I'll try to draft a good response for you shortly. The short answer is you have some great ideas, but there are some gotchas to be careful with.

Specifically:

Scala and session beans = great

Scala and jndi = good (i wish there were more async apis here...)

Scala and Rmi/serialization  = a few things to be careful of.

Scala and Entity Beans = ok

Actors and Mdb = dangerous

I'll follow up with details later today.

On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:


Razvan and Viktor - thanks for your comments!

But what I get to hear still sounds very fuzzy for me. See below.

>> Having said that, if you have sufficient experience and can carefully
>> tippy
>> toe your way a...

This sounds like a "try to do it yourself - and may be you are lucky"
exercise

>> As for the distributed part, I think both akka and Stambecco have support
>> for remote actors, ...

OK. This means that this mechanism is not quite JavaEE compliant.


What I'm trying to figure out in this whole thread is some sort of best
practices or direction for an (average) JavaEE developer for starting using
Scala and its libraries and frameworks in her daily work. Often, in big
JavaEE shops (e.g. enterprises and telcos), you have to use a huge existing
JavaEE-based codebase. Rewriting everything the Scala-way or to become
Web-based is often out of question. Therefore, the natural basic
requirements for using Scala in this context are:

- that Scala code should be able to at least interwork (bind, invoke,
consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).

- In a perfect world, it should be possible to implement JavaEE components
(servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
standards for packaging and deployment. Is it possible now? If not, is it
envisioned?

- It would be useful to expose Scala specific components (e.g. actors) in a
JavaEE friendly way, so that they can be consumed by other JavaEE
components. What about application packaging in this case?

- What is a guideline for Scala components development? Does it make sense
to strive for integration with JavaEE? Or is it better to use Scala-specific
frameworks? Should Scala be used as Java++ for JavaEE components development
(i.e. only new and better language constructs, but e.g. no actors, as they
depend too much on multithreading implementation and ability to explicitly
control it). Or does it make sense to use actors and the like to implement
JavaEE components?

I think having clear statements (somewhere on major scala-related sites?)
answering all these questions would be a great help for many JavaEE
developers planning to switch to Scala.

At the moment this kind of information is lacking. I've spent quite some
time to find out any information about what is possible and what's not. It
seems to be a "black art" at the moment. I could collect some picies here
and there (e.g. how to develop @WebService annotated Web Services, how to
use actors with MDBs, how to use actors with HTTP servlets by means of
Lift), but there is no clear picture explained anywhere.

My current understanding based on what I've found is:

 - It is always possible to use Scala as Java++, i.e. as mainly a syntactic
improvement with better modularity, type ineference and functional features.
Any constructs, concepts like actors, which require some sort of run-time
support are useful only in those cases, where explicit control over
multithreading is allowed (e.g. HTTP servlets). If this control is not
possible (e.g. EJB containers), actors cannot be used so easily without
further tweaks.

- Overall, there are Scala frameworks provide alternative (compared to
JavaEE) ways for persistance, distribution, discovery of components and
methods for their deployment, e.g. Akka. While these methods are eventually
even better than JavaEE, they are not compliant with it in many cases. In
this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
from a JavaEE perspective. I don't know if this is an intended positioning
of such components or is it due to the fact that Scala integration with
JavaEE was not too much in focus till now.

- At the moment, most Scala apps (except for HTTP based Lift apps?) are
mostly standalone components or need a Scala specific container for their
deployment (Akka core?) as opposed to deployment on JavaEE application
servers.

- It is not clear if Scala components can easily access existing remote and
local EJBs. It is not clear, if existing EJBs can easily access Scala
components (e.g. actors) out of the box.

With all that in mind, I'd say that the state of Scala<->JavaEE integration
is RATHER POOR and not clearly explained anywhere with all its pros and
cons. There seems to be also a rather low interest in making this kind of
integration easier. This is probably because currently major adopters of
Scala are using it rather for a "from scratch" apps development, e.g. for
Web apps development (e.g. Lift-based) or for something like financial
services (e.g. Akka). In such cases, integration with legacy JavaEE systems
is probably not so important.

Please correct me, if this analysis is very wrong. I'd be happy to see that
integration with/into JavaEE apps is much better and much easier.

Thanks,

-Leo


Razvan Cojocaru-2 wrote:
>

> Indeed - I assumed you'd run a J2EE container because you're using
> "heavy" J2EE services like XA...

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html

Sent from the Scala - User mailing list archive at Nabble.com.

Josh, what's the loaner pattern?

On 10/6/10, Josh Suereth wrote:
> To follow up:
>
> Remember that inside the J2EE container you should not create your own
> threads. If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.
>
> In our usage, we had a 100% Scala application that ran load-testing on our
> J2EE server. This was a simple application made up of a bunch of Actors
> representing physical entities. They were driven with a discrete event
> model and would communicate to the j2EE server via JNDI/RMI. As stated
> before, Session Beans and Entity beans were fine within Scala. This actually
> worked great!
>
> We also had a Scala layer inside EJB. We used the "loaner" pattern to
> control JTA Transactions within EJB. (i.e. transaction { foo } was a lot
> easier to write!).
>
> We had a set of MDBs for our processing pipeline. Upon reflection, I think
> we would have benefited from dropping j2ee and just using Akka, however
> hindsight is 20/20. I will state this:
>
> MDBs have a high overhead in usage. They interact with transactions and
> your EJB container in strange ways. Actors have a low overhead.
> MDBs are parallelized by default. Actors are conceptually single-threaded.
> This is *huge*. An MDB is not an actor, and you can't treat it like one.
> You can sort of make MDBs be actors, but it can be painful. One of the
> biggest issues we had in our application was threading issues relating to
> MDBs and the overhead of asynchronously modifying a big ugly RDBMS. Once
> again, given enough time I would have written something using Akka that was
> far better.
>
>
> So basically, my recommendation for EJB is to use it as designed when
> required. You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
> them, etc.). However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka, that
> interacts via JNDI + RMI. This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still use
> advanced Scala libraries.
>
> The reality of the situation is the EJBs are very heavyweight abstraction
> with high overhead. Although the parallelism is crazy for them, it assumes
> each session/mdb call is complete and large. Actors have a different feel
> to them and are far lighter-weight concepts. Akka provides you access to
> the light-weight as well as layers to build the complexity you need. I
> learned, from Spring, that this approach is far more scalable in the long
> run. I'd recommend working up an Akka->EJB interface for the rest of your
> team ;)
>
>
> - Josh
> .
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb. I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>> - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>

The loaner pattern is when I control a resource and you pass me a closure that makes use of it.  I will ensure the resource lifecycle.

For example, we had:

trait EjbHelper { self : OurEjbSessionBaseClass =>

  def transactional[A](f : => A) = {
     try {       // We would manipulate JTA here.   Picture a try catch block, where this opens it
        f
     } finally {
       // Again, with the JTA manipulation.   This would be the "commit" block.   For extra credit, you can change this to a while(!doneRetry) block and catch exceptions, revert the transaction and retry the f closure.      }
  }
}
Then in our Ejbs:

class MyEjbClass extends MyEjbRemote with MyEjbLocal with EjbHelper {
   def foo() : Unit = {
       transactional {           // Manipulate the database        }       sendJms(new Jms Notifying that things are actually commited)
   }
}

The JMS Processing, if it happened immediately, required the Database transaction to be completed in case it needed to re-modify the database.  Yes, horrible design, but the Scala certainly helped us make the code work until we had a chance to rework the EJB interactions.
In any case, the transactional method abided by the loaner pattern.   For another example, see my scala-arm library.  (P.S.  the ARM library was designed from my attempts to make JNDI usage easier.   I'm going to add a simplistic JNDI portion to the library shortly.   Essentially imagine reference EJBs as ManagedResource[EjbRemoteInterface].   You can then lookup and use the EJB lazily, even providing retry behavior as desired.   This is what I was alluding to with the performance benchmark testing actor-based code.
- Josh
On Wed, Oct 6, 2010 at 12:21 PM, Razvan (Pub) Cojocaru <pub@razie.com> wrote:

Josh, what's the loaner pattern?

On 10/6/10, Josh Suereth <joshua.suereth@gmail.com> wrote:
> To follow up:
>
> Remember that inside the J2EE container you should not create your own
> threads.   If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.
>
> In our usage, we had a 100% Scala application that ran load-testing on our
> J2EE server.   This was a simple application made up of a bunch of Actors
> representing physical entities.   They were driven with a discrete event
> model and would communicate to the j2EE server via JNDI/RMI.   As stated
> before, Session Beans and Entity beans were fine within Scala. This actually
> worked great!
>
> We also had a Scala layer inside EJB.   We used the "loaner" pattern to
> control JTA Transactions within EJB.   (i.e. transaction { foo } was a lot
> easier to write!).
>
> We had a set of MDBs for our processing pipeline.  Upon reflection, I think
> we would have benefited from dropping j2ee and just using Akka, however
> hindsight is 20/20.  I will state this:
>
> MDBs have a high overhead in usage.   They interact with transactions and
> your EJB container in strange ways.   Actors have a low overhead.
> MDBs are parallelized by default.   Actors are conceptually single-threaded.
>  This is *huge*.   An MDB is not an actor, and you can't treat it like one.
>   You can sort of make MDBs  be actors, but it can be painful.   One of the
> biggest issues we had in our application was threading issues relating to
> MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once
> again, given enough time I would have written something using Akka that was
> far better.
>
>
> So basically, my recommendation for EJB is to use it as designed when
> required.   You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
> them, etc.).  However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka, that
> interacts via JNDI + RMI.   This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still use
> advanced Scala libraries.
>
> The reality of the situation is the EJBs are very heavyweight abstraction
> with high overhead.   Although the parallelism is crazy for them, it assumes
> each session/mdb call is complete and large.   Actors have a different feel
> to them and are far lighter-weight concepts.   Akka provides you access to
> the light-weight as well as layers to build the complexity you need.   I
> learned, from Spring, that this approach is far more scalable in the long
> run.   I'd recommend working up an Akka->EJB interface for the rest of your
> team ;)
>
>
> - Josh
> .
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> <joshua.suereth@gmail.com>wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb.  I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization  = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>>  - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>

--
Sent from my mobile device

The loaner pattern is when I control a resource and you pass me a closure that makes use of it.  I will ensure the resource lifecycle.

For example, we had:

trait EjbHelper { self : OurEjbSessionBaseClass =>

  def transactional[A](f : => A) = {
     try {       // We would manipulate JTA here.   Picture a try catch block, where this opens it
        f
     } finally {
       // Again, with the JTA manipulation.   This would be the "commit" block.   For extra credit, you can change this to a while(!doneRetry) block and catch exceptions, revert the transaction and retry the f closure.      }
  }
}
Then in our Ejbs:

class MyEjbClass extends MyEjbRemote with MyEjbLocal with EjbHelper {
   def foo() : Unit = {
       transactional {           // Manipulate the database        }       sendJms(new Jms Notifying that things are actually commited)
   }
}

The JMS Processing, if it happened immediately, required the Database transaction to be completed in case it needed to re-modify the database.  Yes, horrible design, but the Scala certainly helped us make the code work until we had a chance to rework the EJB interactions.
In any case, the transactional method abided by the loaner pattern.   For another example, see my scala-arm library.  (P.S.  the ARM library was designed from my attempts to make JNDI usage easier.   I'm going to add a simplistic JNDI portion to the library shortly.   Essentially imagine reference EJBs as ManagedResource[EjbRemoteInterface].   You can then lookup and use the EJB lazily, even providing retry behavior as desired.   This is what I was alluding to with the performance benchmark testing actor-based code.
- Josh
On Wed, Oct 6, 2010 at 12:21 PM, Razvan (Pub) Cojocaru <pub@razie.com> wrote:

Josh, what's the loaner pattern?

On 10/6/10, Josh Suereth <joshua.suereth@gmail.com> wrote:
> To follow up:
>
> Remember that inside the J2EE container you should not create your own
> threads.   If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.
>
> In our usage, we had a 100% Scala application that ran load-testing on our
> J2EE server.   This was a simple application made up of a bunch of Actors
> representing physical entities.   They were driven with a discrete event
> model and would communicate to the j2EE server via JNDI/RMI.   As stated
> before, Session Beans and Entity beans were fine within Scala. This actually
> worked great!
>
> We also had a Scala layer inside EJB.   We used the "loaner" pattern to
> control JTA Transactions within EJB.   (i.e. transaction { foo } was a lot
> easier to write!).
>
> We had a set of MDBs for our processing pipeline.  Upon reflection, I think
> we would have benefited from dropping j2ee and just using Akka, however
> hindsight is 20/20.  I will state this:
>
> MDBs have a high overhead in usage.   They interact with transactions and
> your EJB container in strange ways.   Actors have a low overhead.
> MDBs are parallelized by default.   Actors are conceptually single-threaded.
>  This is *huge*.   An MDB is not an actor, and you can't treat it like one.
>   You can sort of make MDBs  be actors, but it can be painful.   One of the
> biggest issues we had in our application was threading issues relating to
> MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once
> again, given enough time I would have written something using Akka that was
> far better.
>
>
> So basically, my recommendation for EJB is to use it as designed when
> required.   You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
> them, etc.).  However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka, that
> interacts via JNDI + RMI.   This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still use
> advanced Scala libraries.
>
> The reality of the situation is the EJBs are very heavyweight abstraction
> with high overhead.   Although the parallelism is crazy for them, it assumes
> each session/mdb call is complete and large.   Actors have a different feel
> to them and are far lighter-weight concepts.   Akka provides you access to
> the light-weight as well as layers to build the complexity you need.   I
> learned, from Spring, that this approach is far more scalable in the long
> run.   I'd recommend working up an Akka->EJB interface for the rest of your
> team ;)
>
>
> - Josh
> .
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> <joshua.suereth@gmail.com>wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb.  I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization  = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>>  - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>

--
Sent from my mobile device

The loaner pattern is when I control a resource and you pass me a closure that makes use of it.  I will ensure the resource lifecycle.

For example, we had:

trait EjbHelper { self : OurEjbSessionBaseClass =>

  def transactional[A](f : => A) = {
     try {       // We would manipulate JTA here.   Picture a try catch block, where this opens it
        f
     } finally {
       // Again, with the JTA manipulation.   This would be the "commit" block.   For extra credit, you can change this to a while(!doneRetry) block and catch exceptions, revert the transaction and retry the f closure.      }
  }
}
Then in our Ejbs:

class MyEjbClass extends MyEjbRemote with MyEjbLocal with EjbHelper {
   def foo() : Unit = {
       transactional {           // Manipulate the database        }       sendJms(new Jms Notifying that things are actually commited)
   }
}

The JMS Processing, if it happened immediately, required the Database transaction to be completed in case it needed to re-modify the database.  Yes, horrible design, but the Scala certainly helped us make the code work until we had a chance to rework the EJB interactions.
In any case, the transactional method abided by the loaner pattern.   For another example, see my scala-arm library.  (P.S.  the ARM library was designed from my attempts to make JNDI usage easier.   I'm going to add a simplistic JNDI portion to the library shortly.   Essentially imagine reference EJBs as ManagedResource[EjbRemoteInterface].   You can then lookup and use the EJB lazily, even providing retry behavior as desired.   This is what I was alluding to with the performance benchmark testing actor-based code.
- Josh
On Wed, Oct 6, 2010 at 12:21 PM, Razvan (Pub) Cojocaru <pub@razie.com> wrote:

Josh, what's the loaner pattern?

On 10/6/10, Josh Suereth <joshua.suereth@gmail.com> wrote:
> To follow up:
>
> Remember that inside the J2EE container you should not create your own
> threads.   If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.
>
> In our usage, we had a 100% Scala application that ran load-testing on our
> J2EE server.   This was a simple application made up of a bunch of Actors
> representing physical entities.   They were driven with a discrete event
> model and would communicate to the j2EE server via JNDI/RMI.   As stated
> before, Session Beans and Entity beans were fine within Scala. This actually
> worked great!
>
> We also had a Scala layer inside EJB.   We used the "loaner" pattern to
> control JTA Transactions within EJB.   (i.e. transaction { foo } was a lot
> easier to write!).
>
> We had a set of MDBs for our processing pipeline.  Upon reflection, I think
> we would have benefited from dropping j2ee and just using Akka, however
> hindsight is 20/20.  I will state this:
>
> MDBs have a high overhead in usage.   They interact with transactions and
> your EJB container in strange ways.   Actors have a low overhead.
> MDBs are parallelized by default.   Actors are conceptually single-threaded.
>  This is *huge*.   An MDB is not an actor, and you can't treat it like one.
>   You can sort of make MDBs  be actors, but it can be painful.   One of the
> biggest issues we had in our application was threading issues relating to
> MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once
> again, given enough time I would have written something using Akka that was
> far better.
>
>
> So basically, my recommendation for EJB is to use it as designed when
> required.   You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
> them, etc.).  However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka, that
> interacts via JNDI + RMI.   This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still use
> advanced Scala libraries.
>
> The reality of the situation is the EJBs are very heavyweight abstraction
> with high overhead.   Although the parallelism is crazy for them, it assumes
> each session/mdb call is complete and large.   Actors have a different feel
> to them and are far lighter-weight concepts.   Akka provides you access to
> the light-weight as well as layers to build the complexity you need.   I
> learned, from Spring, that this approach is far more scalable in the long
> run.   I'd recommend working up an Akka->EJB interface for the rest of your
> team ;)
>
>
> - Josh
> .
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> <joshua.suereth@gmail.com>wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb.  I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization  = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>>  - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>

--
Sent from my mobile device

The loaner pattern is when I control a resource and you pass me a closure that makes use of it.  I will ensure the resource lifecycle.

For example, we had:

trait EjbHelper { self : OurEjbSessionBaseClass =>

  def transactional[A](f : => A) = {
     try {       // We would manipulate JTA here.   Picture a try catch block, where this opens it
        f
     } finally {
       // Again, with the JTA manipulation.   This would be the "commit" block.   For extra credit, you can change this to a while(!doneRetry) block and catch exceptions, revert the transaction and retry the f closure.      }
  }
}
Then in our Ejbs:

class MyEjbClass extends MyEjbRemote with MyEjbLocal with EjbHelper {
   def foo() : Unit = {
       transactional {           // Manipulate the database        }       sendJms(new Jms Notifying that things are actually commited)
   }
}

The JMS Processing, if it happened immediately, required the Database transaction to be completed in case it needed to re-modify the database.  Yes, horrible design, but the Scala certainly helped us make the code work until we had a chance to rework the EJB interactions.
In any case, the transactional method abided by the loaner pattern.   For another example, see my scala-arm library.  (P.S.  the ARM library was designed from my attempts to make JNDI usage easier.   I'm going to add a simplistic JNDI portion to the library shortly.   Essentially imagine reference EJBs as ManagedResource[EjbRemoteInterface].   You can then lookup and use the EJB lazily, even providing retry behavior as desired.   This is what I was alluding to with the performance benchmark testing actor-based code.
- Josh
On Wed, Oct 6, 2010 at 12:21 PM, Razvan (Pub) Cojocaru <pub@razie.com> wrote:

Josh, what's the loaner pattern?

On 10/6/10, Josh Suereth <joshua.suereth@gmail.com> wrote:
> To follow up:
>
> Remember that inside the J2EE container you should not create your own
> threads.   If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.
>
> In our usage, we had a 100% Scala application that ran load-testing on our
> J2EE server.   This was a simple application made up of a bunch of Actors
> representing physical entities.   They were driven with a discrete event
> model and would communicate to the j2EE server via JNDI/RMI.   As stated
> before, Session Beans and Entity beans were fine within Scala. This actually
> worked great!
>
> We also had a Scala layer inside EJB.   We used the "loaner" pattern to
> control JTA Transactions within EJB.   (i.e. transaction { foo } was a lot
> easier to write!).
>
> We had a set of MDBs for our processing pipeline.  Upon reflection, I think
> we would have benefited from dropping j2ee and just using Akka, however
> hindsight is 20/20.  I will state this:
>
> MDBs have a high overhead in usage.   They interact with transactions and
> your EJB container in strange ways.   Actors have a low overhead.
> MDBs are parallelized by default.   Actors are conceptually single-threaded.
>  This is *huge*.   An MDB is not an actor, and you can't treat it like one.
>   You can sort of make MDBs  be actors, but it can be painful.   One of the
> biggest issues we had in our application was threading issues relating to
> MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once
> again, given enough time I would have written something using Akka that was
> far better.
>
>
> So basically, my recommendation for EJB is to use it as designed when
> required.   You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
> them, etc.).  However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka, that
> interacts via JNDI + RMI.   This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still use
> advanced Scala libraries.
>
> The reality of the situation is the EJBs are very heavyweight abstraction
> with high overhead.   Although the parallelism is crazy for them, it assumes
> each session/mdb call is complete and large.   Actors have a different feel
> to them and are far lighter-weight concepts.   Akka provides you access to
> the light-weight as well as layers to build the complexity you need.   I
> learned, from Spring, that this approach is far more scalable in the long
> run.   I'd recommend working up an Akka->EJB interface for the rest of your
> team ;)
>
>
> - Josh
> .
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> <joshua.suereth@gmail.com>wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb.  I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization  = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" <romixlev@yahoo.com> wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>>  - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>

--
Sent from my mobile device

Josh,

First of all, thanks a lot for such a great reply! Very informative and
based on a real-world experience.

> Remember that inside the J2EE container you should not create your own
> threads. If you are creating your own threads, you begin to look for
> container-specific extensions and hacks and things get ugly quick.

Absolutely! Though there is a certain possibility to escape this limitation
in a leagl way. Please see my comments below, when I talk about MDBs.

>In our usage, we had a 100% Scala application that ran load-testing on our
>J2EE server. This was a simple application made up of a bunch of Actors
>representing physical entities. They were driven with a discrete event
>model and would communicate to the j2EE server via JNDI/RMI. As stated
>before, Session Beans and Entity beans were fine within Scala. This
actually
>worked great!

>We also had a Scala layer inside EJB. We used the "loaner" pattern to
>control JTA Transactions within EJB. (i.e. transaction { foo } was a lot
>easier to write!).

Thanks for this example. Interesting! As I mentioned in my analysis message,
here you use Scala as Java++, i.e. make use of type system and traits. This
makes EJB implementation look much more elegant and concise.

>We had a set of MDBs for our processing pipeline. Upon reflection, I think
>we would have benefited from dropping j2ee and just using Akka, however
>hindsight is 20/20. I will state this:

>MDBs have a high overhead in usage. They interact with transactions and
> your EJB container in strange ways.

I agree. Especially for JMS-based MDBs.

> Actors have a low overhead. MDBs are parallelized by default. Actors are
> conceptually single-threaded.
> This is *huge*. An MDB is not an actor, and you can't treat it like one.
> You can sort of make MDBs be actors, but it can be painful.

Can you elaborate a bit more? IMHO, an instance of MDB is supposed to
process one message at a time, and therefore it is similar to an Actor. But
container keeps N instances of the same MDB up and running, so they process
messages in parallel. But the same could be done, if you would start
multiple instances of an actor (or alternatively, would reduce the number of
MDB instances to 1).

>One of the
>biggest issues we had in our application was threading issues relating to
>MDBs and the overhead of asynchronously modifying a big ugly RDBMS. Once
>again, given enough time I would have written something using Akka that was
>far better.

> So basically, my recommendation for EJB is to use it as designed when
> required. You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
>them, etc.).

> However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka,
> that
> interacts via JNDI + RMI.
> This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still
> use
> advanced Scala libraries.

This is a very interesting comment! This is exactly the impression that I
got and expressed in my analysis. Any Scala features that demand run-time
support are "foreign" for the JaveEE and live in their own (better?) world.
It often means that such Scala components cannot be (easily) deployed on a
JavaEE container, cannot be easily packaged in a portable way (e.g. like EAR
or WAR). This makes the whole maintenance much more complex and different.

On reimplementing using Akka or the like:
You and Razvan mention this possibility and this was also my impression.
Doing so is straight forward and easy, at least in short term. But it would
lead to having and maintaining two (or more?) different runtimes,
deployments, etc. One for JavaEE, one for Scala/Akka. While I understand
that this is what is technically possible today, I also understand that this
is not an ideal situation. More runtimes and ways of deployment mean also
more time and money spent on that. Scala's way of deploying and archiving
for Scala-based "containers" (like Akka) is not even quite established and
standardized yet. It will take quite some time, I guess. At the same time
JavaEE has provided means for doing it since years. There are proven tools
supporting all those stages (building enterprise acrhives, deployment and
monitoring). Therefore I'm asking questions about achieving integration of
Scala components with JavaEE model, instead of mostly co-existance in
parallel worlds. To get there, some changes on the Scala (libraries) side,
but also on the JavaEE side may be necessary.

Few questions:
- You mention that you use RMI for interaction with external standalone
Scala actors. That's fine. Have you ever experiences
serialization/deserialization problems, e.g. when Java could not deserialize
complex Scala types? Is RMI performant enough in your experience? What if
you need a more asynchronous style of communication between JavaEE and
actors?

- It is not quite clear for me, if were also able to publish your Scala
actors in JNDI? If so, how?

- Your Scala components were also exposed via RMI, if I understand
correctly. Does it mean that Actors were exposed directly via RMI (how?) or
have you used some sort of a frontend that would accept requests and send
messages to Actors?

- Regarding MDBs and control over multithreading for Actors:

There is something called Resource Adapters (RAs) in JavaEE. RAs are
deployed on the JavaEE application servers. They are allowed to explicitly
control threads, which can be useful for running Actors. More over, RAs may
define alternative than JMS ways for delivering message to/from MDBs. For
example, one could think of implementing more efficient application server
internal way of triggering MDBs from Actors or vice versa. I never tried it,
but it could work. My colleagues also told me, that creation of new RAs is
much simpler nowadays. One can use special annotations. It is also possible
to redefine onMessae methods to have different names, number of parameters,
types of parameters. Eventually, this approach could provide a way to run
Actors on an application server and do it in a reasonably portable and
efficient way.
What do you think? Any comments on that?

Thanks again for your reply! Need to think more about it. May be I'll come
with more questions.

Cheers,
-Leo

On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
wrote:

> Leo,
>
> I just left a shop where we mixed scala and ejb. I'll try to draft a good
> response for you shortly. The short answer is you have some great ideas,
> but
> there are some gotchas to be careful with.
>
> Specifically:
>
> Scala and session beans = great
>
> Scala and jndi = good (i wish there were more async apis here...)
>
> Scala and Rmi/serialization = a few things to be careful of.
>
> Scala and Entity Beans = ok
>
> Actors and Mdb = dangerous
>
> I'll follow up with details later today.
>
> On Oct 2, 2010 7:09 AM, "Leo Romanoff" wrote:
>
>
> Razvan and Viktor - thanks for your comments!
>
> But what I get to hear still sounds very fuzzy for me. See below.
>
>
> >> Having said that, if you have sufficient experience and can carefully
> >> tippy
> >> toe your way a...
>
> This sounds like a "try to do it yourself - and may be you are lucky"
> exercise
>
>
> >> As for the distributed part, I think both akka and Stambecco have
> support
> >> for remote actors, ...
>
> OK. This means that this mechanism is not quite JavaEE compliant.
>
>
> What I'm trying to figure out in this whole thread is some sort of best
> practices or direction for an (average) JavaEE developer for starting
> using
> Scala and its libraries and frameworks in her daily work. Often, in big
> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
> existing
> JavaEE-based codebase. Rewriting everything the Scala-way or to become
> Web-based is often out of question. Therefore, the natural basic
> requirements for using Scala in this context are:
>
> - that Scala code should be able to at least interwork (bind, invoke,
> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>
> - In a perfect world, it should be possible to implement JavaEE components
> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
> standards for packaging and deployment. Is it possible now? If not, is it
> envisioned?
>
> - It would be useful to expose Scala specific components (e.g. actors) in
> a
> JavaEE friendly way, so that they can be consumed by other JavaEE
> components. What about application packaging in this case?
>
> - What is a guideline for Scala components development? Does it make sense
> to strive for integration with JavaEE? Or is it better to use
> Scala-specific
> frameworks? Should Scala be used as Java++ for JavaEE components
> development
> (i.e. only new and better language constructs, but e.g. no actors, as they
> depend too much on multithreading implementation and ability to explicitly
> control it). Or does it make sense to use actors and the like to implement
> JavaEE components?
>
> I think having clear statements (somewhere on major scala-related sites?)
> answering all these questions would be a great help for many JavaEE
> developers planning to switch to Scala.
>
> At the moment this kind of information is lacking. I've spent quite some
> time to find out any information about what is possible and what's not. It
> seems to be a "black art" at the moment. I could collect some picies here
> and there (e.g. how to develop @WebService annotated Web Services, how to
> use actors with MDBs, how to use actors with HTTP servlets by means of
> Lift), but there is no clear picture explained anywhere.
>
> My current understanding based on what I've found is:
>
> - It is always possible to use Scala as Java++, i.e. as mainly a
> syntactic
> improvement with better modularity, type ineference and functional
> features.
> Any constructs, concepts like actors, which require some sort of run-time
> support are useful only in those cases, where explicit control over
> multithreading is allowed (e.g. HTTP servlets). If this control is not
> possible (e.g. EJB containers), actors cannot be used so easily without
> further tweaks.
>
> - Overall, there are Scala frameworks provide alternative (compared to
> JavaEE) ways for persistance, distribution, discovery of components and
> methods for their deployment, e.g. Akka. While these methods are
> eventually
> even better than JavaEE, they are not compliant with it in many cases. In
> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
> from a JavaEE perspective. I don't know if this is an intended positioning
> of such components or is it due to the fact that Scala integration with
> JavaEE was not too much in focus till now.
>
> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
> mostly standalone components or need a Scala specific container for their
> deployment (Akka core?) as opposed to deployment on JavaEE application
> servers.
>
> - It is not clear if Scala components can easily access existing remote
> and
> local EJBs. It is not clear, if existing EJBs can easily access Scala
> components (e.g. actors) out of the box.
>
> With all that in mind, I'd say that the state of Scala<->JavaEE
> integration
> is RATHER POOR and not clearly explained anywhere with all its pros and
> cons. There seems to be also a rather low interest in making this kind of
> integration easier. This is probably because currently major adopters of
> Scala are using it rather for a "from scratch" apps development, e.g. for
> Web apps development (e.g. Lift-based) or for something like financial
> services (e.g. Akka). In such cases, integration with legacy JavaEE
> systems
> is probably not so important.
>
> Please correct me, if this analysis is very wrong. I'd be happy to see
> that
> integration with/into JavaEE apps is much better and much easier.
>
> Thanks,
>
> -Leo
>
>
> Razvan Cojocaru-2 wrote:
> >
>
> > Indeed - I assumed you'd run a J2EE container because you're using
> > "heavy" J2EE services like XA...
>
> --
>
> View this message in context:
> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>
> Sent from the Scala - User mailing list archive at Nabble.com.
>
>

Resource adapter - you mean JCAA - yeah, that's the only legal ticket
out of the j2ee constraints :) but there's no point mixing the two
just to use actors...do not forget the manageability aspect of why
people use j2ee...you either need that and then use jms with MDB or
you don't - then forget j2ee, as being outdated ;)

Cheers

On 10/7/10, Leo Romanoff wrote:
>
> Josh,
>
> First of all, thanks a lot for such a great reply! Very informative and
> based on a real-world experience.
>
>> Remember that inside the J2EE container you should not create your own
>> threads. If you are creating your own threads, you begin to look for
>> container-specific extensions and hacks and things get ugly quick.
>
> Absolutely! Though there is a certain possibility to escape this limitation
> in a leagl way. Please see my comments below, when I talk about MDBs.
>
>>In our usage, we had a 100% Scala application that ran load-testing on our
>>J2EE server. This was a simple application made up of a bunch of Actors
>>representing physical entities. They were driven with a discrete event
>>model and would communicate to the j2EE server via JNDI/RMI. As stated
>>before, Session Beans and Entity beans were fine within Scala. This
> actually
>>worked great!
>
>>We also had a Scala layer inside EJB. We used the "loaner" pattern to
>>control JTA Transactions within EJB. (i.e. transaction { foo } was a lot
>>easier to write!).
>
> Thanks for this example. Interesting! As I mentioned in my analysis message,
> here you use Scala as Java++, i.e. make use of type system and traits. This
> makes EJB implementation look much more elegant and concise.
>
>>We had a set of MDBs for our processing pipeline. Upon reflection, I think
>>we would have benefited from dropping j2ee and just using Akka, however
>>hindsight is 20/20. I will state this:
>
>>MDBs have a high overhead in usage. They interact with transactions and
>> your EJB container in strange ways.
>
> I agree. Especially for JMS-based MDBs.
>
>> Actors have a low overhead. MDBs are parallelized by default. Actors are
>> conceptually single-threaded.
>> This is *huge*. An MDB is not an actor, and you can't treat it like one.
>> You can sort of make MDBs be actors, but it can be painful.
>
> Can you elaborate a bit more? IMHO, an instance of MDB is supposed to
> process one message at a time, and therefore it is similar to an Actor. But
> container keeps N instances of the same MDB up and running, so they process
> messages in parallel. But the same could be done, if you would start
> multiple instances of an actor (or alternatively, would reduce the number of
> MDB instances to 1).
>
>>One of the
>>biggest issues we had in our application was threading issues relating to
>>MDBs and the overhead of asynchronously modifying a big ugly RDBMS. Once
>>again, given enough time I would have written something using Akka that was
>>far better.
>
>
>> So basically, my recommendation for EJB is to use it as designed when
>> required. You can add some nice Scala features (JNDI lookups via loaner
>> pattern, Transactions via loaner pattern if you need to take control of
>>them, etc.).
>
>> However if you want to use things like Actors, I would highly
>> recommend creating another component in the system, perhaps using Akka,
>> that
>> interacts via JNDI + RMI.
>> This would allow what you create to look like
>> another EJB, inter-operate with the RMI dependency injection, and still
>> use
>> advanced Scala libraries.
>
> This is a very interesting comment! This is exactly the impression that I
> got and expressed in my analysis. Any Scala features that demand run-time
> support are "foreign" for the JaveEE and live in their own (better?) world.
> It often means that such Scala components cannot be (easily) deployed on a
> JavaEE container, cannot be easily packaged in a portable way (e.g. like EAR
> or WAR). This makes the whole maintenance much more complex and different.
>
> On reimplementing using Akka or the like:
> You and Razvan mention this possibility and this was also my impression.
> Doing so is straight forward and easy, at least in short term. But it would
> lead to having and maintaining two (or more?) different runtimes,
> deployments, etc. One for JavaEE, one for Scala/Akka. While I understand
> that this is what is technically possible today, I also understand that this
> is not an ideal situation. More runtimes and ways of deployment mean also
> more time and money spent on that. Scala's way of deploying and archiving
> for Scala-based "containers" (like Akka) is not even quite established and
> standardized yet. It will take quite some time, I guess. At the same time
> JavaEE has provided means for doing it since years. There are proven tools
> supporting all those stages (building enterprise acrhives, deployment and
> monitoring). Therefore I'm asking questions about achieving integration of
> Scala components with JavaEE model, instead of mostly co-existance in
> parallel worlds. To get there, some changes on the Scala (libraries) side,
> but also on the JavaEE side may be necessary.
>
>
> Few questions:
> - You mention that you use RMI for interaction with external standalone
> Scala actors. That's fine. Have you ever experiences
> serialization/deserialization problems, e.g. when Java could not deserialize
> complex Scala types? Is RMI performant enough in your experience? What if
> you need a more asynchronous style of communication between JavaEE and
> actors?
>
> - It is not quite clear for me, if were also able to publish your Scala
> actors in JNDI? If so, how?
>
> - Your Scala components were also exposed via RMI, if I understand
> correctly. Does it mean that Actors were exposed directly via RMI (how?) or
> have you used some sort of a frontend that would accept requests and send
> messages to Actors?
>
> - Regarding MDBs and control over multithreading for Actors:
>
> There is something called Resource Adapters (RAs) in JavaEE. RAs are
> deployed on the JavaEE application servers. They are allowed to explicitly
> control threads, which can be useful for running Actors. More over, RAs may
> define alternative than JMS ways for delivering message to/from MDBs. For
> example, one could think of implementing more efficient application server
> internal way of triggering MDBs from Actors or vice versa. I never tried it,
> but it could work. My colleagues also told me, that creation of new RAs is
> much simpler nowadays. One can use special annotations. It is also possible
> to redefine onMessae methods to have different names, number of parameters,
> types of parameters. Eventually, this approach could provide a way to run
> Actors on an application server and do it in a reasonably portable and
> efficient way.
> What do you think? Any comments on that?
>
> Thanks again for your reply! Need to think more about it. May be I'll come
> with more questions.
>
> Cheers,
> -Leo
>
>
>
> On Wed, Oct 6, 2010 at 8:37 AM, Josh Suereth
> wrote:
>
>> Leo,
>>
>> I just left a shop where we mixed scala and ejb. I'll try to draft a good
>> response for you shortly. The short answer is you have some great ideas,
>> but
>> there are some gotchas to be careful with.
>>
>> Specifically:
>>
>> Scala and session beans = great
>>
>> Scala and jndi = good (i wish there were more async apis here...)
>>
>> Scala and Rmi/serialization = a few things to be careful of.
>>
>> Scala and Entity Beans = ok
>>
>> Actors and Mdb = dangerous
>>
>> I'll follow up with details later today.
>>
>> On Oct 2, 2010 7:09 AM, "Leo Romanoff" wrote:
>>
>>
>> Razvan and Viktor - thanks for your comments!
>>
>> But what I get to hear still sounds very fuzzy for me. See below.
>>
>>
>> >> Having said that, if you have sufficient experience and can carefully
>> >> tippy
>> >> toe your way a...
>>
>> This sounds like a "try to do it yourself - and may be you are lucky"
>> exercise
>>
>>
>> >> As for the distributed part, I think both akka and Stambecco have
>> support
>> >> for remote actors, ...
>>
>> OK. This means that this mechanism is not quite JavaEE compliant.
>>
>>
>> What I'm trying to figure out in this whole thread is some sort of best
>> practices or direction for an (average) JavaEE developer for starting
>> using
>> Scala and its libraries and frameworks in her daily work. Often, in big
>> JavaEE shops (e.g. enterprises and telcos), you have to use a huge
>> existing
>> JavaEE-based codebase. Rewriting everything the Scala-way or to become
>> Web-based is often out of question. Therefore, the natural basic
>> requirements for using Scala in this context are:
>>
>> - that Scala code should be able to at least interwork (bind, invoke,
>> consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).
>>
>> - In a perfect world, it should be possible to implement JavaEE components
>> (servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
>> standards for packaging and deployment. Is it possible now? If not, is it
>> envisioned?
>>
>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
>> components. What about application packaging in this case?
>>
>> - What is a guideline for Scala components development? Does it make sense
>> to strive for integration with JavaEE? Or is it better to use
>> Scala-specific
>> frameworks? Should Scala be used as Java++ for JavaEE components
>> development
>> (i.e. only new and better language constructs, but e.g. no actors, as they
>> depend too much on multithreading implementation and ability to explicitly
>> control it). Or does it make sense to use actors and the like to implement
>> JavaEE components?
>>
>> I think having clear statements (somewhere on major scala-related sites?)
>> answering all these questions would be a great help for many JavaEE
>> developers planning to switch to Scala.
>>
>> At the moment this kind of information is lacking. I've spent quite some
>> time to find out any information about what is possible and what's not. It
>> seems to be a "black art" at the moment. I could collect some picies here
>> and there (e.g. how to develop @WebService annotated Web Services, how to
>> use actors with MDBs, how to use actors with HTTP servlets by means of
>> Lift), but there is no clear picture explained anywhere.
>>
>> My current understanding based on what I've found is:
>>
>> - It is always possible to use Scala as Java++, i.e. as mainly a
>> syntactic
>> improvement with better modularity, type ineference and functional
>> features.
>> Any constructs, concepts like actors, which require some sort of run-time
>> support are useful only in those cases, where explicit control over
>> multithreading is allowed (e.g. HTTP servlets). If this control is not
>> possible (e.g. EJB containers), actors cannot be used so easily without
>> further tweaks.
>>
>> - Overall, there are Scala frameworks provide alternative (compared to
>> JavaEE) ways for persistance, distribution, discovery of components and
>> methods for their deployment, e.g. Akka. While these methods are
>> eventually
>> even better than JavaEE, they are not compliant with it in many cases. In
>> this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
>> from a JavaEE perspective. I don't know if this is an intended positioning
>> of such components or is it due to the fact that Scala integration with
>> JavaEE was not too much in focus till now.
>>
>> - At the moment, most Scala apps (except for HTTP based Lift apps?) are
>> mostly standalone components or need a Scala specific container for their
>> deployment (Akka core?) as opposed to deployment on JavaEE application
>> servers.
>>
>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>
>> With all that in mind, I'd say that the state of Scala<->JavaEE
>> integration
>> is RATHER POOR and not clearly explained anywhere with all its pros and
>> cons. There seems to be also a rather low interest in making this kind of
>> integration easier. This is probably because currently major adopters of
>> Scala are using it rather for a "from scratch" apps development, e.g. for
>> Web apps development (e.g. Lift-based) or for something like financial
>> services (e.g. Akka). In such cases, integration with legacy JavaEE
>> systems
>> is probably not so important.
>>
>> Please correct me, if this analysis is very wrong. I'd be happy to see
>> that
>> integration with/into JavaEE apps is much better and much easier.
>>
>> Thanks,
>>
>> -Leo
>>
>>
>> Razvan Cojocaru-2 wrote:
>> >
>>
>> > Indeed - I assumed you'd run a J2EE container because you're using
>> > "heavy" J2EE services like XA...
>>
>> --
>>
>> View this message in context:
>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
>>
>> Sent from the Scala - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> View this message in context:
> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-i...
> Sent from the Scala - User mailing list archive at Nabble.com.
>

Hi,

I am answering the Akka-related stuff since I'm on a time budget.

On Sat, Oct 2, 2010 at 1:09 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

Razvan and Viktor - thanks for your comments!

But what I get to hear still sounds very fuzzy for me. See below.

>> Having said that, if you have sufficient experience and can carefully
>> tippy
>> toe your way around that problem, you could...well, use them.

This sounds like a "try to do it yourself - and may be you are lucky"
exercise

>> As for the distributed part, I think both akka and Stambecco have support
>> for remote actors, probably not via JNDI though.

OK. This means that this mechanism is not quite JavaEE compliant.


What I'm trying to figure out in this whole thread is some sort of best
practices or direction for an (average) JavaEE developer for starting using
Scala and its libraries and frameworks in her daily work. Often, in big
JavaEE shops (e.g. enterprises and telcos), you have to use a huge existing
JavaEE-based codebase. Rewriting everything the Scala-way or to become
Web-based is often out of question. Therefore, the natural basic
requirements for using Scala in this context are:

- that Scala code should be able to at least interwork (bind, invoke,
consume) with existing JavaEE components (HTTP servlets, EJBs, MDBs, etc).

It interworks in a lot of different ways, you have a vast array of IPC to resort to if you don't want to host both in the same container.
 

- In a perfect world, it should be possible to implement JavaEE components
(servlets, EJBs, MDBs, etc) in Scala and deploy them according to JavaEE
standards for packaging and deployment. Is it possible now? If not, is it
envisioned?

- It would be useful to expose Scala specific components (e.g. actors) in a
JavaEE friendly way, so that they can be consumed by other JavaEE
components. What about application packaging in this case?

http://doc.akkasource.org/camel
http://doc.akkasource.org/amqp
http://doc.akkasource.org/rest

 

- What is a guideline for Scala components development? Does it make sense
to strive for integration with JavaEE? Or is it better to use Scala-specific
frameworks? Should Scala be used as Java++ for JavaEE components development
(i.e. only new and better language constructs, but e.g. no actors, as they
depend too much on multithreading implementation and ability to explicitly
control it). Or does it make sense to use actors and the like to implement
JavaEE components?

I think having clear statements (somewhere on major scala-related sites?)
answering all these questions would be a great help for many JavaEE
developers planning to switch to Scala.

At the moment this kind of information is lacking. I've spent quite some
time to find out any information about what is possible and what's not. It
seems to be a "black art" at the moment. I could collect some picies here
and there (e.g. how to develop @WebService annotated Web Services, how to
use actors with MDBs, how to use actors with HTTP servlets by means of
Lift), but there is no clear picture explained anywhere.

http://doc.akkasource.org/rest
 

My current understanding based on what I've found is:

 - It is always possible to use Scala as Java++, i.e. as mainly a syntactic
improvement with better modularity, type ineference and functional features.
Any constructs, concepts like actors, which require some sort of run-time
support are useful only in those cases, where explicit control over
multithreading is allowed (e.g. HTTP servlets). If this control is not
possible (e.g. EJB containers), actors cannot be used so easily without
further tweaks.

- Overall, there are Scala frameworks provide alternative (compared to
JavaEE) ways for persistance, distribution, discovery of components and
methods for their deployment, e.g. Akka. While these methods are eventually
even better than JavaEE, they are not compliant with it in many cases. In
this sense, _Scala components exists in THEIR OWN (may be BETTER?) WORLD_,
from a JavaEE perspective. I don't know if this is an intended positioning
of such components or is it due to the fact that Scala integration with
JavaEE was not too much in focus till now.

You can run Akka as just another jar in your app server, or if you want it to be standalone, you can run it in microkernel mode:
http://doc.akkasource.org/microkernel
 

- At the moment, most Scala apps (except for HTTP based Lift apps?) are
mostly standalone components or need a Scala specific container for their
deployment (Akka core?) as opposed to deployment on JavaEE application
servers.

See answer above
 

- It is not clear if Scala components can easily access existing remote and
local EJBs. It is not clear, if existing EJBs can easily access Scala
components (e.g. actors) out of the box.

http://doc.akkasource.org/camel
 

With all that in mind, I'd say that the state of Scala<->JavaEE integration
is RATHER POOR and not clearly explained anywhere with all its pros and
cons. There seems to be also a rather low interest in making this kind of
integration easier. This is probably because currently major adopters of
Scala are using it rather for a "from scratch" apps development, e.g. for
Web apps development (e.g. Lift-based) or for something like financial
services (e.g. Akka). In such cases, integration with legacy JavaEE systems
is probably not so important.

In my experience people see to run AS->App in a 1:1 ratio to avoid weakest-link problems.
 

Please correct me, if this analysis is very wrong. I'd be happy to see that
integration with/into JavaEE apps is much better and much easier.

I've been creating and maintaining my own Threads in EE environments for 8 years,
I have never had any problems, and I'm not the next Knuth, so it should be repeatable ;-)



Hope this clears a bit of things up regarding Akka :-)

Cheers,

 

Thanks,
 -Leo


Razvan Cojocaru-2 wrote:
>
> Indeed - I assumed you'd run a J2EE container because you're using
> "heavy" J2EE services like XA transactions, authentication contexts,
> fail-over etc.
>
>
> On 10/1/10, Viktor Klang <viktor.klang@gmail.com> wrote:
>> On Fri, Oct 1, 2010 at 8:48 PM, Razvan Cojocaru <pub@razie.com> wrote:
>>
>>> I think my original reply stands.
>>>
>>> You should not use actors or threads in a container, as it will easily
>>> mangle contexts, transactions and whatnot.
>>>
>>
>> I disagree.
>> It totally depends what you're running in that container.
>>
>>
>>>
>>> Having said that, if you have sufficient experience and can carefully
>>> tippy
>>> toe your way around that problem, you could...well, use them.
>>>
>>> As for the distributed part, I think both akka and Stambecco have
>>> support
>>> for remote actors, probably not via JNDI though.
>>>
>>> Cheers,
>>> Razie
>>>
>>> -----Original Message-----
>>> From: Leo Romanoff [mailto:romixlev@yahoo.com]
>>> Sent: Friday, October 01, 2010 1:12 PM
>>> To: scala-user@listes.epfl.ch
>>> Subject: Re: Fwd: [scala-user] Re: Usage of Scala in JavaEE environments
>>>
>>>
>>> Any feedback on:
>>>
>>> - usage of Actors inside EJB containers and possible multithreading
>>> issues?
>>> - ability to publish (via JNDI?), discover and invoke Actors deployed as
>>> part of different JavaEE apps running on the same or different nodes?
>>>
>>> Thanks,
>>>  -Leo
>>>
>>>
>>> Leo Romanoff wrote:
>>> >
>>> > After all those continuations discussions, let's come back to the
>>> > original questions about Scala use in JavaEE environments.
>>> >
>>> >
>>> > dpp wrote:
>>> >>
>>> >>
>>> >>>> But what about usage of Actors inside an EJB container, e.g. if an
>>> >>>> EJB would call it? AFAIK, according to the EJB spec, EJB containers
>>> >>>> should not permit explicit thread-management (e.g. thread creation)
>>> >>>> by user-code as it may lead to conflicts.
>>> >>
>>> >>
>>> >>> If there's a way to transfer an EJB across a thread (e.g., to a
>>> >>> worker thread that does background processing), the very same
>>> >>> mechanism would be used in Actors.
>>> >>
>>> >
>>> > Well, there is no direct access to threads or java.concurrent inside
>>> > EJB components. There is no such thing like Thread at all. EJBs are
>>> "actors"
>>> > in the JavaEE world. And the EJB specification forbids direct
>>> > manipulations with threads by means of Threads API ;-( This is a
>>> > serious limitation. Some vendors provide certain kinds of workarounds.
>>> > For example, BEA and IBM provide a special WorkManager API.
>>> > Glassfish has something similar. You can get a WorkManager, which is
>>> > something like an Executor factory. But they are not standardized and
>>> > not compatible with each other. These APIs are modeled after
>>> > j.u.concurrent, but are not 100% the same thing. Nevertheless,
>>> > supporting these major app servers is very important. May be something
>>> > can be done about it? May be Scala actor's support and library can be
>>> > extended to become configurable so that ThreadPools factories and the
>>> > like can be provided by an application server as well?
>>> >
>>> > Another thing, I'd like to understand is about publishing and
>>> > discovery of Scala actors in JavaEE environments. Most JavaEE
>>> > components (EJBs, MDBs, JMS queues, etc) published themselves in JNDI
>>> > directory, so that other JavaEE applications running on the same or
>>> > other application servers can find them. I think, it would be nice to
>>> > have the same for Scala actors, so that other JavaEE apps can discover
>>> > them and send messages to them. So, here is a question:
>>> >  - Is it possible to publish an Actor in the JNDI?
>>> >  - Is it possible to invoke an Actor that runs inside a different
>>> > JavaEE application, i.e. it is part of another EAR/WAR file and was
>>> > loaded in a different classloader, as it is always the case in the
>>> > JavaEE world. Each app runs in the same JVM, but in its own "sandbox".
>>> >
>>> > These questions covered just the most obvious things. I guess there
>>> > are more issues of this kind. Please add them, if I haven't mentioned
>>> > them yet.
>>> >
>>> > I think, many Java professionals would really like to be able to use
>>> > Scala for JavaEE development instead of plain old Java. Scala is much
>>> > nicer and more powerful as a language. To be able to do it, it is
>>> > important to play the game according to JavaEE rules, to be compliant
>>> > with it and to have a seamless integration of Scala components into
>>> > JavaEE world as first-class citizens. If this kind of support is not
>>> > in place yet, what are the plans of Scala developers or promoters? Is
>>> > anyone working on it? Is it on the roadmap?
>>> >
>>> > * appeal *
>>> > I know that implementing this kind of features may seem not very sexy
>>> > for Scala language developers. It is not so interesting as type
>>> > systems, functional features, etc.  But having a clear picture and
>>> > even better a good support for JavaEE may be crucial for adoption of
>>> > Scala in the enterprise world. And this adoption by the enterprise
>>> > development segment may become the most important factor for Scala for
>>> > moving from a "research/academic language league" to the mainstream
>>> language.
>>> > Enterprises and telcos are also the places where the real money are.
>>> > So, having them on board is very useful from many different
>>> > perspectives.
>>> > * End of appeal *
>>> >
>>> > Thanks,
>>> >   Leo
>>> >
>>> >
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>>
>>> http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-Ja
>>> vaEE-environments-tp2553702p2936337.html<http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-Ja%0AvaEE-environments-tp2553702p2936337.html>
>>> Sent from the Scala - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>> --
>> Viktor Klang,
>> Code Connoisseur
>> Work:   www.akkasource.com
>> Code:   github.com/viktorklang
>> Follow: twitter.com/viktorklang
>> Read:   klangism.tumblr.com
>>
>
> --
> Sent from my mobile device
>
>

--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2952339.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

Hi Viktor,

Thank you for finding a time to answer my questions.

> You can run Akka as just another jar in your app server, or if you want it
> to be standalone, you can run it in microkernel mode:
> http://doc.akkasource.org/microkernel

OK. In case I deploy it on my app server, is it usually exposed via
HTTP-based protocols (REST?)

>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
> > components. What about application packaging in this case?
>>

> http://doc.akkasource.org/camel
> http://doc.akkasource.org/amqp
> http://doc.akkasource.org/rest

I'm very interested in high-performance asynchronous communication between
components on the same app.server. While HTTP is a possible solution, I'd be
very interested in something that is more high-performance and able to
exchange very complex Java objects a-la RMI or EJB invocations using object
serialization mechanisms or passing by reference. Even better, something as
efficient as local EJB invocations could be very valuable. Can I do this
using Akka deployed on my app.server and using any specific connectors (like
Camel)?

>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>

> http://doc.akkasource.org/camel

Do you say that any of Camel transports is supported with Akka? Including
EJB? On the documentation page at http://doc.akkasource.org/camel, EJBs are
not explicitly listed.

> I've been creating and maintaining my own Threads in EE environments for 8
> years,
> I have never had any problems, and I'm not the next Knuth, so it should be
> repeatable ;-)

You are right in the sense, that even though the EJB spec forbids thread
maintenance and it is not guaranteed to work, in practice it usually works.
But at your own risk! ;-)

New Akka-specific questions:
- Does Akka support clustered deployment (on app.servers) with a load
balancing front-end for requests? What about adding/removing cluster
elements dynamically?
- Does Akka support (HTTP, but not only) session replication as known from
app.server world? If so, what is used to achieve it: home-grown solution,
Terracotta or something else?
- What about geo-redundancy capable solutions?

Thanks,
-Leo

Hi Leo,

answers below!

On Fri, Oct 8, 2010 at 11:42 AM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi Viktor,

Thank you for finding a time to answer my questions.

> You can run Akka as just another jar in your app server, or if you want it
> to be standalone, you can run it in microkernel mode:
> http://doc.akkasource.org/microkernel

OK. In case I deploy it on my app server, is it usually exposed via
HTTP-based protocols (REST?)

Exposed to whom?
 

>> - It would be useful to expose Scala specific components (e.g. actors) in
>> a
>> JavaEE friendly way, so that they can be consumed by other JavaEE
> > components. What about application packaging in this case?
>>

> http://doc.akkasource.org/camel
> http://doc.akkasource.org/amqp
> http://doc.akkasource.org/rest

I'm very interested in high-performance asynchronous communication between
components on the same app.server. While HTTP is a possible solution, I'd be
very interested in something that is more high-performance and able to
exchange very complex Java objects a-la RMI or EJB invocations using object
serialization mechanisms or passing by reference. Even better, something as
efficient as local EJB invocations could be very valuable. Can I do this
using Akka deployed on my app.server and using any specific connectors (like
Camel)?

It depends on whos calling who, if you're in the same classloader you can use ActorRegistry to obtain references to alive actors and send messages to them. If you need your actors to call into your EE components you'll have to use your EE facilities to provide that plumbing.
 

>> - It is not clear if Scala components can easily access existing remote
>> and
>> local EJBs. It is not clear, if existing EJBs can easily access Scala
>> components (e.g. actors) out of the box.
>>

> http://doc.akkasource.org/camel

Do you say that any of Camel transports is supported with Akka? Including
EJB? On the documentation page at http://doc.akkasource.org/camel, EJBs are
not explicitly listed.

Akka Camel exposes Actors as camel endpoints with consumers and producers, if you expose your ejbs as camel enpoints you can wire them together.
 

> I've been creating and maintaining my own Threads in EE environments for 8
> years,
> I have never had any problems, and I'm not the next Knuth, so it should be
> repeatable ;-)

You are right in the sense, that even though the EJB spec forbids thread
maintenance and it is not guaranteed to work, in practice it usually works.
But at your own risk! ;-)

New Akka-specific questions:
 - Does Akka support clustered deployment (on app.servers) with a load
balancing front-end for requests? What about adding/removing cluster
elements dynamically?

What kinds of requests? Http? Akka clustering is a commercial offering, however, you can use Remote Actors (less dynamic)
 

 - Does Akka support (HTTP, but not only) session replication as known from
app.server world?

Akka does not use Http Sessions
 

If so, what is used to achieve it: home-grown solution,
Terracotta or something else?

If you want persistence, we support redis, voldemort, cassandra, hbase and mongodb, also couchDB is upcoming.

 

 - What about geo-redundancy capable solutions?

What about them?

Cheers,

 

Thanks,
 -Leo
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2968061.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

Hi Viktor,

> answers below!

That was fast!

>> OK. In case I deploy it on my app server, is it usually exposed via
>> HTTP-based protocols (REST?)

> Exposed to whom?

I meant Akka-based components (actors, etc) exposed to other (external) Java
components.

> >> - It would be useful to expose Scala specific components (e.g. actors)
> in
> >> a
> >> JavaEE friendly way, so that they can be consumed by other JavaEE
> > > components. What about application packaging in this case?
> >>
>
> > http://doc.akkasource.org/camel
> > http://doc.akkasource.org/amqp
> > http://doc.akkasource.org/rest
>
> I'm very interested in high-performance asynchronous communication between
> components on the same app.server. While HTTP is a possible solution, I'd
> be
> very interested in something that is more high-performance and able to
> exchange very complex Java objects a-la RMI or EJB invocations using
> object
> serialization mechanisms or passing by reference. Even better, something
> as
> efficient as local EJB invocations could be very valuable. Can I do this
> using Akka deployed on my app.server and using any specific connectors
> (like
> Camel)?
>

> It depends on whos calling who, if you're in the same classloader you can
> use ActorRegistry to obtain references to alive actors and send messages
> to
> them.

OK. This is nice.

> If you need your actors to call into your EE components you'll have to
> use your EE facilities to provide that plumbing.

I see.

> New Akka-specific questions:
> - Does Akka support clustered deployment (on app.servers) with a load
> balancing front-end for requests? What about adding/removing cluster
> elements dynamically?
>

> What kinds of requests? Http?

Mostly HTTP. But we also use some exotic telco protocols handled by a
special dedicated container (implementing a dedicated protocol stack) inside
J2EE app. server.

> Akka clustering is a commercial offering, however, you can use Remote
> Actors (less dynamic)

OK.

> - Does Akka support (HTTP, but not only) session replication as known
> from
> app.server world?

> Akka does not use Http Sessions

Aha. Let's put a question in a more generic form. Does Akka support any kind
of long-living application session? Or it is assumed that everything is
modeled in a stateless way?
What is the most natural way to model something using HttpSessions and the
like in the J2EE world in Akka?

>> If so, what is used to achieve it: home-grown solution,
>> Terracotta or something else?

>If you want persistence, we support redis, voldemort, cassandra, hbase and
> mongodb, also couchDB is upcoming.

More precisely, I'd like to have transparent session persistence/replication
for the sake of fault tolerance and error recovery. If one of my nodes
fails, another one should be able to pick up the state and process further
requests targeting the original node.

>> - What about geo-redundancy capable solutions?
>>
> What about them?

;-) This question was similar to the question about session replication.
Does Akka support geo-redundancy out of the box? I guess it can be for sure
modeled by hand using remote actors and clustering support. But may be Akka
Enterprise has special modules/support for it already, which allows easy
setup, deployment, monitoring and maintenance of such solutions?

Cheers,
Leo

On Fri, Oct 8, 2010 at 12:37 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

Hi Viktor,

> answers below!

That was fast!

>> OK. In case I deploy it on my app server, is it usually exposed via
>> HTTP-based protocols (REST?)

> Exposed to whom?

I meant Akka-based components (actors, etc) exposed to other (external) Java
components.

Since Akka has a Java API we kind of enourage to use that, but of course, you could either use camel http endpoints or go via our Jersey integration.
 

> >> - It would be useful to expose Scala specific components (e.g. actors)
> in
> >> a
> >> JavaEE friendly way, so that they can be consumed by other JavaEE
> > > components. What about application packaging in this case?
> >>
>
> > http://doc.akkasource.org/camel
> > http://doc.akkasource.org/amqp
> > http://doc.akkasource.org/rest
>
> I'm very interested in high-performance asynchronous communication between
> components on the same app.server. While HTTP is a possible solution, I'd
> be
> very interested in something that is more high-performance and able to
> exchange very complex Java objects a-la RMI or EJB invocations using
> object
> serialization mechanisms or passing by reference. Even better, something
> as
> efficient as local EJB invocations could be very valuable. Can I do this
> using Akka deployed on my app.server and using any specific connectors
> (like
> Camel)?
>

> It depends on whos calling who, if you're in the same classloader you can
> use ActorRegistry to obtain references to alive actors and send messages
> to
> them.

OK. This is nice.

> If you need your actors to call into your EE components you'll have to
> use your EE facilities to provide that plumbing.

I see.

Or Camel
 

> New Akka-specific questions:
>  - Does Akka support clustered deployment (on app.servers) with a load
> balancing front-end for requests? What about adding/removing cluster
> elements dynamically?
>

Coming and going is supported by the commercial clustering :-)
 

> What kinds of requests? Http?

Mostly HTTP. But we also use some exotic telco protocols handled by a
special dedicated container (implementing a dedicated protocol stack) inside
J2EE app. server.

That should be fairly straightforward to integrate/intercommunicate
 

> Akka clustering is a commercial offering, however, you can use Remote
> Actors (less dynamic)

OK.

>  - Does Akka support (HTTP, but not only) session replication as known
> from
> app.server world?

> Akka does not use Http Sessions

Aha. Let's put a question in a more generic form. Does Akka support any kind
of long-living application session?

Yes, actors, they live for as long as you want and have state and communicate asynchrnously.
 

Or it is assumed that everything is
modeled in a stateless way?

Stateless or without shared state?
 

What is the most natural way to model something using HttpSessions and the
like in the J2EE world  in Akka?

Actors.
 

>> If so, what is used to achieve it: home-grown solution,
>> Terracotta or something else?

>If you want persistence, we support redis, voldemort, cassandra, hbase and
> mongodb, also couchDB is upcoming.

More precisely, I'd like to have transparent session persistence/replication
for the sake of fault tolerance and error recovery. If one of my nodes
fails, another one should be able to pick up the state and process further
requests targeting the original node.

Yes, that is targeted for our commercial offering. Moving actors around in a dynamic cluster, along with persistent mailboxes etc.
 

>>  - What about geo-redundancy capable solutions?
>>
> What about them?

;-) This question was similar to the question about session replication.
Does Akka support geo-redundancy out of the box? I guess it can be for sure
modeled by hand using remote actors and clustering support. But may be Akka
Enterprise has special modules/support for it already, which allows easy
setup, deployment, monitoring and maintenance of such solutions?

Yes, your hunch is correct. :-)

 

Cheers,
 Leo
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Usage-of-Scala-in-JavaEE-environments-tp2553702p2968113.html
Sent from the Scala - User mailing list archive at Nabble.com.

--
Viktor Klang,
Code Connoisseur
Work:   www.akkasource.com
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

Leo, There's a lot here, so I tried to snip the the useful answers.

On Thu, Oct 7, 2010 at 4:25 PM, Leo Romanoff <romixlev@yahoo.com> wrote:

<snip/>

> Actors have a low overhead. MDBs are parallelized by default.   Actors are
> conceptually single-threaded.
> This is *huge*.   An MDB is not an actor, and you can't treat it like one.
> You can sort of make MDBs  be actors, but it can be painful.

Can you elaborate a bit more? IMHO, an instance of MDB is supposed to
process one message at a time, and therefore it is similar to an Actor. But
container keeps N instances of the same MDB up and running, so they process
messages in parallel. But the same could be done, if you would start
multiple instances of an actor (or alternatively, would reduce the number of
MDB instances to 1).

Yes, the MDB will have mutliple (N) instances "live" in the container, with a potentially higher upper bound (M).  if you receive > M messages, you could have M separate MDBs processing all of those messages.   When coding an Actor, you do not expect more than one message to be handled at the same time.

Now, if we try to simulate Actors with MDBs by limited the instance count to 1, then we also loose the ability to deploy them to more than one EJB container, as each container would instantiate one instance.   If you wanted to implement supervisor schemes (like those found out of the box in Akka), you would be doing so outside the standard EJB conventions.   IMHO -> This is ugly.   We should learn from Erlang, errr.... Akka.
 

>One of the
>biggest issues we had in our application was threading issues relating to
>MDBs and the overhead of asynchronously modifying a big ugly RDBMS.   Once
>again, given enough time I would have written something using Akka that was
>far better.


> So basically, my recommendation for EJB is to use it as designed when
> required.   You can add some nice Scala features (JNDI lookups via loaner
> pattern, Transactions via loaner pattern if you need to take control of
>them, etc.).

> However if you want to use things like Actors, I would highly
> recommend creating another component in the system, perhaps using Akka,
> that
> interacts via JNDI + RMI.
> This would allow what you create to look like
> another EJB, inter-operate with the RMI dependency injection, and still
> use
> advanced Scala libraries.

This is a very interesting comment! This is exactly the impression that I
got and expressed in my analysis. Any Scala features that demand run-time
support are "foreign" for the JaveEE and live in their own (better?) world.
It often means that such Scala components cannot be (easily) deployed on a
JavaEE container, cannot be easily packaged in a portable way (e.g. like EAR
or WAR). This makes the whole maintenance much more complex and different.

On reimplementing using Akka or the like:
You and Razvan mention this possibility and this was also my impression.
Doing so is straight forward and easy, at least in short term. But it would
lead to having and maintaining two (or more?) different runtimes,
deployments, etc. One for JavaEE, one for Scala/Akka. While I understand
that this is what is technically possible today, I also understand that this
is not an ideal situation. More runtimes and ways of deployment mean also
more time and money spent on that. Scala's way of deploying and archiving
for Scala-based "containers" (like Akka) is not even quite established and
standardized yet. It will take quite some time, I guess. At the same time
JavaEE has provided means for doing it since years. There are proven tools
supporting all those stages (building enterprise acrhives, deployment and
monitoring). Therefore I'm asking questions about achieving integration of
Scala components with JavaEE model, instead of mostly co-existance in
parallel worlds. To get there, some changes on the Scala (libraries) side,
but also on the JavaEE side may be necessary.

Yes, the deployment issue is frustrating.   At my workplace we had a way (at least in JBoss) to create a 'service' that would be started automatically with EAR deployment.   You could embed your Akka component this way.   If you really wanted, you could even request Akka provide an MBean for the kernel.

Few questions:
 - You mention that you use RMI for interaction with external standalone
Scala actors. That's fine. Have you ever experiences
serialization/deserialization problems, e.g. when Java could not deserialize
complex Scala types? Is RMI performant enough in your experience? What if
you need a more asynchronous style of communication between JavaEE and
actors?

RMI is performant for what it is.   For asynchronous communication, we used RMI to send and receive JMS messages.
 

- It is not quite clear for me, if were also able to publish your Scala
actors in JNDI? If so, how?

We communicated through JMS/RMI for certain pieces and used Session Beans/RMI for others.   You could publish Scala actors via RMI if desired, but you would have to create a remote interface and do all the RMI machinery there.   You could then create a RemoteActor trait to match the interface (not quite the remote actor) and use this to communicate with RMI.  If needed, I can try to whip up a sample.
 

- Your Scala components were also exposed via RMI, if I understand
correctly. Does it mean that Actors were exposed directly via RMI (how?) or
have you used some sort of a frontend that would accept requests and send
messages to Actors?

I had a JMS->Actor layer complete with a "router" actor.  This was the path of least resistance as we were already using JMS as our central means of asynchronous communication.
 

- Regarding MDBs and control over multithreading for Actors:

 There is something called Resource Adapters (RAs) in JavaEE. RAs are
deployed on the JavaEE application servers. They are allowed to explicitly
control threads, which can be useful for running Actors. More over, RAs may
define alternative than JMS ways for delivering message to/from MDBs. For
example, one could think of implementing more efficient application server
internal way of triggering MDBs from Actors or vice versa. I never tried it,
but it could work. My colleagues also told me, that creation of new RAs is
much simpler nowadays. One can use special annotations. It is also possible
to redefine onMessae methods to have different names, number of parameters,
types of parameters. Eventually, this approach could provide a way to run
Actors on an application server and do it in a reasonably portable and
efficient way.
What do you think? Any comments on that?

I've used Resource Adapters.   I was using EJBs around the time these were becoming more popular.   We used the Resource Adapter that allowed Quartz "Job" to be treated as an MDB.  I think that these things are useful, but I'm unsure what they mean in the latest EJB spec.   If they are like JBoss's "service" configuration (which I think they do), then they are very useful and this would be a great way to embed an Akka component.   You probably need to jump through some hoops (via RMI/JNDI) to get everything exposed properly, but you'd be surprised how "native" you can make it feel.