RE: Question about Threads

You seem to be equating "native" (ie, provided by the OS) memory and
thread management with JVM's memory and thread management. They are
not the same thing -- nor could they be, given the need to guarantee
the Java memory model over whatever's the CPUs memory model.

On Tue, May 3, 2011 at 07:56, stecca michele wrote:
> Thank you for your replies. I have some additional comments/questions:
>
> 1. About Thread Context switch overhead. Last days I've been reading a lot of articles about this topic. The problem is that they are usually outdated
> and in particular their have been usually written before the usage of the optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.
> Have you got any experience/benchmark about the actual cost of context switch on modern CPUs?
>
> 2. About Memory Occupation. In the link provided by Chris (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm...) I see a lot of confusion.
> Each Java Thread allocates a stack whose size is given by the Xss JVM option param. It is important to note that we are talking about VIRTUAL MEMORY thus
> we have to remember that according to the "demand paging" algorithm (http://en.wikipedia.org/wiki/Demand_paging) physical memory pages are allocated iff the corresponding virtual pages are touched.
> It means that from a physical memory point of view, we are allocating only the pages that are actually needed.
> I see only the risk of running out of virtual address space because the maximum number of thread is given by the theoretical
> limit suggested in (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm...), i.e., the user address space divided by the thread stack size. The point is that in
> 64bits system this limit isn't a problem anymore. Here comes my questions: which are the negative effects on the memory of using many threads?
>
> 3. About resource contention / consumption. Could you please elaborate about that? For instance, are you saying that the are some
> locks/monitors mechanisms in the OS that are needed for sharing files, sockets, etc. that may influence performance? Are you also talking about
> resource exhaustion?
>
> 4. Has the scala lang been designed with the aim (among the others) of minimizing the number of threads in the system?
>
> 5. I leave you a couple of pointers where you can find some discussions about the problems related to the inefficient cache usage
> in multithreaded systems. Hope it is of interest for you. (www.cs.virginia.edu/~skadron/Papers/meng_dissertation.pdf) (http://www.cs.virginia.edu/~skadron/Papers/meng_iccd09.pdf).
>
> Cheers,
>
> Michele
>

2011/5/3 stecca michele <mstecca@yahoo.it>

Thank you for your replies. I have some additional comments/questions:

1. About Thread Context switch overhead. Last days I've been reading a lot of articles about this topic. The problem is that they are usually outdated
and in particular their have been usually written before the usage of the optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.
Have you got any experience/benchmark about the actual cost of context switch on modern CPUs?

2. About Memory Occupation. In the link provided by Chris (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support) I see a lot of confusion.
Each Java Thread allocates a stack whose size is given by the Xss JVM option param. It is important to note that we are talking about VIRTUAL MEMORY thus
we have to remember that according to the "demand paging" algorithm (http://en.wikipedia.org/wiki/Demand_paging) physical memory pages are allocated iff the corresponding virtual pages are touched.
It means that from a physical memory point of view, we are allocating only the pages that are actually needed.
I see only the risk of running out of virtual address space because the maximum number of thread is given by the theoretical
limit suggested in (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support), i.e., the user address space divided by the thread stack size. The point is that in
64bits system this limit isn't a problem anymore. Here comes my questions: which are the negative effects on the memory of using many threads?

3. About resource contention / consumption. Could you please elaborate about that? For instance, are you saying that the are some
locks/monitors mechanisms in the OS that are needed for sharing files, sockets, etc. that may influence performance? Are you also talking about
resource exhaustion?

4. Has the scala lang been designed with the aim (among the others) of minimizing the number of threads in the system?

5. I leave you a couple of pointers where you can find some discussions about the problems related to the inefficient cache usage
in multithreaded systems. Hope it is of interest for you. (www.cs.virginia.edu/~skadron/Papers/meng_dissertation.pdf) (http://www.cs.virginia.edu/~skadron/Papers/meng_iccd09.pdf).

Cheers,

Michele

The more native threads, the more contention and therefore the more risks to suspend/resume threads blocked on a lock, which is an expensive operation. If you run the thread ring benchmark in Go, Scala Actors, Haskell or any other language with user-level threading support you'll see an important performance drop as soon as you use more than one native thread. Moreover, these interactive programming paradigms promote the use of numerous interactive components, no much bigger than regular objects, for which native threading is definitively overkill.

Scala's syntactic sugar for function closures make it easier to program in lightweight event-driven (or continuation-passing) style. There is no other language specific support than that, see
"Scala actors: Unifying thread-based and event-based programming" for an example of creative usage.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.164.7047&rep=rep1&type=pdf

Cheers,
Sébastien

On Tue, May 3, 2011 at 12:59 PM, Daniel Sobral wrote:
> You seem to be equating "native" (ie, provided by the OS) memory and
> thread management with JVM's memory and thread management. They are
> not the same thing -- nor could they be, given the need to guarantee
> the Java memory model over whatever's the CPUs memory model.

HotSpot has always used native threads (green threads were used in
Sun's JDK 1.1 implementation) and that doesn't cause any problems in
terms of the memory model. I am not sure what problems you envisage.
Can you expand on that?

Best,
Ismael

Daniel, could you please elaborate your comment? Are you suggesting
that the JVM's Memory Management system introduce some overhead on the
top the OS' Memory Management system? if so, why does it happen? Is it
because of
the management of the shared heap? What else?

Thank you,
cheers.

Michele

On May 3, 2:08 pm, Sébastien Bocq wrote:
> 2011/5/3 stecca michele
>
>
>
> > Thank you for your replies. I have some additional comments/questions:
>
> > 1. About Thread Context switch overhead. Last days I've been reading a lot
> > of articles about this topic. The problem is that they are usually outdated
> > and in particular their have been usually written before the usage of the
> > optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.
> > Have you got any experience/benchmark about the actual cost of context
> > switch on modern CPUs?
>
> > 2. About Memory Occupation. In the link provided by Chris (
> >http://stackoverflow.com/questions/763579/how-many-threads-can-a-java...)
> > I see a lot of confusion.
> > Each Java Thread allocates a stack whose size is given by the Xss JVM
> > option param. It is important to note that we are talking about VIRTUAL
> > MEMORY thus
> > we have to remember that according to the "demand paging" algorithm (
> >http://en.wikipedia.org/wiki/Demand_paging) physical memory pages are
> > allocated iff the corresponding virtual pages are touched.
> > It means that from a physical memory point of view, we are allocating only
> > the pages that are actually needed.
> > I see only the risk of running out of virtual address space because the
> > maximum number of thread is given by the theoretical
> > limit suggested in (
> >http://stackoverflow.com/questions/763579/how-many-threads-can-a-java...),
> > i.e., the user address space divided by the thread stack size. The point is
> > that in
> > 64bits system this limit isn't a problem anymore. Here comes my questions:
> > which are the negative effects on the memory of using many threads?
>
> > 3. About resource contention / consumption. Could you please elaborate
> > about that? For instance, are you saying that the are some
> > locks/monitors mechanisms in the OS that are needed for sharing files,
> > sockets, etc. that may influence performance? Are you also talking about
> > resource exhaustion?
>
> > 4. Has the scala lang been designed with the aim (among the others) of
> > minimizing the number of threads in the system?
>
> > 5. I leave you a couple of pointers where you can find some discussions
> > about the problems related to the inefficient cache usage
> > in multithreaded systems. Hope it is of interest for you. (
> >www.cs.virginia.edu/~skadron/Papers/meng_dissertation.pdf)
> > (http://www.cs.virginia.edu/~skadron/Papers/meng_iccd09.pdf).
>
> > Cheers,
>
> > Michele
>
> The more native threads, the more contention and therefore the more risks to
> suspend/resume threads blocked on a lock, which is an expensive operation.
> If you run the thread ring benchmark in Go, Scala Actors, Haskell or any
> other language with user-level threading support you'll see an important
> performance drop as soon as you use more than one native thread. Moreover,
> these interactive programming paradigms promote the use of numerous
> interactive components, no much bigger than regular objects, for which
> native threading is definitively overkill.
>
> Scala's syntactic sugar for function closures make it easier to program in
> lightweight event-driven (or continuation-passing) style. There is no other
> language specific support than that, see
> "Scala actors: Unifying thread-based and event-based programming" for an
> example of creative usage.
>
> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.164.7047&rep...
>
> Cheers,
> Sébastien

On Tue, May 3, 2011 at 09:14, Ismael Juma wrote:
> On Tue, May 3, 2011 at 12:59 PM, Daniel Sobral wrote:
>> You seem to be equating "native" (ie, provided by the OS) memory and
>> thread management with JVM's memory and thread management. They are
>> not the same thing -- nor could they be, given the need to guarantee
>> the Java memory model over whatever's the CPUs memory model.
>
> HotSpot has always used native threads (green threads were used in
> Sun's JDK 1.1 implementation) and that doesn't cause any problems in
> terms of the memory model. I am not sure what problems you envisage.
> Can you expand on that?

Using native threads is one thing, but is a Java thread a simple
native thread with absolutely no overhead at all to deal with things
like volatile and synchronized (whether they get used or not), JIT or
plain virtual machine requirements?

Then, there is http://blog.egilh.com/2006/06/2811aspx.html. It doesn't
apply to 64 bits JVM running on Unix, but it is a "nice" example of
the kind of complexity that may come into play. It stumped me and
paulp for a couple of hours, as well (paulp found the link explaining
why decreasing memory let one run more threads).

On Tue, May 3, 2011 at 3:43 PM, Daniel Sobral wrote:
> Using native threads is one thing, but is a Java thread a simple
> native thread with absolutely no overhead at all to deal with things
> like volatile and synchronized (whether they get used or not), JIT or
> plain virtual machine requirements?

If there is overhead, it's negligible relatively speaking. The
overhead for synchronized is at the object-level, not thread-level. I
don't see why volatile would impose any overhead at thread or object
level. In fact, volatile reads are free in the x86 architecture and
writes rely on memory fences or similar operations.

> Then, there is http://blog.egilh.com/2006/06/2811aspx.html. It doesn't
> apply to 64 bits JVM running on Unix, but it is a "nice" example of
> the kind of complexity that may come into play. It stumped me and
> paulp for a couple of hours, as well (paulp found the link explaining
> why decreasing memory let one run more threads).

I don't think this is particularly relevant for places where you'd
want to have thousands of threads. I don't see why you'd use a 32-bit
JVM in that case.

Best,
Ismael

Quote

3. About resource contention / consumption. Could you please elaborate about that? For instance, are you saying that the are some locks/monitors mechanisms in the OS that are needed for sharing files, sockets, etc. that may influence performance? Are you also talking about resource exhaustion?

 

 

Both.

 

Consumption:

 

If you for instance spawn thousands of receiver threads from a socket listener, to process a SOAP request, each of those threads will say try to query a database. You then have to be smart about that and use a pool for DB connections. But that only helps you so much, because each of those threads in progress has already opened a transaction, which is now holding a bunch of other resources and is itself a resource, not to mention the sockets themselves, which are a configurable but limited system resource.

 

Disregard for and mismatching the number and type of resources (in pools or elsewhere) to the number of threads and other instances of stuff will trigger some nasty system deadlocks or freeze..es.

 

Pools of resources now will cause contention in accessing these resources…

 

 

 

Contention:

 

It's about statistics. Assume each thread logs to a file every 5 lines of code. Regardless of the implementation, there will be contention (either to the buffer or flushing to the file), every X concurrent logs. Every contention will cause at least a thread to yield and a core to switch away to another thread.

 

Increasing the number of threads, will increase the number of concurrent logs per second and thus the number of contentions per second and thus increase the time wasted by cores just switching contexts for no good reason, decreasing the overall time the cores spend doing useful stuff.

 

This also explains why increasing blindly the number of cores will not help, in most cases J AND also why it’s a lot better to run 4 JVMs instead of one on a T2000 with 64 cores. While the JVM is a lot smarter about per thread optimization (including thread local heap etc) there are still shared resources causing contention.

 

There is no universal optimal number of threads. It depends a lot on the actual application logic, its architecture, components and technologies.

 

 

 

-----Original Message-----
From: scala-internals@googlegroups.com [mailto:scala-internals@googlegroups.com] On Behalf Of stecca michele
Sent: May-03-11 6:56 AM
To: scala-internals@googlegroups.com
Subject: [scala-internals] RE: Question about Threads

 

Thank you for your replies. I have some additional comments/questions:

 

1. About Thread Context switch overhead. Last days I've been reading a lot of articles about this topic. The problem is that they are usually outdated and in particular their have been usually written before the usage of the optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.

Have you got any experience/benchmark about the actual cost of context switch on modern CPUs?

 

2. About Memory Occupation. In the link provided by Chris (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support) I see a lot of confusion.

Each Java Thread allocates a stack whose size is given by the Xss JVM option param. It is important to note that we are talking about VIRTUAL MEMORY thus we have to remember that according to the "demand paging" algorithm (http://en.wikipedia.org/wiki/Demand_paging) physical memory pages are allocated iff the corresponding virtual pages are touched.

It means that from a physical memory point of view, we are allocating only the pages that are actually needed.

I see only the risk of running out of virtual address space because the maximum number of thread is given by the theoretical limit suggested in (http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support), i.e., the user address space divided by the thread stack size. The point is that in 64bits system this limit isn't a problem anymore. Here comes my questions: which are the negative effects on the memory of using many threads?

 

3. About resource contention / consumption. Could you please elaborate about that? For instance, are you saying that the are some locks/monitors mechanisms in the OS that are needed for sharing files, sockets, etc. that may influence performance? Are you also talking about resource exhaustion?

 

4. Has the scala lang been designed with the aim (among the others) of minimizing the number of threads in the system?

 

5. I leave you a couple of pointers where you can find some discussions about the problems related to the inefficient cache usage in multithreaded systems. Hope it is of interest for you. (www.cs.virginia.edu/~skadron/Papers/meng_dissertation.pdf) (http://www.cs.virginia.edu/~skadron/Papers/meng_iccd09.pdf).

 

Cheers,

 

Michele

Quote

1. About Thread Context switch overhead. Last days I've been reading a lot of articles about this topic. The problem is that they are usually outdated and in particular their have been usually written before the usage of the optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.

Have you got any experience/benchmark about the actual cost of context switch on modern CPUs?

 

No, I don’t. However, switching threads is not just about switching contexts and pipelines, which should be faster now than in the past (???) but also wreaks havoc on caches and cache optimization strategies etc. and the more threads you have switching for no reason, the bigger the havoc caused on the low level caches and pipes and shadow registries and whatever highly organized chunks of silicon you can think of.

 

Whatever you do, don’t log() inside a loop J

 

 

-----Original Message-----
From: scala-internals@googlegroups.com [mailto:scala-internals@googlegroups.com] On Behalf Of steccami
Sent: May-03-11 8:28 AM
To: scala-internals
Subject: [scala-internals] Re: Question about Threads

 

Daniel, could you please elaborate your comment? Are you suggesting that the JVM's Memory Management system introduce some overhead on the top the OS' Memory Management system? if so, why does it happen? Is it because of the management of the shared heap? What else?

 

Thank you,

  cheers.

 

Michele

 

On May 3, 2:08 pm, Sébastien Bocq <sebastien.b...@gmail.com> wrote:

> 2011/5/3 stecca michele <mste...@yahoo.it>

> 

> 

> 

> > Thank you for your replies. I have some additional comments/questions:

> 

> > 1. About Thread Context switch overhead. Last days I've been reading

> > a lot of articles about this topic. The problem is that they are

> > usually outdated and in particular their have been usually written

> > before the usage of the optimized NPTL (Native POSIX Thread Library) library in the Linux Kernel.

> > Have you got any experience/benchmark about the actual cost of

> > context switch on modern CPUs?

> 

> > 2. About Memory Occupation. In the link provided by Chris (

> >http://stackoverflow.com/questions/763579/how-many-threads-can-a-java

> >...)

> > I see a lot of confusion.

> > Each Java Thread allocates a stack whose size is given by the Xss

> >JVM  option param. It is important to note that we are talking about

> >VIRTUAL  MEMORY thus  we have to remember that according to the

> >"demand paging" algorithm (

> >http://en.wikipedia.org/wiki/Demand_paging) physical memory pages are 

> >allocated iff the corresponding virtual pages are touched.

> > It means that from a physical memory point of view, we are

> >allocating only  the pages that are actually needed.

> > I see only the risk of running out of virtual address space because

> >the  maximum number of thread is given by the theoretical  limit

> >suggested in (

> >http://stackoverflow.com/questions/763579/how-many-threads-can-a-java

> >...),  i.e., the user address space divided by the thread stack size.

> >The point is  that in  64bits system this limit isn't a problem

> >anymore. Here comes my questions:

> > which are the negative effects on the memory of using many threads?

> 

> > 3. About resource contention / consumption. Could you please

> > elaborate about that? For instance, are you saying that the are some

> > locks/monitors mechanisms in the OS that are needed for sharing

> > files, sockets, etc. that may influence performance? Are you also

> > talking about resource exhaustion?

> 

> > 4. Has the scala lang been designed with the aim (among the others)

> > of minimizing the number of threads in the system?

> 

> > 5. I leave you a couple of pointers where you can find some

> >discussions  about the problems related to the inefficient cache

> >usage  in multithreaded systems. Hope it is of interest for you. (

> >www.cs.virginia.edu/~skadron/Papers/meng_dissertation.pdf<http://www.

> >cs.virginia.edu/%7Eskadron/Papers/meng_dissertation.pdf>)

> > (http://www.cs.virginia.edu/~skadron/Papers/meng_iccd09.pdf).

> 

> > Cheers,

> 

> > Michele

> 

> The more native threads, the more contention and therefore the more

> risks to suspend/resume threads blocked on a lock, which is an expensive operation.

> If you run the thread ring benchmark in Go, Scala Actors, Haskell or

> any other language with user-level threading support you'll see an

> important performance drop as soon as you use more than one native

> thread. Moreover, these interactive programming paradigms promote the

> use of numerous interactive components, no much bigger than regular

> objects, for which native threading is definitively overkill.

> 

> Scala's syntactic sugar for function closures make it easier to

> program in lightweight event-driven (or continuation-passing) style.

> There is no other language specific support than that, see "Scala

> actors: Unifying thread-based and event-based programming" for an

> example of creative usage.

> 

> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.164.7047&rep...

> 

> Cheers,

> Sébastien

Thank you Razvan. In the past I tried to perform some evaluation tests
in order to discover the performance issues related to a cache
inefficient usage but I wasn't able to discover any problem. By the
way, thank you very much for replies.
Just one last question. I'm currently investigating the threadless-
approach allowed by the usage of the react primitive (e.g., like in
the PingPong example). Was the react primitive introduced to reduce
the number of threads in the system? Which is the difference between
the usage of "react" and the "continuation" (i.e., the "shift"
primitive)?
Thank you.
Cheers,
Michele

Be very careful with micro performance benchmarks, they lie more often than
not! It depends a lot on what the threads actually do. In an "average" J2EE
app, you can bet they move/use a lot of unrelated objects/data :)

Sorry - what "react" is this? I don't think it's a scala keyword but an
example of its flexibility :) In what library? Actors?

I would guess it's a matter for the Dispatcher but it could just loop
through a pile of messages in the same thread rather than switching threads
between "reacts", creating that efficiency we were talking about...

cheers

-----Original Message-----
From: scala-internals@googlegroups.com
[mailto:scala-internals@googlegroups.com] On Behalf Of steccami
Sent: May-04-11 8:39 AM
To: scala-internals
Subject: [scala-internals] Re: Question about Threads

Thank you Razvan. In the past I tried to perform some evaluation tests in
order to discover the performance issues related to a cache inefficient
usage but I wasn't able to discover any problem. By the way, thank you very
much for replies.
Just one last question. I'm currently investigating the threadless- approach
allowed by the usage of the react primitive (e.g., like in the PingPong
example). Was the react primitive introduced to reduce the number of threads
in the system? Which is the difference between the usage of "react" and the
"continuation" (i.e., the "shift"
primitive)?
Thank you.
Cheers,
Michele

can we move this discussion over to scala-user or scala-debate?
- Tiark

On May 4, 2011, at 6:09 PM, Razvan Cojocaru wrote:

> Be very careful with micro performance benchmarks, they lie more often than
> not! It depends a lot on what the threads actually do. In an "average" J2EE
> app, you can bet they move/use a lot of unrelated objects/data :)
>
>
> Sorry - what "react" is this? I don't think it's a scala keyword but an
> example of its flexibility :) In what library? Actors?
>
> I would guess it's a matter for the Dispatcher but it could just loop
> through a pile of messages in the same thread rather than switching threads
> between "reacts", creating that efficiency we were talking about...
>
> cheers
>
> -----Original Message-----
> From: scala-internals@googlegroups.com
> [mailto:scala-internals@googlegroups.com] On Behalf Of steccami
> Sent: May-04-11 8:39 AM
> To: scala-internals
> Subject: [scala-internals] Re: Question about Threads
>
> Thank you Razvan. In the past I tried to perform some evaluation tests in
> order to discover the performance issues related to a cache inefficient
> usage but I wasn't able to discover any problem. By the way, thank you very
> much for replies.
> Just one last question. I'm currently investigating the threadless- approach
> allowed by the usage of the react primitive (e.g., like in the PingPong
> example). Was the react primitive introduced to reduce the number of threads
> in the system? Which is the difference between the usage of "react" and the
> "continuation" (i.e., the "shift"
> primitive)?
> Thank you.
> Cheers,
> Michele
>