Hating myself for a bit of imperative code :)

first idea: "yourstring".view.zipWithIndex.flatMap(returnReplacementOrReturnTheCharItself).mkString

second idea, since the first one might become rather memory consuming and/or slow:
val target = new StringBuilder
"yourstring" view zipWithIndex foreach (e => target.append(returnReplacementOrReturnTheCharItself(e))

not functional, but "functional enough", i'd say.



Am 24.06.2011 16:29, schrieb Jon Steelman:

BANLkTikskBQ70FPuw+x+fpeXawZ75dDgdg [at] mail [dot] gmail [dot] com" type="cite"> Dear Scala Functionalists,
Can this imperative Scala code readily and clearly be written purely functionally or very close to purely? This code surrounds matches in a source string with some before/after strings. The match is case insensitive but the replacement preserves source case. I'm doing simple bits in Scala functionally and feel like this could have been easier to write functionally, but doing this one functionally is not yet quite so obvious to me.
Thanks! Jon
  // Matches in source are case insensitive but preserving case of source matches   def surroundMatches(find: String, beforeFind: String, afterFind: String, source: String) : String = {     if (find == null || beforeFind == null || afterFind == null || source==null) return null     val sourceLc = source.toLowerCase     val findLc = find.toLowerCase
    val sb = new StringBuilder(source)
    var i = source.length     while (i > 0) {       i = sourceLc.lastIndexOf(findLc,i)       if (i >= 0) {         sb.replace(i, i+find.length, beforeFind + source.substring(i, i+find.length) + afterFind)       }       i = i - 1     }     sb.toString()   }

.view and zipWithIndex combined are rather inefficient, at least in 2.8.1 and earlier.   Might as well just do "yourString".zipWithIndex.view.flatMap(returnReplacementOrReturnTheCharItself).mkString

On Fri, Jun 24, 2011 at 10:40 AM, HamsterofDeath <h-star@gmx.de> wrote:

first idea: "yourstring".view.zipWithIndex.flatMap(returnReplacementOrReturnTheCharItself).mkString

second idea, since the first one might become rather memory consuming and/or slow:
val target = new StringBuilder
"yourstring" view zipWithIndex foreach (e => target.append(returnReplacementOrReturnTheCharItself(e))

not functional, but "functional enough", i'd say.



Am 24.06.2011 16:29, schrieb Jon Steelman:

Dear Scala Functionalists,
Can this imperative Scala code readily and clearly be written purely functionally or very close to purely? This code surrounds matches in a source string with some before/after strings. The match is case insensitive but the replacement preserves source case. I'm doing simple bits in Scala functionally and feel like this could have been easier to write functionally, but doing this one functionally is not yet quite so obvious to me.
Thanks! Jon
  // Matches in source are case insensitive but preserving case of source matches   def surroundMatches(find: String, beforeFind: String, afterFind: String, source: String) : String = {     if (find == null || beforeFind == null || afterFind == null || source==null) return null     val sourceLc = source.toLowerCase     val findLc = find.toLowerCase
    val sb = new StringBuilder(source)
    var i = source.length     while (i > 0) {       i = sourceLc.lastIndexOf(findLc,i)       if (i >= 0) {         sb.replace(i, i+find.length, beforeFind + source.substring(i, i+find.length) + afterFind)       }       i = i - 1     }     sb.toString()   }

source.replaceAll(find ,beforeFind + find + afterFind)

On Fri, Jun 24, 2011 at 3:29 PM, Jon Steelman wrote:
> Dear Scala Functionalists,
> Can this imperative Scala code readily and clearly be written purely
> functionally or very close to purely? This code surrounds matches in a
> source string with some before/after strings. The match is case insensitive
> but the replacement preserves source case. I'm doing simple bits in Scala
> functionally and feel like this could have been easier to write
> functionally, but doing this one functionally is not yet quite so obvious to
> me.
> Thanks!
> Jon
>   // Matches in source are case insensitive but preserving case of source
> matches
>   def surroundMatches(find: String, beforeFind: String, afterFind: String,
> source: String) : String = {
>     if (find == null || beforeFind == null || afterFind == null ||
> source==null) return null
>     val sourceLc = source.toLowerCase
>     val findLc = find.toLowerCase
>     val sb = new StringBuilder(source)
>     var i = source.length
>     while (i > 0) {
>       i = sourceLc.lastIndexOf(findLc,i)
>       if (i >= 0) {
>         sb.replace(i, i+find.length, beforeFind + source.substring(i,
> i+find.length) + afterFind)
>       }
>       i = i - 1
>     }
>     sb.toString()
>   }

exactly. that's why i waited until a second and third idea popped up

Am 24.06.2011 16:43, schrieb Josh Suereth:

BANLkTimgQvJg1rcM94rXvnhhrOu2QLSGaYsxptoL_c5smzOicg [at] mail [dot] gmail [dot] com" type="cite">.view and zipWithIndex combined are rather inefficient, at least in 2.8.1 and earlier.   Might as well just do "yourString".zipWithIndex.view.flatMap(returnReplacementOrReturnTheCharItself).mkString

On Fri, Jun 24, 2011 at 10:40 AM, HamsterofDeath <h-star [at] gmx [dot] de" rel="nofollow">h-star@gmx.de> wrote:

first idea: "yourstring".view.zipWithIndex.flatMap(returnReplacementOrReturnTheCharItself).mkString

second idea, since the first one might become rather memory consuming and/or slow:
val target = new StringBuilder
"yourstring" view zipWithIndex foreach (e => target.append(returnReplacementOrReturnTheCharItself(e))

not functional, but "functional enough", i'd say.



Am 24.06.2011 16:29, schrieb Jon Steelman:

Dear Scala Functionalists,
Can this imperative Scala code readily and clearly be written purely functionally or very close to purely? This code surrounds matches in a source string with some before/after strings. The match is case insensitive but the replacement preserves source case. I'm doing simple bits in Scala functionally and feel like this could have been easier to write functionally, but doing this one functionally is not yet quite so obvious to me.
Thanks! Jon
  // Matches in source are case insensitive but preserving case of source matches   def surroundMatches(find: String, beforeFind: String, afterFind: String, source: String) : String = {     if (find == null || beforeFind == null || afterFind == null || source==null) return null     val sourceLc = source.toLowerCase     val findLc = find.toLowerCase
    val sb = new StringBuilder(source)
    var i = source.length     while (i > 0) {       i = sourceLc.lastIndexOf(findLc,i)       if (i >= 0) {         sb.replace(i, i+find.length, beforeFind + source.substring(i, i+find.length) + afterFind)       }       i = i - 1     }     sb.toString()   }

Maybe
("(?i)" + find).r.replaceAllIn(source, beforeFind + find + afterFind)
Just be sure to not include any regex special chars in find.

On Jun 24, 7:29 am, Jon Steelman wrote:
> Dear Scala Functionalists,
>
> Can this imperative Scala code readily and clearly be written purely
> functionally or very close to purely? This code surrounds matches in a
> source string with some before/after strings. The match is case insensitive
> but the replacement preserves source case. I'm doing simple bits in Scala
> functionally and feel like this could have been easier to write
> functionally, but doing this one functionally is not yet quite so obvious to
> me.
>
> Thanks!
> Jon
>
>   // Matches in source are case insensitive but preserving case of source
> matches
>   def surroundMatches(find: String, beforeFind: String, afterFind: String,
> source: String) : String = {
>     if (find == null || beforeFind == null || afterFind == null ||
> source==null) return null
>     val sourceLc = source.toLowerCase
>     val findLc = find.toLowerCase
>
>     val sb = new StringBuilder(source)
>
>     var i = source.length
>     while (i > 0) {
>       i = sourceLc.lastIndexOf(findLc,i)
>       if (i >= 0) {
>         sb.replace(i, i+find.length, beforeFind + source.substring(i,
> i+find.length) + afterFind)
>       }
>       i = i - 1
>     }
>     sb.toString()
>   }

On Fri, Jun 24, 2011 at 7:44 PM, Joshua Gooding wrote:
> Maybe
> ("(?i)" + find).r.replaceAllIn(source, beforeFind + find + afterFind)
> Just be sure to not include any regex special chars in find.

\Q and \E [1] are your friends here.

-jason

[1] http://download.oracle.com/javase/1.4.2/docs/api/java/util/regex/Pattern...

An issue with the several fleshed out proposals is that they don't preserve source's string case on the replace. In other words, we don't want the surrounded find String to be placed literally into the new source as that woud clobber the case we want to preserve from source. (If going with the regex approach, it is almost like we would need capture group back references that are dynamic.)
Thanks,Jon

On Fri, Jun 24, 2011 at 1:44 PM, Joshua Gooding <skavookie@gmail.com> wrote:

Maybe
("(?i)" + find).r.replaceAllIn(source, beforeFind + find + afterFind)
Just be sure to not include any regex special chars in find.

On Jun 24, 7:29 am, Jon Steelman <jon.steel...@gmail.com> wrote:
> Dear Scala Functionalists,
>
> Can this imperative Scala code readily and clearly be written purely
> functionally or very close to purely? This code surrounds matches in a
> source string with some before/after strings. The match is case insensitive
> but the replacement preserves source case. I'm doing simple bits in Scala
> functionally and feel like this could have been easier to write
> functionally, but doing this one functionally is not yet quite so obvious to
> me.
>
> Thanks!
> Jon
>
>   // Matches in source are case insensitive but preserving case of source
> matches
>   def surroundMatches(find: String, beforeFind: String, afterFind: String,
> source: String) : String = {
>     if (find == null || beforeFind == null || afterFind == null ||
> source==null) return null
>     val sourceLc = source.toLowerCase
>     val findLc = find.toLowerCase
>
>     val sb = new StringBuilder(source)
>
>     var i = source.length
>     while (i > 0) {
>       i = sourceLc.lastIndexOf(findLc,i)
>       if (i >= 0) {
>         sb.replace(i, i+find.length, beforeFind + source.substring(i,
> i+find.length) + afterFind)
>       }
>       i = i - 1
>     }
>     sb.toString()
>   }

On Fri, Jun 24, 2011 at 11:29, Jon Steelman wrote:
> Dear Scala Functionalists,
> Can this imperative Scala code readily and clearly be written purely
> functionally or very close to purely? This code surrounds matches in a
> source string with some before/after strings. The match is case insensitive
> but the replacement preserves source case. I'm doing simple bits in Scala
> functionally and feel like this could have been easier to write
> functionally, but doing this one functionally is not yet quite so obvious to
> me.
> Thanks!
> Jon
>   // Matches in source are case insensitive but preserving case of source
> matches
>   def surroundMatches(find: String, beforeFind: String, afterFind: String,
> source: String) : String = {

import scala.util.matching.Regex.Groups

def surroundMatches(find: String, beforeFind: String, afterFind:
String, source: String): String = {
val pattern = ("""(?i)(\Q%s\E)""" format find).r
pattern replaceAllIn (source, x => x match {
case Groups(found) => beforeFind + found + afterFind
})
}

Append these three strings doesn't have a good performance -- if
Replace accepted a CharSequence instead of String one could maybe do
better, but it uses Matcher's appendReplacement which only takes
String underneath.

You could also use a findAllIn and then get the MatchIterator, and
reimplement the whole logic behind replaceAllIn, but appending three
times instead of just once. In fact, you might want to look at how
that's implemented. Not particularly functional -- it uses string
buffers and iterators, after all -- but...

Anyway, note the use of \Q and \E to match a plain string (well, one
that doesn't include \E) inside a regex pattern, parenthesis around it
to mark an extraction group, and (?i) to do a case insensitive match.

Why use scala.util.matching.Regex.Groups?
This would do as well I suppose:

def surroundMatches(find: String, beforeFind: String, afterFind:
String, source: String): String = {
val pattern = ("""(?i)(\Q%s\E)""" format find).r
pattern replaceAllIn (source, (beforeFind + _ + afterFind))
}

On Mon, Jun 27, 2011 at 10:59, Jan van der Vorst wrote:
> Why use scala.util.matching.Regex.Groups?
> This would do as well I suppose:
>
>  def surroundMatches(find: String, beforeFind: String, afterFind:
> String, source: String): String = {
>    val pattern = ("""(?i)(\Q%s\E)""" format find).r
>    pattern replaceAllIn (source, (beforeFind + _ + afterFind))
>  }

If the toString of Match returns the right thing, then indeed!

Dear Scala User,
I have a numerically intensive code in Java/Scala which makes large usage of array. I am thinking about wrapping the array around a class which provides easier access to its elements, but I would need more understanding of which performance faults I would pay.

In my use case, an array typically represent a curve, where the y is given by the value of each point, while the x is basically represented by the index * a interval.

For example, if my interval is = 3 months. I can have an array like that: (please accept the java notation:) )

Array[0] = 0.5 => 0.5 is the y value for time 0
Array[1] = 0.6 => 0.6 is the y value for time 3 months
Array[2] = 0.7 => 0.7 is the y value for time 6 months
and so on...

So at the certain moment, if I want to access the value at X months I should do something like Array[X/Interval]. I end up therefore to bring around my Interval value in my code, and this lead of course to indexing errors :)

So I thought as defining a class like the following

class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
  def get(val time:Int) = array(time/interval)
}

However, this would transform direct access to array element which I do in my code into a method call ...which performance impact may I expect?

Thank you very much for your help
Best Regards

Edmondo Porcu

JVM inlines this kind of access, so there will be no penalty. You should make your wrapper class final or at least make your fields and the access methods final. Also, consider using the apply() method, then your wrapper can be used like a regular sequence. In fact, you should extend one of the Scala sequence interfaces.

import scala.collection._
class FixedTimeArray[T <: AnyVal](final val array: Array[T], final val interval: Int)extends mutable.IndexedSeq[T] with mutable.IndexedSeqOptimized[T, mutable.IndexedSeq[T]] {   final def length = array.length  final def apply(time: Int) = array(time*interval)  final def update(time: Int, value: T) { array(time*interval) = value }}

On Wed, Sep 28, 2011 at 2:30 AM, Edmondo Porcu <edmondo.porcu@gmail.com> wrote:

Dear Scala User,
I have a numerically intensive code in Java/Scala which makes large usage of array. I am thinking about wrapping the array around a class which provides easier access to its elements, but I would need more understanding of which performance faults I would pay.

In my use case, an array typically represent a curve, where the y is given by the value of each point, while the x is basically represented by the index * a interval.

For example, if my interval is = 3 months. I can have an array like that: (please accept the java notation:) )

Array[0] = 0.5 => 0.5 is the y value for time 0
Array[1] = 0.6 => 0.6 is the y value for time 3 months
Array[2] = 0.7 => 0.7 is the y value for time 6 months
and so on...

So at the certain moment, if I want to access the value at X months I should do something like Array[X/Interval]. I end up therefore to bring around my Interval value in my code, and this lead of course to indexing errors :)

So I thought as defining a class like the following

class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
  def get(val time:Int) = array(time/interval)
}

However, this would transform direct access to array element which I do in my code into a method call ...which performance impact may I expect?

Thank you very much for your help
Best Regards

Edmondo Porcu

The JVM will inline your method calls regardless of the methods/class
being final, at least Hotspot server will (and I guess client too).

But what will hurt your performance is the boxing and unboxing that
will happen with the values that you get from the array, because of
your use of generics. Either try using specialization or write
separate wrapper classes for all the types of arrays you use. The
latter is more work but probably also more reliable.

But actually, I would recommend something else entirely. First,
implement you proposed changes and benchmark with real code, what the
performance impact is. Only if the performance penalty is really too
big, begin optimizing.

From my experience performance problems are very often caused by
boxing and unboxing, so when optimizing, I recommend to first look for
these.

Regards,
Rüdiger

2011/9/28 Aleksey Nikiforov :
> JVM inlines this kind of access, so there will be no penalty. You should
> make your wrapper class final or at least make your fields and the access
> methods final. Also, consider using the apply() method, then your wrapper
> can be used like a regular sequence. In fact, you should extend one of the
> Scala sequence interfaces.
>
> import scala.collection._
> class FixedTimeArray[T <: AnyVal](final val array: Array[T], final val
> interval: Int)
> extends mutable.IndexedSeq[T] with mutable.IndexedSeqOptimized[T,
> mutable.IndexedSeq[T]] {
>   final def length = array.length
>   final def apply(time: Int) = array(time*interval)
>   final def update(time: Int, value: T) { array(time*interval) = value }
> }
>
> On Wed, Sep 28, 2011 at 2:30 AM, Edmondo Porcu
> wrote:
>>
>> Dear Scala User,
>> I have a numerically intensive code in Java/Scala which makes large usage
>> of array. I am thinking about wrapping the array around a class which
>> provides easier access to its elements, but I would need more understanding
>> of which performance faults I would pay.
>>
>> In my use case, an array typically represent a curve, where the y is given
>> by the value of each point, while the x is basically represented by the
>> index * a interval.
>>
>> For example, if my interval is = 3 months. I can have an array like that:
>> (please accept the java notation:) )
>>
>> Array[0] = 0.5 => 0.5 is the y value for time 0
>> Array[1] = 0.6 => 0.6 is the y value for time 3 months
>> Array[2] = 0.7 => 0.7 is the y value for time 6 months
>> and so on...
>>
>> So at the certain moment, if I want to access the value at X months I
>> should do something like Array[X/Interval]. I end up therefore to bring
>> around my Interval value in my code, and this lead of course to indexing
>> errors :)
>>
>> So I thought as defining a class like the following
>>
>> class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
>>   def get(val time:Int) = array(time/interval)
>> }
>>
>> However, this would transform direct access to array element which I do in
>> my code into a method call ...which performance impact may I expect?
>>
>> Thank you very much for your help
>> Best Regards
>>
>> Edmondo Porcu
>>
>
>

I was misinformed about final for a long time because of the general misconception surrounding it. When I did my own benchmarks, final helped.


On Wed, Sep 28, 2011 at 7:22 AM, Ruediger Keller <ruediger.keller@rk42.de> wrote:

The JVM will inline your method calls regardless of the methods/class
being final, at least Hotspot server will (and I guess client too).

But what will hurt your performance is the boxing and unboxing that
will happen with the values that you get from the array, because of
your use of generics. Either try using specialization or write
separate wrapper classes for all the types of arrays you use. The
latter is more work but probably also more reliable.

But actually, I would recommend something else entirely. First,
implement you proposed changes and benchmark with real code, what the
performance impact is. Only if the performance penalty is really too
big, begin optimizing.

From my experience performance problems are very often caused by
boxing and unboxing, so when optimizing, I recommend to first look for
these.

Regards,
Rüdiger


2011/9/28 Aleksey Nikiforov <lexn82@gmail.com>:
> JVM inlines this kind of access, so there will be no penalty. You should
> make your wrapper class final or at least make your fields and the access
> methods final. Also, consider using the apply() method, then your wrapper
> can be used like a regular sequence. In fact, you should extend one of the
> Scala sequence interfaces.
>
> import scala.collection._
> class FixedTimeArray[T <: AnyVal](final val array: Array[T], final val
> interval: Int)
> extends mutable.IndexedSeq[T] with mutable.IndexedSeqOptimized[T,
> mutable.IndexedSeq[T]] {
>   final def length = array.length
>   final def apply(time: Int) = array(time*interval)
>   final def update(time: Int, value: T) { array(time*interval) = value }
> }
>
> On Wed, Sep 28, 2011 at 2:30 AM, Edmondo Porcu <edmondo.porcu@gmail.com>
> wrote:
>>
>> Dear Scala User,
>> I have a numerically intensive code in Java/Scala which makes large usage
>> of array. I am thinking about wrapping the array around a class which
>> provides easier access to its elements, but I would need more understanding
>> of which performance faults I would pay.
>>
>> In my use case, an array typically represent a curve, where the y is given
>> by the value of each point, while the x is basically represented by the
>> index * a interval.
>>
>> For example, if my interval is = 3 months. I can have an array like that:
>> (please accept the java notation:) )
>>
>> Array[0] = 0.5 => 0.5 is the y value for time 0
>> Array[1] = 0.6 => 0.6 is the y value for time 3 months
>> Array[2] = 0.7 => 0.7 is the y value for time 6 months
>> and so on...
>>
>> So at the certain moment, if I want to access the value at X months I
>> should do something like Array[X/Interval]. I end up therefore to bring
>> around my Interval value in my code, and this lead of course to indexing
>> errors :)
>>
>> So I thought as defining a class like the following
>>
>> class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
>>   def get(val time:Int) = array(time/interval)
>> }
>>
>> However, this would transform direct access to array element which I do in
>> my code into a method call ...which performance impact may I expect?
>>
>> Thank you very much for your help
>> Best Regards
>>
>> Edmondo Porcu
>>
>
>

I have been relying on hotspot until i found that it would sometimes refuse to inline methods in my benchmarks (non-static non-final instance methods). And I am talking methods that were run million times per second in a while loop.
When it comes to method overriding in subclasses, hotspot tends to deoptimize everything, just to be sure. So final really helps there.
From what I have gathered many hotspot optimizations are not applied consistenly. When you start you application, you may or may not get them. Using final results in a more consistent behavior. Also it will help hotspot to optimize your code faster, reducing the warmup time. Of course the last two are empirical findings.

On Wed, Sep 28, 2011 at 7:45 AM, Ruediger Keller <ruediger.keller@rk42.de> wrote:

It has been said for a long time that Hotspot will do optimistic
inlining even if the class/method is not final. And I have often
observed this.

Only if a class is loaded that actually overrides the method in
question, Hotspot will deoptimize the overly optimistic code. I never
found contrary evidence in my benchmarks. Which doesn't mean that such
cases can't exist, but I never stumbled over them, if they do.

Regards,
Rüdiger

Okay, it seems your experience in this regard is broader than mine. I
will look out for this in the future.

Regards,
Rüdiger

2011/9/28 Aleksey Nikiforov :
> I have been relying on hotspot until i found that it would sometimes refuse
> to inline methods in my benchmarks (non-static non-final instance methods).
> And I am talking methods that were run million times per second in a while
> loop.
> When it comes to method overriding in subclasses, hotspot tends to
> deoptimize everything, just to be sure. So final really helps there.
> From what I have gathered many hotspot optimizations are not applied
> consistenly. When you start you application, you may or may not get them.
> Using final results in a more consistent behavior. Also it will help hotspot
> to optimize your code faster, reducing the warmup time. Of course the last
> two are empirical findings.
>
> On Wed, Sep 28, 2011 at 7:45 AM, Ruediger Keller
> wrote:
>>
>> It has been said for a long time that Hotspot will do optimistic
>> inlining even if the class/method is not final. And I have often
>> observed this.
>>
>> Only if a class is loaded that actually overrides the method in
>> question, Hotspot will deoptimize the overly optimistic code. I never
>> found contrary evidence in my benchmarks. Which doesn't mean that such
>> cases can't exist, but I never stumbled over them, if they do.
>>
>> Regards,
>> Rüdiger
>>
>

If you are certain that you need this to be as fast as possible, you will want to either make the code non-generic:

  class FixedTimeFloats(val array: Array[Float], val interval: Float) {
    def apply(t: Time) = array(t/interval)
  }

or you will want to make the code specialized, with your own implementation of a specialized version of Numeric (too long to include here).

If you do this, you'll (almost surely) get full performance.  Otherwise, you'll waste a lot of time unboxing; generic code has to wrap each value in an object for it to work (unless you use specialization).

  --Rex

On Wed, Sep 28, 2011 at 3:30 AM, Edmondo Porcu <edmondo.porcu@gmail.com> wrote:

Dear Scala User,
I have a numerically intensive code in Java/Scala which makes large usage of array. I am thinking about wrapping the array around a class which provides easier access to its elements, but I would need more understanding of which performance faults I would pay.

In my use case, an array typically represent a curve, where the y is given by the value of each point, while the x is basically represented by the index * a interval.

For example, if my interval is = 3 months. I can have an array like that: (please accept the java notation:) )

Array[0] = 0.5 => 0.5 is the y value for time 0
Array[1] = 0.6 => 0.6 is the y value for time 3 months
Array[2] = 0.7 => 0.7 is the y value for time 6 months
and so on...

So at the certain moment, if I want to access the value at X months I should do something like Array[X/Interval]. I end up therefore to bring around my Interval value in my code, and this lead of course to indexing errors :)

So I thought as defining a class like the following

class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
  def get(val time:Int) = array(time/interval)
}

However, this would transform direct access to array element which I do in my code into a method call ...which performance impact may I expect?

Thank you very much for your help
Best Regards

Edmondo Porcu

On Wed, Sep 28, 2011 at 3:15 PM, Rex Kerr <ichoran@gmail.com> wrote:

If you are certain that you need this to be as fast as possible, you will want to either make the code non-generic:

  class FixedTimeFloats(val array: Array[Float], val interval: Float) {
    def apply(t: Time) = array(t/interval)
  }

Function1 is specialized in Float:

(t: Time) => array(t/interval) //closing over array and interval
 

or you will want to make the code specialized, with your own implementation of a specialized version of Numeric (too long to include here).

If you do this, you'll (almost surely) get full performance.  Otherwise, you'll waste a lot of time unboxing; generic code has to wrap each value in an object for it to work (unless you use specialization).

  --Rex

On Wed, Sep 28, 2011 at 3:30 AM, Edmondo Porcu <edmondo.porcu@gmail.com> wrote:

Dear Scala User,
I have a numerically intensive code in Java/Scala which makes large usage of array. I am thinking about wrapping the array around a class which provides easier access to its elements, but I would need more understanding of which performance faults I would pay.

In my use case, an array typically represent a curve, where the y is given by the value of each point, while the x is basically represented by the index * a interval.

For example, if my interval is = 3 months. I can have an array like that: (please accept the java notation:) )

Array[0] = 0.5 => 0.5 is the y value for time 0
Array[1] = 0.6 => 0.6 is the y value for time 3 months
Array[2] = 0.7 => 0.7 is the y value for time 6 months
and so on...

So at the certain moment, if I want to access the value at X months I should do something like Array[X/Interval]. I end up therefore to bring around my Interval value in my code, and this lead of course to indexing errors :)

So I thought as defining a class like the following

class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val interval:Int){
  def get(val time:Int) = array(time/interval)
}

However, this would transform direct access to array element which I do in my code into a method call ...which performance impact may I expect?

Thank you very much for your help
Best Regards

Edmondo Porcu

--
Viktor Klang

Akka Tech LeadTypesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

Of course, by looking out for final, I have missed that the original code could benefit from specialization, like you have pointed out. So look out for looking out for final ;)

On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller <ruediger.keller@rk42.de> wrote:

Okay, it seems your experience in this regard is broader than mine. I
will look out for this in the future.

Regards,
Rüdiger

Dear all,
what about scala MAPS and primitive types? Are they efficient or one should worry about autoboxing?


Best Regards

2011/9/28 Aleksey Nikiforov <lexn82@gmail.com>

Of course, by looking out for final, I have missed that the original code could benefit from specialization, like you have pointed out. So look out for looking out for final ;)

On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller <ruediger.keller@rk42.de> wrote:

Okay, it seems your experience in this regard is broader than mine. I
will look out for this in the future.

Regards,
Rüdiger

It has been said for a long time that Hotspot will do optimistic
inlining even if the class/method is not final. And I have often
observed this.

Only if a class is loaded that actually overrides the method in
question, Hotspot will deoptimize the overly optimistic code. I never
found contrary evidence in my benchmarks. Which doesn't mean that such
cases can't exist, but I never stumbled over them, if they do.

Regards,
Rüdiger

2011/9/28 Aleksey Nikiforov :
> I was misinformed about final for a long time because of the general
> misconception surrounding it. When I did my own benchmarks, final helped.
>
>
> On Wed, Sep 28, 2011 at 7:22 AM, Ruediger Keller
> wrote:
>>
>> The JVM will inline your method calls regardless of the methods/class
>> being final, at least Hotspot server will (and I guess client too).
>>
>> But what will hurt your performance is the boxing and unboxing that
>> will happen with the values that you get from the array, because of
>> your use of generics. Either try using specialization or write
>> separate wrapper classes for all the types of arrays you use. The
>> latter is more work but probably also more reliable.
>>
>> But actually, I would recommend something else entirely. First,
>> implement you proposed changes and benchmark with real code, what the
>> performance impact is. Only if the performance penalty is really too
>> big, begin optimizing.
>>
>> From my experience performance problems are very often caused by
>> boxing and unboxing, so when optimizing, I recommend to first look for
>> these.
>>
>> Regards,
>> Rüdiger
>>
>>
>> 2011/9/28 Aleksey Nikiforov :
>> > JVM inlines this kind of access, so there will be no penalty. You should
>> > make your wrapper class final or at least make your fields and the
>> > access
>> > methods final. Also, consider using the apply() method, then your
>> > wrapper
>> > can be used like a regular sequence. In fact, you should extend one of
>> > the
>> > Scala sequence interfaces.
>> >
>> > import scala.collection._
>> > class FixedTimeArray[T <: AnyVal](final val array: Array[T], final val
>> > interval: Int)
>> > extends mutable.IndexedSeq[T] with mutable.IndexedSeqOptimized[T,
>> > mutable.IndexedSeq[T]] {
>> >   final def length = array.length
>> >   final def apply(time: Int) = array(time*interval)
>> >   final def update(time: Int, value: T) { array(time*interval) = value }
>> > }
>> >
>> > On Wed, Sep 28, 2011 at 2:30 AM, Edmondo Porcu
>> > wrote:
>> >>
>> >> Dear Scala User,
>> >> I have a numerically intensive code in Java/Scala which makes large
>> >> usage
>> >> of array. I am thinking about wrapping the array around a class which
>> >> provides easier access to its elements, but I would need more
>> >> understanding
>> >> of which performance faults I would pay.
>> >>
>> >> In my use case, an array typically represent a curve, where the y is
>> >> given
>> >> by the value of each point, while the x is basically represented by the
>> >> index * a interval.
>> >>
>> >> For example, if my interval is = 3 months. I can have an array like
>> >> that:
>> >> (please accept the java notation:) )
>> >>
>> >> Array[0] = 0.5 => 0.5 is the y value for time 0
>> >> Array[1] = 0.6 => 0.6 is the y value for time 3 months
>> >> Array[2] = 0.7 => 0.7 is the y value for time 6 months
>> >> and so on...
>> >>
>> >> So at the certain moment, if I want to access the value at X months I
>> >> should do something like Array[X/Interval]. I end up therefore to bring
>> >> around my Interval value in my code, and this lead of course to
>> >> indexing
>> >> errors :)
>> >>
>> >> So I thought as defining a class like the following
>> >>
>> >> class FixedTimeArray[T<:AnyVal] ( val array:Array[T], val
>> >> interval:Int){
>> >>   def get(val time:Int) = array(time/interval)
>> >> }
>> >>
>> >> However, this would transform direct access to array element which I do
>> >> in
>> >> my code into a method call ...which performance impact may I expect?
>> >>
>> >> Thank you very much for your help
>> >> Best Regards
>> >>
>> >> Edmondo Porcu
>> >>
>> >
>> >
>
>

Scala's maps in the standard library are not specialized and will
always box/unbox primitives.

Regards,
Rüdiger

2011/9/28 Edmondo Porcu :
> Dear all,
> what about scala MAPS and primitive types? Are they efficient or one should
> worry about autoboxing?
>
>
> Best Regards
>
> 2011/9/28 Aleksey Nikiforov
>>
>> Of course, by looking out for final, I have missed that the original code
>> could benefit from specialization, like you have pointed out. So look out
>> for looking out for final ;)
>>
>> On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller
>> wrote:
>>>
>>> Okay, it seems your experience in this regard is broader than mine. I
>>> will look out for this in the future.
>>>
>>> Regards,
>>> Rüdiger
>>
>
>

They are boxed (as are the standard Java collections).  Use Trove (for Java) for primitive collections, or something similar.

  --Rex

On Wed, Sep 28, 2011 at 9:39 AM, Edmondo Porcu <edmondo.porcu@gmail.com> wrote:

Dear all,
what about scala MAPS and primitive types? Are they efficient or one should worry about autoboxing?


Best Regards

2011/9/28 Aleksey Nikiforov <lexn82@gmail.com>

Of course, by looking out for final, I have missed that the original code could benefit from specialization, like you have pointed out. So look out for looking out for final ;)

On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller <ruediger.keller@rk42.de> wrote:

Okay, it seems your experience in this regard is broader than mine. I
will look out for this in the future.

Regards,
Rüdiger

how big would the collection framework become if every collection was fully specialized?

-------- Original-Nachricht --------
> Datum: Wed, 28 Sep 2011 10:36:53 -0400
> Von: Rex Kerr
> An: Edmondo Porcu
> CC: Aleksey Nikiforov , Ruediger Keller , scala-user
> Betreff: Re: [scala-user] Help from the masters of JVM - Arrays :)

> They are boxed (as are the standard Java collections). Use Trove (for
> Java)
> for primitive collections, or something similar.
>
> --Rex
>
> On Wed, Sep 28, 2011 at 9:39 AM, Edmondo Porcu
> wrote:
>
> > Dear all,
> > what about scala MAPS and primitive types? Are they efficient or one
> should
> > worry about autoboxing?
> >
> >
> > Best Regards
> >
> > 2011/9/28 Aleksey Nikiforov
> >
> >> Of course, by looking out for final, I have missed that the original
> code
> >> could benefit from specialization, like you have pointed out. So look
> out
> >> for looking out for final ;)
> >>
> >>
> >> On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller
> >> > wrote:
> >>
> >>> Okay, it seems your experience in this regard is broader than mine. I
> >>> will look out for this in the future.
> >>>
> >>> Regards,
> >>> Rüdiger
> >>>
> >>
> >>
> >

Some of it becomes about 10x bigger, and some of it about 50x, and a lot of it doesn't work at all due to bugs in the implementation of specialized.  (The 50x comes from things like map and fold which need two levels of specialization.)

  --Rex

On Wed, Sep 28, 2011 at 11:24 AM, Dennis Haupt <h-star@gmx.de> wrote:

how big would the collection framework become if every collection was fully specialized?

-------- Original-Nachricht --------
> Datum: Wed, 28 Sep 2011 10:36:53 -0400
> Von: Rex Kerr <ichoran@gmail.com>
> An: Edmondo Porcu <edmondo.porcu@gmail.com>
> CC: Aleksey Nikiforov <lexn82@gmail.com>, Ruediger Keller <ruediger.keller@rk42.de>, scala-user <scala-user@googlegroups.com>
> Betreff: Re: [scala-user] Help from the masters of JVM - Arrays :)

> They are boxed (as are the standard Java collections).  Use Trove (for
> Java)
> for primitive collections, or something similar.
>
>   --Rex
>
> On Wed, Sep 28, 2011 at 9:39 AM, Edmondo Porcu
> <edmondo.porcu@gmail.com>wrote:
>
> > Dear all,
> > what about scala MAPS and primitive types? Are they efficient or one
> should
> > worry about autoboxing?
> >
> >
> > Best Regards
> >
> > 2011/9/28 Aleksey Nikiforov <lexn82@gmail.com>
> >
> >> Of course, by looking out for final, I have missed that the original
> code
> >> could benefit from specialization, like you have pointed out. So look
> out
> >> for looking out for final ;)
> >>
> >>
> >> On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller
> <ruediger.keller@rk42.de
> >> > wrote:
> >>
> >>> Okay, it seems your experience in this regard is broader than mine. I
> >>> will look out for this in the future.
> >>>
> >>> Regards,
> >>> Rüdiger
> >>>
> >>
> >>
> >

sounds like a no-go for the next 5 years.

Am 28.09.2011 17:45, schrieb Rex Kerr:

g [at] mail [dot] gmail [dot] com" type="cite">Some of it becomes about 10x bigger, and some of it about 50x, and a lot of it doesn't work at all due to bugs in the implementation of specialized.  (The 50x comes from things like map and fold which need two levels of specialization.)

  --Rex

On Wed, Sep 28, 2011 at 11:24 AM, Dennis Haupt <h-star [at] gmx [dot] de" rel="nofollow">h-star@gmx.de> wrote:

how big would the collection framework become if every collection was fully specialized?

-------- Original-Nachricht --------
> Datum: Wed, 28 Sep 2011 10:36:53 -0400
> Von: Rex Kerr <ichoran [at] gmail [dot] com" rel="nofollow">ichoran@gmail.com>
> An: Edmondo Porcu <edmondo [dot] porcu [at] gmail [dot] com" rel="nofollow">edmondo.porcu@gmail.com>
> CC: Aleksey Nikiforov <lexn82 [at] gmail [dot] com" rel="nofollow">lexn82@gmail.com>, Ruediger Keller <ruediger [dot] keller [at] rk42 [dot] de" rel="nofollow">ruediger.keller@rk42.de>, scala-user <scala-user [at] googlegroups [dot] com" rel="nofollow">scala-user@googlegroups.com>
> Betreff: Re: [scala-user] Help from the masters of JVM - Arrays :)

> They are boxed (as are the standard Java collections).  Use Trove (for
> Java)
> for primitive collections, or something similar.
>
>   --Rex
>
> On Wed, Sep 28, 2011 at 9:39 AM, Edmondo Porcu
> <edmondo [dot] porcu [at] gmail [dot] com" rel="nofollow">edmondo.porcu@gmail.com>wrote:
>
> > Dear all,
> > what about scala MAPS and primitive types? Are they efficient or one
> should
> > worry about autoboxing?
> >
> >
> > Best Regards
> >
> > 2011/9/28 Aleksey Nikiforov <lexn82 [at] gmail [dot] com" rel="nofollow">lexn82@gmail.com>
> >
> >> Of course, by looking out for final, I have missed that the original
> code
> >> could benefit from specialization, like you have pointed out. So look
> out
> >> for looking out for final ;)
> >>
> >>
> >> On Wed, Sep 28, 2011 at 8:09 AM, Ruediger Keller
> <ruediger [dot] keller [at] rk42 [dot] de" rel="nofollow">ruediger.keller@rk42.de
> >> > wrote:
> >>
> >>> Okay, it seems your experience in this regard is broader than mine. I
> >>> will look out for this in the future.
> >>>
> >>> Regards,
> >>> Rüdiger
> >>>
> >>
> >>
> >