wrappedRefArray classType - how to avoid?

Without giving examples of the code in question, it's *highly* unlikely that anyone will try to help you here.  At the bare minimum, any answer would require hours of contriving scenarios, just on the off-chance of rediscovering the particular issue that you've encountered.


On 27 June 2011 18:15, Chris Twiner <chris.twiner@gmail.com> wrote:

Hi all,

I have almost identical code in two separate places. One compiles without reflection and the other not.

The code with the unwanted added reflection unsuprisingly adds significant slowdown.

Does anyone know how to stop the compiler adding these bits of reflection?

Cheers
Chris

--
Kevin Wright

gtalk / msn : kev.lee.wright@gmail.comkev.lee.wright@gmail.commail: kevin.wright@scalatechnology.com
vibe / skype: kev.lee.wrightquora: http://www.quora.com/Kevin-Wright
twitter: @thecoda

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

I was assuming there are a given set of circumstances where that can occur.

Providing code examples will be tough but I am working on isolation.

If there was a reference somebody could point me to aside from nsc I would be happy to look through it.

Aside from that do you have anything helpful to contribute?

On 27 Jun 2011 19:32, "Kevin Wright" <kev.lee.wright@gmail.com> wrote:

Without giving examples of the code in question, it's *highly* unlikely that anyone will try to help you here.  At the bare minimum, any answer would require hours of contriving scenarios, just on the off-chance of rediscovering the particular issue that you've encountered.

On 27 June 2011 18:15, Chris Twiner <chris.twiner@gmail.com> wrote:
>
> Hi all,
>
> I have almost ...

For those interested this seems to occur when the compiler generates a
CanBuildFrom for you, it doesn't just do it in one spot in a function
but converts all of that function to reflective code, or at least
seems that way. Importing a cbf removed the reflective code. I'm
still trying to find a small reproduction of the problem...

On Mon, Jun 27, 2011 at 8:23 PM, Chris Twiner wrote:
> I was assuming there are a given set of circumstances where that can occur.
>
> Providing code examples will be tough but I am working on isolation.
>
> If there was a reference somebody could point me to aside from nsc I would
> be happy to look through it.
>
> Aside from that do you have anything helpful to contribute?
>
> On 27 Jun 2011 19:32, "Kevin Wright" wrote:
>
> Without giving examples of the code in question, it's *highly* unlikely that
> anyone will try to help you here.  At the bare minimum, any answer would
> require hours of contriving scenarios, just on the off-chance of
> rediscovering the particular issue that you've encountered.
>
>
> On 27 June 2011 18:15, Chris Twiner wrote:
>>
>> Hi all,
>>
>> I have almost ...
>
> --
> Kevin Wright
>
> gtalk / msn : kev.lee.wright@gmail.com
> mail: kevin.wright@scalatechnology.com
> vibe / skype: kev.lee.wright
> quora: http://www.quora.com/Kevin-Wright
> twitter: @thecoda
>
> "My point today is that, if we wish to count lines of code, we should not
> regard them as "lines produced" but as "lines spent": the current
> conventional wisdom is so foolish as to book that count on the wrong side of
> the ledger" ~ Dijkstra
>

On Mon, Jun 27, 2011 at 10:08 PM, Chris Twiner wrote:
> For those interested this seems to occur when the compiler generates a
> CanBuildFrom for you, it doesn't just do it in one spot in a function
> but converts all of that function to reflective code, or at least
> seems that way.  Importing a cbf removed the reflective code.  I'm
> still trying to find a small reproduction of the problem...

I'm curious, but still have no idea what you mean. Can you post a
snippet from from javap -v on the class that makes the reflective
call?

-jason

Sure its a bit out of context though (via jad), I'm working on a
contained example (where ScalesXml$.MODULE$.xmlCBF() is imported):

public Path elementEnd(Path xml)
{
if(xml.node().index() >= 32) goto _L2; else goto _L1
_L1:
xml.zipUp();
goto _L3
_L2:
if(!xml.top().isLeft()) goto _L5; else goto _L4
_L4:
xml;
goto _L3
_L5:
Path path = (Path)xml.top().right().get();
if(xml.parentTree().children().apply(xml.node().index()) !=
xml.node().focus()) goto _L7; else goto _L6
_L6:
path;
goto _L3
_L7:
Either parentFocus;
scala.collection.TraversableOnce v =
((ImmutableArrayProxy)xml.parentTree().children()).ar();
ImmutableArrayProxy n =
(ImmutableArrayProxy)xml.parentTree().children().updated(xml.node().index(),
xml.node().focus(), ScalesXml$.MODULE$.xmlCBF());
parentFocus =
package$.MODULE$.subtree(xml.parentTree().section(), n,
ScalesXml$.MODULE$.xmlCBF());
new Path(path.top(), new Node(path.node().index(),
parentFocus), ScalesXml$.MODULE$.xmlCBF());
_L3:
return;
}

when not imported:

if(xml.node().index() >= 32) goto _L2; else goto _L1
_L1:
xml.zipUp();
goto _L3
_L2:
if(!xml.top().isLeft()) goto _L5; else goto _L4
_L4:
xml;
goto _L3
_L5:
Path path = (Path)xml.top().right().get();
if(xml.parentTree().children().apply(xml.node().index()) !=
xml.node().focus()) goto _L7; else goto _L6
_L6:
path;
goto _L3
_L7:
Either parentFocus;
scala.collection.TraversableOnce v =
((ImmutableArrayProxy)xml.parentTree().children()).ar();
VectorImpl n = new
VectorImpl((Vector)Extras$.MODULE$.replaceLast((Vector)v,
xml.node().index(), xml.node().focus(),
Vector$.MODULE$.canBuildFrom()));
parentFocus =
package$.MODULE$.subtree(xml.parentTree().section(), n,
ImmutableArrayProxy$.MODULE$.canBuildFrom(ClassManifest$.MODULE$.classType(scala/Either,
ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
ClassManifest$.MODULE$.classType(scales/utils/Tree,
ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
ClassManifest$.MODULE$.classType(scales/xml/Elem),
NoManifest$.MODULE$
})))
})))));
new Path(path.top(), new Node(path.node().index(),
parentFocus), ImmutableArrayProxy$.MODULE$.canBuildFrom(ClassManifest$.MODULE$.classType(scala/Either,
ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
ClassManifest$.MODULE$.classType(scales/utils/Tree,
ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
ClassManifest$.MODULE$.classType(scales/xml/Elem),
NoManifest$.MODULE$
})))
})))));
_L3:

Sorry, it seems its only the code that has any direct relation to a
cbf that is hit (thought it was the whole code but simply read it
wrong), but I don't see any warnings generated for it hinting that it
will occur.

I was using yourkit to profile, where I saw a hotspot for the
wrapRefArray calls.

On Mon, Jun 27, 2011 at 10:16 PM, Jason Zaugg wrote:
> On Mon, Jun 27, 2011 at 10:08 PM, Chris Twiner wrote:
>> For those interested this seems to occur when the compiler generates a
>> CanBuildFrom for you, it doesn't just do it in one spot in a function
>> but converts all of that function to reflective code, or at least
>> seems that way.  Importing a cbf removed the reflective code.  I'm
>> still trying to find a small reproduction of the problem...
>
> I'm curious, but still have no idea what you mean. Can you post a
> snippet from from javap -v on the class that makes the reflective
> call?
>
> -jason
>

On Mon, Jun 27, 2011 at 10:27 PM, Chris Twiner wrote:
> Sure its a bit out of context though (via jad), I'm working on a
> contained example (where ScalesXml$.MODULE$.xmlCBF() is imported):
>
[snip]
>ImmutableArrayProxy$.MODULE$.canBuildFrom(ClassManifest$.MODULE$.classType(scala/Either,
> ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
> Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
>            ClassManifest$.MODULE$.classType(scales/utils/Tree,
> ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
> Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
>                ClassManifest$.MODULE$.classType(scales/xml/Elem),
> NoManifest$.MODULE$
>            })))
>        })))));
>        new Path(path.top(), new Node(path.node().index(),
> parentFocus), ImmutableArrayProxy$.MODULE$.canBuildFrom(ClassManifest$.MODULE$.classType(scala/Either,
> ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
> Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
>            ClassManifest$.MODULE$.classType(scales/utils/Tree,
> ClassManifest$.MODULE$.classType(scales/xml/XmlItem),
> Predef$.MODULE$.wrapRefArray((Object[])(new OptManifest[] {
>                ClassManifest$.MODULE$.classType(scales/xml/Elem),
> NoManifest$.MODULE$
>            })))
>        })))));
> _L3:

Okay, there aren't any reflective method invocations there, just
regular method calls to construct a scala.reflect.Manifest.

It's very easy to be fooled by YourKit with hotspots like these that
turn out to be ghosts. But it is feasible that you could measure some
overhead in constructing the manifest inside a tight loop, and adding
an import to use "one I prepared earlier" could speed things up.

Here's a smaller example to demonstrate Manifest creation works:

$ scala -Xprint:typer -e "manifest[List[Int]]"

scala.this.Predef.manifest[List[Int]](
reflect.this.Manifest.classType[List[Int]](
classOf[scala.collection.immutable.List],
reflect.this.Manifest.Int
)
)

Or, for the bytecode fans:

scala> class A { manifest[List[String]] }
defined class A

scala> :javap -v A

-jason

On Mon, Jun 27, 2011 at 11:00 PM, Jason Zaugg wrote:
> On Mon, Jun 27, 2011 at 10:27 PM, Chris Twiner wrote:
[more snippage]

> Okay, there aren't any reflective method invocations there, just
> regular method calls to construct a scala.reflect.Manifest.

true that, blame that on low sleep, I'm strictly interested in the
scala."reflect" calls not java reflection Method style fun, which it
closely resembles.

> It's very easy to be fooled by YourKit with hotspots like these that
> turn out to be ghosts. But it is feasible that you could measure some
> overhead in constructing the manifest inside a tight loop, and adding
> an import to use "one I prepared earlier" could speed things up.

[snippage]

Thanks for the response Jason, in particular this is called as part of
an xml parse, each "end element" event being a call, for larger
documents this equates to a 12-14% slowdown: