- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
scala and NIO
Mon, 2009-04-20, 23:36
This bug:
https://lampsvn.epfl.ch/trac/scala/ticket/1883
"BufferUnderflowException when Source.fromURL applied to UTF-8 content"
...revealed to me that different versions of fromURL do very different
things:
// this one uses regular I/O, and is deprecated
def fromURL(url: URL): Source = {
val it = new Iterator[Char] { etc. }
}
// this one uses NIO
def fromURL(url: URL, enc:String): Source =
fromInputStream(url.openStream(), enc)
// since it calls this
def fromInputStream(is: InputStream, enc: String): Source =
BufferedSource.fromInputStream(is, enc, Source.DefaultBufSize, { () => fromInputStream(is, enc) })
I'm sure it goes without saying that this is not acceptable, these two
functions have to use the same code path.
The version which uses old skool IO has the benefit of not suffering
from bug #1883, so I vote for that one.
I am far from an expert on java I/O but the whisperings in my head tell
me that NIO requires a certain amount of devotion to get right, which it
seems to me it is lacking here. Outside scala.tools.nsc, NIO is used
only in Source/BufferedSource and a couple XML spots, all apparently
authored by buraq.
I propose to remove all the NIO uses until such time as someone can
demonstrate a benefit and is willing to take responsibility for keeping
it working. I know we have at least one NIO expert here, but I think
his hands are full with other things.
Tue, 2009-04-21, 00:27
#2
Re: scala and NIO
On Mon, Apr 20, 2009 at 11:36 PM, Paul Phillips wrote:
> I propose to remove all the NIO uses until such time as someone can
> demonstrate a benefit and is willing to take responsibility for keeping
> it working. I know we have at least one NIO expert here, but I think
> his hands are full with other things.
NIO really doesn't have very much extra to offer where simple blocking
stream IO is involved, and as Robert Fischer pointed out
InputStreamReaders use the relevant bits of NIO internally anyway.
However, the mention of Buraq suggests that the use of NIO is quite
likely related to Scala's XML support. XML processors have to be able
to sniff character encodings from a prefix of the stream that an XML
document instance is being parsed from, which entails switching on the
fly from an initial least common denominator US ASCII encoding to the
one specified in the XML declaration.
IME this is a bit of a pain to do with java.io.{ InputStream,
InputStreamReader } and it could very well be that Buraq has put
together something in terms of NIO's explicit Charset{Encoder,
Decoder} classes and BufferedSource has inherited this as a result.
But aside from that, I'd tend to agree. We definitely want good NIO
support in a future set of scala.io.* packages, but that's orthogonal
to BufferedSource.
Cheers,
Miles
Note that Java's old I/O impl has been retrofitted to be a facade over NIO anyway, so you're not
losing much performance by using the old version. In general, NIO is to be avoided unless you
absolutely, positively know it, and can prove you *need* it. Usually this means needing the NIO API.
If you go ahead with using NIO, expect lots and lots of subtle, annoying bugs.
~~ Robert.
Paul Phillips wrote:
> This bug:
>
> https://lampsvn.epfl.ch/trac/scala/ticket/1883
> "BufferUnderflowException when Source.fromURL applied to UTF-8 content"
>
[...]
>
> The version which uses old skool IO has the benefit of not suffering
> from bug #1883, so I vote for that one.
>
> I am far from an expert on java I/O but the whisperings in my head tell
> me that NIO requires a certain amount of devotion to get right, which it
> seems to me it is lacking here. Outside scala.tools.nsc, NIO is used
> only in Source/BufferedSource and a couple XML spots, all apparently
> authored by buraq.
>
> I propose to remove all the NIO uses until such time as someone can
> demonstrate a benefit and is willing to take responsibility for keeping
> it working. I know we have at least one NIO expert here, but I think
> his hands are full with other things.
>
~~ Robert Fischer.
Grails Training http://GroovyMag.com/training
Smokejumper Consulting http://SmokejumperIT.com
Enfranchised Mind Blog http://EnfranchisedMind.com/blog
Check out my book, "Grails Persistence with GORM and GSQL"!
http://www.smokejumperit.com/redirect.html