- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
Character encoding issue...
Mon, 2009-03-23, 23:19
Hi
I'm dipping my toe into the Scala waters (being from a C/C++ but mainly now Python background), and I'm having an issue regarding something I thought should be trivial.
I'm trying to read a file (that isn't strictly 100% ASCII) with the code:
var fd = Source.fromFile( filename )
for ( line <- fd.getLines ) {
println(line)}
This ends up reading the plain ASCII data, but once it gets to a "strange" character I get:
ava.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:398) at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:117) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:86) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:74) at scala.io.Source$$anon$6.next(Source.scala:307) at scala.io.Source$$anon$6.next(Source.scala:301) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.io.Source$$anon$6.foreach(Source.scala:301) at index2scala.Main$.loadFile(Main.scala:32) at index2scala.Main$$anonfun$main$1.apply(Main.scala:55) at index2scala.Main$$anonfun$main$1.apply(Main.scala:49) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.runtime.BoxedArray$AnyIterator.foreach(BoxedArray.scala:45) at scala.Iterable$class.foreach(Iterable.scala:256) at scala.runtime.BoxedArray.foreach(BoxedArray.scala:24) at index2scala.Main$.main(Main.scala:49) at index2scala.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at scala.tools.nsc.ObjectRunner$$anonfun$run$1.apply(ObjectRunner.scala:75) at scala.tools.nsc.ObjectRunner$.withContextClassLoader(ObjectRunner.scala:49) at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:74) at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:154) at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
It appears that the problem is caused by more than one input line, since I might have 10 lines in my document. If I erase the last line the error no longer appears, but if I change the file to ONLY include that last line, the problem doesn't appear either. It seems that some combination of lines is required to trigger this.
I can try and create a smaller test document and paste here if it helps. Just wondering if this is a known problem?
also, I've noticed if the test document is:
foo · barblah
then the program will print out "foo" and end gracefully. No exception appears to be thrown.
Thanks
Ken
I'm dipping my toe into the Scala waters (being from a C/C++ but mainly now Python background), and I'm having an issue regarding something I thought should be trivial.
I'm trying to read a file (that isn't strictly 100% ASCII) with the code:
var fd = Source.fromFile( filename )
for ( line <- fd.getLines ) {
println(line)}
This ends up reading the plain ASCII data, but once it gets to a "strange" character I get:
ava.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:398) at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:117) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:86) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:74) at scala.io.Source$$anon$6.next(Source.scala:307) at scala.io.Source$$anon$6.next(Source.scala:301) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.io.Source$$anon$6.foreach(Source.scala:301) at index2scala.Main$.loadFile(Main.scala:32) at index2scala.Main$$anonfun$main$1.apply(Main.scala:55) at index2scala.Main$$anonfun$main$1.apply(Main.scala:49) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.runtime.BoxedArray$AnyIterator.foreach(BoxedArray.scala:45) at scala.Iterable$class.foreach(Iterable.scala:256) at scala.runtime.BoxedArray.foreach(BoxedArray.scala:24) at index2scala.Main$.main(Main.scala:49) at index2scala.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at scala.tools.nsc.ObjectRunner$$anonfun$run$1.apply(ObjectRunner.scala:75) at scala.tools.nsc.ObjectRunner$.withContextClassLoader(ObjectRunner.scala:49) at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:74) at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:154) at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
It appears that the problem is caused by more than one input line, since I might have 10 lines in my document. If I erase the last line the error no longer appears, but if I change the file to ONLY include that last line, the problem doesn't appear either. It seems that some combination of lines is required to trigger this.
I can try and create a smaller test document and paste here if it helps. Just wondering if this is a known problem?
also, I've noticed if the test document is:
foo · barblah
then the program will print out "foo" and end gracefully. No exception appears to be thrown.
Thanks
Ken