This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Character encoding issue...

No replies
Ken Faulkner
Joined: 2009-03-23,
User offline. Last seen 42 years 45 weeks ago.
Hi
I'm dipping my toe into the Scala waters (being from a C/C++ but mainly now Python background), and I'm having an issue regarding something I thought should be trivial.
I'm trying to read a file (that isn't strictly 100% ASCII) with the code:

var fd = Source.fromFile( filename )
for ( line <- fd.getLines ) {
  println(line)}
This ends up reading the plain ASCII data, but once it gets to a "strange" character I get:
ava.nio.BufferUnderflowException at java.nio.Buffer.nextGetIndex(Buffer.java:398) at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:117) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:86) at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:74) at scala.io.Source$$anon$6.next(Source.scala:307) at scala.io.Source$$anon$6.next(Source.scala:301) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.io.Source$$anon$6.foreach(Source.scala:301) at index2scala.Main$.loadFile(Main.scala:32) at index2scala.Main$$anonfun$main$1.apply(Main.scala:55) at index2scala.Main$$anonfun$main$1.apply(Main.scala:49) at scala.Iterator$class.foreach(Iterator.scala:414) at scala.runtime.BoxedArray$AnyIterator.foreach(BoxedArray.scala:45) at scala.Iterable$class.foreach(Iterable.scala:256) at scala.runtime.BoxedArray.foreach(BoxedArray.scala:24) at index2scala.Main$.main(Main.scala:49) at index2scala.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at scala.tools.nsc.ObjectRunner$$anonfun$run$1.apply(ObjectRunner.scala:75) at scala.tools.nsc.ObjectRunner$.withContextClassLoader(ObjectRunner.scala:49) at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:74) at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:154) at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

It appears that the problem is caused by more than one input line, since I might have 10 lines in my document. If I erase the last line the error no longer appears, but if I change the file to ONLY include that last line, the problem doesn't appear either. It seems that some combination of lines is required to trigger this.
I can try and create a smaller test document and paste here if it helps. Just wondering if this is a known problem?

also,  I've noticed if the test document is:
foo · barblah

then the program will print out "foo" and end gracefully. No exception appears to be thrown. 
Thanks
Ken

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland