Scala REPL in Cygwin

On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
> Using scala's REPL functionality in Cygwin always caused me problems.

You guys will be happy to know that despite my below-zero interest in
windows I installed windows 7 64bit this weekend and did a bunch of work
trying to close windows repl tickets.

However the real test will be when I check something in and it breaks
something somewhere in a way I can't see or understand, which is the
usual outcome when I touch anything windows. Should this happen,
somebody who cares about windows MUST rise to the occasion and help me.
I find it a great distance short of fair that this task falls on me if
there are indeed so many windows users. (This is not directed at you
Christopher, but at the fact that most of these bugs go back for years.)

I feel for you, Paul.

I tested Christopher's patch, and it works fine WHEN RUNNING WITH rxvt
(TERM=rxvt-cygwin-native)

The stty commands are necessary to make command line editing and tab
keys work. I think dropping the exec is necessary to capture the exit
code. Maybe someone else knows more about that?

But when you run with the plain old Windows shell (i.e. bash --login
-i inside a cmd shell, TERM=cygwin), then the patch does NOT work. In
that case, the unpatched file works.

I looked at a bunch of Cygwin/JLine error reports, and I am not sure
that this distinction is well understood. It certainly surprised me.
So, a better check would be this:

rxvt=false
if $cygwin ; then
case "$TERM" in
rxvt*) rxvt=true ;;
esac
fi

if $rxvt ; then
JAVA_OPTS="$JAVA_OPTS -Djline.terminal=jline.UnixTerminal"
stty -icanon min 1 -echo > /dev/null 2>&1
"${JAVACMD:=java}" $JAVA_OPTS -cp "$TOOL_CLASSPATH" \
-Dscala.home="$SCALA_HOME" -Denv.emacs="$EMACS" \
scala.tools.nsc.MainGenericRunner "$@"
# Record the exit status immediately, or it will be overridden.
SCALA_STATUS=$?
stty icanon echo > /dev/null 2>&1
exit $SCALA_STATUS
else
exec "${JAVACMD:=java}" $JAVA_OPTS -cp "$TOOL_CLASSPATH" \
-Dscala.home="$SCALA_HOME" -Denv.emacs="$EMACS" \
scala.tools.nsc.MainGenericRunner "$@"
fi

I'll be glad to try out whatever you come up with.

And I have another question. What is the significance of
-Denv.emacs="$EMACS"? I grepped for "emacs" in the jline source, and
they know nothing about it. Is this something for the Scala
interpreter? Tab expansion in Emacs is something that I still can't
get to work.

Cheers,

Cay

2010/11/23 Paul Phillips :
> On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
>> Using scala's REPL functionality in Cygwin always caused me problems.
>
> You guys will be happy to know that despite my below-zero interest in
> windows I installed windows 7 64bit this weekend and did a bunch of work
> trying to close windows repl tickets.
>
> However the real test will be when I check something in and it breaks
> something somewhere in a way I can't see or understand, which is the
> usual outcome when I touch anything windows.  Should this happen,
> somebody who cares about windows MUST rise to the occasion and help me.
> I find it a great distance short of fair that this task falls on me if
> there are indeed so many windows users.  (This is not directed at you
> Christopher, but at the fact that most of these bugs go back for years.)
>
> --
> Paul Phillips      | A national political campaign is better than the
> Moral Alien        | best circus ever heard of, with a mass baptism and
> Empiricist         | a couple of hangings thrown in.
> slap pi uphill!    |     -- H. L. Mencken
>

I feel for you, Paul.

I tested Christopher's patch, and it works fine WHEN RUNNING WITH rxvt
(TERM=rxvt-cygwin-native)

The stty commands are necessary to make command line editing and tab
keys work. I think dropping the exec is necessary to capture the exit
code. Maybe someone else knows more about that?

But when you run with the plain old Windows shell (i.e. bash --login
-i inside a cmd shell, TERM=cygwin), then the patch does NOT work. In
that case, the unpatched file works.

I looked at a bunch of Cygwin/JLine error reports, and I am not sure
that this distinction is well understood. It certainly surprised me.
So, a better check would be this:

rxvt=false
if $cygwin ; then
  case "$TERM" in
      rxvt*) rxvt=true ;;
  esac
fi

if $rxvt ; then
  JAVA_OPTS="$JAVA_OPTS -Djline.terminal=jline.UnixTerminal"
  stty -icanon min 1 -echo > /dev/null 2>&1
  "${JAVACMD:=java}" $JAVA_OPTS -cp "$TOOL_CLASSPATH" \
      -Dscala.home="$SCALA_HOME" -Denv.emacs="$EMACS" \
      scala.tools.nsc.MainGenericRunner  "$@"
  # Record the exit status immediately, or it will be overridden.
  SCALA_STATUS=$?
  stty icanon echo > /dev/null 2>&1
  exit $SCALA_STATUS
else
  exec "${JAVACMD:=java}" $JAVA_OPTS -cp "$TOOL_CLASSPATH" \
      -Dscala.home="$SCALA_HOME" -Denv.emacs="$EMACS" \
      scala.tools.nsc.MainGenericRunner  "$@"
fi

I'll be glad to try out whatever you come up with.

And I have another question. What is the significance of
-Denv.emacs="$EMACS"? I grepped for "emacs" in the jline source, and
they know nothing about it. Is this something for the Scala
interpreter? Tab expansion in Emacs is something that I still can't
get to work.

Cheers,

Cay

2010/11/23 Paul Phillips <paulp [at] improving [dot] org>:
> On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
>> Using scala's REPL functionality in Cygwin always caused me problems.
>
> You guys will be happy to know that despite my below-zero interest in
> windows I installed windows 7 64bit this weekend and did a bunch of work
> trying to close windows repl tickets.
>
> However the real test will be when I check something in and it breaks
> something somewhere in a way I can't see or understand, which is the
> usual outcome when I touch anything windows.  Should this happen,
> somebody who cares about windows MUST rise to the occasion and help me.
> I find it a great distance short of fair that this task falls on me if
> there are indeed so many windows users.  (This is not directed at you
> Christopher, but at the fact that most of these bugs go back for years.)
>
> --
> Paul Phillips      | A national political campaign is better than the
> Moral Alien        | best circus ever heard of, with a mass baptism and
> Empiricist         | a couple of hangings thrown in.
> slap pi uphill!    |     -- H. L. Mencken
>

On Tue, Nov 23, 2010 at 2:40 PM, Christopher Farnham
wrote:
> BTW:  Where is the source code for the 'scala' shell script managed?  I
> downloaded the source from
> 'http://lampsvn.epfl.ch/svn-repos/scala/scala/trunk' but couldn't find the
> script.

Scripts are generated from a template:

http://lampsvn.epfl.ch/trac/scala/browser/scala/trunk/src/compiler/scala...

2010/11/23 Cay Horstmann :
> And I have another question. What is the significance of
> -Denv.emacs="$EMACS"? I grepped for "emacs" in the jline source, and
> they know nothing about it. Is this something for the Scala
> interpreter? Tab expansion in Emacs is something that I still can't
> get to work.

I found this out myself by grepping the Scala source. When $EMACS is
set to a non-empty string, then JLine is deactivated in
compiler/scala/tools/nsc/InterpreterLoop.scala.

This is probably not a good idea. When running inside a M-x term
shell, EMACS is set to something like "23.1.1 (term:0.96)". In
terminal mode, jline works ok when you run
EMACS="" scala. That is, the arrow keys and tab completion work fine.

If you run with M-x shell or M-x scala-run-scala, EMACS is set to "t".
In that case, I don't think there is much hope to get tab completion
to work. So, it would probably be a good idea to disable jline only
when EMACS == "t".

Cheers,

Cay

Actually, I just noted a patch on
http://lampsvn.epfl.ch/trac/scala/ticket/2097 that is very much along
the lines of my suggestion, but it factors out the common behavior
more nicely.

2010/11/23 Christopher Farnham :
> Cay,
> Thank you for improving my fix.  I probably should have tested a few
> different Cygwin configurations before passing it on to the group.    If a
> patch is made I will be sure to test it on my Windows box in a couple
> different environments to ensure that it's robust.

This thread has sparked back my interest in figuring why cygwin or cmd.exe
does not show the cursor when moving left or backspacing. Also since I've
been using the REPL I've been limiting myself to lines of code of less than
80 chars, because cmd.exe/jline does not handle wrapping well.

i've posted a modified snapshot of the jline jar at
http://github.com/downloads/huynhjl/jline/jline-0.9.95-SNAPSHOT.jar forked
from gnodet/jline on github. It includes new windows dll to make native
calls to move the cursor around. It seems to fix the two issues mentioned
above on JDK/JRE 1.6 and Windows 7 64 bits. Haven't had a chance to test
other combinations yet.

I'll be glad to provide a patch to scala's jline (which seems to have
diverged a bit) if that works for others.

--huynhjl

Hi Paul,

Paul Phillips-3 wrote:
>
> On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
>> Using scala's REPL functionality in Cygwin always caused me problems.
>
> You guys will be happy to know that despite my below-zero interest in
> windows I installed windows 7 64bit this weekend and did a bunch of work
> trying to close windows repl tickets.
>
> However the real test will be when I check something in and it breaks
> something somewhere in a way I can't see or understand, which is the
> usual outcome when I touch anything windows. Should this happen,
> somebody who cares about windows MUST rise to the occasion and help me.
> I find it a great distance short of fair that this task falls on me if
> there are indeed so many windows users.
>

I have 3 different Windows environments that I use daily and I can help test
on those (WinXP 32 bits, Vista 32 bits, Windows 7 64 bits).

I've been hacking with jline for about a week now (see
http://github.com/huynhjl/jline). I think I've addressed wrapping issues for
xterm (on ubuntu) and cursor display and wrapping for Windows (cygwin in
cmd.exe but not rxvt or mintty). My latest snapshot is at
http://github.com/huynhjl/jline/downloads.

I've also added some incremental reverse search and it's good enough that
I've replaced the jline bundled with scala on all my computers. It works
like readline's reverse-i-search:

scala> val foo = "bar"
foo: java.lang.String = bar

(reverse-i-search)`va': val foo = "bar"

At this point, I'm interested to look at multi line history support in jline
but I would need some help on the REPL side of this.

If there is no chance for my fork to be included with scala, I'll probably
stop here and look to submit just the Windows fix into the hypothetical next
official release of jline.

Jean-Laurent

On Thu, Dec 2, 2010 at 9:25 AM, huynhjl wrote:
> At this point, I'm interested to look at multi line history support in jline
> but I would need some help on the REPL side of this.
>
> If there is no chance for my fork to be included with scala, I'll probably
> stop here and look to submit just the Windows fix into the hypothetical next
> official release of jline.

Regardless of how this is accepted for Scala, submit your changes in
any case to the JLine community. I'm pretty sure, there's a bunch of
people who will love you for contributing something like this (JRuby,
Jython etc.).

You fixed the infamous "cursor turns invisible when moved" bug on Windows?

Great! I really hope your fixes will be accepted into Scala.

Regards,
Ruediger

2010/12/2 huynhjl :
>
> Hi Paul,
>
>
> Paul Phillips-3 wrote:
>>
>> On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
>>> Using scala's REPL functionality in Cygwin always caused me problems.
>>
>> You guys will be happy to know that despite my below-zero interest in
>> windows I installed windows 7 64bit this weekend and did a bunch of work
>> trying to close windows repl tickets.
>>
>> However the real test will be when I check something in and it breaks
>> something somewhere in a way I can't see or understand, which is the
>> usual outcome when I touch anything windows.  Should this happen,
>> somebody who cares about windows MUST rise to the occasion and help me.
>> I find it a great distance short of fair that this task falls on me if
>> there are indeed so many windows users.
>>
>
> I have 3 different Windows environments that I use daily and I can help test
> on those (WinXP 32 bits, Vista 32 bits, Windows 7 64 bits).
>
> I've been hacking with jline for about a week now (see
> http://github.com/huynhjl/jline). I think I've addressed wrapping issues for
> xterm (on ubuntu) and cursor display and wrapping for Windows (cygwin in
> cmd.exe but not rxvt or mintty).  My latest snapshot is at
> http://github.com/huynhjl/jline/downloads.
>
> I've also added some incremental reverse search and it's good enough that
> I've replaced the jline bundled with scala on all my computers. It works
> like readline's reverse-i-search:
>
> scala> val foo = "bar"
> foo: java.lang.String = bar
>
> (reverse-i-search)`va': val foo = "bar"
>
> At this point, I'm interested to look at multi line history support in jline
> but I would need some help on the REPL side of this.
>
> If there is no chance for my fork to be included with scala, I'll probably
> stop here and look to submit just the Windows fix into the hypothetical next
> official release of jline.
>
> Jean-Laurent
> --
> View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Scala-REPL-in-Cy...
> Sent from the Scala mailing list archive at Nabble.com.
>

Jean-Laurent,

You can release and deploy your fork on scala-tools.org or
oss.sonatype.org, (change the groupId and version number).
When done, ping me, I'll modify maven-scala-plugin to allow user to
choose your deployed version for "mvn scala:console"

/davidB

On Thu, Dec 2, 2010 at 15:38, Ruediger Keller wrote:
> You fixed the infamous "cursor turns invisible when moved" bug on Windows?
>
> Great! I really hope your fixes will be accepted into Scala.
>
> Regards,
> Ruediger
>
>
> 2010/12/2 huynhjl :
>>
>> Hi Paul,
>>
>>
>> Paul Phillips-3 wrote:
>>>
>>> On Mon, Nov 22, 2010 at 12:01:41PM -0500, Christopher Farnham wrote:
>>>> Using scala's REPL functionality in Cygwin always caused me problems.
>>>
>>> You guys will be happy to know that despite my below-zero interest in
>>> windows I installed windows 7 64bit this weekend and did a bunch of work
>>> trying to close windows repl tickets.
>>>
>>> However the real test will be when I check something in and it breaks
>>> something somewhere in a way I can't see or understand, which is the
>>> usual outcome when I touch anything windows.  Should this happen,
>>> somebody who cares about windows MUST rise to the occasion and help me.
>>> I find it a great distance short of fair that this task falls on me if
>>> there are indeed so many windows users.
>>>
>>
>> I have 3 different Windows environments that I use daily and I can help test
>> on those (WinXP 32 bits, Vista 32 bits, Windows 7 64 bits).
>>
>> I've been hacking with jline for about a week now (see
>> http://github.com/huynhjl/jline). I think I've addressed wrapping issues for
>> xterm (on ubuntu) and cursor display and wrapping for Windows (cygwin in
>> cmd.exe but not rxvt or mintty).  My latest snapshot is at
>> http://github.com/huynhjl/jline/downloads.
>>
>> I've also added some incremental reverse search and it's good enough that
>> I've replaced the jline bundled with scala on all my computers. It works
>> like readline's reverse-i-search:
>>
>> scala> val foo = "bar"
>> foo: java.lang.String = bar
>>
>> (reverse-i-search)`va': val foo = "bar"
>>
>> At this point, I'm interested to look at multi line history support in jline
>> but I would need some help on the REPL side of this.
>>
>> If there is no chance for my fork to be included with scala, I'll probably
>> stop here and look to submit just the Windows fix into the hypothetical next
>> official release of jline.
>>
>> Jean-Laurent
>> --
>> View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Scala-REPL-in-Cy...
>> Sent from the Scala mailing list archive at Nabble.com.
>>
>

Yes, I'm planning to join the discussion. I can see from the mailing list
archive that some discussion happened about a month ago. I just have to
reactivate my sourceforge account and join the mailing list. My guess is the
Windows fix is simpler to merge.

Yes, that required to regenerate the DLL to call
the SetConsoleCursorPosition win32 function.

I'm happy to see all this interest in jline, but people shouldn't get
their hopes up too high too fast. A bunch of us from a number of
different jvm languages have been trying to update jline, for a long
while now. We get bursts of activity and stall. It's easy enough to
get one's own environment doing something sensible, but then (assuming
you can get anyone else to even try your stuff) you will be assaulted by
a range of tedious details because you will always have broken someone.

For starters here's xterm, rxvt, cygwin, the windows command shell,
iterm, osx terminal, screen, and god knows how many things on linux:
each of these have custom settings, each of them run a bunch of
different shells with different quirks, each of them is used in many
different countries. I've lost entire days to ridiculous things like
how ^H is handled.

Also and separately, I've rewritten most of jline in scala along with
new features which must be seen to be believed, but this along with the
project which motivated it have also stalled because I felt like I
couldn't take any more time away from nurturing scala trunk. That code
is not yet public.

In May headius from jruby managed to get some momentum going, so there
are people on the mailing list and we have (unfortunately) at least ten
different forks going on github and elsewhere.

jline-users [at] lists [dot] sourceforge [dot] net

Here are the remotes I see in my current sline repo:

% git remote -v show |grep fetch
bgerm git://github.com/bgerm/jline.git (fetch)
gnodet git://github.com/gnodet/jline.git (fetch)
hns git://github.com/hns/jline.git (fetch)
mikiobraun git://github.com/mikiobraun/jline-fork.git (fetch)
origin git [at] github [dot] com:paulp/jline.git (fetch)
sourceforge git://jline.git.sourceforge.net/gitroot/jline/jline (fetch)

Those are just the ones which had something material different from the
others at the time; it's out of date.

On 12/2/10 11:17 AM, Paul Phillips wrote:
> I'm happy to see all this interest in jline, but people shouldn't get
> their hopes up too high too fast. A bunch of us from a number of
> different jvm languages have been trying to update jline, for a long
> while now. We get bursts of activity and stall. It's easy enough to
> get one's own environment doing something sensible, but then (assuming
> you can get anyone else to even try your stuff) you will be assaulted by
> a range of tedious details because you will always have broken someone.

It's perhaps not relevant to this particular discussion, but I have a Scala
readline-like front-end that supports GNU Readline, JLine, and Editline (the
BSD-licensed readline replacement used on Mac OS X and the BSDs). Basically, it
finds and uses whatever is there, providing a common interface. The Editline
support, which only works on Unix-like systems, is via a java-editline JNI
layer I wrote.

Details are at these links. Direct comments, questions, abuse to me.

http://bmc.github.com/javaeditline/
http://bmc.github.com/grizzled-scala/api/grizzled/readline/package.html

Thank you for the feedback. I've subscribed to the jline mailing list and will
try to see how to have our efforts add up to something.

Scala trunk is important and I hope we'll see your jline in scala work come to
fruition but that should not take away from the compiler work.

Jean-Laurent

On Fri, Dec 03, 2010 at 12:35:54AM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> Thank you for the feedback. I've subscribed to the jline mailing list
> and will try to see how to have our efforts add up to something.

To find out where things were I tried building the jline2 repo mentioned
on the jline list. I see you also tried this, but there are a ton of
changes against regular jline: when you start the repl with that jar
it's not using jline at all, it's catching the failure to create
JLineReader using a simple jline free reader.

In any case, I ported trunk to match its changes and built a
distribution, which is here:

https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23676-b201012...

A few seconds test is all I've given it but it looks great: all the
normal things work, I'm not able to induce the usual cursor location
flailing yet (but I'm not booking that win for a long while) and ctrl-R
search works. On my platform anyway. Let us know about windows.

Thank you Paul!

cmd.exe or cygwin on cmd.exe:
* what works:
- history works
- tab completion works
- basic line editing and cursor movement works (CTRL-A, CTRL-E, CTRL-U, CRTL-U)
- reverse search with CTRL-R works (new feature compared to before)
* issues:
- cursor is not visible when moving around (same as before)
- cannot move back or backspace to first line after line wrap (same as before)
- moving back through history with one line wraps causes the window to scroll
and leaves one line of junk (same as before)
- type some chars, move back with left arrow, pres ESC will cause
StringIndexOutOfBoundsException (same as before but that also affects reverse
search)

cygwin rxvt/mintty: unusable (same as before)

In summary I would say, we are not worse off with jline2 than with jline and we
would gain reverse search. jline2 is active so that's a plus. Though I'm still
digging through the code I'm expecting to figure out the wrapping and the cursor
display. I'll let the jline2 people know when I have something to merge back.

Jean-Laurent

On Sat, Dec 04, 2010 at 06:32:08AM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> In summary I would say, we are not worse off with jline2 than with
> jline and we would gain reverse search. jline2 is active so that's a
> plus. Though I'm still digging through the code I'm expecting to
> figure out the wrapping and the cursor display. I'll let the jline2
> people know when I have something to merge back.

I agree it's highly unlikely we're worse off. I'll see about shipping
it with 2.9 if we can get the ducks lined up. Based on the git history
the maintainers have done a fine job integrating the relevant patches,
which were pretty widely dispersed. There are more gratuitous
incompatibilities than I'd choose but it's a tiny price to pay in the
grand scheme of things (and besides, I already paid it.)

> I've been hacking with jline for about a week now (see
> http://github.com/huynhjl/jline). I think I've addressed wrapping issues for
> xterm (on ubuntu) and cursor display and wrapping for Windows

I had to delete this repository to facilitate the merge to the jline2 branch. My
changes are now at http://github.com/huynhjl/jline2. The original jar snapshot
from my jline fork can be downloaded here:
https://github.com/huynhjl/jline2/downloads

Jean-Laurent

On Sat, Dec 04, 2010 at 11:02:53PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> I had to delete this repository to facilitate the merge to the jline2
> branch. My changes are now at http://github.com/huynhjl/jline2. The
> original jar snapshot from my jline fork can be downloaded here:
> https://github.com/huynhjl/jline2/downloads

I rebuilt jline2 with all your changes and built a new trunk
distribution with it.

https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2....

On the presumption that we can't possibly be the only people with jline
issues, let me just say: people, it would really behoove you to try this
out and report issues, especially if you're on any platform but OSX.

Hi Paul,

I'm running on Linux, should I try it too? :-)

The scala command line is basically unusable here, because it doesn't
wrap lines.
If you wrap too many lines by mistake, it will mess up the remaining
session.

You have to kill it and restart.

Bye,

Simon

On Mon, Dec 06, 2010 at 07:06:31PM +0100, Simon Ochsenreither wrote:
> I'm running on Linux, should I try it too? :-)
>
> The scala command line is basically unusable here, because it doesn't
> wrap lines. If you wrap too many lines by mistake, it will mess up the
> remaining session.
>
> You have to kill it and restart.

I'm familiar with the behavior, it does the same thing for me, that's
why I don't think I should have to work too hard to convince people to
try it. Yes, linux is in the category of not-OSX. Also "unusable" is a
stretch: I spent all my time in the repl, so I know it's usable. But it
could be a hell of a lot more usable, thus the effort you see taking
place before your eyes!

Tested under Windows XP under cmd.exe and cygwin (TERM=cygwin).

Working:

Cursor visible as it moves
line wrapping
backspace to previous line
scroll through history containing multi-line entries without
unnecessary scrolling
CTRL-R i-search.
CTRL-W/A/E/H for editing

Broken:

Nothing I could find

Great work!

-jason

On Mon, Dec 6, 2010 at 6:46 PM, Paul Phillips wrote:
> I rebuilt jline2 with all your changes and built a new trunk
> distribution with it.
>
> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2....
>
> On the presumption that we can't possibly be the only people with jline
> issues, let me just say: people, it would really behoove you to try this
> out and report issues, especially if you're on any platform but OSX.

2010/12/6 Paul Phillips :
> On Sat, Dec 04, 2010 at 11:02:53PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
>> I had to delete this repository to facilitate the merge to the jline2
>> branch. My changes are now at http://github.com/huynhjl/jline2. The
>> original jar snapshot from my jline fork can be downloaded here:
>> https://github.com/huynhjl/jline2/downloads
>
> I rebuilt jline2 with all your changes and built a new trunk
> distribution with it.
>
> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2....
>
> On the presumption that we can't possibly be the only people with jline
> issues, let me just say: people, it would really behoove you to try this
> out and report issues, especially if you're on any platform but OSX.
>

I tested this under Win7 x64 with cmd.exe and it seems much improved.
No invisible cursor and backspace to previous line, too. I did not
find any regressions.

Good work!

Regards,
Ruediger

2010/12/6 Paul Phillips <paulp [at] improving [dot] org>:
> On Sat, Dec 04, 2010 at 11:02:53PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
>> I had to delete this repository to facilitate the merge to the jline2
>> branch. My changes are now at http://github.com/huynhjl/jline2. The
>> original jar snapshot from my jline fork can be downloaded here:
>> https://github.com/huynhjl/jline2/downloads
>
> I rebuilt jline2 with all your changes and built a new trunk
> distribution with it.
>
> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2.tar.gz
>
> On the presumption that we can't possibly be the only people with jline
> issues, let me just say: people, it would really behoove you to try this
> out and report issues, especially if you're on any platform but OSX.
>

I tested this under Win7 x64 with cmd.exe and it seems much improved.
No invisible cursor and backspace to previous line, too. I did not
find any regressions.

Good work!

Regards,
Ruediger

On 6 December 2010 19:48, Ruediger Keller <ruediger [at] rk42 [dot] de> wrote:

2010/12/6 Paul Phillips <paulp [at] improving [dot] org>:
> On Sat, Dec 04, 2010 at 11:02:53PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
>> I had to delete this repository to facilitate the merge to the jline2
>> branch. My changes are now at http://github.com/huynhjl/jline2. The
>> original jar snapshot from my jline fork can be downloaded here:
>> https://github.com/huynhjl/jline2/downloads
>
> I rebuilt jline2 with all your changes and built a new trunk
> distribution with it.
>
> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2.tar.gz
>
> On the presumption that we can't possibly be the only people with jline
> issues, let me just say: people, it would really behoove you to try this
> out and report issues, especially if you're on any platform but OSX.
>

I tested this under Win7 x64 with cmd.exe and it seems much improved.
No invisible cursor and backspace to previous line, too. I did not
find any regressions.

Good work!

Regards,
Ruediger

Looking good on Windows XP 32bit with cmd.exe (don't ask, it's a corporate thing)At least, as good as anything *can* look on a locked down company laptop running XP...


--
Kevin Wright

mail / gtalk / msn : kev [dot] lee [dot] wright [at] gmail [dot] com
pulse / skype: kev.lee.wright
twitter: @thecoda

Paul> I rebuilt jline2 with all your changes and built a new trunk
Paul> distribution with it.
Paul> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23690-jline2....

Christopher> Unfortunately, I'm experiencing some issues with your build. I'm
not sure how I
Christopher> might be able to help diagnose the problem except to show you a
quick example:
Christopher> CYGWIN=tty notitle glob binmode ntsec

I can reproduce with CYGWIN set to the same value. I'll troubleshoot more
tonight. I can tell that if I remove tty before starting cygwin it works fine,
but that if I just set it before starting scala like CYGWIN="notitle glob
binmode ntsec" ./scala; I still have the problem.

Paul, I've just noticed that the build with jline2 doesn't save history
permanently. When I start a new session it does fetch the history from my 2.8.x
session but not from the jline2 custom 2.9.0 build past sessions.

On other note I did more testing with Ubuntu and the non graphical console
(CTRL-ALT-F1) and with putty/ssh ssh-ing over from Win7 to Ubuntu and things
worked well. I also tested on Win7 with java 32 bits and 64 bits and things
worked fine too.

Jean-Laurent

I can tell that if I remove tty before starting cygwin it works fine,

but that if I just set it before starting scala like CYGWIN="notitle glob

binmode ntsec" ./scala; I still have the problem.

I can tell that if I remove tty before starting cygwin it works fine,
but that if I just set it before starting scala like CYGWIN="notitle glob
binmode ntsec" ./scala; I still have the problem.

On Mon, Dec 06, 2010 at 09:46:43AM -0800, Paul Phillips wrote:
> On the presumption that we can't possibly be the only people with jline
> issues, let me just say: people, it would really behoove you to try this
> out and report issues, especially if you're on any platform but OSX.

I just tested this out under Ubuntu (inside screen on Gnome Terminal).
All the long-line stuff (wrapping, backspace, history) seem to work
fine. I tested it with the assumption that it should work like
bash/readline, since it shares many of those key bindings, and it's
what *I* expect ;)

Overall it worked great and seemed like a big improvement!

Here are all the things that worked as I expected:

C-a, C-b, C-d, C-e, C-f, C-h, TAB (C-i), C-k (sort of), C-l, C-n,
C-p, C-r, C-u, C-w

Things that didn't work that I really miss:

C-y (should yank (paste) text killed via C-k back into buffer)
M-f (move forward by word)
M-b (move backward by word)
M-d (delete word right)
M-Backspace (delete word left)

Things that did something other than I expect (but are maybe ok):

C-c (quits the program, instead of cancelling the current command)
C-s (should cycle back forward through histroy, like C-r in reverse)
C-x (repeats char, expected set mark/jump between mark and cursor)

Absolutely great work!

This is a difference between day and night here (Ubuntu 10.10,
gnome-terminal + zsh)!
Everything works as expected even the line breaks!

Thanks for that great work, this is absolutely brilliant!

Bye,

Simon

PS: Are there any official plans to replace the current JLine library
with the new one in the next release?
Or is it possible to just repalce the jar file?

Hi Christopher,

I tested under 2.8.1 (and the old jline) and the issue is there too. So this
does not look like a regression issue with jline2 and this is still the same
issue you mentioned when you started this thread, right? I haven't given up hope
to find some way to address it but this may take longer.

Overall it looks like we are still ahead with jline2.

Let me know how the mintty helps (from my quick try the other day, it was not
much better).

Jean-Laurent

________________________________
From: Christopher Farnham
To: jlh276-gh [at] yahoo [dot] com
Cc: Paul Phillips ; scala [at] listes [dot] epfl [dot] ch
Sent: Tue, December 7, 2010 8:02:53 AM
Subject: Re: [scala] Re: Scala REPL in Cygwin

I can tell that if I remove tty before starting cygwin it works fine,
but that if I just set it before starting scala like CYGWIN="notitle globbinmode

ntsec" ./scala; I still have the problem.
Perhaps, then, the best way to fix this is via the 'scala' bash script where we
unset then reset tty via 'stty' like I did in my initial, candidate patch. That
is how the JRuby folks are currently handling tty.

I'm also thinking of moving to 'mintty' in order to avoid these problems. I
began using Cygwin back in '99 and i'm still dealing with the same problems.
Time to move to something a little better. (old habits die hard) (i'm old)

Christopher Farnham
chris [dot] farnham [at] gmail [dot] com

On Tue, Dec 7, 2010 at 10:39 AM, wrote:

I can tell that if I remove tty before starting cygwin it works fine,
>but that if I just set it before starting scala like CYGWIN="notitle glob
>binmode ntsec" ./scala; I still have the problem.

So this
does not look like a regression issue with jline2 and this is still the same
issue you mentioned when you started this thread, right?

I've manually tested the patch in 2097 against the jline2 build that Paul
created (I edited the scala script) and it seems to work nicely in:
- rxvt under cygwin (rxvt-cygwin-native)
- mintty under cygwin
- cygwin
- It does not affect Ubuntu xterm adversely

I tried to see if I could modify jline2 itself to do the same stty settings, but
I'm unable to make stty calls from the Java process. (ioctl complains).

I would vote to apply that patch for 2.9.

Jean-Laurent

I tried scala-2.9.0.r23690-b20101205221300 and from
https://github.com/paulp/scala-full/downloads with rxvt in Cygwin.

Everything works, PROVIDED you set stty and
-Djline.terminal=jline.UnixTerminal in the usual way
(http://www.java.net/blog/cayhorstmann/archive/2010/11/24/report-sewer-ho...).

If you don't do that, nothing works at all, just like with 2.8.

Cheers,

Cay

On Fri, Dec 10, 2010 at 11:03:49PM -0800, Cay Horstmann wrote:
> If you don't do that, nothing works at all, just like with 2.8.

I have a new build up which maybe fixes that, and other little things.
I don't have parallels where I am right now so it's completely untested
on windows: if I can get a sanity check from someone, I'll check it into
trunk.

https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23736-jline2....

I'm on Window 7 x64. scala.bat isn't working for me in that build. It
seems there are several places where the quoting is wrong.

For example in all checks of this style:
if "%_JAVACMD%"=="" (
the quoting around the variable seems wrong. Running it turns these
checks into if """"=="", making them fail. But it seems they should
succeed.

Regards,
Ruediger

2010/12/11 Paul Phillips :
> On Fri, Dec 10, 2010 at 11:03:49PM -0800, Cay Horstmann wrote:
>> If you don't do that, nothing works at all, just like with 2.8.
>
> I have a new build up which maybe fixes that, and other little things.
> I don't have parallels where I am right now so it's completely untested
> on windows: if I can get a sanity check from someone, I'll check it into
> trunk.
>
> https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23736-jline2....
>
> --
> Paul Phillips      | You think you know when you learn, are more sure
> Analgesic          | when you can write, even more when you can teach,
> Empiricist         | but certain when you can program.  -- Alan Perlis
> i pull his palp!   |----------* http://www.improving.org/paulp/ *----------
>

Cygwin seems to work fine under cmd.exe or rxvt (Win7 64 bits) - testing
includes reverse search, wrapping, cursor visibility, tab completion. So it's
good on that end.

But scala.bat under cmd.exe prints this:

C:\setup\scala\scala-2.9.0.r23736-b20101211103605\bin>scala.bat
Files\Java\jre6\bin\java.exe""=="" was unexpected at this time.

After changing the path to a JAVA_HOME that does not contain spaces:
C:\setup\scala\scala-2.9.0.r23736-b20101211103605\bin>set
JAVA_HOME=C:\util\java\jdk1.6.0_21

C:\setup\scala\scala-2.9.0.r23736-b20101211103605\bin>scala
Exception in thread "main" java.lang.NoClassDefFoundError:
Caused by: java.lang.ClassNotFoundException:
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: . Program will exit.

So it seems there are some issues with the scala.bat. Is there anything specific
you'd like me to try out in particular?

Jean-Laurent

On Sat, Dec 11, 2010 at 12:12:29PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> But scala.bat under cmd.exe prints this:

Well that figures, it looks like I merged some bad juju. I will cook
another one.

Here is another try. If I've booched this one too I'll put it on hold
until tuesday when I can try it in windows. Let me know.

https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23736-jline.t...

On Sat, Dec 11, 2010 at 12:12:29PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> So it seems there are some issues with the scala.bat. Is there
> anything specific you'd like me to try out in particular?

I'm looking especially (maybe only) for any behavior which represents a
regression against 2.8.1. If it appears to be "strictly superior" even
if only marginally then I'll check it in and work out the rest as I go.

This one works fine with rxvt. It's a definite improvement over 2.8. I
peeked at the script, and it looks fine too--at least the
rxvt-specific parts.

Cheers,

Cay

2010/12/11 Paul Phillips :
> Here is another try.  If I've booched this one too I'll put it on hold
> until tuesday when I can try it in windows.  Let me know.
>
>  https://github.com/downloads/paulp/scala-full/scala-2.9.0.r23736-jline.tar.gz
>
> On Sat, Dec 11, 2010 at 12:12:29PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
>> So it seems there are some issues with the scala.bat. Is there
>> anything specific you'd like me to try out in particular?
>
> I'm looking especially (maybe only) for any behavior which represents a
> regression against 2.8.1.  If it appears to be "strictly superior" even
> if only marginally then I'll check it in and work out the rest as I go.
>
> --
> Paul Phillips      | We must respect the other fellow's religion, but only
> Vivid              | in the sense and to the extent that we respect his
> Empiricist         | theory that his wife is beautiful and his children smart.
> slap pi uphill!    |     -- H. L. Mencken
>

OK, new jline is checked in. If I can avoid breaking the build between
now and a few hours from now, it will be in tonight's nightly.

http://www.scala-lang.org/node/212/distributions

Awesome, thanks for merging this in.

Jean-Laurent

On Sun, Dec 12, 2010 at 12:51:06PM -0800, jlh276-gh [at] yahoo [dot] com wrote:
> Awesome, thanks for merging this in.

Sure, but let's not count too many chickens yet. I already have a
reasonably serious problem: if I paste some code into the repl which is
longer than one line, I get either only the one line, or sometimes a
truncated version: but definitely not the nice predictable "paste a
giant object and it just works" behavior I've come to like. Do you
not-on-OSXers experience this?

I eventually figured out my paste issue only arose on lines longer than
the terminal width, and pinned it on "newlineAtWrap()" which is one of
your additions. I rebuilt jline with it disabled and committed that to
trunk. Can you assess the situation? If that method is doing only what
I think it is I can easily live without it, but I'm not sure.

Ok, newlineAtWrap was added by the jline2 folks before my fork to (I think)
handle terminals where the cursor does not wrap when a character is printed in
the last row. I only added an interface method to disable it for Win32.

On terminals where the cursor does not wrap, the behavior is not quite right for
wrapping (as well when backspacing on a wrapped line, when the cursor is in the
middle of the line). I will assess the situation further to see if I can
pinpoint that terminal behavior better.

Paul,

I submitted a "fix" on github. Following your lead, I totally got rid of
newlineAtWrap and addressed the wrapping issues in a different way. The issues
were behavior on backspace on wrapped line (whether the cursor while backspacing
is on the second line or the first line).

The result is that the cursor does not move immediately to the next line when
you reach the end of the line, but on the other hand, I think I fixed a few bugs
I hadn't detected before as well.

Here is a snapshot
https://github.com/downloads/huynhjl/jline2/jline-2.6-SNAPSHOT.jar if you want
to do a quick test before merging.

Jean-Laurent