- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
Updating jline2
Wed, 2012-01-18, 03:09
Hi,
I'd really like to vi-mode line editing at the Scala console. I can use rlwrap to get readline (running 'rlwrap scala -Xnojline'), but then other features of the REPL don't work (such as tab-completion).
I looked into jline, and AFAICT Scala is using the jline2 here
https://github.com/paulp/jline2
which is a fork of the 'official' jline2 here:
https://github.com/jline/jline2
This 'official' jline2 implemented vi-mode several months ago, but unfortunately the fork doesn't include any recent upstream changes. Running git merge-base on the two project masters shows the most recent common commit as 6c4271a4949f79d46c17bafb0da31c8372eb2d0b from 1 year ago.
Can we update the jline2 (including the copy in Scala itself) to incorporate upstream changes to jline2? (More generally, it might be nice -- if possible -- to separate out changes to jline2 that Scala needs from the jline2 project to make the fork unnecessary).
-Caleb Spare
I'd really like to vi-mode line editing at the Scala console. I can use rlwrap to get readline (running 'rlwrap scala -Xnojline'), but then other features of the REPL don't work (such as tab-completion).
I looked into jline, and AFAICT Scala is using the jline2 here
https://github.com/paulp/jline2
which is a fork of the 'official' jline2 here:
https://github.com/jline/jline2
This 'official' jline2 implemented vi-mode several months ago, but unfortunately the fork doesn't include any recent upstream changes. Running git merge-base on the two project masters shows the most recent common commit as 6c4271a4949f79d46c17bafb0da31c8372eb2d0b from 1 year ago.
Can we update the jline2 (including the copy in Scala itself) to incorporate upstream changes to jline2? (More generally, it might be nice -- if possible -- to separate out changes to jline2 that Scala needs from the jline2 project to make the fork unnecessary).
-Caleb Spare
Wed, 2012-01-18, 14:21
#2
Re: Updating jline2
+ scala-internals
This email most likely belongs on Scala Internals.
Paul Phillips would probably be able to update jline2 used in Scala, since it is his fork and he knows which bugs he needed to fix. If you created a merge of the new features with paulp's branch such that the interface stays the same, I'm sure he'd gladly accept that pull request.
One thing I'm trying to do is update the Scala build to use a jline that is deployed on maven-central rather than our own. This would make Paul's fork be *the* Scala jline and would most likely get deployed under org.scala-lang:jline:x.y.z and used for scala releases. Haven't quite gotten to this point yet, but we're almost there.
- Josh
On Tue, Jan 17, 2012 at 9:08 PM, Caleb Spare <cespare@gmail.com> wrote:
This email most likely belongs on Scala Internals.
Paul Phillips would probably be able to update jline2 used in Scala, since it is his fork and he knows which bugs he needed to fix. If you created a merge of the new features with paulp's branch such that the interface stays the same, I'm sure he'd gladly accept that pull request.
One thing I'm trying to do is update the Scala build to use a jline that is deployed on maven-central rather than our own. This would make Paul's fork be *the* Scala jline and would most likely get deployed under org.scala-lang:jline:x.y.z and used for scala releases. Haven't quite gotten to this point yet, but we're almost there.
- Josh
On Tue, Jan 17, 2012 at 9:08 PM, Caleb Spare <cespare@gmail.com> wrote:
Hi,
I'd really like to vi-mode line editing at the Scala console. I can use rlwrap to get readline (running 'rlwrap scala -Xnojline'), but then other features of the REPL don't work (such as tab-completion).
I looked into jline, and AFAICT Scala is using the jline2 here
https://github.com/paulp/jline2
which is a fork of the 'official' jline2 here:
https://github.com/jline/jline2
This 'official' jline2 implemented vi-mode several months ago, but unfortunately the fork doesn't include any recent upstream changes. Running git merge-base on the two project masters shows the most recent common commit as 6c4271a4949f79d46c17bafb0da31c8372eb2d0b from 1 year ago.
Can we update the jline2 (including the copy in Scala itself) to incorporate upstream changes to jline2? (More generally, it might be nice -- if possible -- to separate out changes to jline2 that Scala needs from the jline2 project to make the fork unnecessary).
-Caleb Spare
Thu, 2012-01-19, 00:31
#3
Re: Updating jline2
-scala-user
Thanks Josh. I didn't originally post this to scala-internals because the group description says
and I am not a Scala contributor.
I am looking into doing the merge myself, but unfortunately the merge conflicts will require a fair amount of massaging to correct. Hopefully I'll get a pull request together this week.
Has anyone talked to the creator of jline2 to see what he thinks about the situation? It's unfortunate that he's busy adding useful features to it while Scala has resorted to an independent fork. Wouldn't it be better if the two projects were unified, instead of trying to just promote the Scala jline?
-Caleb
On Wed, Jan 18, 2012 at 5:15 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
Thanks Josh. I didn't originally post this to scala-internals because the group description says
Please only post new threads if you commit to the Scala code base, (...)
and I am not a Scala contributor.
I am looking into doing the merge myself, but unfortunately the merge conflicts will require a fair amount of massaging to correct. Hopefully I'll get a pull request together this week.
Has anyone talked to the creator of jline2 to see what he thinks about the situation? It's unfortunate that he's busy adding useful features to it while Scala has resorted to an independent fork. Wouldn't it be better if the two projects were unified, instead of trying to just promote the Scala jline?
-Caleb
On Wed, Jan 18, 2012 at 5:15 AM, Josh Suereth <joshua.suereth@gmail.com> wrote:
+ scala-internals
This email most likely belongs on Scala Internals.
Paul Phillips would probably be able to update jline2 used in Scala, since it is his fork and he knows which bugs he needed to fix. If you created a merge of the new features with paulp's branch such that the interface stays the same, I'm sure he'd gladly accept that pull request.
One thing I'm trying to do is update the Scala build to use a jline that is deployed on maven-central rather than our own. This would make Paul's fork be *the* Scala jline and would most likely get deployed under org.scala-lang:jline:x.y.z and used for scala releases. Haven't quite gotten to this point yet, but we're almost there.
- Josh
On Tue, Jan 17, 2012 at 9:08 PM, Caleb Spare <cespare@gmail.com> wrote:
Hi,
I'd really like to vi-mode line editing at the Scala console. I can use rlwrap to get readline (running 'rlwrap scala -Xnojline'), but then other features of the REPL don't work (such as tab-completion).
I looked into jline, and AFAICT Scala is using the jline2 here
https://github.com/paulp/jline2
which is a fork of the 'official' jline2 here:
https://github.com/jline/jline2
This 'official' jline2 implemented vi-mode several months ago, but unfortunately the fork doesn't include any recent upstream changes. Running git merge-base on the two project masters shows the most recent common commit as 6c4271a4949f79d46c17bafb0da31c8372eb2d0b from 1 year ago.
Can we update the jline2 (including the copy in Scala itself) to incorporate upstream changes to jline2? (More generally, it might be nice -- if possible -- to separate out changes to jline2 that Scala needs from the jline2 project to make the fork unnecessary).
-Caleb Spare
Thu, 2012-01-19, 06:41
#4
Re: Re: Updating jline2
Someone should reword that, it's too aggressive.
On Wed, Jan 18, 2012 at 3:22 PM, Caleb Spare <cespare@gmail.com> wrote:
I will root for you. I tried using the jline2 repo out of the box and though it compiled, the repl dies on startup in a mysterious way. Perhaps you will have better luck.
There isn't a situation, and there aren't really two projects. We went in circles with jline a long, long time before this, and I wasn't going through another major release without working arrow keys if I could help it. Having a repository for it is just sensible development practice. Now that the pendulum has swung the other way, yes I'd rather use stock jline if we can. Do send that pull request if you can navigate the waters.
On Wed, Jan 18, 2012 at 3:22 PM, Caleb Spare <cespare@gmail.com> wrote:
I am looking into doing the merge myself, but unfortunately the merge conflicts will require a fair amount of massaging to correct. Hopefully I'll get a pull request together this week.
I will root for you. I tried using the jline2 repo out of the box and though it compiled, the repl dies on startup in a mysterious way. Perhaps you will have better luck.
Has anyone talked to the creator of jline2 to see what he thinks about the situation? It's unfortunate that he's busy adding useful features to it while Scala has resorted to an independent fork. Wouldn't it be better if the two projects were unified, instead of trying to just promote the Scala jline?
There isn't a situation, and there aren't really two projects. We went in circles with jline a long, long time before this, and I wasn't going through another major release without working arrow keys if I could help it. Having a repository for it is just sensible development practice. Now that the pendulum has swung the other way, yes I'd rather use stock jline if we can. Do send that pull request if you can navigate the waters.
Thu, 2012-01-19, 20:41
#5
Re: Re: Updating jline2
On Wed, 18 Jan 2012 21:40:58 -0800
Paul Phillips wrote:
> Someone should reword that, it's too aggressive.
>
> On Wed, Jan 18, 2012 at 3:22 PM, Caleb Spare wrote:
>
> > I am looking into doing the merge myself, but unfortunately the merge
> > conflicts will require a fair amount of massaging to correct. Hopefully
> > I'll get a pull request together this week.
> >
>
> I will root for you. I tried using the jline2 repo out of the box and
> though it compiled, the repl dies on startup in a mysterious way. Perhaps
> you will have better luck.
For what it is worth, there is a jline 1.0 that appears compatible with jline 0.9.94 but fixes some of the more visible issues. I upgraded to that for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on linux, and doesn't leak resources as much. I think it I assume there are improvements in jline2 that Scala uses and you are aware of 1.0, but I wasn't, so I figured I'd mention it.
-Mark
Thu, 2012-01-19, 22:21
#6
Re: Re: Updating jline2
On Thu, Jan 19, 2012 at 11:29 AM, Mark Harrah <dmharrah@gmail.com> wrote:
For what it is worth, there is a jline 1.0 that appears compatible with jline 0.9.94 but fixes some of the more visible issues. I upgraded to that for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on linux, and doesn't leak resources as much. I think it I assume there are improvements in jline2 that Scala uses and you are aware of 1.0, but I wasn't, so I figured I'd mention it.
No, I didn't realize the multiplicity of choices. That makes for an interesting situation.
I wonder who decided that "threshold" wasn't sufficiently misspelled and fixed it. (Or maybe it was only changed in the jline2 fork and it persisted from 0.9.94, hopefully it's that.)
[scalacfork] /scala/trunk/src/compiler/scala/tools/nsc/interpreter/JLineReader.scala:66: error: value setAutoprintThreshold is not a member of JLineReader.this.JLineConsoleReader[scalacfork] this setAutoprintThreshold 400 // max completion candidates without warning [scalacfork] ^
// Why that is
public void setAutoprintThreshhold(final int autoprintThreshhold)
Fri, 2012-01-20, 16:01
#7
Re: Re: Updating jline2
On Thu, Jan 19, 2012 at 11:29 AM, Mark Harrah <dmharrah@gmail.com> wrote:
For what it is worth, there is a jline 1.0 that appears compatible with jline 0.9.94 but fixes some of the more visible issues. I upgraded to that for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on linux, and doesn't leak resources as much. I think it I assume there are improvements in jline2 that Scala uses and you are aware of 1.0, but I wasn't, so I figured I'd mention it.
I have performed the necessary surgeries to eliminate the scala jline fork and use jline 1.0 as a binary dependency. This isn't going to make the original guy that much happier since he was after something which was in jline2, but this will also make an eventual move to jline2 easier.
In the meantime, it should mean sbt and scala can use the same jline. Does this create any new opportunities or eliminate categories of problems? Is there anything in particular I should do on the scala side to realize them?
Fri, 2012-01-20, 16:21
#8
Re: Re: Updating jline2
On Fri, Jan 20, 2012 at 3:58 PM, Paul Phillips <paulp@improving.org> wrote:
I couldn't find a way to switch both the SBT and REPL JLines to use UnsupportedTerminal, as they were both configured by the system property `jline.terminal`.
To future proof against this next time around, the REPL could include it's own configuration property (e.g. scala.jline.terminal), and configure JLine programmatically.
-jason
https://github.com/orfjackal/idea-sbt-plugin/commit/ead8117642202a9cfe994469b142dc966dae3554
On Thu, Jan 19, 2012 at 11:29 AM, Mark Harrah <dmharrah@gmail.com> wrote:
For what it is worth, there is a jline 1.0 that appears compatible with jline 0.9.94 but fixes some of the more visible issues. I upgraded to that for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on linux, and doesn't leak resources as much. I think it I assume there are improvements in jline2 that Scala uses and you are aware of 1.0, but I wasn't, so I figured I'd mention it.
I have performed the necessary surgeries to eliminate the scala jline fork and use jline 1.0 as a binary dependency. This isn't going to make the original guy that much happier since he was after something which was in jline2, but this will also make an eventual move to jline2 easier.
In the meantime, it should mean sbt and scala can use the same jline. Does this create any new opportunities or eliminate categories of problems? Is there anything in particular I should do on the scala side to realize them?
I couldn't find a way to switch both the SBT and REPL JLines to use UnsupportedTerminal, as they were both configured by the system property `jline.terminal`.
To future proof against this next time around, the REPL could include it's own configuration property (e.g. scala.jline.terminal), and configure JLine programmatically.
-jason
https://github.com/orfjackal/idea-sbt-plugin/commit/ead8117642202a9cfe994469b142dc966dae3554
Fri, 2012-01-20, 22:11
#9
Re: Re: Updating jline2
This isn't going to make the original guy that much happier since he was after something which was in jline2, but this will also make an eventual move to jline2 easier.
Can you explain this? What is the actual relationship between jline, jline2, and the jline2 scala fork, and how would moving to jline make using jline2 easier?
At this point, I'm wondering whether making my own standalone scala console based on jline2 would be the easiest way to get the readline features I need. (I also kind of wonder whether just using readline through jna would be better. In my limited experience, even established readline replacements like editline seem quirky, so my hopes are fairly low that jline2 will behave the way I want.)
-Caleb
On Fri, Jan 20, 2012 at 7:11 AM, Jason Zaugg <jzaugg@gmail.com> wrote:
On Fri, Jan 20, 2012 at 3:58 PM, Paul Phillips <paulp@improving.org> wrote:
On Thu, Jan 19, 2012 at 11:29 AM, Mark Harrah <dmharrah@gmail.com> wrote:
For what it is worth, there is a jline 1.0 that appears compatible with jline 0.9.94 but fixes some of the more visible issues. I upgraded to that for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on linux, and doesn't leak resources as much. I think it I assume there are improvements in jline2 that Scala uses and you are aware of 1.0, but I wasn't, so I figured I'd mention it.
I have performed the necessary surgeries to eliminate the scala jline fork and use jline 1.0 as a binary dependency. This isn't going to make the original guy that much happier since he was after something which was in jline2, but this will also make an eventual move to jline2 easier.
In the meantime, it should mean sbt and scala can use the same jline. Does this create any new opportunities or eliminate categories of problems? Is there anything in particular I should do on the scala side to realize them?
I couldn't find a way to switch both the SBT and REPL JLines to use UnsupportedTerminal, as they were both configured by the system property `jline.terminal`.
To future proof against this next time around, the REPL could include it's own configuration property (e.g. scala.jline.terminal), and configure JLine programmatically.
-jason
https://github.com/orfjackal/idea-sbt-plugin/commit/ead8117642202a9cfe994469b142dc966dae3554
Fri, 2012-01-20, 22:21
#10
Re: Re: Updating jline2
On Fri, Jan 20, 2012 at 1:05 PM, Caleb Spare <cespare@gmail.com> wrote:
Can you explain this? What is the actual relationship between jline, jline2, and the jline2 scala fork, and how would moving to jline make using jline2 easier?
jline is a long-abandoned project recently revived long enough to implement some key bugfixes and desirable features in a backward-compatible way. jline2 is what various people have worked on for years because jline was dead. Moving to jline makes eventually using jline2 easier because it puts us back in the mainstream of jvm languages where we have some chance of benefiting from the efforts of others rather than drifting away on our private island.
At this point, I'm wondering whether making my own standalone scala console based on jline2 would be the easiest way to get the readline features I need. (I also kind of wonder whether just using readline through jna would be better. In my limited experience, even established readline replacements like editline seem quirky, so my hopes are fairly low that jline2 will behave the way I want.)
All I can tell you is that if you go all the way back to the drawing board you will have your work cut out for you. I guess it's easier when you don't have to worry about anyone who isn't you, but not enough easier.
Fri, 2012-01-20, 23:31
#11
Re: Re: Updating jline2
On Fri, 20 Jan 2012 06:58:26 -0800
Paul Phillips wrote:
> On Thu, Jan 19, 2012 at 11:29 AM, Mark Harrah wrote:
>
> > For what it is worth, there is a jline 1.0 that appears compatible with
> > jline 0.9.94 but fixes some of the more visible issues. I upgraded to that
> > for the upcoming sbt 0.12.0 and it fixes line wrapping, the delete key on
> > linux, and doesn't leak resources as much. I think it I assume there are
> > improvements in jline2 that Scala uses and you are aware of 1.0, but I
> > wasn't, so I figured I'd mention it.
> >
>
> I have performed the necessary surgeries to eliminate the scala jline fork
> and use jline 1.0 as a binary dependency. This isn't going to make the
> original guy that much happier since he was after something which was in
> jline2, but this will also make an eventual move to jline2 easier.
Right.
> In the meantime, it should mean sbt and scala can use the same jline. Does
> this create any new opportunities or eliminate categories of problems? Is
> there anything in particular I should do on the scala side to realize them?
I think it should work ok. sbt embeds jline inside the launcher because it uses native code, so you can test it out by just rebuilding the launcher (the 'proguard' action) and using that launcher for a project. The 0.12 branch has the jline version bumped to 1.0 or you can do it yourself (in project/Util.scala I think) on another branch.
I expect the major opportunity for Scala is as you say: moving to the mainstream jline and going from there. I know you want to work towards unifying the repl and sbt and one thing I think would make this easier is if it is possible to pull the repl out of the Scala build. That way, if we have a common library, say wrappers around jline like sbt's tab completion parser combinators or APIs on top of the compiler, we aren't tied to the Scala release cycle. I'm not exactly sure that moving to mainstream jline 1.0 is necessary for this, but syncing the version used by sbt and the repl seems like it would help.
-Mark
This email most likely belongs on Scala Internals.
Paul Phillips would probably be able to update jline2 used in Scala, since it is his fork and he knows which bugs he needed to fix. If you created a merge of the new features with paulp's branch such that the interface stays the same, I'm sure he'd gladly accept that pull request.
One thing I'm trying to do is update the Scala build to use a jline that is deployed on maven-central rather than our own. This would make Paul's fork be *the* Scala jline and would most likely get deployed under org.scala-lang:jline:x.y.z and used for scala releases. Haven't quite gotten to this point yet, but we're almost there.
- Josh
On Tue, Jan 17, 2012 at 9:08 PM, Caleb Spare <cespare@gmail.com> wrote: