get, or else / plus: times minus

On Sun, Aug 30, 2009 at 4:00 PM, Paul Phillips wrote:
> As long as everyone is gathered around the API bikeshed warming their
> hands against the cold, it was suggested by some wise soul that:
>
>  getOrElse should be named orElse
>  orElse should be named or

And maybe ...

get should be named apply

I'm not being entirely serious, but I would love to be able to say
goodbye to get and set prefixes, or at least relegate them to the role
of indicating that "Here be Java" ;-)

Cheers,

Miles

+1 on or and orElse

I just showed a coworker how to refactor algorithms using Option.
This would have helped I think.

Sent from my iPhone

On Aug 30, 2009, at 11:00 AM, Paul Phillips wrote:

> As long as everyone is gathered around the API bikeshed warming their
> hands against the cold, it was suggested by some wise soul that:
>
> getOrElse should be named orElse
> orElse should be named or
>
> I realize this will never happen, but oh it'd be so nice.
>
> On the "times" subject, I will revert it. I assume those of you
> unable
> to locate any use cases never looked at the benchmark I sent, which
> used
> it for two distinct purposes: once to run the benchmark, and once as
> one
> of the subjects of the benchmark, implementing this method from
> RichString (which is what led me to investigate it in the first place,
> as I was trying to find the fastest ways to do string-as-seq ops.)
>
> def * (n: Int): String = {
> val buf = new StringBuilder
> for (i <- 0 until n) buf append self // current implementation
> // n times (buf append self) // alternatively...
> buf.toString
> }
>
> Martin is correct that with -optimise on, the speed difference
> disappears. So that's good.
>
> In my view the meaningful question was neglected: why does someone who
> writes an outside library feel like they have to intervene in the
> design
> of the standard library? I mean, I know why: but can we seek a design
> that reduces the need of people working on other projects to lobby?
> The
> question of "times" is irrelevant, that cannot scale: and if the
> standard library is to be constrained by DSLs it will suffer for it.
>

Oh, also, as I already stated, I think the scala library should always
get precedence for common utility methods and namespaces.

I do like Bill's suggestion for somehow limited scope of some of these
(or controlling which are available), but in general I agree with Paul.

Sent from my iPhone

On Aug 30, 2009, at 11:00 AM, Paul Phillips wrote:

> As long as everyone is gathered around the API bikeshed warming their
> hands against the cold, it was suggested by some wise soul that:
>
> getOrElse should be named orElse
> orElse should be named or
>
> I realize this will never happen, but oh it'd be so nice.
>
> On the "times" subject, I will revert it. I assume those of you
> unable
> to locate any use cases never looked at the benchmark I sent, which
> used
> it for two distinct purposes: once to run the benchmark, and once as
> one
> of the subjects of the benchmark, implementing this method from
> RichString (which is what led me to investigate it in the first place,
> as I was trying to find the fastest ways to do string-as-seq ops.)
>
> def * (n: Int): String = {
> val buf = new StringBuilder
> for (i <- 0 until n) buf append self // current implementation
> // n times (buf append self) // alternatively...
> buf.toString
> }
>
> Martin is correct that with -optimise on, the speed difference
> disappears. So that's good.
>
> In my view the meaningful question was neglected: why does someone who
> writes an outside library feel like they have to intervene in the
> design
> of the standard library? I mean, I know why: but can we seek a design
> that reduces the need of people working on other projects to lobby?
> The
> question of "times" is irrelevant, that cannot scale: and if the
> standard library is to be constrained by DSLs it will suffer for it.
>

I also know it's a pipe dream, but I want to speak in favor of getOrElse becoming orElse and orElse becoming or.

I think it makes composing control structures with Option a lot nicer. As a case study, consider my experimentation with a Lock API for Scala (http://uncountablymany.blogspot.com/2009/07/pimp-my-lock-case-study-in-scala-api.html).

There are two fundamental interfaces in my experimental Scala Lock API (Lock and TryingLock) compared to Java's one interface (Lock). However, the benefit comes from the Scala API having only three fundamental methods ("def lock: Unit" on Lock, "def lock: Boolean" on TryingLock, and "def unlock: Unit" on both) for these interfaces, versus five fundamental methods in Java's lock API (void lock(), void lockInterruptibly(), boolean tryLock(), boolean tryLock(long time, TimeUnit unit), and void unlock()).

In my Scala API, Java's lock/lockInterruptibly just become two different "modes" on Scala's Lock#lock, and Java's tryLock/tryLock(long, TimeUnit) just become two different "modes" on Scala's TryingLock#lock.

What does this have to do with Option? Well, aside from the "fundamental" methods, there's "nicer abstraction" methods for Locks and TryingLocks, namely map, foreach, and apply. Let's just look at "apply", which executes a block of code while holding the given lock.

  val lock = new Lock
  lock {
    ... stuff ...
  }

In this code, apply takes a (block: => T) of code and returns T (whatever the block returns). If we switch "modes" by calling attemptFor, which returns a TryingLock:

  lock.attemptFor(10.minutes) {
     ... stuff ...
  }

Then TryingLock's apply needs to take a (block: => T) of code... but can only return an Option[T] (because acquiring the lock might fail). If it returns an Option[T], then composing control structures looks much better with "orElse" than with "getOrElse".

  lock.attemptFor(10.minutes) {
    ... stuff ...
  } orElse {
    ... stuff if we can't acquire the lock ...
  }

It's minor, but I think not having the "get" removes the mental burden of having to know there's an Option under the hood.

(You can see the (very much in progress) Lock code here: http://github.com/joshcough/scala-parallel/tree/master/src/main/scala/scala/util/concurrent/locks)

* * *

With regards to Paul's other point about being constrained by DSLs, I agree. I think there are certain things that are reinvented over and over and over again, and worse, which might clash with each other if trying to use two libraries. Take, for example, my use above of the "10.minutes" idiom instead of the usual (for Java) pair of parameters (10, TimeUnit.MINUTES). I know at least four projects that reinvent this exact sugar: Scalax, Lift, scala-time (a wrapper for Joda Time), and scala-parallel. If there aren't more libraries reinventing this sugar already, there will be in the future. Now, reinventing is fine. They're all properly scoped and you have to import the relevant implicits to bring them in scope. It's fine, that is, until you want to use any two of these libraries within the same scope. You can only have one of these sugars in scope at the same time, otherwise they will clash with each other.

What's worse, each library has very different semantics. In Scalax, "1.year" means 1 "average" year (so three times the milliseconds in a standard year plus the milliseconds in a leap year divided by four). In scala-time, "1.year" means one calendar year, and so has no exact meaning in milliseconds until you put it in some context, or call "standardDuration" on it, in which case you'll get the number of milliseconds in a standard year. Lift and scala-parallel don't even have "1.year" (Lift only goes up to "weeks", and scala-parallel only to "days").

I don't know what the end-game for this is. Right now it doesn't seem to be much of a problem as few people use several Scala libraries together. (And the other hassles of using several Scala libraries together (like binary compatibility) are much greater.) Ideally, Scala would have a standard date/time library that was flexible enough to handle all these different use cases. However, I, for one, don't have the time nor the expertise necessary to develop such a library. Another approach would be for the standard Scala library to have the minimal API necessary to support the sweet & sugary syntax, but allowing each separate library to interpret that syntax with their own semantics.

Regardless, I think there's an opportunity for the standard Scala distribution to play a coordinating role. The nightmare scenario, I think, is what has happened in Java for lack of a standard Function interface: practically every library defines their own Runnable/Callable/Predicate/Extractor/Processor/Generator/Handler/Converter/Formatter. If it weren't for checked exceptions (huge proviso, I know), I think Java would have benefited enormously from a standard Function interface. I think with some things (like syntax for "3.minutes"), Scala also has some places where it could benefit from central coordination.

--j

On Sun, Aug 30, 2009 at 8:00 AM, Paul Phillips <paulp@improving.org> wrote:

As long as everyone is gathered around the API bikeshed warming their
hands against the cold, it was suggested by some wise soul that:

 getOrElse should be named orElse
 orElse should be named or

I realize this will never happen, but oh it'd be so nice.

On the "times" subject, I will revert it.  I assume those of you unable
to locate any use cases never looked at the benchmark I sent, which used
it for two distinct purposes: once to run the benchmark, and once as one
of the subjects of the benchmark, implementing this method from
RichString (which is what led me to investigate it in the first place,
as I was trying to find the fastest ways to do string-as-seq ops.)

 def * (n: Int): String = {
   val buf = new StringBuilder
   for (i <- 0 until n) buf append self // current implementation
   // n times (buf append self) // alternatively...
   buf.toString
 }

Martin is correct that with -optimise on, the speed difference
disappears.  So that's good.

In my view the meaningful question was neglected: why does someone who
writes an outside library feel like they have to intervene in the design
of the standard library? I mean, I know why: but can we seek a design
that reduces the need of people working on other projects to lobby? The
question of "times" is irrelevant, that cannot scale: and if the
standard library is to be constrained by DSLs it will suffer for it.

--
Paul Phillips      | It's better to have gloved and tossed than never to
In Theory          | have played baseball.
Empiricist         |
pull his pi pal!   |----------* http://www.improving.org/paulp/ *----------

On Sun, Aug 30, 2009 at 7:28 PM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:

I also know it's a pipe dream, but I want to speak in favor of getOrElse becoming orElse and orElse becoming or.

I think it makes composing control structures with Option a lot nicer. As a case study, consider my experimentation with a Lock API for Scala (http://uncountablymany.blogspot.com/2009/07/pimp-my-lock-case-study-in-scala-api.html).

There are two fundamental interfaces in my experimental Scala Lock API (Lock and TryingLock) compared to Java's one interface (Lock). However, the benefit comes from the Scala API having only three fundamental methods ("def lock: Unit" on Lock, "def lock: Boolean" on TryingLock, and "def unlock: Unit" on both) for these interfaces, versus five fundamental methods in Java's lock API (void lock(), void lockInterruptibly(), boolean tryLock(), boolean tryLock(long time, TimeUnit unit), and void unlock()).

In my Scala API, Java's lock/lockInterruptibly just become two different "modes" on Scala's Lock#lock, and Java's tryLock/tryLock(long, TimeUnit) just become two different "modes" on Scala's TryingLock#lock.

What does this have to do with Option? Well, aside from the "fundamental" methods, there's "nicer abstraction" methods for Locks and TryingLocks, namely map, foreach, and apply. Let's just look at "apply", which executes a block of code while holding the given lock.

  val lock = new Lock
  lock {
    ... stuff ...
  }

In this code, apply takes a (block: => T) of code and returns T (whatever the block returns). If we switch "modes" by calling attemptFor, which returns a TryingLock:

  lock.attemptFor(10.minutes) {
     ... stuff ...
  }

Then TryingLock's apply needs to take a (block: => T) of code... but can only return an Option[T] (because acquiring the lock might fail). If it returns an Option[T], then composing control structures looks much better with "orElse" than with "getOrElse".

  lock.attemptFor(10.minutes) {
    ... stuff ...
  } orElse {
    ... stuff if we can't acquire the lock ...
  }

It's minor, but I think not having the "get" removes the mental burden of having to know there's an Option under the hood.

(You can see the (very much in progress) Lock code here: http://github.com/joshcough/scala-parallel/tree/master/src/main/scala/scala/util/concurrent/locks)

* * *

With regards to Paul's other point about being constrained by DSLs, I agree. I think there are certain things that are reinvented over and over and over again, and worse, which might clash with each other if trying to use two libraries. Take, for example, my use above of the "10.minutes" idiom instead of the usual (for Java) pair of parameters (10, TimeUnit.MINUTES). I know at least four projects that reinvent this exact sugar: Scalax, Lift, scala-time (a wrapper for Joda Time), and scala-parallel. If there aren't more libraries reinventing this sugar already, there will be in the future. Now, reinventing is fine. They're all properly scoped and you have to import the relevant implicits to bring them in scope. It's fine, that is, until you want to use any two of these libraries within the same scope. You can only have one of these sugars in scope at the same time, otherwise they will clash with each other.

What's worse, each library has very different semantics. In Scalax, "1.year" means 1 "average" year (so three times the milliseconds in a standard year plus the milliseconds in a leap year divided by four). In scala-time, "1.year" means one calendar year, and so has no exact meaning in milliseconds until you put it in some context, or call "standardDuration" on it, in which case you'll get the number of milliseconds in a standard year. Lift and scala-parallel don't even have "1.year" (Lift only goes up to "weeks", and scala-parallel only to "days").

I don't know what the end-game for this is. Right now it doesn't seem to be much of a problem as few people use several Scala libraries together. (And the other hassles of using several Scala libraries together (like binary compatibility) are much greater.) Ideally, Scala would have a standard date/time library that was flexible enough to handle all these different use cases. However, I, for one, don't have the time nor the expertise necessary to develop such a library. Another approach would be for the standard Scala library to have the minimal API necessary to support the sweet & sugary syntax, but allowing each separate library to interpret that syntax with their own semantics.

Regardless, I think there's an opportunity for the standard Scala distribution to play a coordinating role. The nightmare scenario, I think, is what has happened in Java for lack of a standard Function interface: practically every library defines their own Runnable/Callable/Predicate/Extractor/Processor/Generator/Handler/Converter/Formatter. If it weren't for checked exceptions (huge proviso, I know), I think Java would have benefited enormously from a standard Function interface. I think with some things (like syntax for "3.minutes"), Scala also has some places where it could benefit from central coordination.

--j

Agreed!  I know the targets I'd have for helping develop/push new libraries into core are the following (in order of the amount of re-use I get out of my private implementations of these):

1) Automatic-Resource-Management
    * Binding for I/O
    * Bindings for JTA/JPA
2) Time
   * Joda time is more than useful in many situations.  If it makes it into Java 7, perhaps then we could think about a wrapper around it?
3) Simple I/O for small utilities (i.e. I don't care about nio yet)
4) Properties for "Beanish" objects and Binding
    * Can listen on an object for mutated state
    * receive information on specific property that was mutated
    * can hook validation rules into mutation
    * can bind property values from two distinct objects such that their state is updated together
5) Graph Library
     * Node traversal
     * Coloring
6) Linear Algebra

I know that some work is going into each of these.  I'm willing to contribute whatever time I can to help develop/vet/prototype APIs and try them out in testing utilities at work to see what flys and what doesn't.  I understand this is a "wish list",  but I'm more than willing to put time/effort into prototyping each of these potentials for inclusing in the Scala core.   My opinion is that if something can exist well as a stand-alone library, then hopefully we can think about adapting it into the scala core libraries.   ARM, Time and I/O certainly exist in several public-facing external libraries.  Perhaps we can start formalizing these into the core?  I feel we're starting to get grips on which designs work, and which don't.   Personally I use scalax ARM for several different reasons at work (primarily Transactions and I/O).  I'm actively toying with various API designs in an attempt to create something that could eventually be added to scala core.

Martin,

If we wanted to have libraries be seriously considered for inclusion in Scala core, what's the appropriate format for this?   Would a SIP be the right way to outline the design of an ARM library, along with a prototype implementation, that could then be negotiated/adapted into core?  What are the pieces that you feel are missing from core that you would like to see the community help define?


Thanks!

- Josh

I, for one, would like to see JODA-like time&date library on Scala, Java 7 or not. First, Scala lacks a proper date&time library, so it isn't encumbered by an existing standard. Second, Java alternatives are awful, particularly from a functional language perspective. Third, date&time are necessary, or even important, for many interactive applications, as timestamping, scheduling and similar tasks are common. Fourth, JODA is a very good and proven library. Fifth and last, there doesn't seem to be alternatives to it.

On Sun, Aug 30, 2009 at 10:22 PM, Josh Suereth <joshua.suereth@gmail.com> wrote:

On Sun, Aug 30, 2009 at 7:28 PM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:

I also know it's a pipe dream, but I want to speak in favor of getOrElse becoming orElse and orElse becoming or.

I think it makes composing control structures with Option a lot nicer. As a case study, consider my experimentation with a Lock API for Scala (http://uncountablymany.blogspot.com/2009/07/pimp-my-lock-case-study-in-scala-api.html).

There are two fundamental interfaces in my experimental Scala Lock API (Lock and TryingLock) compared to Java's one interface (Lock). However, the benefit comes from the Scala API having only three fundamental methods ("def lock: Unit" on Lock, "def lock: Boolean" on TryingLock, and "def unlock: Unit" on both) for these interfaces, versus five fundamental methods in Java's lock API (void lock(), void lockInterruptibly(), boolean tryLock(), boolean tryLock(long time, TimeUnit unit), and void unlock()).

In my Scala API, Java's lock/lockInterruptibly just become two different "modes" on Scala's Lock#lock, and Java's tryLock/tryLock(long, TimeUnit) just become two different "modes" on Scala's TryingLock#lock.

What does this have to do with Option? Well, aside from the "fundamental" methods, there's "nicer abstraction" methods for Locks and TryingLocks, namely map, foreach, and apply. Let's just look at "apply", which executes a block of code while holding the given lock.

  val lock = new Lock
  lock {
    ... stuff ...
  }

In this code, apply takes a (block: => T) of code and returns T (whatever the block returns). If we switch "modes" by calling attemptFor, which returns a TryingLock:

  lock.attemptFor(10.minutes) {
     ... stuff ...
  }

Then TryingLock's apply needs to take a (block: => T) of code... but can only return an Option[T] (because acquiring the lock might fail). If it returns an Option[T], then composing control structures looks much better with "orElse" than with "getOrElse".

  lock.attemptFor(10.minutes) {
    ... stuff ...
  } orElse {
    ... stuff if we can't acquire the lock ...
  }

It's minor, but I think not having the "get" removes the mental burden of having to know there's an Option under the hood.

(You can see the (very much in progress) Lock code here: http://github.com/joshcough/scala-parallel/tree/master/src/main/scala/scala/util/concurrent/locks)

* * *

With regards to Paul's other point about being constrained by DSLs, I agree. I think there are certain things that are reinvented over and over and over again, and worse, which might clash with each other if trying to use two libraries. Take, for example, my use above of the "10.minutes" idiom instead of the usual (for Java) pair of parameters (10, TimeUnit.MINUTES). I know at least four projects that reinvent this exact sugar: Scalax, Lift, scala-time (a wrapper for Joda Time), and scala-parallel. If there aren't more libraries reinventing this sugar already, there will be in the future. Now, reinventing is fine. They're all properly scoped and you have to import the relevant implicits to bring them in scope. It's fine, that is, until you want to use any two of these libraries within the same scope. You can only have one of these sugars in scope at the same time, otherwise they will clash with each other.

What's worse, each library has very different semantics. In Scalax, "1.year" means 1 "average" year (so three times the milliseconds in a standard year plus the milliseconds in a leap year divided by four). In scala-time, "1.year" means one calendar year, and so has no exact meaning in milliseconds until you put it in some context, or call "standardDuration" on it, in which case you'll get the number of milliseconds in a standard year. Lift and scala-parallel don't even have "1.year" (Lift only goes up to "weeks", and scala-parallel only to "days").

I don't know what the end-game for this is. Right now it doesn't seem to be much of a problem as few people use several Scala libraries together. (And the other hassles of using several Scala libraries together (like binary compatibility) are much greater.) Ideally, Scala would have a standard date/time library that was flexible enough to handle all these different use cases. However, I, for one, don't have the time nor the expertise necessary to develop such a library. Another approach would be for the standard Scala library to have the minimal API necessary to support the sweet & sugary syntax, but allowing each separate library to interpret that syntax with their own semantics.

Regardless, I think there's an opportunity for the standard Scala distribution to play a coordinating role. The nightmare scenario, I think, is what has happened in Java for lack of a standard Function interface: practically every library defines their own Runnable/Callable/Predicate/Extractor/Processor/Generator/Handler/Converter/Formatter. If it weren't for checked exceptions (huge proviso, I know), I think Java would have benefited enormously from a standard Function interface. I think with some things (like syntax for "3.minutes"), Scala also has some places where it could benefit from central coordination.

--j

Agreed!  I know the targets I'd have for helping develop/push new libraries into core are the following (in order of the amount of re-use I get out of my private implementations of these):

1) Automatic-Resource-Management
    * Binding for I/O
    * Bindings for JTA/JPA
2) Time
   * Joda time is more than useful in many situations.  If it makes it into Java 7, perhaps then we could think about a wrapper around it?
3) Simple I/O for small utilities (i.e. I don't care about nio yet)
4) Properties for "Beanish" objects and Binding
    * Can listen on an object for mutated state
    * receive information on specific property that was mutated
    * can hook validation rules into mutation
    * can bind property values from two distinct objects such that their state is updated together
5) Graph Library
     * Node traversal
     * Coloring
6) Linear Algebra

I know that some work is going into each of these.  I'm willing to contribute whatever time I can to help develop/vet/prototype APIs and try them out in testing utilities at work to see what flys and what doesn't.  I understand this is a "wish list",  but I'm more than willing to put time/effort into prototyping each of these potentials for inclusing in the Scala core.   My opinion is that if something can exist well as a stand-alone library, then hopefully we can think about adapting it into the scala core libraries.   ARM, Time and I/O certainly exist in several public-facing external libraries.  Perhaps we can start formalizing these into the core?  I feel we're starting to get grips on which designs work, and which don't.   Personally I use scalax ARM for several different reasons at work (primarily Transactions and I/O).  I'm actively toying with various API designs in an attempt to create something that could eventually be added to scala core.

Martin,

If we wanted to have libraries be seriously considered for inclusion in Scala core, what's the appropriate format for this?   Would a SIP be the right way to outline the design of an ARM library, along with a prototype implementation, that could then be negotiated/adapted into core?  What are the pieces that you feel are missing from core that you would like to see the community help define?


Thanks!

- Josh

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

I agree with Daniel.

1) Scala has a clean slate to start with re Date/Time. It should be taken advantage of.
2) New Joda-inspired Date/Time is NOT making it into Java 7.
3) Even if new Date/Time library DID make it into Java 7, Scala JUST dropped compatibility for Java 1.4, so the odds of us ever making it to 1.7 anytime soon are pretty much nonexistent.
4) Everything else Daniel said.

I wrote a small Scala wrapper around Joda Time, but I think a full-fledged standard Scala Date/Time library would be better http://github.com/jorgeortiz85/scala-time/tree/master

--j

On Sun, Aug 30, 2009 at 6:44 PM, Daniel Sobral <dcsobral@gmail.com> wrote:

I, for one, would like to see JODA-like time&date library on Scala, Java 7 or not. First, Scala lacks a proper date&time library, so it isn't encumbered by an existing standard. Second, Java alternatives are awful, particularly from a functional language perspective. Third, date&time are necessary, or even important, for many interactive applications, as timestamping, scheduling and similar tasks are common. Fourth, JODA is a very good and proven library. Fifth and last, there doesn't seem to be alternatives to it.

On Sun, Aug 30, 2009 at 10:22 PM, Josh Suereth <joshua.suereth@gmail.com> wrote:

On Sun, Aug 30, 2009 at 7:28 PM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:

I also know it's a pipe dream, but I want to speak in favor of getOrElse becoming orElse and orElse becoming or.

I think it makes composing control structures with Option a lot nicer. As a case study, consider my experimentation with a Lock API for Scala (http://uncountablymany.blogspot.com/2009/07/pimp-my-lock-case-study-in-scala-api.html).

There are two fundamental interfaces in my experimental Scala Lock API (Lock and TryingLock) compared to Java's one interface (Lock). However, the benefit comes from the Scala API having only three fundamental methods ("def lock: Unit" on Lock, "def lock: Boolean" on TryingLock, and "def unlock: Unit" on both) for these interfaces, versus five fundamental methods in Java's lock API (void lock(), void lockInterruptibly(), boolean tryLock(), boolean tryLock(long time, TimeUnit unit), and void unlock()).

In my Scala API, Java's lock/lockInterruptibly just become two different "modes" on Scala's Lock#lock, and Java's tryLock/tryLock(long, TimeUnit) just become two different "modes" on Scala's TryingLock#lock.

What does this have to do with Option? Well, aside from the "fundamental" methods, there's "nicer abstraction" methods for Locks and TryingLocks, namely map, foreach, and apply. Let's just look at "apply", which executes a block of code while holding the given lock.

  val lock = new Lock
  lock {
    ... stuff ...
  }

In this code, apply takes a (block: => T) of code and returns T (whatever the block returns). If we switch "modes" by calling attemptFor, which returns a TryingLock:

  lock.attemptFor(10.minutes) {
     ... stuff ...
  }

Then TryingLock's apply needs to take a (block: => T) of code... but can only return an Option[T] (because acquiring the lock might fail). If it returns an Option[T], then composing control structures looks much better with "orElse" than with "getOrElse".

  lock.attemptFor(10.minutes) {
    ... stuff ...
  } orElse {
    ... stuff if we can't acquire the lock ...
  }

It's minor, but I think not having the "get" removes the mental burden of having to know there's an Option under the hood.

(You can see the (very much in progress) Lock code here: http://github.com/joshcough/scala-parallel/tree/master/src/main/scala/scala/util/concurrent/locks)

* * *

With regards to Paul's other point about being constrained by DSLs, I agree. I think there are certain things that are reinvented over and over and over again, and worse, which might clash with each other if trying to use two libraries. Take, for example, my use above of the "10.minutes" idiom instead of the usual (for Java) pair of parameters (10, TimeUnit.MINUTES). I know at least four projects that reinvent this exact sugar: Scalax, Lift, scala-time (a wrapper for Joda Time), and scala-parallel. If there aren't more libraries reinventing this sugar already, there will be in the future. Now, reinventing is fine. They're all properly scoped and you have to import the relevant implicits to bring them in scope. It's fine, that is, until you want to use any two of these libraries within the same scope. You can only have one of these sugars in scope at the same time, otherwise they will clash with each other.

What's worse, each library has very different semantics. In Scalax, "1.year" means 1 "average" year (so three times the milliseconds in a standard year plus the milliseconds in a leap year divided by four). In scala-time, "1.year" means one calendar year, and so has no exact meaning in milliseconds until you put it in some context, or call "standardDuration" on it, in which case you'll get the number of milliseconds in a standard year. Lift and scala-parallel don't even have "1.year" (Lift only goes up to "weeks", and scala-parallel only to "days").

I don't know what the end-game for this is. Right now it doesn't seem to be much of a problem as few people use several Scala libraries together. (And the other hassles of using several Scala libraries together (like binary compatibility) are much greater.) Ideally, Scala would have a standard date/time library that was flexible enough to handle all these different use cases. However, I, for one, don't have the time nor the expertise necessary to develop such a library. Another approach would be for the standard Scala library to have the minimal API necessary to support the sweet & sugary syntax, but allowing each separate library to interpret that syntax with their own semantics.

Regardless, I think there's an opportunity for the standard Scala distribution to play a coordinating role. The nightmare scenario, I think, is what has happened in Java for lack of a standard Function interface: practically every library defines their own Runnable/Callable/Predicate/Extractor/Processor/Generator/Handler/Converter/Formatter. If it weren't for checked exceptions (huge proviso, I know), I think Java would have benefited enormously from a standard Function interface. I think with some things (like syntax for "3.minutes"), Scala also has some places where it could benefit from central coordination.

--j

Agreed!  I know the targets I'd have for helping develop/push new libraries into core are the following (in order of the amount of re-use I get out of my private implementations of these):

1) Automatic-Resource-Management
    * Binding for I/O
    * Bindings for JTA/JPA
2) Time
   * Joda time is more than useful in many situations.  If it makes it into Java 7, perhaps then we could think about a wrapper around it?
3) Simple I/O for small utilities (i.e. I don't care about nio yet)
4) Properties for "Beanish" objects and Binding
    * Can listen on an object for mutated state
    * receive information on specific property that was mutated
    * can hook validation rules into mutation
    * can bind property values from two distinct objects such that their state is updated together
5) Graph Library
     * Node traversal
     * Coloring
6) Linear Algebra

I know that some work is going into each of these.  I'm willing to contribute whatever time I can to help develop/vet/prototype APIs and try them out in testing utilities at work to see what flys and what doesn't.  I understand this is a "wish list",  but I'm more than willing to put time/effort into prototyping each of these potentials for inclusing in the Scala core.   My opinion is that if something can exist well as a stand-alone library, then hopefully we can think about adapting it into the scala core libraries.   ARM, Time and I/O certainly exist in several public-facing external libraries.  Perhaps we can start formalizing these into the core?  I feel we're starting to get grips on which designs work, and which don't.   Personally I use scalax ARM for several different reasons at work (primarily Transactions and I/O).  I'm actively toying with various API designs in an attempt to create something that could eventually be added to scala core.

Martin,

If we wanted to have libraries be seriously considered for inclusion in Scala core, what's the appropriate format for this?   Would a SIP be the right way to outline the design of an ARM library, along with a prototype implementation, that could then be negotiated/adapted into core?  What are the pieces that you feel are missing from core that you would like to see the community help define?


Thanks!

- Josh

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

Hi Paul,

On Sun, Aug 30, 2009 at 8:00 AM, Paul Phillips wrote:
> In my view the meaningful question was neglected: why does someone who
> writes an outside library feel like they have to intervene in the design
> of the standard library? I mean, I know why: but can we seek a design
> that reduces the need of people working on other projects to lobby? The
> question of "times" is irrelevant, that cannot scale: and if the
> standard library is to be constrained by DSLs it will suffer for it.
>
Let me first reiterate that I do agree the standard library should get
first shot at the best syntax, as Vlad posted in the earlier thread.
Especially right now, even if code is broken, if it's right for the
standard library, then I think we should bite the bullet and break
code in 2.8 and get it over with. There aren't that many users, and
every user knows that part of the deal is 2.8 will likely break their
code. My concern about the times addition you made was only partly
that it messed up the DSL I had sketched out on a napkin. I was also
concerned that it broke specs itself, and would break any specs users
code that used Eric's times syntax. If it were to affect a lot of
specs users, then that's something you should be aware of and
consider. If not, then how needed would this addition be in the first
place? I myself have never needed it, so I asked for use cases. I got
only a few back, which although good use cases in my mind don't
justify its addition to RichInt.

I realized when Eric in his reply said he thought it should be an
import that my main concern was not that this was added to the
standard library so much as to RichInt. When you add times to RichInt,
it becomes globally in scope in a way that is hard to deal with if
there's a conflict with prior code. Adding things to Predef in general
is a bit like adding a new keyword to the language, except that I
think local legacy definitions of the same name will often hide the
new ones in Predef. But with an implicit, it just becomes ambiguous
and won't compile anymore. I'm not aware of a way to hide or rename
things automatically imported from Predef. Maybe there's a way, and if
not, perhaps there should be a way. But even if there is a way today
or tomorrow, I would never design something into a library that
required my users to hide or rename something imported from Predef. So
in my mind anything added to Predef, especially implicits, needs to
meet a high standard of need.

Because of this problem of conflicting implicits, I have tried to be
very conservative about implicits in ScalaTest. (Because ScalaTest
will be used together will lots and lots of different kinds of apps,
many of which will have their own implicits.) If you use a ScalaTest
Suite, you only get one implicit conversion by default, the one that
adds === on everything. I have in the documentation an example of how
to turn if off if you have a conflict with a different implicit
conversion for === in the application you're testing. So I only offer
one implicit conversion by default, and made sure there's an easy way
to turn it off. Any other implicits you must explicitly invite into
your code with an import or via mixins.

The other thing I do is that for traits that don't have a self type, I
often make a companion object that extends the trait. This gives
people three choices: they can import the syntax, mixin the syntax, or
neither. I've been thinking of calling this the "selfless trait"
pattern, if there isn't another name for it. You can only do it if the
trait doesn't have a self type, and it is "selfless" in that it is
serving its users by giving them all three choices. If you had added
the N times feature to the standard library in this way, then you
wouldn't have heard a peep out of me or anyone else, because you
wouldn't have broken any existing code on disks or future DSLs
sketched on napkins. Here's what it would look like:

trait TimesOnInt {

class TimesOnIntWrapper(times: Int) {

def times(f: => Unit) {
var i = 0
while (i < times) {
f
i = i + 1
}
}

def times(f: Int => Unit) {
var i = 0
while (i < times) {
f(i)
i = i + 1
}
}
}

implicit def convertToTimesOnIntWrapper(i: Int) = new TimesOnIntWrapper(i)
}

object TimesOnInt extends TimesOnInt

Because it is a trait, library designers who want to offer it to their
users can just mix it into a trait they offer. Eric could just have
his Specification class extend TimesOnInt from the standard library,
for example. Then there would be no "import tax" for his users. One
nice use case for importing from the companion object is trying things
out in the REPL:

scala> import TimesOnInt._
import TimesOnInt._

scala> 3 times { println("hi") }
hi
hi
hi

scala> 3 times { i => println(i) }
0
1
2

I believe this is a "design that reduces the need of people working on
other projects to lobby."

By the way, I'm sorry you felt I was "intervening." I would hope I
came across more as expressing concerns and making suggestions. It is
EPFL's library and your decision what goes in and what doesn't, but
what you do affects us. We are your users. For better or for worse,
Martin wanted this to be more than a research project. He wanted
people to actually use Scala. That seems to be happening, and as Scala
gets more users, you will likely get more and more "suggestions" from
people when you make changes and additions. Your most important job in
my opinion is to say NO to people who ask you to add things. But if 20
people have asked you for times on Int, then they might be on to
something. If you collect their use cases and keep them handy, then
you'll be ready the next time a curmudgeon like me asks you for use
cases that justify a change.

Bill

> --
> Paul Phillips      | It's better to have gloved and tossed than never to
> In Theory          | have played baseball.
> Empiricist         |
> pull his pi pal!   |----------* http://www.improving.org/paulp/ *----------
>

On Sun, Aug 30, 2009 at 5:00 PM, Paul Phillips wrote:
> In my view the meaningful question was neglected: why does someone who
> writes an outside library feel like they have to intervene in the design
> of the standard library? I mean, I know why: but can we seek a design
> that reduces the need of people working on other projects to lobby?

Maybe the real solution to this is to make DSLs as libraries more
integrated into the compiler.

To me the current approach of using methods as keywords seems like a
hack. A more natural approach would be to have the library responsible
for it's own grammar and directly manipulate the AST (maybe an
approach like what Project Lombok does is the correct direction). This
way the library could do what ever it wants and not worry about what
the Scala library does.

It seems to me that people still prefer to write their DSL as
libraries rather than compiler plug-ins. Without knowing much about
how compiler plug-ins is supposed to work I take this as sign as the
plug-in stuff isn't quite there yet.

Is it possible to have the compiler provide the basic building blocks
like a general syntax, lexer and parser, and let the DSL designer
construct a language from the bits and pieces the Scala compiler makes
available?

For example maybe it could be possible to define "methods" that take
it's argument as a grammar.

grammar TimeGrammar {
TimeType = 'Seconds | 'Minutes | 'Hours
Time = i:Int t:TimeType
}

def sleep(t:TimeGrammar.Time) = t.t match {
case 'Seconds => sleep(i * 1000)
...
}

sleep(5 Seconds)

BR,
John

The compiler plugins do its stuff after lexing/parsing, so they aren't all that useful as DSL.
Anyway, the whole point of Scala is not needing to introduce extraneous ad-hoc rules to handle different stuff. What you consider a "hack" is by design and intention.
If you truly need to exceed what's possible within the language, consider an external DSL with Scala's parser library instead.
On Wed, Sep 2, 2009 at 7:09 PM, John Nilsson <john@milsson.nu> wrote:

On Sun, Aug 30, 2009 at 5:00 PM, Paul Phillips<paulp@improving.org> wrote:
> In my view the meaningful question was neglected: why does someone who
> writes an outside library feel like they have to intervene in the design
> of the standard library? I mean, I know why: but can we seek a design
> that reduces the need of people working on other projects to lobby?

Maybe the real solution to this is to make DSLs as libraries more
integrated into the compiler.

To me the current approach of using methods as keywords seems like a
hack. A more natural approach would be to have the library responsible
for it's own grammar and directly manipulate the AST (maybe an
approach like what Project Lombok does is the correct direction). This
way the library could do what ever it wants and not worry about what
the Scala library does.

It seems to me that people still prefer to write their DSL as
libraries rather than compiler plug-ins. Without knowing much about
how compiler plug-ins is supposed to work I take this as sign as the
plug-in stuff isn't quite there yet.

Is it possible to have the compiler provide the basic building blocks
like a general syntax, lexer and parser, and let the DSL designer
construct a language from the bits and pieces the Scala compiler makes
available?

For example maybe it could be possible to define "methods" that take
it's argument as a grammar.

grammar TimeGrammar  {
  TimeType = 'Seconds | 'Minutes | 'Hours
  Time = i:Int t:TimeType
}

def sleep(t:TimeGrammar.Time) = t.t match {
   case 'Seconds => sleep(i * 1000)
   ...
}

sleep(5 Seconds)

BR,
John

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

On Thu, Sep 03, 2009 at 12:09:01AM +0200, John Nilsson wrote:
> Is it possible to have the compiler provide the basic building blocks
> like a general syntax, lexer and parser, and let the DSL designer
> construct a language from the bits and pieces the Scala compiler makes
> available?

I had something similar in mind when I wrote a scalac parser with
combinators, but the basic building blocks are combinator rules. I
reached the point where I was parsing the whole language (except XML)
which was no small feat given semicolon inference, but I stopped there.
The plan was that you could override productions at will somehow, either
with a command line directive or some kind of metasyntax at the file
level. For instance here are some rules from it taken at random:

lazy val funDcl = funSig ~ COLON ~ sType
lazy val funDef = funSig ~ opt(COLON ~> sType) ~< EQUALS ~ expr
lazy val funSig = id ~ opt(funTypeParamClause) ~ paramClauses
lazy val funTypeParamClause = brackets(rep1sep(typeParam, COMMA))

lazy val tmplDef: Parser[ImplDef] = (
( CLASS ~!> classDef )
| ( CASECLASS ~!> classDef )
| ( OBJECT ~!> objectDef )
| ( CASEOBJECT ~!> objectDef )
| ( TRAIT ~!> traitDef )
)

and you could say, ignoring the fact that you can't call super when
overriding a lazy val (they should be defs for this purpose, but are
lazy vals for performance):

MyParser extends ScalaParser {
override lazy val tmplDef: Parser[ImplDef] =
super.tmplDef | MyNuttyCustomSyntasx
}

I think it's totally doable but there is a lot to do and figure out I
wasn't at all sure it'd end up fast enough to be usable. Right now I'm
focused only on things that are realistic for 2.8. But there are a ton
of neat ideas here for the exploring.

On Thu, Sep 3, 2009 at 2:48 AM, Daniel Sobral wrote:
> Anyway, the whole point of Scala is not needing to introduce extraneous
> ad-hoc rules to handle different stuff. What you consider a "hack" is by
> design and intention.

I think there is a middle road here. Maybe "hack" was the wrong word.
What I was thinking was to take the same intention and push the design
further down the chain so that it is syntactic rather than semantic,
compile time rather then runtime, but in other respects follow the
same intention of providing a few basic building blocks that can be
combined in interesting ways in libraris, instead of adding ad-hoc
rules into the language as such.

F.ex. I guess it will be totaly enough to only be able to define
syntactic sugare this way, there doesn't have to be any facilities to
change the core languages as such.

> If you truly need to exceed what's possible within the language, consider an
> external DSL with Scala's parser library instead.

To much overhead, the cost of doing it this way far exceeds the pain
of ugly api's. You'll have no tool support. By having it as an
integrated part of the language hopefully you could piggy back on the
available syntax coloring, refactoring support and such.

On Thu, Sep 3, 2009 at 3:10 AM, Paul Phillips wrote:
> I think it's totally doable but there is a lot to do and figure out I
> wasn't at all sure it'd end up fast enough to be usable. Right now I'm
> focused only on things that are realistic for 2.8. But there are a ton
> of neat ideas here for the exploring.

Understandable, and as it should be. But hopefully the discussion
could sparc som interst among researches and practitioners looking for
something fun to do. =)
(If I ever get something going I was actually planning to implement a
toy language with such a feature one of theese days...)

BR,
John

On Thu, Sep 3, 2009 at 4:04 AM, John Nilsson <john@milsson.nu> wrote:

On Thu, Sep 3, 2009 at 2:48 AM, Daniel Sobral<dcsobral@gmail.com> wrote:
> Anyway, the whole point of Scala is not needing to introduce extraneous
> ad-hoc rules to handle different stuff. What you consider a "hack" is by
> design and intention.

I think there is a middle road here. Maybe "hack" was the wrong word.
What I was thinking was to take the same intention and push the design
further down the chain so that it is syntactic rather than semantic,
compile time rather then runtime, but in other respects follow the
same intention of providing a few basic building blocks that can be
combined in interesting ways in libraris, instead of adding ad-hoc
rules into the language as such.

The whole -point- of a "scalable language" is that the language itself supports all these use cases, rather than having to hack the compiler. I also don't see how this solves the problem at all. Instead of having libraries clobber all over each other with similarly named methods and classes, we'll have libraries with special -syntax- clobbering each other's syntax. It'll be Ruby's open classes mess all over again.

F.ex. I guess it will be totaly enough to only be able to define
syntactic sugare this way, there doesn't have to be any facilities to
change the core languages as such.


> If you truly need to exceed what's possible within the language, consider an
> external DSL with Scala's parser library instead.

To much overhead, the cost of doing it this way far exceeds the pain
of ugly api's. You'll have no tool support. By having it as an
integrated part of the language hopefully you could piggy back on the
available syntax coloring, refactoring support and such.



On Thu, Sep 3, 2009 at 3:10 AM, Paul Phillips<paulp@improving.org> wrote:
> I think it's totally doable but there is a lot to do and figure out I
> wasn't at all sure it'd end up fast enough to be usable.  Right now I'm
> focused only on things that are realistic for 2.8.  But there are a ton
> of neat ideas here for the exploring.

Understandable, and as it should be. But hopefully the discussion
could sparc som interst among researches and practitioners looking for
something fun to do. =)
(If I ever get something going I was actually planning to implement a
toy language with such a feature one of theese days...)

BR,
John

On Thu, Sep 03, 2009 at 10:54:50AM -0700, Jorge Ortiz wrote:
> The whole -point- of a "scalable language" is that the language itself
> supports all these use cases, rather than having to hack the compiler.
> I also don't see how this solves the problem at all. Instead of having
> libraries clobber all over each other with similarly named methods and
> classes, we'll have libraries with special -syntax- clobbering each
> other's syntax. It'll be Ruby's open classes mess all over again.

I realize you're not directly addressing me, but with the combinator
based parser, I thought of it partly as a simple resolution to some
eternal fights, like: regexp literals, string interpolation, um I had
lots more examples when I was doing it. No, these tweaks wouldn't
necessarily be naively composable, but that's not a reason to prevent
people from doing what they want with their own code. And it provides a
much more accessible proving ground for adding features to the core
language. We can't all be as inherently lovable as XML literals...

And, there is research in this area, and things have been learned.
Fascinating to me is: "Katahdin is a programming language where the
syntax and semantics are mutable at runtime."

http://www.chrisseaton.com/katahdin/

"What could possibly go wrong?"

Having programmed in Forth, a language in which you quite literally defined the language as you went(*), I feel entitled to answer that question.   While business rules part of the program becomes a thing of pure, undiluted beauty -- assuming the programmer is good -- to understand what's going on you HAVE to read the whole program, as no basic rules can be trusted to be valid. I'll finish with a Forth example of such thing, and leave to the reader's imagination the meaning of it.   : 0 1 ;   (*) The good practice when programming in Forth was evolving the language until it became a DSL for the domain of the problem. You could programmatically take over parsing and code generation.

On Thu, Sep 3, 2009 at 3:12 PM, Paul Phillips <paulp@improving.org> wrote:

On Thu, Sep 03, 2009 at 10:54:50AM -0700, Jorge Ortiz wrote:
> The whole -point- of a "scalable language" is that the language itself
> supports all these use cases, rather than having to hack the compiler.
> I also don't see how this solves the problem at all. Instead of having
> libraries clobber all over each other with similarly named methods and
> classes, we'll have libraries with special -syntax- clobbering each
> other's syntax. It'll be Ruby's open classes mess all over again.

I realize you're not directly addressing me, but with the combinator
based parser, I thought of it partly as a simple resolution to some
eternal fights, like: regexp literals, string interpolation, um I had
lots more examples when I was doing it.  No, these tweaks wouldn't
necessarily be naively composable, but that's not a reason to prevent
people from doing what they want with their own code.  And it provides a
much more accessible proving ground for adding features to the core
language.  We can't all be as inherently lovable as XML literals...

And, there is research in this area, and things have been learned.
Fascinating to me is: "Katahdin is a programming language where the
syntax and semantics are mutable at runtime."

 http://www.chrisseaton.com/katahdin/

"What could possibly go wrong?"

--
Paul Phillips      | Every normal man must be tempted at times
Stickler           | to spit on his hands, hoist the black flag,
Empiricist         | and begin to slit throats.
pp: i haul pills   |     -- H. L. Mencken

--
Daniel C. Sobral

Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.

I'm totally in favor of having the capability of extending the syntax of the language. It would be great to have string interpolation, JSON literals, perhaps other goodies not currently possible with Scala's usual "just a library" approach to syntax. (I do think triple-quoted strings and .r work fairly well as regex literals, though.)

However, such a capability
  1) is not needed to add syntax for dates/times (as demonstrated by several Scala libraries doing it as a library); and,
  2) in no way solves the coordination problem of several libraries making up their own syntax (if anything, the problem becomes worse)

We still need some coordination and standardization of commonly used extensions, whether they be merely library extensions or full-on compiler extensions.

--j

On Thu, Sep 3, 2009 at 11:12 AM, Paul Phillips <paulp@improving.org> wrote:

On Thu, Sep 03, 2009 at 10:54:50AM -0700, Jorge Ortiz wrote:
> The whole -point- of a "scalable language" is that the language itself
> supports all these use cases, rather than having to hack the compiler.
> I also don't see how this solves the problem at all. Instead of having
> libraries clobber all over each other with similarly named methods and
> classes, we'll have libraries with special -syntax- clobbering each
> other's syntax. It'll be Ruby's open classes mess all over again.

I realize you're not directly addressing me, but with the combinator
based parser, I thought of it partly as a simple resolution to some
eternal fights, like: regexp literals, string interpolation, um I had
lots more examples when I was doing it.  No, these tweaks wouldn't
necessarily be naively composable, but that's not a reason to prevent
people from doing what they want with their own code.  And it provides a
much more accessible proving ground for adding features to the core
language.  We can't all be as inherently lovable as XML literals...

And, there is research in this area, and things have been learned.
Fascinating to me is: "Katahdin is a programming language where the
syntax and semantics are mutable at runtime."

 http://www.chrisseaton.com/katahdin/

"What could possibly go wrong?"

--
Paul Phillips      | Every normal man must be tempted at times
Stickler           | to spit on his hands, hoist the black flag,
Empiricist         | and begin to slit throats.
pp: i haul pills   |     -- H. L. Mencken

On Thu, Sep 3, 2009 at 11:29 PM, Jorge Ortiz wrote:
>   1) is not needed to add syntax for dates/times (as demonstrated by several
> Scala libraries doing it as a library);
What I'm looking for is the possibility to have my abstractions be a
compile-time thing rather then taking up runtime resources. Things
like compiling regular expressions, SQL queries and such. This is hard
to do as a library ATM.

and,
>   2) in no way solves the coordination problem of several libraries making
> up their own syntax (if anything, the problem becomes worse)
I was thinking of a very constrained support for embedded DSL to avoid
problems of too liberal rules interfering with each other.

If you, as in my earlier example, only allow DSL syntax within a Scala
specific syntax for delimiting it, you at least constraint it. If you
further more only allow the DSL to interface with it's surroundings
through Scala constructs you have full control over the problem.

ex.

val someTable = 'myTable;
val qty = #SQL(SELECT SUM(someNum) FROM ?(someTable))

Here "#SQL()" and "?(someTable)" is all Scala. With some sensible
rules for handling shadowing of identifiers maybe one could write the
same as:

val someTable = 'myTable;
val qty = SQL(SELECT SUM(someNum) FROM someTable)

But then the implementer of SQL() would decide wether "someTable" is
an identifier within the DSL or if it is to be imported and resolved
as a Scala identifier. Maybe this will introduce the same coordination
headaches that spawned this thread from the beginning.

BR,
John