zipped is not enough: we need zipWith (well, I want zipWith, anyway)

On 16/02/12 23:15, Chris Marshall wrote:
> Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
> def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]
> It's extremely common for me to do these things:
> 1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
> This looks much nicer as:
> (cc zipWith f).toMap
> Than
> (cc map (a => a -> f(b))).toMap
> 2. Zip all the elements of a collection to a single element
> cc zipWith ( _ => b)
> Is much, much safer than:
> cc zipAll (Seq(b), null, b)
> Ugh! Look, there's a null in there! The arguments aren't even by-name for me to provide a sys.error in case the "null" ever gets used accidentally (i.e. I cannot do cc zipAll (Seq(b), sys.error("Ugh, null"), b))
> Looking in JIRA, Martin suggests https://issues.scala-lang.org/browse/SI-1512 that the issue is "fixed" by Tuple2.zipped but this seems wrong to me as it addresses neither of the above points (and it pertains to a slightly different signature of zipWith, suggested by Dan Spiewak)
> Chris
zipWith generalises to any applicative functor. That's what you really
need/want.

(cc <*> f.pure[CC]).toMap

and

cc <*> Seq( (a: A) => a -> b) 

Or for the latter

def pairTo[A, B](b: B): A => (A, B) = a => a -> b

And then 

cc <*> pairTo(b).pure[CC]

That's nice, but I still think they deserve addition into the standard library
Chris

> Date: Thu, 16 Feb 2012 23:16:22 +1000
> From: tonymorris@gmail.com
> To: scala-user@googlegroups.com
> Subject: Re: [scala-user] zipped is not enough: we need zipWith (well, I want zipWith, anyway)
>
> On 16/02/12 23:15, Chris Marshall wrote:
> > Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
> > def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]
> > It's extremely common for me to do these things:
> > 1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
> > This looks much nicer as:
> > (cc zipWith f).toMap
> > Than
> > (cc map (a => a -> f(b))).toMap
> > 2. Zip all the elements of a collection to a single element
> > cc zipWith ( _ => b)
> > Is much, much safer than:
> > cc zipAll (Seq(b), null, b)
> > Ugh! Look, there's a null in there! The arguments aren't even by-name for me to provide a sys.error in case the "null" ever gets used accidentally (i.e. I cannot do cc zipAll (Seq(b), sys.error("Ugh, null"), b))
> > Looking in JIRA, Martin suggests https://issues.scala-lang.org/browse/SI-1512 that the issue is "fixed" by Tuple2.zipped but this seems wrong to me as it addresses neither of the above points (and it pertains to a slightly different signature of zipWith, suggested by Dan Spiewak)
> > Chris
> zipWith generalises to any applicative functor. That's what you really
> need/want.
>
> --
> Tony Morris
> http://tmorris.net/
>
>

On Thu, Feb 16, 2012 at 8:15 AM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:

def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]

It's extremely common for me to do these things:
1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
This looks much nicer as:

(cc zipWith f).toMap

Than

(cc map (a => a -> f(b))).toMap

It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)

2. Zip all the elements of a collection to a single element

cc zipWith ( _ => b)

What's wrong with cc map (_ -> b)?

  --Rex

> It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)

Actually I did but then removed it again when 2.8 arrived because I'm not keen on pimping the collections library for the same reason that Yammer isn't

> What's wrong with cc map (_ -> b)?

Nothing at all: I'm an idiot - I have had zips and zippers on the brain for the last week and now feel rather foolish

>   --Rex

Thanks

On Thu, Feb 16, 2012 at 1:26 PM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

> It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)

Actually I did but then removed it again when 2.8 arrived because I'm not keen on pimping the collections library for the same reason that Yammer isn't

I thought Yammer's point was that they didn't use collections because they were not consistently as fast as Java collections.  I didn't recall them not pimping collections for any reason.  Did you mean Yang Zhang's "true Scala complexity" post?

Anyway, the entry barrier to enriching collections is high, but once you understand the three or four critical pieces and accept that there's no getting around a decent gob of boilerplate for a conceptually almost trivial task, it's not that hard to actually use.  Here:

import collection.generic.CanBuildFrom
class MappableCollection[A,C](ca: C)(implicit c2s: C => Iterable[A]) {
  def mapTo[B,D[_,_] <: Map[_,_]](f: A => B)(implicit cbf: CanBuildFrom[Nothing,(A,B),D[A,B]]): D[A,B] = {
    val mapped = cbf()
    var i = c2s(ca).iterator
    while (i.hasNext) {
      val a = i.next
      mapped += (a -> f(a))
    }
    mapped.result
  }
}
implicit def collections_can_be_mapped[A, C[A]](ca: C[A])(implicit c2s: C[A] => Iterable[A]) =
  new MappableCollection[A,C[A]](ca)(c2s)
implicit def strings_can_be_mapped(s: String)(implicit c2s: String => Iterable[Char]) =
  new MappableCollection[Char,String](s)(c2s)

Took just a couple minutes of alterations from something I already had.

Now:

scala> List(1,2,3).mapTo(_.toString*2)
res32: scala.collection.immutable.Map[Int,String] = Map(1 -> 11, 2 -> 22, 3 -> 33)

scala> "wish" mapTo(_.toInt)
res33: scala.collection.immutable.Map[Char,Int] = Map(w -> 119, i -> 105, s -> 115, h -> 104)

I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

  --Rex

>  Did you mean Yang Zhang's "true Scala complexity" post?

Ah yes; I meant that. I totally agree with your example - i.e. it's really not that hard. At the time I thought it was not worth that amount of code to save a few characters given the addition of the toMap method (previously I'd hated the fact that you had to do Map(xs map (x => x -> f(x)).toSeq : _ *) - ugh). Also given that "map" was starting to mean "Functor" to me, I didn't like using it as part of the "mapTo" name
> I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

That and specializing the library. I think a working, specialized collections library would get a lot of people to sit up and take notice. At the moment, specialization feels a lot like we brought a Lion to deal with a rogue mouse when there's a hyena in the bedroom terrorising the missus.
>  --Rex

On Thu, Feb 16, 2012 at 3:26 PM, Chris Marshall <oxbow_lakes@hotmail.com> wrote:

> I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

That and specializing the library. I think a working, specialized collections library would get a lot of people to sit up and take notice. At the moment, specialization feels a lot like we brought a Lion to deal with a rogue mouse when there's a hyena in the bedroom terrorising the missus.

Well, the value added to functions is quite high, even if right now it's really only functions that benefit.

But I agree--specialized collections would be a big deal, and I don't see how to get there without significant language enhancements (maybe scala-virtualized or macros will do the trick).  In addition to the existing bugs and limitations in the implementation of specialization, there is an issue with code bloat for things like map and fold, and an issue with multiple dispatch eating up much of the savings that you thought you had from lack of boxing.

I attempted to create a decent mutable primitive collections library, and even with writing code generators to produce lots of annoying boilerplate in the library, I couldn't reliably both improve performance significantly and keep the code clean.  I never was satisfied that it was a promising direction.  I'm going to try again when more specialization bugs are fixed and some other appropriate feature appears (any of MyType, scala-virtualized, or macros would change things enough to make it worth having another go).

  --Rex