- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
New Collections: can we decrease the noise level?
Sat, 2009-11-14, 20:49
Here are a couple of functions I wrote lately:
These are simplified already from algorithms I thought cleaner, but which required even more boilerplate. Also, they require importing stuff from scala.collection and scala.collection.generic. Here are some thoughts:
1. To fully use CC[X] <: Traversable[X], I need Traversable[X] with TraversableLike[X, CC[X]] with GenericTraversableTemplate[X, CC].
Without Traversable, I can't get companion (ie, no "empty" or "apply"). Without GenericTraversableTemplate, I can't get a companion that returns CC. Without TraversableLike, I can't get stuff like "++" to return CC.
This pretty much applies down the hierarchy of classes, but it is particularly important for the *generic* ones: Traversable, Iterable, Map, Set and Seq.
2. The implicit CanBuildFrom is pretty much required for anything. Having it means, as well, I can't use <% to simplify declarations.
I'm talking about a pattern that you, designers of Scala, have worked hard to achieve, because of how powerful it is. I suggest some thought must be put on how to simplify the usage of this pattern a bit. A couple of suggestions:
* Define something like:
type TraversableType[A, CC] = Traversable[A] with TraversableLike[A, CC[A]] with GenericTraversableTemplate[A, CC]
to make the type parameter declaration briefer and less redundant.
* Accept mixing <% and implicit parameters. Just append the implicitly declared implicits to the implicit parameter list.
Accept them or not, but please do something about the verbosity of these declarations.
About CanBuildFrom, if we had something like A : T for it, that would help a lot.
On a minor note, I haven't found any way to get at a CC[A]-returning companion except casting coll to GenericTraversableTemplate before getting the companion. Is there anything that can be done about it?
I'll just finish by stating the use case. I have CC[T] <: Traversable[T], with perhaps a T : Numeric or T <% Ordering. I want operations on coll: CC[T] to return CC[T]. There ought to be some way to state so in a brief manner.
-- Daniel C. Sobral
Veni, vidi, veterni.
def quicksort
[T, CC[X] <: Traversable[X] with TraversableLike[X, CC[X]]] // My type parameters
(coll: CC[T]) // My explicit parameter
(implicit o: T => Ordered[T], cbf: CanBuildFrom[CC[T], T, CC[T]]) // My implicit parameters
: CC[T] = // My return type
if (coll.isEmpty) {
coll
} else {
val (smaller, bigger) = coll.tail partition (_ < coll.head)
quicksort(smaller) ++ coll.companion(coll.head) ++ quicksort(bigger)
}
import scala.collection.breakOut
def mode
[T, CC[X] <: Seq[X]](coll: CC[T])
(implicit o: T => Ordered[T], cbf: CanBuildFrom[Nothing, T, CC[T]])
: CC[T] = {
val grouped = coll.groupBy(x => x).mapValues(_.size).toSeq
val max = grouped.map(_._2).max
grouped.filter(_._2 == max).map(_._1)(breakOut)
}
These are simplified already from algorithms I thought cleaner, but which required even more boilerplate. Also, they require importing stuff from scala.collection and scala.collection.generic. Here are some thoughts:
1. To fully use CC[X] <: Traversable[X], I need Traversable[X] with TraversableLike[X, CC[X]] with GenericTraversableTemplate[X, CC].
Without Traversable, I can't get companion (ie, no "empty" or "apply"). Without GenericTraversableTemplate, I can't get a companion that returns CC. Without TraversableLike, I can't get stuff like "++" to return CC.
This pretty much applies down the hierarchy of classes, but it is particularly important for the *generic* ones: Traversable, Iterable, Map, Set and Seq.
2. The implicit CanBuildFrom is pretty much required for anything. Having it means, as well, I can't use <% to simplify declarations.
I'm talking about a pattern that you, designers of Scala, have worked hard to achieve, because of how powerful it is. I suggest some thought must be put on how to simplify the usage of this pattern a bit. A couple of suggestions:
* Define something like:
type TraversableType[A, CC] = Traversable[A] with TraversableLike[A, CC[A]] with GenericTraversableTemplate[A, CC]
to make the type parameter declaration briefer and less redundant.
* Accept mixing <% and implicit parameters. Just append the implicitly declared implicits to the implicit parameter list.
Accept them or not, but please do something about the verbosity of these declarations.
About CanBuildFrom, if we had something like A : T for it, that would help a lot.
On a minor note, I haven't found any way to get at a CC[A]-returning companion except casting coll to GenericTraversableTemplate before getting the companion. Is there anything that can be done about it?
I'll just finish by stating the use case. I have CC[T] <: Traversable[T], with perhaps a T : Numeric or T <% Ordering. I want operations on coll: CC[T] to return CC[T]. There ought to be some way to state so in a brief manner.
-- Daniel C. Sobral
Veni, vidi, veterni.
Sat, 2009-11-14, 23:47
#2
Re: New Collections: can we decrease the noise level?
I don't want to defend the ugliness that is 2.8 collections boilerplate, but you'll note that your underlying request
is fundamentally impossible in 2.7.x
--j
I want operations on coll: CC[T] to return CC[T]
is fundamentally impossible in 2.7.x
--j
Sun, 2009-11-15, 01:47
#3
Test cases
Hello everyone,
this is my first time posting on this mailing list, so I hope that
I took the right mailing list for my question. I'm an exchange
student at CMU, Pittsburgh from EPFL, Lausanne and for
one of our courses we are asked to engage with an open source
project. Together with my team mate Stéphanie Falbriard
we decided to work on the scala project because we already
have some experience with scala from working with it at the
EPFL. We contacted Martin Odersky and he mentioned that
the test cases would be a good thing to work on. We already
compiled scala, ran the provided test cases and examined how
adding test cases works technically. Now our questions are:
- how can we determine which test cases are needed?
- how is the creation of a test case assigned to someone?
It would also be nice to know if someone has a good test
case to begin with.
Thanks a lot,
-Andres Nötzli
Tue, 2009-11-17, 00:07
#4
Re: Test cases
On Sat, Nov 14, 2009 at 07:38:52PM -0500, Andres Nötzli wrote:
> - how can we determine which test cases are needed?
Take a look at the open tickets in trac, that'll give you plenty of
ideas. Anything which lends itself to being tested in a self contained
way should have a pile of unit tests. I suggest getting familiar with
scalacheck as it gives you the best chance of flushing out the bugs you
didn't specifically think to test.
The collections are creaking under the weight of straightforward to
write tests which aren't there. Take a look at
sequenceComparisons.scala for an incomplete sample of the sort of test
which should exist for and between every collection.
If you want to undertake an immediately useful task, there are a whole
bunch of test cases which still use SUnit, which is supposed to go away.
Those tests are all quite old, they'd love a refresh. If you expunged
SUnit that's one more thing we could delete.
> - how is the creation of a test case assigned to someone?
I can't speak for anyone else, but I program myself to dream about tests
and I wake up knowing what I'm supposed to do. I assume that's how all
test cases are assigned, but I've been wrong before.
Tue, 2009-11-17, 03:57
#5
Re: Test cases
>>>>> "Paul" == Paul Phillips writes:
Paul> On Sat, Nov 14, 2009 at 07:38:52PM -0500, Andres Nötzli wrote:
>> - how can we determine which test cases are needed?
Paul> Take a look at the open tickets in trac, that'll give you plenty
Paul> of ideas.
Not to mention the tickets closed as "fixed". Especially if when it was
closed, it wasn't closed with a "fixed in r12345" remark, or if it was
closed with such a remark, you look at revision 12345 and it doesn't
include any new test cases.
Paul> If you want to undertake an immediately useful task, there are a
Paul> whole bunch of test cases which still use SUnit, which is
Paul> supposed to go away. Those tests are all quite old, they'd love
Paul> a refresh. If you expunged SUnit that's one more thing we could
Paul> delete.
+1
Tue, 2009-11-17, 10:17
#6
Re: Test cases
On Tue, Nov 17, 2009 at 3:51 AM, Seth Tisue wrote:
>>>>>> "Paul" == Paul Phillips writes:
>
> Paul> On Sat, Nov 14, 2009 at 07:38:52PM -0500, Andres Nötzli wrote:
> >> - how can we determine which test cases are needed?
>
> Paul> Take a look at the open tickets in trac, that'll give you plenty
> Paul> of ideas.
>
> Not to mention the tickets closed as "fixed". Especially if when it was
> closed, it wasn't closed with a "fixed in r12345" remark, or if it was
> closed with such a remark, you look at revision 12345 and it doesn't
> include any new test cases.
yes, but one needs to be careful there. Sometimes the test case came
later. And sometimes it is bunched with other tests in one file. Maybe
try to find a file with the ticket number in its name and
alternatively grep for the ticket number in all existing test files.
Cheers
Fri, 2009-11-20, 19:47
#7
Re: Test cases
First of all, thank you for all the useful answers.
We decided that we will do the conversion of the
SUnit test. Now we are not sure if the conversion
should use something like ScalaCheck which could
be useful for things like the test cases for BigInts/BigDecimals
or if we should do them like most other test cases
with a .check file. Until now there seem to be only
two tests using ScalaCheck in the test\files\scalacheck.
Is ScalaCheck already fully integrated in the testing process?
Fri, 2009-11-20, 20:07
#8
Re: Test cases
On Fri, Nov 20, 2009 at 01:37:14PM -0500, Andres Nötzli wrote:
> Until now there seem to be only two tests using ScalaCheck in the
> test\files\scalacheck. Is ScalaCheck already fully integrated in the
> testing process?
Only two tests because that's where I stopped hoping I had blazed enough
trail. At the time I checked them in with the working scalacheck jar,
they were being run as part of ant test. So in theory it is fully
integrated, but since I haven't been monitoring it it's possible it is
not. Also, I have some patches to scalacheck which I really need to
check in over there but it's always a bit of a brain jumbler to switch
my focus between projects.
Sat, 2009-11-21, 19:17
#9
Re: Test cases
On Fri, Nov 20, 2009 at 01:37:14PM -0500, Andres Nötzli wrote:
>> Until now there seem to be only two tests using ScalaCheck in the
>> test\files\scalacheck. Is ScalaCheck already fully integrated in the
>> testing process?
>>
>
> Only two tests because that's where I stopped hoping I had blazed enough
> trail. At the time I checked them in with the working scalacheck jar,
> they were being run as part of ant test. So in theory it is fully
> integrated, but since I haven't been monitoring it it's possible it is
> not. Also, I have some patches to scalacheck which I really need to
> check in over there but it's always a bit of a brain jumbler to switch
> my focus between projects.
>
>
Since we have to deliver something until the next week (we didn't chose
this narrow time frame...)
it sounds like the method with the .check files is the way to go. Is it
an acceptable solution to rewrite
the SUnit tests this way? We already converted some tests by taking the
old test methods and
defining our own AssertTrue/AssertFalse/AssertEquals methods which would
output a message on the
standard output if the criterion is not met. Is this a good way to
convert the SUnit cases or should we
consider to do it differently?
Thanks,
Andres
Sat, 2009-11-21, 19:37
#10
Re: Test cases
On Sat, Nov 21, 2009 at 01:07:58PM -0500, Andres Nötzli wrote:
> We already converted some tests by taking the old test methods and
> defining our own AssertTrue/AssertFalse/AssertEquals methods which
> would output a message on the standard output if the criterion is not
> met. Is this a good way to convert the SUnit cases or should we
> consider to do it differently?
Maybe the expedient solution is to salvage whatever bits of SUnit you
need to smooth the transition, and put them into some class which is
private to whatever environment partest runs in. That way at least we
can delete SUnit and be left with an encapsulated legacy situation.
There's still no obvious winner for how testing will or should be done
going forward, so converting those tests to use partest and .check files
is probably as good an answer as any.
Tue, 2009-11-24, 08:17
#11
Re: Test cases
>
> On Sat, Nov 21, 2009 at 01:07:58PM -0500, Andres Nötzli wrote:
>
>> We already converted some tests by taking the old test methods and
>> defining our own AssertTrue/AssertFalse/AssertEquals methods which
>> would output a message on the standard output if the criterion is not
>> met. Is this a good way to convert the SUnit cases or should we
>> consider to do it differently?
>>
>
> Maybe the expedient solution is to salvage whatever bits of SUnit you
> need to smooth the transition, and put them into some class which is
> private to whatever environment partest runs in. That way at least we
> can delete SUnit and be left with an encapsulated legacy situation.
> There's still no obvious winner for how testing will or should be done
> going forward, so converting those tests to use partest and .check files
> is probably as good an answer as any.
>
>
We converted now all the test cases using SUnit to partest/.check tests.
We decided not to
write our own AssertTrue/False/Whatever stuff but to replace the asserts
with something
adapted to the test case. I think that this helps to keep the test cases
independent one from
another. We are currently waiting for the approval of our Contributor
Agreement but if
anyone is interested in taking a look at the source before it gets
committed I can upload it
somewhere (I just didn't want to put the whole source on the mailing
list and force everyone
to download it).
Thanks for all the help.
- Andres
Tue, 2009-11-24, 18:37
#12
Re: Test cases
I temporarily uploaded the files now here:
http://www.coccosoft.ch/anderes/no_sunit.zip
On Sat, Nov 14, 2009 at 11:49 AM, Daniel Sobral wrote:
>
> I'm talking about a pattern that you, designers of Scala, have worked hard
> to achieve, because of how powerful it is. I suggest some thought must be
> put on how to simplify the usage of this pattern a bit. A couple of
> suggestions:
> * Define something like:
> type TraversableType[A, CC] = Traversable[A] with TraversableLike[A, CC[A]]
> with GenericTraversableTemplate[A, CC]
+1
> to make the type parameter declaration briefer and less redundant.
> * Accept mixing <% and implicit parameters. Just append the implicitly
> declared implicits to the implicit parameter list.
I'd like this, but I have a guess it's not feasible. Note that
:Ordering is preferred to <% Ordered[T] now (I think...)
> Accept them or not, but please do something about the verbosity of these
> declarations.
> About CanBuildFrom, if we had something like A : T for it, that would help a
> lot.
Some of it can be accomplished:
// Better name here, obviously.
scala> trait Foo[-From,+To] { type BF[-T] = CanBuildFrom[From,T,To] }
defined trait Foo
scala> type TraversableType[A, CC[T] <: Traversable[T] with
TraversableLike[T,CC[T]]] = Traversable[A] with TraversableLike[A,
CC[A]]
defined type alias TraversableType
scala> def foo[T:Foo[CC[T],CC[T]]#BF,CC[X] <:
TraversableType[X,CC]](f: T,cc: CC[T]) = null
foo: [T,CC[X] <: TraversableType[X,CC]](f: T,cc: CC[T])(implicit
evidence$1: scala.collection.generic.CanBuildFrom[CC[T],T,CC[T]])Null
I don't think I have the type-foo attach the Foo type to CC[X] though.
I think it might not be possible.