Unit testing

I apologize if I sound inappropriate, but isn't the very definition of "de facto" that it's not actually a "set" definition, but the "dominating" thing?

"The term de facto may also be used when there is no relevant law or standard, but a common practice is well established, although not universal. For example, English is the most common language in the United States, but is not the official national language, making English the de facto official language of the United States. English is the de jure sole official language in twenty eight individual states of the United States."

http://en.wikipedia.org/wiki/De_facto

On Tue, Feb 24, 2009 at 7:01 PM, Russel Winder <russel.winder@concertant.com> wrote:

If I type "unit test" as a search term on the http://www.scala-lang.org/
page the first mention of a unit test framework is item 8 "ScalaCheck
1.5" and there are only 11 items.

Type "Scala unit test" at Google and the principle item is Ola Bini's
little rant about the lack of unit test framework for Scala.  Well
actually he comes up with SUnit, ScUnit, Rehersal, ScalaCheck and Specs.
To this add ScalaTest.  Also of course there is TestNG and the venerable
JUnit which are the Java unit test frameworks that can be used from
Scala.

So my question is:  has the Scala community decided on which framework
will be deemed the de facto standard one?

Unit testing became the accepted norm in Java programming partly because
there was one standard framework, JUnit.  TestNG arose because of
JUnit's post Java 1.5 deficiencies, and JUnit 4 is really a pale
imitation of TestNG.  Unit testing is still relatively rare in C and C++
and one can probably show that one of the factors is that there is a
plethora of unit testing frameworks and none of them are looked at as de
facto standard.

So for unit testing to become strong in Scala programming it would be
good if there was one or perhaps two de facto or actual standard
frameworks.  SUnit clearly fails to be this for whatever reason.

Is the de facto standard therefore actually TestNG?

I suspect I should immediately ask that all flames and trolls be
directed to /dev/null.  I want to move development from Java to Scala
and genuinely need an answer to "Which unit test framework is the
standard one".

Thanks.
--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077

--
Viktor Klang
Senior Systems Analyst

Based on popularity* I would say that Scala has 3 major test frameworks with different focus areas:

ScalaTest -- focuses on basic Unit Testing; the closest JUnit equivalent

ScalaSpecs -- focuses on specifications and behavior-driven development

ScalaCheck -- also does Unit Testing but focusses on automatic test case generation

Thankfully they all work together nicely so you don't have to chose between them.  You can start with the one you're most comfortable with and then add the others to suit your needs.

alex

* My very own perception of popularity with no substantive data to support it and no offence intended to other frameworks not mentioned.

On Tue, Feb 24, 2009 at 10:01 AM, Russel Winder <russel.winder@concertant.com> wrote:

If I type "unit test" as a search term on the http://www.scala-lang.org/
page the first mention of a unit test framework is item 8 "ScalaCheck
1.5" and there are only 11 items.

Type "Scala unit test" at Google and the principle item is Ola Bini's
little rant about the lack of unit test framework for Scala.  Well
actually he comes up with SUnit, ScUnit, Rehersal, ScalaCheck and Specs.
To this add ScalaTest.  Also of course there is TestNG and the venerable
JUnit which are the Java unit test frameworks that can be used from
Scala.

So my question is:  has the Scala community decided on which framework
will be deemed the de facto standard one?

Unit testing became the accepted norm in Java programming partly because
there was one standard framework, JUnit.  TestNG arose because of
JUnit's post Java 1.5 deficiencies, and JUnit 4 is really a pale
imitation of TestNG.  Unit testing is still relatively rare in C and C++
and one can probably show that one of the factors is that there is a
plethora of unit testing frameworks and none of them are looked at as de
facto standard.

So for unit testing to become strong in Scala programming it would be
good if there was one or perhaps two de facto or actual standard
frameworks.  SUnit clearly fails to be this for whatever reason.

Is the de facto standard therefore actually TestNG?

I suspect I should immediately ask that all flames and trolls be
directed to /dev/null.  I want to move development from Java to Scala
and genuinely need an answer to "Which unit test framework is the
standard one".

Thanks.
--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077

Hi Russel,

There are three production-ready testing frameworks for Scala, and
they are each aimed at different goals:

ScalaCheck enables property-based testing, in which you write a
"property" in one or a few lines of code, and it automatically
generates test cases for you, including attempting to test edge
conditions.

Specs enables a behavior-driven development style of testing, in which
you write tests first using specification like terms, and the output
of the test reads as an informal specification of the system under
test.

ScalaTest is designed to facilitate different styles of testing. It
also facilitates a behavior-driven development style of testing
similar to Specs, but with a different syntax. (And in the next
release, coming this week, will have its own matchers DSL which really
I think shows off the awesomeness of Scala's DSL capabilities.) But if
you don't drink the BDD koolaid, you can also use ScalaTest to test in
more traditional styles, or to use nicer syntax when writing JUnit or
TestNG tests in Scala.

You can use ScalaCheck from either Specs or ScalaTest, and you can use
Specs from ScalaTest and vice versa, so everything is fairly
integrated with each other. I think everyone should look at
ScalaCheck, but probably use it from either ScalaTest or Specs,
because they are more general. I find I still need to write a lot of
non-property-based tests even though I use ScalaCheck properties as
much as I can. So if you don't like BDD, or want (or need) to be
writing JUnit or TestNG tests, the ScalaTest is the way to go. If you
like the BDD style, then either ScalaTest or Specs will work, and you
can pick whichever syntax you prefer.

Bill

On Tue, Feb 24, 2009 at 10:01 AM, Russel Winder
wrote:
> If I type "unit test" as a search term on the http://www.scala-lang.org/
> page the first mention of a unit test framework is item 8 "ScalaCheck
> 1.5" and there are only 11 items.
>
> Type "Scala unit test" at Google and the principle item is Ola Bini's
> little rant about the lack of unit test framework for Scala.  Well
> actually he comes up with SUnit, ScUnit, Rehersal, ScalaCheck and Specs.
> To this add ScalaTest.  Also of course there is TestNG and the venerable
> JUnit which are the Java unit test frameworks that can be used from
> Scala.
>
> So my question is:  has the Scala community decided on which framework
> will be deemed the de facto standard one?
>
> Unit testing became the accepted norm in Java programming partly because
> there was one standard framework, JUnit.  TestNG arose because of
> JUnit's post Java 1.5 deficiencies, and JUnit 4 is really a pale
> imitation of TestNG.  Unit testing is still relatively rare in C and C++
> and one can probably show that one of the factors is that there is a
> plethora of unit testing frameworks and none of them are looked at as de
> facto standard.
>
> So for unit testing to become strong in Scala programming it would be
> good if there was one or perhaps two de facto or actual standard
> frameworks.  SUnit clearly fails to be this for whatever reason.
>
> Is the de facto standard therefore actually TestNG?
>
> I suspect I should immediately ask that all flames and trolls be
> directed to /dev/null.  I want to move development from Java to Scala
> and genuinely need an answer to "Which unit test framework is the
> standard one".
>
> Thanks.
> --
> Russel.
> ====================================================
> Dr Russel Winder                 Partner
>
> Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
> 41 Buckmaster Road,              f: +44 8700 516 084
> London SW11 1EN, UK.             m: +44 7770 465 077
>

We must not also not forget that the creator of ScalaCheck, Rickard Nilsson, is an awesome guy.

On Tue, Feb 24, 2009 at 8:02 PM, Bill Venners <bill@artima.com> wrote:

Hi Russel,

There are three production-ready testing frameworks for Scala, and
they are each aimed at different goals:

ScalaCheck enables property-based testing, in which you write a
"property" in one or a few lines of code, and it automatically
generates test cases for you, including attempting to test edge
conditions.

Specs enables a behavior-driven development style of testing, in which
you write tests first using specification like terms, and the output
of the test reads as an informal specification of the system under
test.

ScalaTest is designed to facilitate different styles of testing. It
also facilitates a behavior-driven development style of testing
similar to Specs, but with a different syntax. (And in the next
release, coming this week, will have its own matchers DSL which really
I think shows off the awesomeness of Scala's DSL capabilities.) But if
you don't drink the BDD koolaid, you can also use ScalaTest to test in
more traditional styles, or to use nicer syntax when writing JUnit or
TestNG tests in Scala.

You can use ScalaCheck from either Specs or ScalaTest, and you can use
Specs from ScalaTest and vice versa, so everything is fairly
integrated with each other. I think everyone should look at
ScalaCheck, but probably use it from either ScalaTest or Specs,
because they are more general. I find I still need to write a lot of
non-property-based tests even though I use ScalaCheck properties as
much as I can. So if you don't like BDD, or want (or need) to be
writing JUnit or TestNG tests, the ScalaTest is the way to go. If you
like the BDD style, then either ScalaTest or Specs will work, and you
can pick whichever syntax you prefer.

Bill

On Tue, Feb 24, 2009 at 10:01 AM, Russel Winder
<russel.winder@concertant.com> wrote:
> If I type "unit test" as a search term on the http://www.scala-lang.org/
> page the first mention of a unit test framework is item 8 "ScalaCheck
> 1.5" and there are only 11 items.
>
> Type "Scala unit test" at Google and the principle item is Ola Bini's
> little rant about the lack of unit test framework for Scala.  Well
> actually he comes up with SUnit, ScUnit, Rehersal, ScalaCheck and Specs.
> To this add ScalaTest.  Also of course there is TestNG and the venerable
> JUnit which are the Java unit test frameworks that can be used from
> Scala.
>
> So my question is:  has the Scala community decided on which framework
> will be deemed the de facto standard one?
>
> Unit testing became the accepted norm in Java programming partly because
> there was one standard framework, JUnit.  TestNG arose because of
> JUnit's post Java 1.5 deficiencies, and JUnit 4 is really a pale
> imitation of TestNG.  Unit testing is still relatively rare in C and C++
> and one can probably show that one of the factors is that there is a
> plethora of unit testing frameworks and none of them are looked at as de
> facto standard.
>
> So for unit testing to become strong in Scala programming it would be
> good if there was one or perhaps two de facto or actual standard
> frameworks.  SUnit clearly fails to be this for whatever reason.
>
> Is the de facto standard therefore actually TestNG?
>
> I suspect I should immediately ask that all flames and trolls be
> directed to /dev/null.  I want to move development from Java to Scala
> and genuinely need an answer to "Which unit test framework is the
> standard one".
>
> Thanks.
> --
> Russel.
> ====================================================
> Dr Russel Winder                 Partner
>
> Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
> 41 Buckmaster Road,              f: +44 8700 516 084
> London SW11 1EN, UK.             m: +44 7770 465 077
>

--
Viktor Klang
Senior Systems Analyst

We use Specs with ScalaCheck, using the JUnit4 runner included in Specs, so that our existing tools don't need retraining.

2009/2/24 Russel Winder <russel.winder@concertant.com>

If I type "unit test" as a search term on the http://www.scala-lang.org/
page the first mention of a unit test framework is item 8 "ScalaCheck
1.5" and there are only 11 items.

Type "Scala unit test" at Google and the principle item is Ola Bini's
little rant about the lack of unit test framework for Scala.  Well
actually he comes up with SUnit, ScUnit, Rehersal, ScalaCheck and Specs.
To this add ScalaTest.  Also of course there is TestNG and the venerable
JUnit which are the Java unit test frameworks that can be used from
Scala.

So my question is:  has the Scala community decided on which framework
will be deemed the de facto standard one?

Unit testing became the accepted norm in Java programming partly because
there was one standard framework, JUnit.  TestNG arose because of
JUnit's post Java 1.5 deficiencies, and JUnit 4 is really a pale
imitation of TestNG.  Unit testing is still relatively rare in C and C++
and one can probably show that one of the factors is that there is a
plethora of unit testing frameworks and none of them are looked at as de
facto standard.

So for unit testing to become strong in Scala programming it would be
good if there was one or perhaps two de facto or actual standard
frameworks.  SUnit clearly fails to be this for whatever reason.

Is the de facto standard therefore actually TestNG?

I suspect I should immediately ask that all flames and trolls be
directed to /dev/null.  I want to move development from Java to Scala
and genuinely need an answer to "Which unit test framework is the
standard one".

Thanks.
--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077

Hi Russel,

I think that you must not forget that JUnit came at a time when Java was
dominant, Agile ideas were picking up, etc... All sorts of factors making it
a no-brainer at the time. Then came TestNG and JUnit4 showing that we could
still find ways to innovate and express ourselves better. Not to mention
rspec which was my original inspiration for creating the specs library.

So eventually I'm not sure that we really need a de jure standard. Some
people may be more comfortable with one paradigm than with the other.

I think that now unit testing or specifications should be strong only
because everyone understands their utility, not because there is "one best
way". Then, in the Scala-land, the offer seems pretty healthy and up to
supporting your requirements. If not, we need to roll-up our sleeves and get
back to work.

Try out those different libraries and please tell us what you like and what
you don't like. Then you'll help us build your own "standard" which I think
doesn't exist yet.

Eric.

PS: The second post from Ola is not that bad!
http://olabini.com/blog/2008/01/scala-testing-with-specs
PS: Yes, ScalaCheck rocks:
http://etorreborre.blogspot.com/search/label/scalacheck

Thanks to everyone who replied to my email about Scala unit testing.
Most constructive and definitely most helpful.

I shall go away, mull, experiment, learn and then be decisive.

Thanks.