Scala test frameworks unified interface

All,

The Mark and the Scala testing community are working hard at a unified interface for the three main test frameworks (and hopefully any others that spring up as well). It would/should/could be usable not only by sbt, but by IDE's and any other tools making use of test frameworks. We could use some help understanding the needs of some of the other tools. Were at a point where we don't want to make it much more complicated, but also want to make sure were serving the needs of the most common tools.

The interfaces are in pure Java, and can be found here: http://github.com/harrah/test-interface. However, those are already slightly out of date based on the conversations below, mostly in the more recent messages in the thread.

For anyone who cares to provide it, input would be greatly appreciated.

-Josh

---------- Forwarded message ----------
From: Mark Harrah <dmharrah@gmail.com>
Date: Mon, Oct 19, 2009 at 10:23 AM
Subject: [sbt] Re: ScalaTest 1.0
To: simple-build-tool@googlegroups.com



Fair enough.  Since we have always intended this to be useful to more
than just sbt, we should involve the other tools and take this
discussion to a common mailing list, perhaps scala-tools or a new
group.  Do you want to take care of this?

-Mark

On 10/19/09, Josh Cough <joshcough@gmail.com> wrote:
> More to come later, but regarding this:
>
>>> On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>>>> I found yet another issue with Runner. It would be nice if tools had
>>>> immediate info of when test is starting, is finished, and what its
> result
>>>> is, as well as an entire suites. I suppose this issue could be resolved
>>>> with the Logger, but thats probably not the best way.
>
>>I don't see a problem with this.  I think it is the user that needs
>>the immediate feedback, not the tool.  The framework can log to the
>>Logger as it runs and inform the tool of results after the suite
>>completes, .
>
> Lets imagine that an IDE is using this API, and it wants to display a little
> green dot for every test that passes. If we use the logger, it would have to
> parse the log output in order to do that. Each framework is going to log in
> whatever way it pleases, so the IDE will have to have three different log
> parses in order to properly display information to the user in real time.
> Logging is good for a command line tool like sbt, and for running your specs
> from the command line, but it doesn't help other tools.
>
> Now, lets also imagine that you have a Suite with 100k tests, and they take
> an hour to run. Logging would be helpful, but if you're in an IDE and you
> cant see that output, then well, this just doesn't work for you. You have to
> wait around for an hour.
>
> Maybe its a pipedream that an IDE would use this, but its a pipedream I've
> had for years:
>
> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-and-in-ides-in.html
>
> On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah <dmharrah@gmail.com> wrote:
>
>>
>> Eric, Josh,
>>
>> On 10/19/09, etorreborre <etorreborre@gmail.com> wrote:
>> > I think that we may go through a too complicated road here, because
>> > frameworks may have very different way to run and report results. For
>> > example in specs you can nest examples and subexamples indefinitely
>> > and I don't know how to use the current interface with that.
>> >
>> > What would be simpler is:
>> >
>> > Framework.java:
>> >
>> > /** return a Runner using the ResultOutput to display
>> >      - descriptions
>> >      - test results
>> >      - run statistics
>> > */
>> > public Runner testRunner(ClassLoader testClassLoader, Logger logger,
>> > ResultOutput output);
>> >
>> > ResultOutput.java
>> >
>> > /** write to the console */
>> > public void write(message: String)
>> >
>> > Runner.java
>> >
>> > /** return true if the tests are not passing for whatever reason */
>> > public boolean run(String testClassName, TestFingerprint fingerprint,
>> > String [] args);
>> >
>> > Do you see any issue with this approach?
>>
>> The Logger is supposed to be used for the purpose of the ResultOutput.
>>  If you would prefer ResultOutput, that is fine with me.  The run
>> method above is basically all sbt needs.  It just needs to know on a
>> test suite level whether the suite passed or failed so that it can
>> implement test-quick and test-failed.  However, it would be useful as
>> a user to have the number of tests passed/failing/whatever and a
>> summary of failures be uniformly reported.  I think that the rest of
>> the logging/output can be framework specific.  So, I like returning a
>> TestResult[] as Josh suggests.
>>
>> As Eric indicates, the problem with the event listener is that each
>> framework will have a different event hierarchy and events complicate
>> the interface.  There could be a basic TestEvent interface and each
>> framework could implement its own hierarchy, but it should be
>> information that a client is not required to handle.  That is, run
>> should still return TestResult[].
>>
>> > On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>> >> I found yet another issue with Runner. It would be nice if tools had
>> >> immediate info of when test is starting, is finished, and what its
>> result
>> >> is, as well as an entire suites. I suppose this issue could be resolved
>> >> with
>> >> the Logger, but thats probably not the best way.
>>
>> I don't see a problem with this.  I think it is the user that needs
>> the immediate feedback, not the tool.  The framework can log to the
>> Logger as it runs and inform the tool of results after the suite
>> completes, .
>>
>> -Mark
>>
>> >
>>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To post to this group, send email to simple-build-tool@googlegroups.com
To unsubscribe from this group, send email to 2Bunsubscribe [at] googlegroups [dot] com" target="_blank" rel="nofollow">simple-build-tool+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en
-~----------~----~----~----~------~----~------~--~---

Why not use "abstract trait"'s instead of java interfaces?  They should compile similarly and not have a dependency on any specific Scala library....

- Josh

On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough <joshcough@gmail.com> wrote:

All,

The Mark and the Scala testing community are working hard at a unified interface for the three main test frameworks (and hopefully any others that spring up as well). It would/should/could be usable not only by sbt, but by IDE's and any other tools making use of test frameworks. We could use some help understanding the needs of some of the other tools. Were at a point where we don't want to make it much more complicated, but also want to make sure were serving the needs of the most common tools.

The interfaces are in pure Java, and can be found here: http://github.com/harrah/test-interface. However, those are already slightly out of date based on the conversations below, mostly in the more recent messages in the thread.

For anyone who cares to provide it, input would be greatly appreciated.

-Josh

---------- Forwarded message ----------
From: Mark Harrah <dmharrah@gmail.com>
Date: Mon, Oct 19, 2009 at 10:23 AM
Subject: [sbt] Re: ScalaTest 1.0
To: simple-build-tool@googlegroups.com



Fair enough.  Since we have always intended this to be useful to more
than just sbt, we should involve the other tools and take this
discussion to a common mailing list, perhaps scala-tools or a new
group.  Do you want to take care of this?

-Mark

On 10/19/09, Josh Cough <joshcough@gmail.com> wrote:
> More to come later, but regarding this:
>
>>> On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>>>> I found yet another issue with Runner. It would be nice if tools had
>>>> immediate info of when test is starting, is finished, and what its
> result
>>>> is, as well as an entire suites. I suppose this issue could be resolved
>>>> with the Logger, but thats probably not the best way.
>
>>I don't see a problem with this.  I think it is the user that needs
>>the immediate feedback, not the tool.  The framework can log to the
>>Logger as it runs and inform the tool of results after the suite
>>completes, .
>
> Lets imagine that an IDE is using this API, and it wants to display a little
> green dot for every test that passes. If we use the logger, it would have to
> parse the log output in order to do that. Each framework is going to log in
> whatever way it pleases, so the IDE will have to have three different log
> parses in order to properly display information to the user in real time.
> Logging is good for a command line tool like sbt, and for running your specs
> from the command line, but it doesn't help other tools.
>
> Now, lets also imagine that you have a Suite with 100k tests, and they take
> an hour to run. Logging would be helpful, but if you're in an IDE and you
> cant see that output, then well, this just doesn't work for you. You have to
> wait around for an hour.
>
> Maybe its a pipedream that an IDE would use this, but its a pipedream I've
> had for years:
>
> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-and-in-ides-in.html
>
> On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah <dmharrah@gmail.com> wrote:
>
>>
>> Eric, Josh,
>>
>> On 10/19/09, etorreborre <etorreborre@gmail.com> wrote:
>> > I think that we may go through a too complicated road here, because
>> > frameworks may have very different way to run and report results. For
>> > example in specs you can nest examples and subexamples indefinitely
>> > and I don't know how to use the current interface with that.
>> >
>> > What would be simpler is:
>> >
>> > Framework.java:
>> >
>> > /** return a Runner using the ResultOutput to display
>> >      - descriptions
>> >      - test results
>> >      - run statistics
>> > */
>> > public Runner testRunner(ClassLoader testClassLoader, Logger logger,
>> > ResultOutput output);
>> >
>> > ResultOutput.java
>> >
>> > /** write to the console */
>> > public void write(message: String)
>> >
>> > Runner.java
>> >
>> > /** return true if the tests are not passing for whatever reason */
>> > public boolean run(String testClassName, TestFingerprint fingerprint,
>> > String [] args);
>> >
>> > Do you see any issue with this approach?
>>
>> The Logger is supposed to be used for the purpose of the ResultOutput.
>>  If you would prefer ResultOutput, that is fine with me.  The run
>> method above is basically all sbt needs.  It just needs to know on a
>> test suite level whether the suite passed or failed so that it can
>> implement test-quick and test-failed.  However, it would be useful as
>> a user to have the number of tests passed/failing/whatever and a
>> summary of failures be uniformly reported.  I think that the rest of
>> the logging/output can be framework specific.  So, I like returning a
>> TestResult[] as Josh suggests.
>>
>> As Eric indicates, the problem with the event listener is that each
>> framework will have a different event hierarchy and events complicate
>> the interface.  There could be a basic TestEvent interface and each
>> framework could implement its own hierarchy, but it should be
>> information that a client is not required to handle.  That is, run
>> should still return TestResult[].
>>
>> > On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>> >> I found yet another issue with Runner. It would be nice if tools had
>> >> immediate info of when test is starting, is finished, and what its
>> result
>> >> is, as well as an entire suites. I suppose this issue could be resolved
>> >> with
>> >> the Logger, but thats probably not the best way.
>>
>> I don't see a problem with this.  I think it is the user that needs
>> the immediate feedback, not the tool.  The framework can log to the
>> Logger as it runs and inform the tool of results after the suite
>> completes, .
>>
>> -Mark
>>
>> >
>>
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To post to this group, send email to simple-build-tool@googlegroups.com
To unsubscribe from this group, send email to 2Bunsubscribe [at] googlegroups [dot] com" target="_blank" rel="nofollow">simple-build-tool+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en
-~----------~----~----~----~------~----~------~--~---

By using Java interfaces, you keep scala-library.jar off of the classpath and get an automatic check that no Scala classes are referenced.

-Mark

On Wednesday 21 October 2009 10:48, Josh Cough wrote:
> Hi Josh,
>
> That sounds fine. Java interfaces were just what Mark proposed and we all
> said okay. I don't think anyone would object to changing it.
>
> Anyone opposed? I can go in and change it tonight.
>
> -Josh
>
> On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth wrote:
>
> > Why not use "abstract trait"'s instead of java interfaces? They should
> > compile similarly and not have a dependency on any specific Scala
> > library....
> >
> > - Josh
> >
> >
> > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough wrote:
> >
> >> All,
> >>
> >> The Mark and the Scala testing community are working hard at a unified
> >> interface for the three main test frameworks (and hopefully any others that
> >> spring up as well). It would/should/could be usable not only by sbt, but by
> >> IDE's and any other tools making use of test frameworks. We could use some
> >> help understanding the needs of some of the other tools. Were at a point
> >> where we don't want to make it much more complicated, but also want to make
> >> sure were serving the needs of the most common tools.
> >>
> >> The interfaces are in pure Java, and can be found here:
> >> http://github.com/harrah/test-interface. However, those are already
> >> slightly out of date based on the conversations below, mostly in the more
> >> recent messages in the thread.
> >>
> >> For anyone who cares to provide it, input would be greatly appreciated.
> >>
> >> -Josh
> >>
> >> ---------- Forwarded message ----------
> >> From: Mark Harrah
> >> Date: Mon, Oct 19, 2009 at 10:23 AM
> >> Subject: [sbt] Re: ScalaTest 1.0
> >> To: simple-build-tool@googlegroups.com
> >>
> >>
> >>
> >> Fair enough. Since we have always intended this to be useful to more
> >> than just sbt, we should involve the other tools and take this
> >> discussion to a common mailing list, perhaps scala-tools or a new
> >> group. Do you want to take care of this?
> >>
> >> -Mark
> >>
> >> On 10/19/09, Josh Cough wrote:
> >> > More to come later, but regarding this:
> >> >
> >> >>> On Oct 19, 3:03 pm, Josh Cough wrote:
> >> >>>> I found yet another issue with Runner. It would be nice if tools had
> >> >>>> immediate info of when test is starting, is finished, and what its
> >> > result
> >> >>>> is, as well as an entire suites. I suppose this issue could be
> >> resolved
> >> >>>> with the Logger, but thats probably not the best way.
> >> >
> >> >>I don't see a problem with this. I think it is the user that needs
> >> >>the immediate feedback, not the tool. The framework can log to the
> >> >>Logger as it runs and inform the tool of results after the suite
> >> >>completes, .
> >> >
> >> > Lets imagine that an IDE is using this API, and it wants to display a
> >> little
> >> > green dot for every test that passes. If we use the logger, it would
> >> have to
> >> > parse the log output in order to do that. Each framework is going to log
> >> in
> >> > whatever way it pleases, so the IDE will have to have three different
> >> log
> >> > parses in order to properly display information to the user in real
> >> time.
> >> > Logging is good for a command line tool like sbt, and for running your
> >> specs
> >> > from the command line, but it doesn't help other tools.
> >> >
> >> > Now, lets also imagine that you have a Suite with 100k tests, and they
> >> take
> >> > an hour to run. Logging would be helpful, but if you're in an IDE and
> >> you
> >> > cant see that output, then well, this just doesn't work for you. You
> >> have to
> >> > wait around for an hour.
> >> >
> >> > Maybe its a pipedream that an IDE would use this, but its a pipedream
> >> I've
> >> > had for years:
> >> >
> >> >
> >> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-an...
> >> >
> >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah
> >> wrote:
> >> >
> >> >>
> >> >> Eric, Josh,
> >> >>
> >> >> On 10/19/09, etorreborre wrote:
> >> >> > I think that we may go through a too complicated road here, because
> >> >> > frameworks may have very different way to run and report results. For
> >> >> > example in specs you can nest examples and subexamples indefinitely
> >> >> > and I don't know how to use the current interface with that.
> >> >> >
> >> >> > What would be simpler is:
> >> >> >
> >> >> > Framework.java:
> >> >> >
> >> >> > /** return a Runner using the ResultOutput to display
> >> >> > - descriptions
> >> >> > - test results
> >> >> > - run statistics
> >> >> > */
> >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger logger,
> >> >> > ResultOutput output);
> >> >> >
> >> >> > ResultOutput.java
> >> >> >
> >> >> > /** write to the console */
> >> >> > public void write(message: String)
> >> >> >
> >> >> > Runner.java
> >> >> >
> >> >> > /** return true if the tests are not passing for whatever reason */
> >> >> > public boolean run(String testClassName, TestFingerprint fingerprint,
> >> >> > String [] args);
> >> >> >
> >> >> > Do you see any issue with this approach?
> >> >>
> >> >> The Logger is supposed to be used for the purpose of the ResultOutput.
> >> >> If you would prefer ResultOutput, that is fine with me. The run
> >> >> method above is basically all sbt needs. It just needs to know on a
> >> >> test suite level whether the suite passed or failed so that it can
> >> >> implement test-quick and test-failed. However, it would be useful as
> >> >> a user to have the number of tests passed/failing/whatever and a
> >> >> summary of failures be uniformly reported. I think that the rest of
> >> >> the logging/output can be framework specific. So, I like returning a
> >> >> TestResult[] as Josh suggests.
> >> >>
> >> >> As Eric indicates, the problem with the event listener is that each
> >> >> framework will have a different event hierarchy and events complicate
> >> >> the interface. There could be a basic TestEvent interface and each
> >> >> framework could implement its own hierarchy, but it should be
> >> >> information that a client is not required to handle. That is, run
> >> >> should still return TestResult[].
> >> >>
> >> >> > On Oct 19, 3:03 pm, Josh Cough wrote:
> >> >> >> I found yet another issue with Runner. It would be nice if tools had
> >> >> >> immediate info of when test is starting, is finished, and what its
> >> >> result
> >> >> >> is, as well as an entire suites. I suppose this issue could be
> >> resolved
> >> >> >> with
> >> >> >> the Logger, but thats probably not the best way.
> >> >>
> >> >> I don't see a problem with this. I think it is the user that needs
> >> >> the immediate feedback, not the tool. The framework can log to the
> >> >> Logger as it runs and inform the tool of results after the suite
> >> >> completes, .
> >> >>
> >> >> -Mark
> >> >>
> >> >> >
> >> >>
> >> >
> >> > >
> >> >
> >>
> >> --~--~---------~--~----~------------~-------~--~----~
> >> You received this message because you are subscribed to the Google Groups
> >> "simple-build-tool" group.
> >> To post to this group, send email to simple-build-tool@googlegroups.com
> >> To unsubscribe from this group, send email to
> >> simple-build-tool+unsubscribe@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/simple-build-tool?hl=en
> >> -~----------~----~----~----~------~----~------~--~---
> >>
> >>
> >>
> >
>

public enum Result{
  Success, Error, Failure, Skipped
}

By using Java interfaces, you keep scala-library.jar off of the classpath and get an automatic check that no Scala classes are referenced.

-Mark

On Wednesday 21 October 2009 10:48, Josh Cough wrote:
> Hi Josh,
>
> That sounds fine. Java interfaces were just what Mark proposed and we all
> said okay. I don't think anyone would object to changing it.
>
> Anyone opposed? I can go in and change it tonight.
>
> -Josh
>
> On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth <joshua.suereth@gmail.com>wrote:
>
> > Why not use "abstract trait"'s instead of java interfaces?  They should
> > compile similarly and not have a dependency on any specific Scala
> > library....
> >
> > - Josh
> >
> >
> > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough <joshcough@gmail.com> wrote:
> >
> >> All,
> >>
> >> The Mark and the Scala testing community are working hard at a unified
> >> interface for the three main test frameworks (and hopefully any others that
> >> spring up as well). It would/should/could be usable not only by sbt, but by
> >> IDE's and any other tools making use of test frameworks. We could use some
> >> help understanding the needs of some of the other tools. Were at a point
> >> where we don't want to make it much more complicated, but also want to make
> >> sure were serving the needs of the most common tools.
> >>
> >> The interfaces are in pure Java, and can be found here:
> >> http://github.com/harrah/test-interface. However, those are already
> >> slightly out of date based on the conversations below, mostly in the more
> >> recent messages in the thread.
> >>
> >> For anyone who cares to provide it, input would be greatly appreciated.
> >>
> >> -Josh
> >>
> >> ---------- Forwarded message ----------
> >> From: Mark Harrah <dmharrah@gmail.com>
> >> Date: Mon, Oct 19, 2009 at 10:23 AM
> >> Subject: [sbt] Re: ScalaTest 1.0
> >> To: simple-build-tool@googlegroups.com
> >>
> >>
> >>
> >> Fair enough.  Since we have always intended this to be useful to more
> >> than just sbt, we should involve the other tools and take this
> >> discussion to a common mailing list, perhaps scala-tools or a new
> >> group.  Do you want to take care of this?
> >>
> >> -Mark
> >>
> >> On 10/19/09, Josh Cough <joshcough@gmail.com> wrote:
> >> > More to come later, but regarding this:
> >> >
> >> >>> On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
> >> >>>> I found yet another issue with Runner. It would be nice if tools had
> >> >>>> immediate info of when test is starting, is finished, and what its
> >> > result
> >> >>>> is, as well as an entire suites. I suppose this issue could be
> >> resolved
> >> >>>> with the Logger, but thats probably not the best way.
> >> >
> >> >>I don't see a problem with this.  I think it is the user that needs
> >> >>the immediate feedback, not the tool.  The framework can log to the
> >> >>Logger as it runs and inform the tool of results after the suite
> >> >>completes, .
> >> >
> >> > Lets imagine that an IDE is using this API, and it wants to display a
> >> little
> >> > green dot for every test that passes. If we use the logger, it would
> >> have to
> >> > parse the log output in order to do that. Each framework is going to log
> >> in
> >> > whatever way it pleases, so the IDE will have to have three different
> >> log
> >> > parses in order to properly display information to the user in real
> >> time.
> >> > Logging is good for a command line tool like sbt, and for running your
> >> specs
> >> > from the command line, but it doesn't help other tools.
> >> >
> >> > Now, lets also imagine that you have a Suite with 100k tests, and they
> >> take
> >> > an hour to run. Logging would be helpful, but if you're in an IDE and
> >> you
> >> > cant see that output, then well, this just doesn't work for you. You
> >> have to
> >> > wait around for an hour.
> >> >
> >> > Maybe its a pipedream that an IDE would use this, but its a pipedream
> >> I've
> >> > had for years:
> >> >
> >> >
> >> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-and-in-ides-in.html
> >> >
> >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah <dmharrah@gmail.com>
> >> wrote:
> >> >
> >> >>
> >> >> Eric, Josh,
> >> >>
> >> >> On 10/19/09, etorreborre <etorreborre@gmail.com> wrote:
> >> >> > I think that we may go through a too complicated road here, because
> >> >> > frameworks may have very different way to run and report results. For
> >> >> > example in specs you can nest examples and subexamples indefinitely
> >> >> > and I don't know how to use the current interface with that.
> >> >> >
> >> >> > What would be simpler is:
> >> >> >
> >> >> > Framework.java:
> >> >> >
> >> >> > /** return a Runner using the ResultOutput to display
> >> >> >      - descriptions
> >> >> >      - test results
> >> >> >      - run statistics
> >> >> > */
> >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger logger,
> >> >> > ResultOutput output);
> >> >> >
> >> >> > ResultOutput.java
> >> >> >
> >> >> > /** write to the console */
> >> >> > public void write(message: String)
> >> >> >
> >> >> > Runner.java
> >> >> >
> >> >> > /** return true if the tests are not passing for whatever reason */
> >> >> > public boolean run(String testClassName, TestFingerprint fingerprint,
> >> >> > String [] args);
> >> >> >
> >> >> > Do you see any issue with this approach?
> >> >>
> >> >> The Logger is supposed to be used for the purpose of the ResultOutput.
> >> >>  If you would prefer ResultOutput, that is fine with me.  The run
> >> >> method above is basically all sbt needs.  It just needs to know on a
> >> >> test suite level whether the suite passed or failed so that it can
> >> >> implement test-quick and test-failed.  However, it would be useful as
> >> >> a user to have the number of tests passed/failing/whatever and a
> >> >> summary of failures be uniformly reported.  I think that the rest of
> >> >> the logging/output can be framework specific.  So, I like returning a
> >> >> TestResult[] as Josh suggests.
> >> >>
> >> >> As Eric indicates, the problem with the event listener is that each
> >> >> framework will have a different event hierarchy and events complicate
> >> >> the interface.  There could be a basic TestEvent interface and each
> >> >> framework could implement its own hierarchy, but it should be
> >> >> information that a client is not required to handle.  That is, run
> >> >> should still return TestResult[].
> >> >>
> >> >> > On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
> >> >> >> I found yet another issue with Runner. It would be nice if tools had
> >> >> >> immediate info of when test is starting, is finished, and what its
> >> >> result
> >> >> >> is, as well as an entire suites. I suppose this issue could be
> >> resolved
> >> >> >> with
> >> >> >> the Logger, but thats probably not the best way.
> >> >>
> >> >> I don't see a problem with this.  I think it is the user that needs
> >> >> the immediate feedback, not the tool.  The framework can log to the
> >> >> Logger as it runs and inform the tool of results after the suite
> >> >> completes, .
> >> >>
> >> >> -Mark
> >> >>
> >> >> >
> >> >>
> >> >
> >> > >
> >> >
> >>
> >> --~--~---------~--~----~------------~-------~--~----~
> >> You received this message because you are subscribed to the Google Groups
> >> "simple-build-tool" group.
> >> To post to this group, send email to simple-build-tool@googlegroups.com
> >> To unsubscribe from this group, send email to
> >> 2Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool+unsubscribe@googlegroups.com<252Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool%2Bunsubscribe@googlegroups.com>
> >> For more options, visit this group at
> >> http://groups.google.com/group/simple-build-tool?hl=en
> >> -~----------~----~----~----~------~----~------~--~---
> >>
> >>
> >>
> >
>

On Wednesday 21 October 2009 10:42, Josh Suereth wrote:
> Why not use "abstract trait"'s instead of java interfaces? They should
> compile similarly and not have a dependency on any specific Scala
> library....

I tried this out and abstract is redundant and has no effect. A trait is
compiled to only a Java interface if there are no concrete members regardless
of 'abstract' being present. The 'abstract' modifier does not check that all
members are abstract. This compiles:

abstract trait A {
def g = 3
}

-Mark

As far as I know, $tag is gone in 2.8, and everything Scala implements
ScalaObject.

2009/10/21 Josh Cough :
> This might be a question for another list, but I'll ask it here since we are
> on the topic. We have this enum:
>
> public enum Result{
> Success, Error, Failure, Skipped
> }
>
> I tried to convert it over, and got some interesting byte code. I tried to
> use what I thought to be the simplest Scala code possible:
>
> abstract trait Result
> object Error extends Result
> object Success extends Result
> object Failure extends Result
> object Skipped extends Result
>
> Among the bytecode was scala/ScalaObject, and java.rmi.RemoteException. Is
> it possible to convert a Java enum to Scala without referencing ScalaObject?
> Maybe I'm going about it all wrong, and hopefully this isn't a stupid
> question.
>
> Here's one example:
>
> javap -c Error$
> Compiled from "Result.scala"
> public final class Error$ extends java.lang.Object implements
> Result,scala.ScalaObject{
> public static final Error$ MODULE$;
>
> public static {};
>   Code:
>    0:    new    #10; //class Error$
>    3:    invokespecial    #13; //Method "":()V
>    6:    return
>
> public Error$();
>   Code:
>    0:    aload_0
>    1:    invokespecial    #17; //Method java/lang/Object."":()V
>    4:    aload_0
>    5:    putstatic    #19; //Field MODULE$:LError$;
>    8:    return
>
> public int $tag()   throws java.rmi.RemoteException;
>   Code:
>    0:    aload_0
>    1:    invokestatic    #26; //Method
> scala/ScalaObject$class.$tag:(Lscala/ScalaObject;)I
>    4:    ireturn
>
> }
>
> -Josh
>
>
> On Wed, Oct 21, 2009 at 10:09 AM, Mark Harrah wrote:
>>
>> By using Java interfaces, you keep scala-library.jar off of the classpath
>> and get an automatic check that no Scala classes are referenced.
>>
>> -Mark
>>
>> On Wednesday 21 October 2009 10:48, Josh Cough wrote:
>> > Hi Josh,
>> >
>> > That sounds fine. Java interfaces were just what Mark proposed and we
>> > all
>> > said okay. I don't think anyone would object to changing it.
>> >
>> > Anyone opposed? I can go in and change it tonight.
>> >
>> > -Josh
>> >
>> > On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth
>> > wrote:
>> >
>> > > Why not use "abstract trait"'s instead of java interfaces?  They
>> > > should
>> > > compile similarly and not have a dependency on any specific Scala
>> > > library....
>> > >
>> > > - Josh
>> > >
>> > >
>> > > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough
>> > > wrote:
>> > >
>> > >> All,
>> > >>
>> > >> The Mark and the Scala testing community are working hard at a
>> > >> unified
>> > >> interface for the three main test frameworks (and hopefully any
>> > >> others that
>> > >> spring up as well). It would/should/could be usable not only by sbt,
>> > >> but by
>> > >> IDE's and any other tools making use of test frameworks. We could use
>> > >> some
>> > >> help understanding the needs of some of the other tools. Were at a
>> > >> point
>> > >> where we don't want to make it much more complicated, but also want
>> > >> to make
>> > >> sure were serving the needs of the most common tools.
>> > >>
>> > >> The interfaces are in pure Java, and can be found here:
>> > >> http://github.com/harrah/test-interface. However, those are already
>> > >> slightly out of date based on the conversations below, mostly in the
>> > >> more
>> > >> recent messages in the thread.
>> > >>
>> > >> For anyone who cares to provide it, input would be greatly
>> > >> appreciated.
>> > >>
>> > >> -Josh
>> > >>
>> > >> ---------- Forwarded message ----------
>> > >> From: Mark Harrah
>> > >> Date: Mon, Oct 19, 2009 at 10:23 AM
>> > >> Subject: [sbt] Re: ScalaTest 1.0
>> > >> To: simple-build-tool@googlegroups.com
>> > >>
>> > >>
>> > >>
>> > >> Fair enough.  Since we have always intended this to be useful to more
>> > >> than just sbt, we should involve the other tools and take this
>> > >> discussion to a common mailing list, perhaps scala-tools or a new
>> > >> group.  Do you want to take care of this?
>> > >>
>> > >> -Mark
>> > >>
>> > >> On 10/19/09, Josh Cough wrote:
>> > >> > More to come later, but regarding this:
>> > >> >
>> > >> >>> On Oct 19, 3:03 pm, Josh Cough wrote:
>> > >> >>>> I found yet another issue with Runner. It would be nice if tools
>> > >> >>>> had
>> > >> >>>> immediate info of when test is starting, is finished, and what
>> > >> >>>> its
>> > >> > result
>> > >> >>>> is, as well as an entire suites. I suppose this issue could be
>> > >> resolved
>> > >> >>>> with the Logger, but thats probably not the best way.
>> > >> >
>> > >> >>I don't see a problem with this.  I think it is the user that needs
>> > >> >>the immediate feedback, not the tool.  The framework can log to the
>> > >> >>Logger as it runs and inform the tool of results after the suite
>> > >> >>completes, .
>> > >> >
>> > >> > Lets imagine that an IDE is using this API, and it wants to display
>> > >> > a
>> > >> little
>> > >> > green dot for every test that passes. If we use the logger, it
>> > >> > would
>> > >> have to
>> > >> > parse the log output in order to do that. Each framework is going
>> > >> > to log
>> > >> in
>> > >> > whatever way it pleases, so the IDE will have to have three
>> > >> > different
>> > >> log
>> > >> > parses in order to properly display information to the user in real
>> > >> time.
>> > >> > Logging is good for a command line tool like sbt, and for running
>> > >> > your
>> > >> specs
>> > >> > from the command line, but it doesn't help other tools.
>> > >> >
>> > >> > Now, lets also imagine that you have a Suite with 100k tests, and
>> > >> > they
>> > >> take
>> > >> > an hour to run. Logging would be helpful, but if you're in an IDE
>> > >> > and
>> > >> you
>> > >> > cant see that output, then well, this just doesn't work for you.
>> > >> > You
>> > >> have to
>> > >> > wait around for an hour.
>> > >> >
>> > >> > Maybe its a pipedream that an IDE would use this, but its a
>> > >> > pipedream
>> > >> I've
>> > >> > had for years:
>> > >> >
>> > >> >
>> > >>
>> > >> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-an...
>> > >> >
>> > >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah
>> > >> wrote:
>> > >> >
>> > >> >>
>> > >> >> Eric, Josh,
>> > >> >>
>> > >> >> On 10/19/09, etorreborre wrote:
>> > >> >> > I think that we may go through a too complicated road here,
>> > >> >> > because
>> > >> >> > frameworks may have very different way to run and report
>> > >> >> > results. For
>> > >> >> > example in specs you can nest examples and subexamples
>> > >> >> > indefinitely
>> > >> >> > and I don't know how to use the current interface with that.
>> > >> >> >
>> > >> >> > What would be simpler is:
>> > >> >> >
>> > >> >> > Framework.java:
>> > >> >> >
>> > >> >> > /** return a Runner using the ResultOutput to display
>> > >> >> >      - descriptions
>> > >> >> >      - test results
>> > >> >> >      - run statistics
>> > >> >> > */
>> > >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger
>> > >> >> > logger,
>> > >> >> > ResultOutput output);
>> > >> >> >
>> > >> >> > ResultOutput.java
>> > >> >> >
>> > >> >> > /** write to the console */
>> > >> >> > public void write(message: String)
>> > >> >> >
>> > >> >> > Runner.java
>> > >> >> >
>> > >> >> > /** return true if the tests are not passing for whatever reason
>> > >> >> > */
>> > >> >> > public boolean run(String testClassName, TestFingerprint
>> > >> >> > fingerprint,
>> > >> >> > String [] args);
>> > >> >> >
>> > >> >> > Do you see any issue with this approach?
>> > >> >>
>> > >> >> The Logger is supposed to be used for the purpose of the
>> > >> >> ResultOutput.
>> > >> >>  If you would prefer ResultOutput, that is fine with me.  The run
>> > >> >> method above is basically all sbt needs.  It just needs to know on
>> > >> >> a
>> > >> >> test suite level whether the suite passed or failed so that it can
>> > >> >> implement test-quick and test-failed.  However, it would be useful
>> > >> >> as
>> > >> >> a user to have the number of tests passed/failing/whatever and a
>> > >> >> summary of failures be uniformly reported.  I think that the rest
>> > >> >> of
>> > >> >> the logging/output can be framework specific.  So, I like
>> > >> >> returning a
>> > >> >> TestResult[] as Josh suggests.
>> > >> >>
>> > >> >> As Eric indicates, the problem with the event listener is that
>> > >> >> each
>> > >> >> framework will have a different event hierarchy and events
>> > >> >> complicate
>> > >> >> the interface.  There could be a basic TestEvent interface and
>> > >> >> each
>> > >> >> framework could implement its own hierarchy, but it should be
>> > >> >> information that a client is not required to handle.  That is, run
>> > >> >> should still return TestResult[].
>> > >> >>
>> > >> >> > On Oct 19, 3:03 pm, Josh Cough wrote:
>> > >> >> >> I found yet another issue with Runner. It would be nice if
>> > >> >> >> tools had
>> > >> >> >> immediate info of when test is starting, is finished, and what
>> > >> >> >> its
>> > >> >> result
>> > >> >> >> is, as well as an entire suites. I suppose this issue could be
>> > >> resolved
>> > >> >> >> with
>> > >> >> >> the Logger, but thats probably not the best way.
>> > >> >>
>> > >> >> I don't see a problem with this.  I think it is the user that
>> > >> >> needs
>> > >> >> the immediate feedback, not the tool.  The framework can log to
>> > >> >> the
>> > >> >> Logger as it runs and inform the tool of results after the suite
>> > >> >> completes, .
>> > >> >>
>> > >> >> -Mark
>> > >> >>
>> > >> >> >
>> > >> >>
>> > >> >
>> > >> > >
>> > >> >
>> > >>
>> > >> --~--~---------~--~----~------------~-------~--~----~
>> > >> You received this message because you are subscribed to the Google
>> > >> Groups
>> > >> "simple-build-tool" group.
>> > >> To post to this group, send email to
>> > >> simple-build-tool@googlegroups.com
>> > >> To unsubscribe from this group, send email to
>> > >>
>> > >> simple-build-tool+unsubscribe@googlegroups.com
>> > >> For more options, visit this group at
>> > >> http://groups.google.com/group/simple-build-tool?hl=en
>> > >> -~----------~----~----~----~------~----~------~--~---
>> > >>
>> > >>
>> > >>
>> > >
>> >
>
>

As far as I know, $tag is gone in 2.8, and everything Scala implements
ScalaObject.

2009/10/21 Josh Cough <joshcough@gmail.com>:
> This might be a question for another list, but I'll ask it here since we are
> on the topic. We have this enum:
>
> public enum Result{
>   Success, Error, Failure, Skipped
> }
>
> I tried to convert it over, and got some interesting byte code. I tried to
> use what I thought to be the simplest Scala code possible:
>
> abstract trait Result
> object Error extends Result
> object Success extends Result
> object Failure extends Result
> object Skipped extends Result
>
> Among the bytecode was scala/ScalaObject, and java.rmi.RemoteException. Is
> it possible to convert a Java enum to Scala without referencing ScalaObject?
> Maybe I'm going about it all wrong, and hopefully this isn't a stupid
> question.
>
> Here's one example:
>
> javap -c Error$
> Compiled from "Result.scala"
> public final class Error$ extends java.lang.Object implements
> Result,scala.ScalaObject{
> public static final Error$ MODULE$;
>
> public static {};
>   Code:
>    0:    new    #10; //class Error$
>    3:    invokespecial    #13; //Method "<init>":()V
>    6:    return
>
> public Error$();
>   Code:
>    0:    aload_0
>    1:    invokespecial    #17; //Method java/lang/Object."<init>":()V
>    4:    aload_0
>    5:    putstatic    #19; //Field MODULE$:LError$;
>    8:    return
>
> public int $tag()   throws java.rmi.RemoteException;
>   Code:
>    0:    aload_0
>    1:    invokestatic    #26; //Method
> scala/ScalaObject$class.$tag:(Lscala/ScalaObject;)I
>    4:    ireturn
>
> }
>
> -Josh
>
>
> On Wed, Oct 21, 2009 at 10:09 AM, Mark Harrah <harrah@bu.edu> wrote:
>>
>> By using Java interfaces, you keep scala-library.jar off of the classpath
>> and get an automatic check that no Scala classes are referenced.
>>
>> -Mark
>>
>> On Wednesday 21 October 2009 10:48, Josh Cough wrote:
>> > Hi Josh,
>> >
>> > That sounds fine. Java interfaces were just what Mark proposed and we
>> > all
>> > said okay. I don't think anyone would object to changing it.
>> >
>> > Anyone opposed? I can go in and change it tonight.
>> >
>> > -Josh
>> >
>> > On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth
>> > <joshua.suereth@gmail.com>wrote:
>> >
>> > > Why not use "abstract trait"'s instead of java interfaces?  They
>> > > should
>> > > compile similarly and not have a dependency on any specific Scala
>> > > library....
>> > >
>> > > - Josh
>> > >
>> > >
>> > > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough <joshcough@gmail.com>
>> > > wrote:
>> > >
>> > >> All,
>> > >>
>> > >> The Mark and the Scala testing community are working hard at a
>> > >> unified
>> > >> interface for the three main test frameworks (and hopefully any
>> > >> others that
>> > >> spring up as well). It would/should/could be usable not only by sbt,
>> > >> but by
>> > >> IDE's and any other tools making use of test frameworks. We could use
>> > >> some
>> > >> help understanding the needs of some of the other tools. Were at a
>> > >> point
>> > >> where we don't want to make it much more complicated, but also want
>> > >> to make
>> > >> sure were serving the needs of the most common tools.
>> > >>
>> > >> The interfaces are in pure Java, and can be found here:
>> > >> http://github.com/harrah/test-interface. However, those are already
>> > >> slightly out of date based on the conversations below, mostly in the
>> > >> more
>> > >> recent messages in the thread.
>> > >>
>> > >> For anyone who cares to provide it, input would be greatly
>> > >> appreciated.
>> > >>
>> > >> -Josh
>> > >>
>> > >> ---------- Forwarded message ----------
>> > >> From: Mark Harrah <dmharrah@gmail.com>
>> > >> Date: Mon, Oct 19, 2009 at 10:23 AM
>> > >> Subject: [sbt] Re: ScalaTest 1.0
>> > >> To: simple-build-tool@googlegroups.com
>> > >>
>> > >>
>> > >>
>> > >> Fair enough.  Since we have always intended this to be useful to more
>> > >> than just sbt, we should involve the other tools and take this
>> > >> discussion to a common mailing list, perhaps scala-tools or a new
>> > >> group.  Do you want to take care of this?
>> > >>
>> > >> -Mark
>> > >>
>> > >> On 10/19/09, Josh Cough <joshcough@gmail.com> wrote:
>> > >> > More to come later, but regarding this:
>> > >> >
>> > >> >>> On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>> > >> >>>> I found yet another issue with Runner. It would be nice if tools
>> > >> >>>> had
>> > >> >>>> immediate info of when test is starting, is finished, and what
>> > >> >>>> its
>> > >> > result
>> > >> >>>> is, as well as an entire suites. I suppose this issue could be
>> > >> resolved
>> > >> >>>> with the Logger, but thats probably not the best way.
>> > >> >
>> > >> >>I don't see a problem with this.  I think it is the user that needs
>> > >> >>the immediate feedback, not the tool.  The framework can log to the
>> > >> >>Logger as it runs and inform the tool of results after the suite
>> > >> >>completes, .
>> > >> >
>> > >> > Lets imagine that an IDE is using this API, and it wants to display
>> > >> > a
>> > >> little
>> > >> > green dot for every test that passes. If we use the logger, it
>> > >> > would
>> > >> have to
>> > >> > parse the log output in order to do that. Each framework is going
>> > >> > to log
>> > >> in
>> > >> > whatever way it pleases, so the IDE will have to have three
>> > >> > different
>> > >> log
>> > >> > parses in order to properly display information to the user in real
>> > >> time.
>> > >> > Logging is good for a command line tool like sbt, and for running
>> > >> > your
>> > >> specs
>> > >> > from the command line, but it doesn't help other tools.
>> > >> >
>> > >> > Now, lets also imagine that you have a Suite with 100k tests, and
>> > >> > they
>> > >> take
>> > >> > an hour to run. Logging would be helpful, but if you're in an IDE
>> > >> > and
>> > >> you
>> > >> > cant see that output, then well, this just doesn't work for you.
>> > >> > You
>> > >> have to
>> > >> > wait around for an hour.
>> > >> >
>> > >> > Maybe its a pipedream that an IDE would use this, but its a
>> > >> > pipedream
>> > >> I've
>> > >> > had for years:
>> > >> >
>> > >> >
>> > >>
>> > >> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-and-in-ides-in.html
>> > >> >
>> > >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah <dmharrah@gmail.com>
>> > >> wrote:
>> > >> >
>> > >> >>
>> > >> >> Eric, Josh,
>> > >> >>
>> > >> >> On 10/19/09, etorreborre <etorreborre@gmail.com> wrote:
>> > >> >> > I think that we may go through a too complicated road here,
>> > >> >> > because
>> > >> >> > frameworks may have very different way to run and report
>> > >> >> > results. For
>> > >> >> > example in specs you can nest examples and subexamples
>> > >> >> > indefinitely
>> > >> >> > and I don't know how to use the current interface with that.
>> > >> >> >
>> > >> >> > What would be simpler is:
>> > >> >> >
>> > >> >> > Framework.java:
>> > >> >> >
>> > >> >> > /** return a Runner using the ResultOutput to display
>> > >> >> >      - descriptions
>> > >> >> >      - test results
>> > >> >> >      - run statistics
>> > >> >> > */
>> > >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger
>> > >> >> > logger,
>> > >> >> > ResultOutput output);
>> > >> >> >
>> > >> >> > ResultOutput.java
>> > >> >> >
>> > >> >> > /** write to the console */
>> > >> >> > public void write(message: String)
>> > >> >> >
>> > >> >> > Runner.java
>> > >> >> >
>> > >> >> > /** return true if the tests are not passing for whatever reason
>> > >> >> > */
>> > >> >> > public boolean run(String testClassName, TestFingerprint
>> > >> >> > fingerprint,
>> > >> >> > String [] args);
>> > >> >> >
>> > >> >> > Do you see any issue with this approach?
>> > >> >>
>> > >> >> The Logger is supposed to be used for the purpose of the
>> > >> >> ResultOutput.
>> > >> >>  If you would prefer ResultOutput, that is fine with me.  The run
>> > >> >> method above is basically all sbt needs.  It just needs to know on
>> > >> >> a
>> > >> >> test suite level whether the suite passed or failed so that it can
>> > >> >> implement test-quick and test-failed.  However, it would be useful
>> > >> >> as
>> > >> >> a user to have the number of tests passed/failing/whatever and a
>> > >> >> summary of failures be uniformly reported.  I think that the rest
>> > >> >> of
>> > >> >> the logging/output can be framework specific.  So, I like
>> > >> >> returning a
>> > >> >> TestResult[] as Josh suggests.
>> > >> >>
>> > >> >> As Eric indicates, the problem with the event listener is that
>> > >> >> each
>> > >> >> framework will have a different event hierarchy and events
>> > >> >> complicate
>> > >> >> the interface.  There could be a basic TestEvent interface and
>> > >> >> each
>> > >> >> framework could implement its own hierarchy, but it should be
>> > >> >> information that a client is not required to handle.  That is, run
>> > >> >> should still return TestResult[].
>> > >> >>
>> > >> >> > On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
>> > >> >> >> I found yet another issue with Runner. It would be nice if
>> > >> >> >> tools had
>> > >> >> >> immediate info of when test is starting, is finished, and what
>> > >> >> >> its
>> > >> >> result
>> > >> >> >> is, as well as an entire suites. I suppose this issue could be
>> > >> resolved
>> > >> >> >> with
>> > >> >> >> the Logger, but thats probably not the best way.
>> > >> >>
>> > >> >> I don't see a problem with this.  I think it is the user that
>> > >> >> needs
>> > >> >> the immediate feedback, not the tool.  The framework can log to
>> > >> >> the
>> > >> >> Logger as it runs and inform the tool of results after the suite
>> > >> >> completes, .
>> > >> >>
>> > >> >> -Mark
>> > >> >>
>> > >> >> >
>> > >> >>
>> > >> >
>> > >> > >
>> > >> >
>> > >>
>> > >> --~--~---------~--~----~------------~-------~--~----~
>> > >> You received this message because you are subscribed to the Google
>> > >> Groups
>> > >> "simple-build-tool" group.
>> > >> To post to this group, send email to
>> > >> simple-build-tool@googlegroups.com
>> > >> To unsubscribe from this group, send email to
>> > >>
>> > >> 2Bunsubscribe [at] googlegroups [dot] com" target="_blank" rel="nofollow">simple-build-tool+unsubscribe@googlegroups.com<252Bunsubscribe [at] googlegroups [dot] com" target="_blank" rel="nofollow">simple-build-tool%2Bunsubscribe@googlegroups.com>
>> > >> For more options, visit this group at
>> > >> http://groups.google.com/group/simple-build-tool?hl=en
>> > >> -~----------~----~----~----~------~----~------~--~---
>> > >>
>> > >>
>> > >>
>> > >
>> >
>
>



--
Ricky Clarkson
Java and Scala Programmer, AD Holdings
+44 1565 770804
Skype: ricky_clarkson
Google Talk: ricky.clarkson@gmail.com
Google Wave: ricky.clarkson@googlewave.com

I think the interface has to stay in Java, unfortunately.

-Mark

On Wednesday 21 October 2009 11:29, Josh Cough wrote:
> This might be a question for another list, but I'll ask it here since we are
> on the topic. We have this enum:
>
> public enum Result{
> Success, Error, Failure, Skipped
> }
>
> I tried to convert it over, and got some interesting byte code. I tried to
> use what I thought to be the simplest Scala code possible:
>
> abstract trait Result
> object Error extends Result
> object Success extends Result
> object Failure extends Result
> object Skipped extends Result
>
> Among the bytecode was scala/ScalaObject, and java.rmi.RemoteException. Is
> it possible to convert a Java enum to Scala without referencing ScalaObject?
> Maybe I'm going about it all wrong, and hopefully this isn't a stupid
> question.
>
> Here's one example:
>
> javap -c Error$
> Compiled from "Result.scala"
> public final class Error$ extends java.lang.Object implements
> Result,scala.ScalaObject{
> public static final Error$ MODULE$;
>
> public static {};
> Code:
> 0: new #10; //class Error$
> 3: invokespecial #13; //Method "":()V
> 6: return
>
> public Error$();
> Code:
> 0: aload_0
> 1: invokespecial #17; //Method java/lang/Object."":()V
> 4: aload_0
> 5: putstatic #19; //Field MODULE$:LError$;
> 8: return
>
> public int $tag() throws java.rmi.RemoteException;
> Code:
> 0: aload_0
> 1: invokestatic #26; //Method
> scala/ScalaObject$class.$tag:(Lscala/ScalaObject;)I
> 4: ireturn
>
> }
>
> -Josh
>
>
> On Wed, Oct 21, 2009 at 10:09 AM, Mark Harrah wrote:
>
> > By using Java interfaces, you keep scala-library.jar off of the classpath
> > and get an automatic check that no Scala classes are referenced.
> >
> > -Mark
> >
> > On Wednesday 21 October 2009 10:48, Josh Cough wrote:
> > > Hi Josh,
> > >
> > > That sounds fine. Java interfaces were just what Mark proposed and we all
> > > said okay. I don't think anyone would object to changing it.
> > >
> > > Anyone opposed? I can go in and change it tonight.
> > >
> > > -Josh
> > >
> > > On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth > >wrote:
> > >
> > > > Why not use "abstract trait"'s instead of java interfaces? They should
> > > > compile similarly and not have a dependency on any specific Scala
> > > > library....
> > > >
> > > > - Josh
> > > >
> > > >
> > > > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough
> > wrote:
> > > >
> > > >> All,
> > > >>
> > > >> The Mark and the Scala testing community are working hard at a unified
> > > >> interface for the three main test frameworks (and hopefully any others
> > that
> > > >> spring up as well). It would/should/could be usable not only by sbt,
> > but by
> > > >> IDE's and any other tools making use of test frameworks. We could use
> > some
> > > >> help understanding the needs of some of the other tools. Were at a
> > point
> > > >> where we don't want to make it much more complicated, but also want to
> > make
> > > >> sure were serving the needs of the most common tools.
> > > >>
> > > >> The interfaces are in pure Java, and can be found here:
> > > >> http://github.com/harrah/test-interface. However, those are already
> > > >> slightly out of date based on the conversations below, mostly in the
> > more
> > > >> recent messages in the thread.
> > > >>
> > > >> For anyone who cares to provide it, input would be greatly
> > appreciated.
> > > >>
> > > >> -Josh
> > > >>
> > > >> ---------- Forwarded message ----------
> > > >> From: Mark Harrah
> > > >> Date: Mon, Oct 19, 2009 at 10:23 AM
> > > >> Subject: [sbt] Re: ScalaTest 1.0
> > > >> To: simple-build-tool@googlegroups.com
> > > >>
> > > >>
> > > >>
> > > >> Fair enough. Since we have always intended this to be useful to more
> > > >> than just sbt, we should involve the other tools and take this
> > > >> discussion to a common mailing list, perhaps scala-tools or a new
> > > >> group. Do you want to take care of this?
> > > >>
> > > >> -Mark
> > > >>
> > > >> On 10/19/09, Josh Cough wrote:
> > > >> > More to come later, but regarding this:
> > > >> >
> > > >> >>> On Oct 19, 3:03 pm, Josh Cough wrote:
> > > >> >>>> I found yet another issue with Runner. It would be nice if tools
> > had
> > > >> >>>> immediate info of when test is starting, is finished, and what
> > its
> > > >> > result
> > > >> >>>> is, as well as an entire suites. I suppose this issue could be
> > > >> resolved
> > > >> >>>> with the Logger, but thats probably not the best way.
> > > >> >
> > > >> >>I don't see a problem with this. I think it is the user that needs
> > > >> >>the immediate feedback, not the tool. The framework can log to the
> > > >> >>Logger as it runs and inform the tool of results after the suite
> > > >> >>completes, .
> > > >> >
> > > >> > Lets imagine that an IDE is using this API, and it wants to display
> > a
> > > >> little
> > > >> > green dot for every test that passes. If we use the logger, it would
> > > >> have to
> > > >> > parse the log output in order to do that. Each framework is going to
> > log
> > > >> in
> > > >> > whatever way it pleases, so the IDE will have to have three
> > different
> > > >> log
> > > >> > parses in order to properly display information to the user in real
> > > >> time.
> > > >> > Logging is good for a command line tool like sbt, and for running
> > your
> > > >> specs
> > > >> > from the command line, but it doesn't help other tools.
> > > >> >
> > > >> > Now, lets also imagine that you have a Suite with 100k tests, and
> > they
> > > >> take
> > > >> > an hour to run. Logging would be helpful, but if you're in an IDE
> > and
> > > >> you
> > > >> > cant see that output, then well, this just doesn't work for you. You
> > > >> have to
> > > >> > wait around for an hour.
> > > >> >
> > > >> > Maybe its a pipedream that an IDE would use this, but its a
> > pipedream
> > > >> I've
> > > >> > had for years:
> > > >> >
> > > >> >
> > > >>
> > http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-an...
> > > >> >
> > > >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah
> > > >> wrote:
> > > >> >
> > > >> >>
> > > >> >> Eric, Josh,
> > > >> >>
> > > >> >> On 10/19/09, etorreborre wrote:
> > > >> >> > I think that we may go through a too complicated road here,
> > because
> > > >> >> > frameworks may have very different way to run and report results.
> > For
> > > >> >> > example in specs you can nest examples and subexamples
> > indefinitely
> > > >> >> > and I don't know how to use the current interface with that.
> > > >> >> >
> > > >> >> > What would be simpler is:
> > > >> >> >
> > > >> >> > Framework.java:
> > > >> >> >
> > > >> >> > /** return a Runner using the ResultOutput to display
> > > >> >> > - descriptions
> > > >> >> > - test results
> > > >> >> > - run statistics
> > > >> >> > */
> > > >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger
> > logger,
> > > >> >> > ResultOutput output);
> > > >> >> >
> > > >> >> > ResultOutput.java
> > > >> >> >
> > > >> >> > /** write to the console */
> > > >> >> > public void write(message: String)
> > > >> >> >
> > > >> >> > Runner.java
> > > >> >> >
> > > >> >> > /** return true if the tests are not passing for whatever reason
> > */
> > > >> >> > public boolean run(String testClassName, TestFingerprint
> > fingerprint,
> > > >> >> > String [] args);
> > > >> >> >
> > > >> >> > Do you see any issue with this approach?
> > > >> >>
> > > >> >> The Logger is supposed to be used for the purpose of the
> > ResultOutput.
> > > >> >> If you would prefer ResultOutput, that is fine with me. The run
> > > >> >> method above is basically all sbt needs. It just needs to know on
> > a
> > > >> >> test suite level whether the suite passed or failed so that it can
> > > >> >> implement test-quick and test-failed. However, it would be useful
> > as
> > > >> >> a user to have the number of tests passed/failing/whatever and a
> > > >> >> summary of failures be uniformly reported. I think that the rest
> > of
> > > >> >> the logging/output can be framework specific. So, I like returning
> > a
> > > >> >> TestResult[] as Josh suggests.
> > > >> >>
> > > >> >> As Eric indicates, the problem with the event listener is that each
> > > >> >> framework will have a different event hierarchy and events
> > complicate
> > > >> >> the interface. There could be a basic TestEvent interface and each
> > > >> >> framework could implement its own hierarchy, but it should be
> > > >> >> information that a client is not required to handle. That is, run
> > > >> >> should still return TestResult[].
> > > >> >>
> > > >> >> > On Oct 19, 3:03 pm, Josh Cough wrote:
> > > >> >> >> I found yet another issue with Runner. It would be nice if tools
> > had
> > > >> >> >> immediate info of when test is starting, is finished, and what
> > its
> > > >> >> result
> > > >> >> >> is, as well as an entire suites. I suppose this issue could be
> > > >> resolved
> > > >> >> >> with
> > > >> >> >> the Logger, but thats probably not the best way.
> > > >> >>
> > > >> >> I don't see a problem with this. I think it is the user that needs
> > > >> >> the immediate feedback, not the tool. The framework can log to the
> > > >> >> Logger as it runs and inform the tool of results after the suite
> > > >> >> completes, .
> > > >> >>
> > > >> >> -Mark
> > > >> >>
> > > >> >> >
> > > >> >>
> > > >> >
> > > >> > >
> > > >> >
> > > >>
> > > >> --~--~---------~--~----~------------~-------~--~----~
> > > >> You received this message because you are subscribed to the Google
> > Groups
> > > >> "simple-build-tool" group.
> > > >> To post to this group, send email to
> > simple-build-tool@googlegroups.com
> > > >> To unsubscribe from this group, send email to
> > > >> simple-build-tool+unsubscribe@googlegroups.com
> >
> > >
> > > >> For more options, visit this group at
> > > >> http://groups.google.com/group/simple-build-tool?hl=en
> > > >> -~----------~----~----~----~------~----~------~--~---
> > > >>
> > > >>
> > > >>
> > > >
> > >
> >
>

I think the interface has to stay in Java, unfortunately.

-Mark

On Wednesday 21 October 2009 11:29, Josh Cough wrote:
> This might be a question for another list, but I'll ask it here since we are
> on the topic. We have this enum:
>
> public enum Result{
>   Success, Error, Failure, Skipped
> }
>
> I tried to convert it over, and got some interesting byte code. I tried to
> use what I thought to be the simplest Scala code possible:
>
> abstract trait Result
> object Error extends Result
> object Success extends Result
> object Failure extends Result
> object Skipped extends Result
>
> Among the bytecode was scala/ScalaObject, and java.rmi.RemoteException. Is
> it possible to convert a Java enum to Scala without referencing ScalaObject?
> Maybe I'm going about it all wrong, and hopefully this isn't a stupid
> question.
>
> Here's one example:
>
> javap -c Error$
> Compiled from "Result.scala"
> public final class Error$ extends java.lang.Object implements
> Result,scala.ScalaObject{
> public static final Error$ MODULE$;
>
> public static {};
>   Code:
>    0:    new    #10; //class Error$
>    3:    invokespecial    #13; //Method "<init>":()V
>    6:    return
>
> public Error$();
>   Code:
>    0:    aload_0
>    1:    invokespecial    #17; //Method java/lang/Object."<init>":()V
>    4:    aload_0
>    5:    putstatic    #19; //Field MODULE$:LError$;
>    8:    return
>
> public int $tag()   throws java.rmi.RemoteException;
>   Code:
>    0:    aload_0
>    1:    invokestatic    #26; //Method
> scala/ScalaObject$class.$tag:(Lscala/ScalaObject;)I
>    4:    ireturn
>
> }
>
> -Josh
>
>
> On Wed, Oct 21, 2009 at 10:09 AM, Mark Harrah <harrah@bu.edu> wrote:
>
> > By using Java interfaces, you keep scala-library.jar off of the classpath
> > and get an automatic check that no Scala classes are referenced.
> >
> > -Mark
> >
> > On Wednesday 21 October 2009 10:48, Josh Cough wrote:
> > > Hi Josh,
> > >
> > > That sounds fine. Java interfaces were just what Mark proposed and we all
> > > said okay. I don't think anyone would object to changing it.
> > >
> > > Anyone opposed? I can go in and change it tonight.
> > >
> > > -Josh
> > >
> > > On Wed, Oct 21, 2009 at 9:42 AM, Josh Suereth <joshua.suereth@gmail.com
> > >wrote:
> > >
> > > > Why not use "abstract trait"'s instead of java interfaces?  They should
> > > > compile similarly and not have a dependency on any specific Scala
> > > > library....
> > > >
> > > > - Josh
> > > >
> > > >
> > > > On Wed, Oct 21, 2009 at 10:30 AM, Josh Cough <joshcough@gmail.com>
> > wrote:
> > > >
> > > >> All,
> > > >>
> > > >> The Mark and the Scala testing community are working hard at a unified
> > > >> interface for the three main test frameworks (and hopefully any others
> > that
> > > >> spring up as well). It would/should/could be usable not only by sbt,
> > but by
> > > >> IDE's and any other tools making use of test frameworks. We could use
> > some
> > > >> help understanding the needs of some of the other tools. Were at a
> > point
> > > >> where we don't want to make it much more complicated, but also want to
> > make
> > > >> sure were serving the needs of the most common tools.
> > > >>
> > > >> The interfaces are in pure Java, and can be found here:
> > > >> http://github.com/harrah/test-interface. However, those are already
> > > >> slightly out of date based on the conversations below, mostly in the
> > more
> > > >> recent messages in the thread.
> > > >>
> > > >> For anyone who cares to provide it, input would be greatly
> > appreciated.
> > > >>
> > > >> -Josh
> > > >>
> > > >> ---------- Forwarded message ----------
> > > >> From: Mark Harrah <dmharrah@gmail.com>
> > > >> Date: Mon, Oct 19, 2009 at 10:23 AM
> > > >> Subject: [sbt] Re: ScalaTest 1.0
> > > >> To: simple-build-tool@googlegroups.com
> > > >>
> > > >>
> > > >>
> > > >> Fair enough.  Since we have always intended this to be useful to more
> > > >> than just sbt, we should involve the other tools and take this
> > > >> discussion to a common mailing list, perhaps scala-tools or a new
> > > >> group.  Do you want to take care of this?
> > > >>
> > > >> -Mark
> > > >>
> > > >> On 10/19/09, Josh Cough <joshcough@gmail.com> wrote:
> > > >> > More to come later, but regarding this:
> > > >> >
> > > >> >>> On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
> > > >> >>>> I found yet another issue with Runner. It would be nice if tools
> > had
> > > >> >>>> immediate info of when test is starting, is finished, and what
> > its
> > > >> > result
> > > >> >>>> is, as well as an entire suites. I suppose this issue could be
> > > >> resolved
> > > >> >>>> with the Logger, but thats probably not the best way.
> > > >> >
> > > >> >>I don't see a problem with this.  I think it is the user that needs
> > > >> >>the immediate feedback, not the tool.  The framework can log to the
> > > >> >>Logger as it runs and inform the tool of results after the suite
> > > >> >>completes, .
> > > >> >
> > > >> > Lets imagine that an IDE is using this API, and it wants to display
> > a
> > > >> little
> > > >> > green dot for every test that passes. If we use the logger, it would
> > > >> have to
> > > >> > parse the log output in order to do that. Each framework is going to
> > log
> > > >> in
> > > >> > whatever way it pleases, so the IDE will have to have three
> > different
> > > >> log
> > > >> > parses in order to properly display information to the user in real
> > > >> time.
> > > >> > Logging is good for a command line tool like sbt, and for running
> > your
> > > >> specs
> > > >> > from the command line, but it doesn't help other tools.
> > > >> >
> > > >> > Now, lets also imagine that you have a Suite with 100k tests, and
> > they
> > > >> take
> > > >> > an hour to run. Logging would be helpful, but if you're in an IDE
> > and
> > > >> you
> > > >> > cant see that output, then well, this just doesn't work for you. You
> > > >> have to
> > > >> > wait around for an hour.
> > > >> >
> > > >> > Maybe its a pipedream that an IDE would use this, but its a
> > pipedream
> > > >> I've
> > > >> > had for years:
> > > >> >
> > > >> >
> > > >>
> > http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-and-in-ides-in.html
> > > >> >
> > > >> > On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah <dmharrah@gmail.com>
> > > >> wrote:
> > > >> >
> > > >> >>
> > > >> >> Eric, Josh,
> > > >> >>
> > > >> >> On 10/19/09, etorreborre <etorreborre@gmail.com> wrote:
> > > >> >> > I think that we may go through a too complicated road here,
> > because
> > > >> >> > frameworks may have very different way to run and report results.
> > For
> > > >> >> > example in specs you can nest examples and subexamples
> > indefinitely
> > > >> >> > and I don't know how to use the current interface with that.
> > > >> >> >
> > > >> >> > What would be simpler is:
> > > >> >> >
> > > >> >> > Framework.java:
> > > >> >> >
> > > >> >> > /** return a Runner using the ResultOutput to display
> > > >> >> >      - descriptions
> > > >> >> >      - test results
> > > >> >> >      - run statistics
> > > >> >> > */
> > > >> >> > public Runner testRunner(ClassLoader testClassLoader, Logger
> > logger,
> > > >> >> > ResultOutput output);
> > > >> >> >
> > > >> >> > ResultOutput.java
> > > >> >> >
> > > >> >> > /** write to the console */
> > > >> >> > public void write(message: String)
> > > >> >> >
> > > >> >> > Runner.java
> > > >> >> >
> > > >> >> > /** return true if the tests are not passing for whatever reason
> > */
> > > >> >> > public boolean run(String testClassName, TestFingerprint
> > fingerprint,
> > > >> >> > String [] args);
> > > >> >> >
> > > >> >> > Do you see any issue with this approach?
> > > >> >>
> > > >> >> The Logger is supposed to be used for the purpose of the
> > ResultOutput.
> > > >> >>  If you would prefer ResultOutput, that is fine with me.  The run
> > > >> >> method above is basically all sbt needs.  It just needs to know on
> > a
> > > >> >> test suite level whether the suite passed or failed so that it can
> > > >> >> implement test-quick and test-failed.  However, it would be useful
> > as
> > > >> >> a user to have the number of tests passed/failing/whatever and a
> > > >> >> summary of failures be uniformly reported.  I think that the rest
> > of
> > > >> >> the logging/output can be framework specific.  So, I like returning
> > a
> > > >> >> TestResult[] as Josh suggests.
> > > >> >>
> > > >> >> As Eric indicates, the problem with the event listener is that each
> > > >> >> framework will have a different event hierarchy and events
> > complicate
> > > >> >> the interface.  There could be a basic TestEvent interface and each
> > > >> >> framework could implement its own hierarchy, but it should be
> > > >> >> information that a client is not required to handle.  That is, run
> > > >> >> should still return TestResult[].
> > > >> >>
> > > >> >> > On Oct 19, 3:03 pm, Josh Cough <joshco...@gmail.com> wrote:
> > > >> >> >> I found yet another issue with Runner. It would be nice if tools
> > had
> > > >> >> >> immediate info of when test is starting, is finished, and what
> > its
> > > >> >> result
> > > >> >> >> is, as well as an entire suites. I suppose this issue could be
> > > >> resolved
> > > >> >> >> with
> > > >> >> >> the Logger, but thats probably not the best way.
> > > >> >>
> > > >> >> I don't see a problem with this.  I think it is the user that needs
> > > >> >> the immediate feedback, not the tool.  The framework can log to the
> > > >> >> Logger as it runs and inform the tool of results after the suite
> > > >> >> completes, .
> > > >> >>
> > > >> >> -Mark
> > > >> >>
> > > >> >> >
> > > >> >>
> > > >> >
> > > >> > >
> > > >> >
> > > >>
> > > >> --~--~---------~--~----~------------~-------~--~----~
> > > >> You received this message because you are subscribed to the Google
> > Groups
> > > >> "simple-build-tool" group.
> > > >> To post to this group, send email to
> > simple-build-tool@googlegroups.com
> > > >> To unsubscribe from this group, send email to
> > > >> 2Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool+unsubscribe@googlegroups.com<252Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool%2Bunsubscribe@googlegroups.com>
> > <252Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool%2Bunsubscribe@googlegroups.com<25252Bunsubscribe [at] googlegroups [dot] com" rel="nofollow">simple-build-tool%252Bunsubscribe@googlegroups.com>
> > >
> > > >> For more options, visit this group at
> > > >> http://groups.google.com/group/simple-build-tool?hl=en
> > > >> -~----------~----~----~----~------~----~------~--~---
> > > >>
> > > >>
> > > >>
> > > >
> > >
> >
>

On Wed, Oct 21, 2009 at 11:35:11AM -0400, Mark Harrah wrote:
> I tried this out and abstract is redundant and has no effect. A trait
> is compiled to only a Java interface if there are no concrete members
> regardless of 'abstract' being present. The 'abstract' modifier does
> not check that all members are abstract.

That seems like a useful thing to change, if abstract is indeed a no-op
on trait at the moment. (I have long labored under the impression that
"abstract trait" did mean something, but if it does I never found it in
the source.)

Or maybe we need a "totally" modifier modifier. People talk a lot about
nouns vs. verbs in programming languages but frankly we're not going to
the next level until we embrace adverbs.

// no concrete members
totally abstract trait

// cannot even be called by other members of "this", only recursively
totally private method

// cannot be invoked directly, must be found implicitly
totally implicit method

// never evaluated, even after you reference it - program hangs
totally lazy value

// re-using memory from garbage collection disallowed, must be fresh
totally new instance

On Wed, Oct 21, 2009 at 11:35:11AM -0400, Mark Harrah wrote:
> I tried this out and abstract is redundant and has no effect.  A trait
> is compiled to only a Java interface if there are no concrete members
> regardless of 'abstract' being present.  The 'abstract' modifier does
> not check that all members are abstract.

That seems like a useful thing to change, if abstract is indeed a no-op
on trait at the moment.  (I have long labored under the impression that
"abstract trait" did mean something, but if it does I never found it in
the source.)

Or maybe we need a "totally" modifier modifier.  People talk a lot about
nouns vs. verbs in programming languages but frankly we're not going to
the next level until we embrace adverbs.

 // no concrete members
 totally abstract trait

 // cannot even be called by other members of "this", only recursively
 totally private method

 // cannot be invoked directly, must be found implicitly
 totally implicit method

 // never evaluated, even after you reference it - program hangs
 totally lazy value

 // re-using memory from garbage collection disallowed, must be fresh
 totally new instance

--
Paul Phillips      | A national political campaign is better than the
Vivid              | best circus ever heard of, with a mass baptism and
Empiricist         | a couple of hangings thrown in.
i pull his palp!   |     -- H. L. Mencken

On Wed, Oct 21, 2009 at 11:13:11AM -0500, Josh Cough wrote:
> totally lazy is really funny. the rest seem quite useful though.

Glad to know I'm not the only one who wants totally private methods
which can only be invoked recursively! What's nicest about them is that
as soon as the logs show they've been invoked, we can all flee screaming
into our terminator shelters.

As the author of Spock (http://www.spockframework.org), let me tell you the
main gripes I have with the JUnit API/SPI. My intention is not to bash
JUnit, but to give you some ideas on how to improve upon this de-facto
standard.

- Simplistic instead of simple
The JUnit API doesn't provide the features needed by IDEs and build tools.
Therefore, every tool invents its own hacks to work around the limitations,
which leads to many problems. Example: IDEs often use reflection to call
into non-public JUnit methods.

- Too loose object model (e.g. class Description), underspecified semantics,
almost no documentation
As a result, every IDE and build tool makes its own assumptions. It's quite
hard to write a fully featured JUnit runner that works well in all popular
environments.
Example: Although JUnit seems to allow arbitrary nesting of DescriptionS,
IDEs assume that there are only two levels: One for test classes, and a
second one for their methods. If your runner uses more than that, IDE
integration will start to break.
Example: Some environments (e.g. Maven) still don't know that JUnit
sometimes throws subclasses of AssertionError.
Example: It's unclear how exactly JUnit runners should interact with
RunNotifier / clients should interpret RunListener callbacks. As a result,
different environments will report different number of tests etc.
Example: Until recently, clients had to parse (!)
Description.getDisplayName() to find out information like a test's method
name.

- Too static execution model
JUnit supports the concept of requesting a detailed Description of the tests
to be run, without actually running them. In particular, the exact number
and names of tests must be known beforehand. Although this may sound useful
at first, I think it imposes more limitations than it helps.
Example: It's impossible to generate/register new tests at runtime (without
breaking the model)
Example: JUnit's data-driven testing feature (@Parameterized) needs to
construct all test data just to provides the tests' DescriptionS. When the
tests are actually run, the test data will be constructed a second time.
Combined with poor IDE integration, I've seen test data being constructed up
to ten times for a single test run. Now imagine that your test data is large
and is loaded from a database...

- Missing interfaces
An API that aims to support many clients and implementors should be largely
composed of interfaces.

One more thought: To widen the potential audience of your API, it would be
great if it was pure Java.

Cheers,
Peter

joshcough wrote:
>
> All,
>
> The Mark and the Scala testing community are working hard at a unified
> interface for the three main test frameworks (and hopefully any others
> that
> spring up as well). It would/should/could be usable not only by sbt, but
> by
> IDE's and any other tools making use of test frameworks. We could use some
> help understanding the needs of some of the other tools. Were at a point
> where we don't want to make it much more complicated, but also want to
> make
> sure were serving the needs of the most common tools.
>
> The interfaces are in pure Java, and can be found here:
> http://github.com/harrah/test-interface. However, those are already
> slightly
> out of date based on the conversations below, mostly in the more recent
> messages in the thread.
>
> For anyone who cares to provide it, input would be greatly appreciated.
>
> -Josh
>
> ---------- Forwarded message ----------
> From: Mark Harrah
> Date: Mon, Oct 19, 2009 at 10:23 AM
> Subject: [sbt] Re: ScalaTest 1.0
> To: simple-build-tool@googlegroups.com
>
>
>
> Fair enough. Since we have always intended this to be useful to more
> than just sbt, we should involve the other tools and take this
> discussion to a common mailing list, perhaps scala-tools or a new
> group. Do you want to take care of this?
>
> -Mark
>
> On 10/19/09, Josh Cough wrote:
>> More to come later, but regarding this:
>>
>>>> On Oct 19, 3:03 pm, Josh Cough wrote:
>>>>> I found yet another issue with Runner. It would be nice if tools had
>>>>> immediate info of when test is starting, is finished, and what its
>> result
>>>>> is, as well as an entire suites. I suppose this issue could be
>>>>> resolved
>>>>> with the Logger, but thats probably not the best way.
>>
>>>I don't see a problem with this. I think it is the user that needs
>>>the immediate feedback, not the tool. The framework can log to the
>>>Logger as it runs and inform the tool of results after the suite
>>>completes, .
>>
>> Lets imagine that an IDE is using this API, and it wants to display a
> little
>> green dot for every test that passes. If we use the logger, it would have
> to
>> parse the log output in order to do that. Each framework is going to log
> in
>> whatever way it pleases, so the IDE will have to have three different log
>> parses in order to properly display information to the user in real time.
>> Logging is good for a command line tool like sbt, and for running your
> specs
>> from the command line, but it doesn't help other tools.
>>
>> Now, lets also imagine that you have a Suite with 100k tests, and they
> take
>> an hour to run. Logging would be helpful, but if you're in an IDE and you
>> cant see that output, then well, this just doesn't work for you. You have
> to
>> wait around for an hour.
>>
>> Maybe its a pipedream that an IDE would use this, but its a pipedream
>> I've
>> had for years:
>>
>>
> http://jackcoughonsoftware.blogspot.com/2007/12/unit-testing-in-scala-an...
>>
>> On Mon, Oct 19, 2009 at 9:06 AM, Mark Harrah wrote:
>>
>>>
>>> Eric, Josh,
>>>
>>> On 10/19/09, etorreborre wrote:
>>> > I think that we may go through a too complicated road here, because
>>> > frameworks may have very different way to run and report results. For
>>> > example in specs you can nest examples and subexamples indefinitely
>>> > and I don't know how to use the current interface with that.
>>> >
>>> > What would be simpler is:
>>> >
>>> > Framework.java:
>>> >
>>> > /** return a Runner using the ResultOutput to display
>>> > - descriptions
>>> > - test results
>>> > - run statistics
>>> > */
>>> > public Runner testRunner(ClassLoader testClassLoader, Logger logger,
>>> > ResultOutput output);
>>> >
>>> > ResultOutput.java
>>> >
>>> > /** write to the console */
>>> > public void write(message: String)
>>> >
>>> > Runner.java
>>> >
>>> > /** return true if the tests are not passing for whatever reason */
>>> > public boolean run(String testClassName, TestFingerprint fingerprint,
>>> > String [] args);
>>> >
>>> > Do you see any issue with this approach?
>>>
>>> The Logger is supposed to be used for the purpose of the ResultOutput.
>>> If you would prefer ResultOutput, that is fine with me. The run
>>> method above is basically all sbt needs. It just needs to know on a
>>> test suite level whether the suite passed or failed so that it can
>>> implement test-quick and test-failed. However, it would be useful as
>>> a user to have the number of tests passed/failing/whatever and a
>>> summary of failures be uniformly reported. I think that the rest of
>>> the logging/output can be framework specific. So, I like returning a
>>> TestResult[] as Josh suggests.
>>>
>>> As Eric indicates, the problem with the event listener is that each
>>> framework will have a different event hierarchy and events complicate
>>> the interface. There could be a basic TestEvent interface and each
>>> framework could implement its own hierarchy, but it should be
>>> information that a client is not required to handle. That is, run
>>> should still return TestResult[].
>>>
>>> > On Oct 19, 3:03 pm, Josh Cough wrote:
>>> >> I found yet another issue with Runner. It would be nice if tools had
>>> >> immediate info of when test is starting, is finished, and what its
>>> result
>>> >> is, as well as an entire suites. I suppose this issue could be
> resolved
>>> >> with
>>> >> the Logger, but thats probably not the best way.
>>>
>>> I don't see a problem with this. I think it is the user that needs
>>> the immediate feedback, not the tool. The framework can log to the
>>> Logger as it runs and inform the tool of results after the suite
>>> completes, .
>>>
>>> -Mark
>>>
>>> >
>>>
>>
>> >
>>
>
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups
> "simple-build-tool" group.
> To post to this group, send email to simple-build-tool@googlegroups.com
> To unsubscribe from this group, send email to
> simple-build-tool+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/simple-build-tool?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
>

Hi Peter,

The API does consist of pure Java interfaces (and an enum). The reasons why
were discussed in other parts of the thread, but in any case it was agreed to
keep them that way. Still, our scope is currently limited to Scala testing
frameworks and command line build tools because that is what we know.

We posted here in hopes that we'd get IDE or test framework developers
involved. So, if you look at the API [1], I think you'll see that we have
similar ideas, just that we don't know what IDEs and other frameworks need.

-Mark

[1] http://github.com/harrah/test-interface

On Thursday 22 October 2009, Peter Niederwieser wrote:
> As the author of Spock (http://www.spockframework.org), let me tell you the
> main gripes I have with the JUnit API/SPI. My intention is not to bash
> JUnit, but to give you some ideas on how to improve upon this de-facto
> standard.
>
> - Simplistic instead of simple
> The JUnit API doesn't provide the features needed by IDEs and build tools.
> Therefore, every tool invents its own hacks to work around the limitations,
> which leads to many problems. Example: IDEs often use reflection to call
> into non-public JUnit methods.
>
> - Too loose object model (e.g. class Description), underspecified
> semantics, almost no documentation
> As a result, every IDE and build tool makes its own assumptions. It's quite
> hard to write a fully featured JUnit runner that works well in all popular
> environments.
> Example: Although JUnit seems to allow arbitrary nesting of DescriptionS,
> IDEs assume that there are only two levels: One for test classes, and a
> second one for their methods. If your runner uses more than that, IDE
> integration will start to break.
> Example: Some environments (e.g. Maven) still don't know that JUnit
> sometimes throws subclasses of AssertionError.
> Example: It's unclear how exactly JUnit runners should interact with
> RunNotifier / clients should interpret RunListener callbacks. As a result,
> different environments will report different number of tests etc.
> Example: Until recently, clients had to parse (!)
> Description.getDisplayName() to find out information like a test's method
> name.
>
> - Too static execution model
> JUnit supports the concept of requesting a detailed Description of the
> tests to be run, without actually running them. In particular, the exact
> number and names of tests must be known beforehand. Although this may sound
> useful at first, I think it imposes more limitations than it helps.
> Example: It's impossible to generate/register new tests at runtime (without
> breaking the model)
> Example: JUnit's data-driven testing feature (@Parameterized) needs to
> construct all test data just to provides the tests' DescriptionS. When the
> tests are actually run, the test data will be constructed a second time.
> Combined with poor IDE integration, I've seen test data being constructed
> up to ten times for a single test run. Now imagine that your test data is
> large and is loaded from a database...
>
> - Missing interfaces
> An API that aims to support many clients and implementors should be largely
> composed of interfaces.
>
> One more thought: To widen the potential audience of your API, it would be
> great if it was pure Java.
>
> Cheers,
> Peter
>
> joshcough wrote:
> > All,
> >
> > The Mark and the Scala testing community are working hard at a unified
> > interface for the three main test frameworks (and hopefully any others
> > that
> > spring up as well). It would/should/could be usable not only by sbt, but
> > by
> > IDE's and any other tools making use of test frameworks. We could use
> > some help understanding the needs of some of the other tools. Were at a
> > point where we don't want to make it much more complicated, but also want
> > to make
> > sure were serving the needs of the most common tools.
> >
> > The interfaces are in pure Java, and can be found here:
> > http://github.com/harrah/test-interface. However, those are already
> > slightly
> > out of date based on the conversations below, mostly in the more recent
> > messages in the thread.
> >
> > For anyone who cares to provide it, input would be greatly appreciated.
> >
> > -Josh

For NetBeans, the test framework should also be integrated in via ant
task, or maven task. So, if there are some kinds of unified ant/maven
tasks, it will be very helpful.

I'll take a look at current APIs to see if it's enough/suitable for
NetBeans to support JUnit, ScalaTest, Spec etc.

-Caoyuan

On Sat, Oct 24, 2009 at 8:45 AM, Mark Harrah wrote:
> Hi Peter,
>
> The API does consist of pure Java interfaces (and an enum).  The reasons why
> were discussed in other parts of the thread, but in any case it was agreed to
> keep them that way.  Still, our scope is currently limited to Scala testing
> frameworks and command line build tools because that is what we know.
>
> We posted here in hopes that we'd get IDE or test framework developers
> involved.  So, if you look at the API [1], I think you'll see that we have
> similar ideas, just that we don't know what IDEs and other frameworks need.
>
> -Mark
>
> [1] http://github.com/harrah/test-interface
>
>
> On Thursday 22 October 2009, Peter Niederwieser wrote:
>> As the author of Spock (http://www.spockframework.org), let me tell you the
>> main gripes I have with the JUnit API/SPI. My intention is not to bash
>> JUnit, but to give you some ideas on how to improve upon this de-facto
>> standard.
>>
>> - Simplistic instead of simple
>> The JUnit API doesn't provide the features needed by IDEs and build tools.
>> Therefore, every tool invents its own hacks to work around the limitations,
>> which leads to many problems. Example: IDEs often use reflection to call
>> into non-public JUnit methods.
>>
>> - Too loose object model (e.g. class Description), underspecified
>> semantics, almost no documentation
>> As a result, every IDE and build tool makes its own assumptions. It's quite
>> hard to write a fully featured JUnit runner that works well in all popular
>> environments.
>> Example: Although JUnit seems to allow arbitrary nesting of DescriptionS,
>> IDEs assume that there are only two levels: One for test classes, and a
>> second one for their methods. If your runner uses more than that, IDE
>> integration will start to break.
>> Example: Some environments (e.g. Maven) still don't know that JUnit
>> sometimes throws subclasses of AssertionError.
>> Example: It's unclear how exactly JUnit runners should interact with
>> RunNotifier / clients should interpret RunListener callbacks. As a result,
>> different environments will report different number of tests etc.
>> Example: Until recently, clients had to parse (!)
>> Description.getDisplayName() to find out information like a test's method
>> name.
>>
>> - Too static execution model
>> JUnit supports the concept of requesting a detailed Description of the
>> tests to be run, without actually running them. In particular, the exact
>> number and names of tests must be known beforehand. Although this may sound
>> useful at first, I think it imposes more limitations than it helps.
>> Example: It's impossible to generate/register new tests at runtime (without
>> breaking the model)
>> Example: JUnit's data-driven testing feature (@Parameterized) needs to
>> construct all test data just to provides the tests' DescriptionS. When the
>> tests are actually run, the test data will be constructed a second time.
>> Combined with poor IDE integration, I've seen test data being constructed
>> up to ten times for a single test run. Now imagine that your test data is
>> large and is loaded from a database...
>>
>> - Missing interfaces
>> An API that aims to support many clients and implementors should be largely
>> composed of interfaces.
>>
>> One more thought: To widen the potential audience of your API, it would be
>> great if it was pure Java.
>>
>> Cheers,
>> Peter
>>
>> joshcough wrote:
>> > All,
>> >
>> > The Mark and the Scala testing community are working hard at a unified
>> > interface for the three main test frameworks (and hopefully any others
>> > that
>> > spring up as well). It would/should/could be usable not only by sbt, but
>> > by
>> > IDE's and any other tools making use of test frameworks. We could use
>> > some help understanding the needs of some of the other tools. Were at a
>> > point where we don't want to make it much more complicated, but also want
>> > to make
>> > sure were serving the needs of the most common tools.
>> >
>> > The interfaces are in pure Java, and can be found here:
>> > http://github.com/harrah/test-interface. However, those are already
>> > slightly
>> > out of date based on the conversations below, mostly in the more recent
>> > messages in the thread.
>> >
>> > For anyone who cares to provide it, input would be greatly appreciated.
>> >
>> > -Josh
>
>