questioning FP

On Sun, Oct 9, 2011 at 10:34 PM, Ittay Dror wrote:
> Hi All,
>
> (I welcome any sort of replies, even containing foul language, as long as
> there's also information inside.)
>
> I'm about to give a lecture on FP. Now, some concepts in FP I love, but for
> some I have a hard time finding a good angle. Specifically, the IO monad.
>
> Consider the following imperative code:
>
>   service.fetchData
>
>   service.close
>
> Obviously, both method calls create side effects.
>
> Now, (discarding iteratees), lets consider making this with the IO monad:
>
>   for (data <- service.fetchData;
>
>          _ <- service.close)
>
>      yield data
>
> where all methods are changed to return IO[Data], IO[Unit] and also the
> calling code must now return an IO[X] for some X.
>
> The question is, what have we gained?
>
> Instead of sequencing by line order, we're explicitly sequencing in the for
> comprehension. Not much of a gain here, since any mistake that causes code
> to be written in the first order in the imperative case can also be done in
> the functional case.
>
> I also don't see anything that makes the code less prone to other mistakes.
> Such as forgetting to close or calling close twise. Obviously, there are
> ways to fix this such as service.withData(f: Data => X), but again, the more
> imperative style of silently calling close inside withData doesn't loose to
> the functional style that must return IO[X].
>
> What am I missing? I'd appreciate any other examples which show how the use
> of IO is superior by making code that is more robust.
>

The IO monad was a neat trick for combining side effects with lazy
evaluation. But your title seems to be wrong. Questioning FP? FP is
much broader than lazy evaluation. In fact, there is only one lazily
evaluated language in wide usage today and even its creators have said
that laziness was probably a mistake.

Strict languages don't need the IO monad, and generally don't have it,
even though they could. Bob Harper's posts in his "existential type"
series are a good explanation on why not.

An interesting quote from Simon Peyton Jones is that "laziness has
kept us pure". That is, it has motivated the use of monads to
represent effects.
I agree that a static representation of effects is very useful, but my
gut feeling is that there must be a more lightweight and polymorphic
way to get effect checking than what monads provide.

You are not missing anything. Monads are really overrated and not essential to FP. 
- Steven
On 09.10.2011, at 21:34, Ittay Dror wrote:

Hi All,
(I welcome any sort of replies, even containing foul language, as long as there's also information inside.)I'm about to give a lecture on FP. Now, some concepts in FP I love, but for some I have a hard time finding a good angle. Specifically, the IO monad.Consider the following imperative code:  service.fetchData  service.closeObviously, both method calls create side effects.Now, (discarding iteratees), lets consider making this with the IO monad:  for (data <- service.fetchData;         _ <- service.close)     yield datawhere all methods are changed to return IO[Data], IO[Unit] and also the calling code must now return an IO[X] for some X.The question is, what have we gained?Instead of sequencing by line order, we're explicitly sequencing in the for comprehension. Not much of a gain here, since any mistake that causes code to be written in the first order in the imperative case can also be done in the functional case.I also don't see anything that makes the code less prone to other mistakes. Such as forgetting to close or calling close twise. Obviously, there are ways to fix this such as service.withData(f: Data => X), but again, the more imperative style of silently calling close inside withData doesn't loose to the functional style that must return IO[X].What am I missing? I'd appreciate any other examples which show how the use of IO is superior by making code that is more robust.Regards,Ittay

Hi Ittay,

I also have yet to see any point to IO monads in the vast majority of cases.  They seem to allow one to pay a tiny little bit less attention after everything is set up in exchange for requiring heaps more abstraction* and boilerplate.

For what it's worth, one of the best discussions I've seen of IO monads--with something of a positive spin on them--is from James Iry:
  http://james-iry.blogspot.com/2007/11/monads-are-elephants-part-4.html

Anyway, I find that adhering to four principles yields robust IO.
  (1) Input that may fail should be wrapped in an option or the like, and manipulated functionally (i.e. maps and such on the option).
  (2) Output that may fail should take a lambda whose job it is to deal with the consequences of failure if it cannot be ignored.
  (3) If multiple steps must happen in sequence, access them only through a method that ensures that all steps happen in sequence.  If this is not possible, create a single entity (class, actor, whatever) whose job it is to supervise all the input or output from one channel.

I _do_ have to keep these things in mind.  The framework does not force me to do it this way.  But this doesn't at all diminish the importance of functional programming approaches; it just uses them in a different way than the IO monad.

For example, my personal library contains things like

object Print {
  def toFile(f: File)(out: PrintWriter => Unit) {
      val p = new java.io.PrintWriter(f)
      try { out(p) } finally { p.close() }
    }
  }
  def toFileSafely(f: File)(err: (File,IOException) => Unit)(out: PrintWriter => Unit) {
    // Same deal, but catch exceptions and have err handle them
  }
}

which make it almost impossible to mess up sending text to a file:

val xs = List(1,3,5,7,9)
Print.toFile(new File("test.txt")) { pr => xs.map(pr.println) }

For cases where resources need to be cleaned up, I also have a variant of the forward pipe that looks something like:

class TidyPipe[A](a: A) {
  def tidy[Z](f: A => Z)(g: A => Any) = { val z = f(a); g(a); z }
}

which with you can then (given implicit conversion in scope):

input tidy { parser } { _.close() }

and a well-written parser(in: Input) method should emit an Option[ParsedResult] of some sort, so then this is manipulated with stuff like:

input tidy { parser } { _.close() } map { result => some; complicated; thing(result) }

These sorts of things make it so easy to do the right thing that I really don't miss IO monads; I already spend almost no time thinking about IO (aside from complex things like parsing, and that is because parsing is inherently complex), and it comes out correct essentially all the time (and the mistakes are _not_ because I lost track of what the state of the world was supposed to be).  Adding more of a framework around things would not help.

  --Rex

* Abstraction is great when the essence of your problem remains behind, and irrelevant but distracting details disappear.  However, with IO I find that abstraction makes the essence of the problem disappear while leaving behind necessary but trivial details.  For example, maybe I need to know it's a byte buffer of size 8192 in order to do my parsing!


On Sun, Oct 9, 2011 at 4:34 PM, Ittay Dror <ittay.dror@gmail.com> wrote:

Hi All,

(I welcome any sort of replies, even containing foul language, as long as there's also information inside.)

I'm about to give a lecture on FP. Now, some concepts in FP I love, but for some I have a hard time finding a good angle. Specifically, the IO monad.

Consider the following imperative code:

  service.fetchData

  service.close

Obviously, both method calls create side effects.

Now, (discarding iteratees), lets consider making this with the IO monad:

  for (data <- service.fetchData;

         _ <- service.close)

     yield data

where all methods are changed to return IO[Data], IO[Unit] and also the calling code must now return an IO[X] for some X.

The question is, what have we gained?

Instead of sequencing by line order, we're explicitly sequencing in the for comprehension. Not much of a gain here, since any mistake that causes code to be written in the first order in the imperative case can also be done in the functional case.

I also don't see anything that makes the code less prone to other mistakes. Such as forgetting to close or calling close twise. Obviously, there are ways to fix this such as service.withData(f: Data => X), but again, the more imperative style of silently calling close inside withData doesn't loose to the functional style that must return IO[X].

What am I missing? I'd appreciate any other examples which show how the use of IO is superior by making code that is more robust.

Regards,

Ittay

martin odersky skrev 2011-10-09 22:52:
> Strict languages don't need the IO monad, and generally don't have it,
> even though they could. Bob Harper's posts in his "existential type"
> series are a good explanation on why not.
>
> An interesting quote from Simon Peyton Jones is that "laziness has
> kept us pure". That is, it has motivated the use of monads to
> represent effects.
> I agree that a static representation of effects is very useful, but my
> gut feeling is that there must be a more lightweight and polymorphic
> way to get effect checking than what monads provide.

Well, the IO monad (or rather the IO type) is a simple and
straightforward way to keep the language pure without introducing an
additional effects system. So there are certainly benefits to using it
regardless of if the language is lazy or strict.

/Jesper Nordenberg

Dear Martin,
It's intriguing to contemplate the possibility of something lighter weight than a three method api!
Best wishes,
--greg

On Sun, Oct 9, 2011 at 3:31 PM, Jesper Nordenberg <megagurka@yahoo.com> wrote:

martin odersky skrev 2011-10-09 22:52:

Strict languages don't need the IO monad, and generally don't have it,
even though they could. Bob Harper's posts in his "existential type"
series are a good explanation on why not.

An interesting quote from Simon Peyton Jones is that "laziness has
kept us pure". That is, it has motivated the use of monads to
represent effects.
I agree that a static representation of effects is very useful, but my
gut feeling is that there must be a more lightweight and polymorphic
way to get effect checking than what monads provide.

Well, the IO monad (or rather the IO type) is a simple and straightforward way to keep the language pure without introducing an additional effects system. So there are certainly benefits to using it regardless of if the language is lazy or strict.

/Jesper Nordenberg

--
L.G. Meredith
Managing Partner
Biosimilarity LLC
7329 39th Ave SWSeattle, WA 98136

+1 206.650.3740

http://biosimilarity.blogspot.com

body p { margin-bottom: 0cm; margin-top: 0pt; }

martin odersky wrote:

CAENVNkZFgs_dkDffJcV2g6HekkzwqTi7HWneJfnJhbeczLf-pg [at] mail [dot] gmail [dot] com" type="cite">The IO monad was a neat trick for combining side effects with lazy evaluation. But your title seems to be wrong. Questioning FP? FP is much broader than lazy evaluation. In fact, there is only one lazily evaluated language in wide usage today and even its creators have said that laziness was probably a mistake. Strict languages don't need the IO monad, and generally don't have it, even though they could. Bob Harper's posts in his "existential type" series are a good explanation on why not. An interesting quote from Simon Peyton Jones is that "laziness has kept us pure". That is, it has motivated the use of monads to represent effects. I agree that a static representation of effects is very useful, but my gut feeling is that there must be a more lightweight and polymorphic way to get effect checking than what monads provide. -- Martin

The title is probably too much of a troll. Sorry about that.

My questioning is specifically about making functions pure. And the hoops one need to jump through to get that (at least the way I know which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they behave better, but i want to find a use case that appeals to everyday java developers. That is, a case where making a function pure removes pitfalls (while the function remains basically the same)
2. what are the alternatives to IO monad? an implicit World argument?

Thank you,
Ittay

> 2. what are the alternatives to IO monad? an implicit World argument?

You can also make the "World" argument not implicit; but in any case,
the type system has to ensure that you don't reuse an "old" world. This
is called "linear types", and there is a language which does something
similar ("uniqueness types"):

On Oct 10, 2011, at 08:39 , Lars Hupel wrote:

>> 2. what are the alternatives to IO monad? an implicit World argument?
>
> You can also make the "World" argument not implicit; but in any case,
> the type system has to ensure that you don't reuse an "old" world. This
> is called "linear types", and there is a language which does something
> similar ("uniqueness types"):
>
How can this ever work in a distributed concurrent setting? I saw “Concurrent Clean” mentioned on that page, but couldn’t (quickly) find further mention. Where can I dig deeper?

Roland Kuhn
Typesafe – Enterprise-Grade Scala from the Experts
twitter: @rolandkuhn

Ittay Dror skrev 2011-10-10 07:16:
> The title is probably too much of a troll. Sorry about that.
>
> My questioning is specifically about making functions pure. And the
> hoops one need to jump through to get that (at least the way I know
> which is the IO monad). More specifically:
> 1. what does it buy me? in theory, pure functions are nice since they
> behave better, but i want to find a use case that appeals to everyday
> java developers. That is, a case where making a function pure removes
> pitfalls (while the function remains basically the same)

That's basically the same question as "what does a type system buy me?".
It enables automatic detection of programming errors at compile time
rather than at runtime.

It also opens up opportunities for performance optimizations and makes
your code a whole lot easier to test.

> 2. what are the alternatives to IO monad? an implicit World argument?

The ones I know are uniqueness typing (for example Clean) and effects
tracking (for example D, DDC, upcoming Scala effects system). So, either
you track effects using the type system, or you build a separate system
for it.

/Jesper Nordenberg

On Mon, Oct 10, 2011 at 12:31 AM, Jesper Nordenberg wrote:
> martin odersky skrev 2011-10-09 22:52:
>>
>> Strict languages don't need the IO monad, and generally don't have it,
>> even though they could. Bob Harper's posts in his "existential type"
>> series are a good explanation on why not.
>>
>> An interesting quote from Simon Peyton Jones is that "laziness has
>> kept us pure". That is, it has motivated the use of monads to
>> represent effects.
>> I agree that a static representation of effects is very useful, but my
>> gut feeling is that there must be a more lightweight and polymorphic
>> way to get effect checking than what monads provide.
>
> Well, the IO monad (or rather the IO type) is a simple and straightforward
> way to keep the language pure without introducing an additional effects
> system. So there are certainly benefits to using it regardless of if the
> language is lazy or strict.
>
It's certainly straightforward conceptually, but it means that every
program fragment has to be rewritten completely once the first effect
is introduced, even if that effect is just a print statement. I don't
find that very easy to use, and would like to shoot for something more
polymorphic.

Cheers

On Mon, Oct 10, 2011 at 7:16 AM, Ittay Dror wrote:
>
>
> martin odersky wrote:
>
> The IO monad was a neat trick for combining side effects with lazy
> evaluation. But your title seems to be wrong. Questioning FP? FP is much
> broader than lazy evaluation. In fact, there is only one lazily evaluated
> language in wide usage today and even its creators have said that laziness
> was probably a mistake. Strict languages don't need the IO monad, and
> generally don't have it, even though they could. Bob Harper's posts in his
> "existential type" series are a good explanation on why not. An interesting
> quote from Simon Peyton Jones is that "laziness has kept us pure". That is,
> it has motivated the use of monads to represent effects. I agree that a
> static representation of effects is very useful, but my gut feeling is that
> there must be a more lightweight and polymorphic way to get effect checking
> than what monads provide. -- Martin
>
> The title is probably too much of a troll. Sorry about that.
>
> My questioning is specifically about making functions pure. And the hoops
> one need to jump through to get that (at least the way I know which is the
> IO monad). More specifically:
> 1. what does it buy me? in theory, pure functions are nice since they behave
> better, but i want to find a use case that appeals to everyday java
> developers. That is, a case where making a function pure removes pitfalls
> (while the function remains basically the same)
> 2. what are the alternatives to IO monad? an implicit World argument?
>
The IO monad does not make a function pure. It just makes it obvious
that it's impure.
An explicit world argument would make the function technically pure,
but again, doing so
is dubious in practice.

If I'd search for good examples where purity is useful, I'd look
elsewhere. For instance, parallel collections allow automatic speedups
in exchange for all computations being pure.

Cheers

body p { margin-bottom: 0cm; margin-top: 0pt; }

Jesper Nordenberg wrote:

4E929566 [dot] 7090409 [at] yahoo [dot] com" type="cite">Ittay Dror skrev 2011-10-10 07:16:

The title is probably too much of a troll. Sorry about that.

My questioning is specifically about making functions pure. And the
hoops one need to jump through to get that (at least the way I know
which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they
behave better, but i want to find a use case that appeals to everyday
java developers. That is, a case where making a function pure removes
pitfalls (while the function remains basically the same)

That's basically the same question as "what does a type system buy me?". It enables automatic detection of programming errors at compile time rather than at runtime.

I'd love to see a code example where the introduction of IO[_] makes more errors detectable. In the code example I used in the first message, I couldn't find where it helps.

4E929566 [dot] 7090409 [at] yahoo [dot] com" type="cite">
It also opens up opportunities for performance optimizations and makes your code a whole lot easier to test.

Performance how? Because of laziness? This can also work against you. In the lowest level it looks to me like working through a bind function is less efficient than working with the value directly.

How is code easier to test? I initially thought that when I have a pure function, it will behave the same every time. So once I test it I'm sure it works the same in production. But then I thought: Ok, lets assume that I have a function that closes a socket. Further, assume that closing it a second time creates an error. If I have code that does 'for (_ <- close(socket); _ <- close(socket))', it will fail just the same as the (simpler) imperative code "socket.close; socket.close", wouldn't it? Side effects are not gone. They are just abstracted over, the question is whether the abstraction helps.

Again, would love some code examples.

4E929566 [dot] 7090409 [at] yahoo [dot] com" type="cite">

2. what are the alternatives to IO monad? an implicit World argument?

The ones I know are uniqueness typing (for example Clean) and effects tracking (for example D, DDC, upcoming Scala effects system). So, either you track effects using the type system, or you build a separate system for it.

/Jesper Nordenberg

On Mon, Oct 10, 2011 at 2:47 AM, Meredith Gregory <lgreg.meredith@gmail.com> wrote:

It's intriguing to contemplate the possibility of something lighter weight than a three method api!

(I know you're aware of all this, Greg, just pointing it out for the sake of the "debate" part of this mailing list)
so, let's see...
something with few methods may sound like it's lightweight, but that's like saying programming straight in binary is even lighterweight: look! there's only ones and zeroes!
then again, don't forget about the monad laws, which, implicit in your program though they may be, are an important constraint on the interface (2 downsides: they are implicit, they add more constraints)
the real deal breaker, I'd say, is that monads don't compose well -- even simply mixing Option (I mean, Maybe) and List in a for (pardon me, do) comprehension is cumbersome! (not so much in Scala, thanks to CanBuildFrom and breakOut). For a recent proposal to solve monad composition, see "the monad zipper" (http://lambda-the-ultimate.org/node/3929), 
I say, monads are starting to sound like a regular straightjacket by now (well, this is a feature, not a bug -- to some degree)
Personally, I think we (as in, a strict language with subtyping, objects,...) are much better off with a (lightweight...) polymorphic effect system (being developed for Scala by Lukas Rytz -- see also Disciple (http://disciple.ouroborus.net/) for something more Haskelly)

cheersadriaan

> The IO monad does not make a function pure. It just makes it obvious
> that it's impure.

I have to disagree here, or at least ask why you think so. Let's
consider the function `putStrLn` from Haskell with the type `String ->
IO ()`. The definition of "purity" I am used to basically requires that
`putStrLn "foo"` does not execute a side-effect (true) and does not
depend on (hidden) state such that subsequent executions yield exactly
the same value (also true). The side-effect is executed by the
underlying system only if it is somehow included in `main`.

On 10 October 2011 08:43, Adriaan Moors <adriaan.moors@epfl.ch> wrote:

On Mon, Oct 10, 2011 at 2:47 AM, Meredith Gregory <lgreg.meredith@gmail.com> wrote:

It's intriguing to contemplate the possibility of something lighter weight than a three method api!

(I know you're aware of all this, Greg, just pointing it out for the sake of the "debate" part of this mailing list)
so, let's see...
something with few methods may sound like it's lightweight, but that's like saying programming straight in binary is even lighterweight: look! there's only ones and zeroes!
then again, don't forget about the monad laws, which, implicit in your program though they may be, are an important constraint on the interface (2 downsides: they are implicit, they add more constraints)
the real deal breaker, I'd say, is that monads don't compose well -- even simply mixing Option (I mean, Maybe) and List in a for (pardon me, do) comprehension is cumbersome! (not so much in Scala, thanks to CanBuildFrom and breakOut). For a recent proposal to solve monad composition, see "the monad zipper" (http://lambda-the-ultimate.org/node/3929), 

In the interests of fairness, it's worth pointing out that Option doesn't play especially nicely alongside collection types when used in a comprehension.  My code is littered with .toSeq calls whenever I'm writing code like this:
    for {      item <- possibleItem.toSeq      entry <- item.subEntries    } yield ...
In the same vein, Maybe is also restricted by the need to explicitly project it when using in a comprehension.  As a certain Mr Morris would doubtless be the first to point out, Maybes would be a lot friendlier if map and flatMap were to take the right projection by default on an otherwise unprojected instance.
Not that any of this is a serious problem.  In the case of Options, both myself and Paul took a look at possible solutions, and basically came to the conclusion that it can only be dealt with by using typeclasses after the fashion of scalaz, or by integrating Option more strongly into the collections hierarchy (thus breaking binary compatibility) .  I have high hopes for the next major release of scala :)  

I say, monads are starting to sound like a regular straightjacket by now (well, this is a feature, not a bug -- to some degree)
Personally, I think we (as in, a strict language with subtyping, objects,...) are much better off with a (lightweight...) polymorphic effect system (being developed for Scala by Lukas Rytz -- see also Disciple (http://disciple.ouroborus.net/) for something more Haskelly)

cheersadriaan

--
Kevin Wright
mail: kevin.wright@scalatechnology.com
gtalk / msn : kev.lee.wright@gmail.com quora: http://www.quora.com/Kevin-Wrightgoogle+: http://gplus.to/thecoda
kev.lee.wright@gmail.com twitter: @thecoda
vibe / skype: kev.lee.wrightsteam: kev_lee_wright
"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

On Mon, Oct 10, 2011 at 10:04 AM, Lars Hupel wrote:
>> The IO monad does not make a function pure. It just makes it obvious
>> that it's impure.
>
> I have to disagree here, or at least ask why you think so. Let's
> consider the function `putStrLn` from Haskell with the type `String ->
> IO ()`. The definition of "purity" I am used to basically requires that
> `putStrLn "foo"` does not execute a side-effect (true) and does not
> depend on (hidden) state such that subsequent executions yield exactly
> the same value (also true). The side-effect is executed by the
> underlying system only if it is somehow included in `main`.
>
You can do the same trick for every language: You can say *all* Scala
functions just "assemble" a computation which then gets executed only
by calling main. So instead of Monadic bind, Scala has

val x = a; b

On the type level, every type T would be interpreted as IO[T]. Every
function application is a bind. And voila! No side effects!

So the only thing IO provides is that code *not* using it is certified
to be pure, whereas in Scala everything is in the IO monad.

Cheers

On Mon, Oct 10, 2011 at 2:49 AM, Jesper Nordenberg <megagurka@yahoo.com> wrote:

Ittay Dror skrev 2011-10-10 07:16:

The title is probably too much of a troll. Sorry about that.

My questioning is specifically about making functions pure. And the
hoops one need to jump through to get that (at least the way I know
which is the IO monad). More specifically:
1. what does it buy me? in theory, pure functions are nice since they
behave better, but i want to find a use case that appeals to everyday
java developers. That is, a case where making a function pure removes
pitfalls (while the function remains basically the same)

That's basically the same question as "what does a type system buy me?". It enables automatic detection of programming errors at compile time rather than at runtime.

It enables automatic detection of _some_ errors; one still would like to know how big of a hassle it is (much more in Java than Scala, to the point of motivating dynamically typed languages!) and how frequent those errors are (not very, when it comes to IO, from what I've seen).
 

It also opens up opportunities for performance optimizations and makes your code a whole lot easier to test.

I could buy the part about testing, but performance is usually better when you pay _less_ attention, not more, to what is allowed and not.  In particular, writing wrappers that are instantiated with real objects in addition to the actual IO you need to do is not a good way to have higher performance.
 

2. what are the alternatives to IO monad? an implicit World argument?

The ones I know are uniqueness typing (for example Clean) and effects tracking (for example D, DDC, upcoming Scala effects system). So, either you track effects using the type system, or you build a separate system for it.

I'm quite skeptical of the value of _any_ of these approaches since the world outside your program does not have type information.  Thus, you end up having to watch for unexpected effects and do run-time identification of your types regardless.  That your type system will catch you if you, say, fail to have a file handle closed at a certain point is usually not  a big deal (and that has lighter-weight solutions anyway for the large majority of cases).

  --Rex

Ittay Dror gmail.com> writes:
> I'd love to see a code example where the introduction of IO[_] makes
> more errors detectable. In the code example I used in the first
> message, I couldn't find where it helps.

Simple:

foo : Int => Int

If your language is impure (like Scala), foo can format your HD, store and
retrieve stuff in a DB etc. If your language is pure, foo will always return the
same value given the same argument and it will do nothing else, thus it's much
easier to verify it's behavior.

You are thinking backwards, it's not the functions that perform side effects
that are easy to test, it's the ones that have no side effects. Problem in a
language without effects tracking is that you have no idea or control over which
functions perform side effects.

Also, knowing that a function is pure gives the compiler more options for
optimization, it can memoize the return value, it can complete remove calls if
the result is not used, it can parallelize calls etc.

So, how do you keep the language pure and still allow side effects? The IO monad
is a quite natural solution to that.

> If I have
> code that does 'for (_ <- close(socket); _ <- close(socket))',
> it will fail just the same as the (simpler) imperative code
> "socket.close; socket.close", wouldn't it? Side effects are not
> gone. They are just abstracted over, the question is whether the
> abstraction helps.

You have just discovered that ';' is shorthand for flatMap. It's the same
abstraction.

/Jesper Nordenberg

body p { margin-bottom: 0cm; margin-top: 0pt; }

Replies inline. Please note I'm replying from the perspective of an every-day developer, not from the theoretical point of view. I want to find a killer argument for using IO.

Jesper Nordenberg wrote:

loom [dot] 20111010T093707-484 [at] post [dot] gmane [dot] org" type="cite">

Ittay Dror <ittay.dror <at> gmail.com> writes:

    I'd love to see a code example where the introduction of IO[_] makes
    more errors detectable. In the code example I used in the first
    message, I couldn't find where it helps.

Simple:

foo : Int => Int

If your language is impure (like Scala), foo can format your HD, store and 
retrieve stuff in a DB etc. If your language is pure, foo will always return the 
same value given the same argument and it will do nothing else, thus it's much 
easier to verify it's behavior.

This is true in theory, but in practice, I usually know what a function does when I use it.  People don't name their functions 'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a function named 'factorial' and inside have code to format the HD.

For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD just as well

If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this approach (it is similar to how Option is an alternative to document the possible return of null), but then why go through the hassle of a monad? I can have:
case class Dangerous[A](value: A)

loom [dot] 20111010T093707-484 [at] post [dot] gmane [dot] org" type="cite">

You are thinking backwards, it's not the functions that perform side effects 
that are easy to test, it's the ones that have no side effects. Problem in a 
language without effects tracking is that you have no idea or control over which 
functions perform side effects.

But when I test I know the function I am testing. It's not a random thing.  Testing is a very big thing in the Java world, people manage to do it fine. Sometime they manage the difference between a test environment and a production environment with DI (so a fetchUser function will work with a real database in production, but be tested with a mock database)

loom [dot] 20111010T093707-484 [at] post [dot] gmane [dot] org" type="cite">

Also, knowing that a function is pure gives the compiler more options for 
optimization, it can memoize the return value, it can complete remove calls if 
the result is not used, it can parallelize calls etc.

True, but this can cause the reverse effect, like large memory consumption, thrashing of the cpu etc. In practice, I'd rather have code that works plainly and then optimize the bottlenecks I find, controlling how I do it. Of course it is easier to optimize pure code (e.g., scala's parallel collections). Btw, when the code is not pure I might need to optimize it manually anyway, so I need to use a profiler anyway.

loom [dot] 20111010T093707-484 [at] post [dot] gmane [dot] org" type="cite">

So, how do you keep the language pure and still allow side effects? The IO monad 
is a quite natural solution to that.

    If I have
    code that does 'for (_ <- close(socket); _ <- close(socket))',
    it will fail just the same as the (simpler) imperative code
    "socket.close; socket.close", wouldn't it? Side effects are not
    gone. They are just abstracted over, the question is whether the
    abstraction helps. 

You have just discovered that ';' is shorthand for flatMap. It's the same 
abstraction.

/Jesper Nordenberg

> This is true in theory, but in practice, I usually know what a function does
> when I use it. People don't name their functions 'foo', they name them
> 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a
> function named 'factorial' and inside have code to format the HD.

Sorry, but that doesn't work. I have stumbled upon so many seemingly non
side-effecting methods with "innocent" names which did some really
fundamental state changes that I came to the conclusion that anything
which is not "documented" in the types harms development.

> For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD
> just as well

You are mixing two problems. Admittedly, putting so many different side
effects into `IO` is not a good idea. IMHO it would be good to separate
different effects into `Filesystem` or `Network` etc.

> If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this
> approach (it is similar to how Option is an alternative to document the possible
> return of null), but then why go through the hassle of a monad? I can have:
> case class Dangerous[A](value: A)

This smells like you want to re-invent monads, as `Dangerous` can be
treated as a monad. I challenge you to write an API using that class and
I bet that you will use some monadic patterns with or without knowing it.

Ittay Dror gmail.com> writes:
> This is true in theory, but in practice, I usually know what a
> function does when I use it.  People don't name their functions
> 'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine
> a situation where I'd write a function named 'factorial' and inside
> have code to format the HD.
> For the same reasons, a function 'printMsg: String => IO[Unit]'
> can format my HD just as well

Fine, you obviously don't need and value compile time checking. Personally,
I value it very high. The IO monad may not be most precise and user friendly
way to encode side effects, but it's a simple solution and compared to having
no effects tracking I would take it any day.

> If IO[_] is just a tag to mean a function is dangerous, then I'm
> fine with this approach (it is similar to how Option is an
> alternative to document the possible return of null), but then why
> go through the hassle of a monad? I can have:
> case class Dangerous[A](value: A)

What's stopping you from doing: def makeSafe[A](d : Dangerous[A]) : A =
d.value? You do realize that many side effects are transitive?

> But when I test I know the function I am testing. It's not a random
> thing.  Testing is a very big thing in the Java world, people manage
> to do it fine. Sometime they manage the difference between a test
> environment and a production environment with DI (so a fetchUser
> function will work with a real database in production, but be tested
> with a mock database)

Really? So you never test an abstract interface method without knowing the
details of the implementation?

That people in the Java world manage to do proper testing is a huge
overstatement. I would rather say that the opposite is true.

> True, but this can cause the reverse effect, like large memory
> consumption, thrashing of the cpu etc. In practice, I'd rather have
> code that works plainly and then optimize the bottlenecks I find,
> controlling how I do it. Of course it is easier to optimize pure
> code (e.g., scala's parallel collections). Btw, when the code is not
> pure I might need to optimize it manually anyway, so I need to use a
> profiler anyway.

So, your argument is that because you can't optimize some parts of the code
you might as well not do it for any part? Or maybe you're totally against
compiler optimizations? The JVM is not the right platform for you then.

/Jesper Nordenberg

martin odersky epfl.ch> writes:
> So the only thing IO provides is that code *not* using it is certified
> to be pure, whereas in Scala everything is in the IO monad.

That's quite a huge thing, don't you agree?

/Jesper Nordenberg

body p { margin-bottom: 0cm; margin-top: 0pt; }

Lars Hupel wrote:

1 [at] dough [dot] gmane [dot] org" type="cite">

This is true in theory, but in practice, I usually know what a function does 
when I use it. People don't name their functions 'foo', they name them 
'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a 
function named 'factorial' and inside have code to format the HD.

Sorry, but that doesn't work. I have stumbled upon so many seemingly non
side-effecting methods with "innocent" names which did some really
fundamental state changes that I came to the conclusion that anything
which is not "documented" in the types harms development.

fair enough, see below.

1 [at] dough [dot] gmane [dot] org" type="cite">

For the same reasons, a function 'printMsg: String => IO[Unit]' can format my HD 
just as well

You are mixing two problems. Admittedly, putting so many different side
effects into `IO` is not a good idea. IMHO it would be good to separate
different effects into `Filesystem` or `Network` etc.

If IO[_] is just a tag to mean a function is dangerous, then I'm fine with this 
approach (it is similar to how Option is an alternative to document the possible 
return of null), but then why go through the hassle of a monad? I can have:
case class Dangerous[A](value: A)

This smells like you want to re-invent monads, as `Dangerous` can be
treated as a monad. I challenge you to write an API using that class and
I bet that you will use some monadic patterns with or without knowing it.

var bar = foo(3)
println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
// an ahha moment, realizing foo is dangerous. check what foo actually does and then, after making sure all is well:
println(bar.value * 2)

So I need to jump through a hoop to use foo, but it is a very small one. I don't need to use bind, or propogate Dangerous up the call trace or anything like that.

1 [at] dough [dot] gmane [dot] org" type="cite">

body p { margin-bottom: 0cm; margin-top: 0pt; }

Jesper Nordenberg wrote:

loom [dot] 20111010T123018-200 [at] post [dot] gmane [dot] org" type="cite">

Ittay Dror <ittay.dror <at> gmail.com> writes:

    This is true in theory, but in practice, I usually know what a
    function does when I use it.  People don't name their functions
    'foo', they name them 'formatHD', 'readFromDB' etc. I can't imagine
    a situation where I'd write a function named 'factorial' and inside
    have code to format the HD.
    For the same reasons, a function 'printMsg: String => IO[Unit]'
    can format my HD just as well

Fine, you obviously don't need and value compile time checking. Personally,
I value it very high. The IO monad may not be most precise and user friendly 
way to encode side effects, but it's a simple solution and compared to having 
no effects tracking I would take it any day.

I value it when it brings a lot of benefit for a small effort (which is why I love type inference: it reduces the effort).

loom [dot] 20111010T123018-200 [at] post [dot] gmane [dot] org" type="cite">

    If IO[_] is just a tag to mean a function is dangerous, then I'm
    fine with this approach (it is similar to how Option is an
    alternative to document the possible return of null), but then why
    go through the hassle of a monad? I can have:
    case class Dangerous[A](value: A)

What's stopping you from doing: def makeSafe[A](d : Dangerous[A]) : A = 
d.value? You do realize that many side effects are transitive?

the fact that i'm a decent programmer. otherwise, even in haskell, i can wreck havoc.

loom [dot] 20111010T123018-200 [at] post [dot] gmane [dot] org" type="cite">

    But when I test I know the function I am testing. It's not a random
    thing.  Testing is a very big thing in the Java world, people manage
    to do it fine. Sometime they manage the difference between a test
    environment and a production environment with DI (so a fetchUser
    function will work with a real database in production, but be tested
    with a mock database)

Really? So you never test an abstract interface method without knowing the
details of the implementation?

when i test a function, i know what it is supposed to do (i don't care how). usually, i'm the one who wrote it. It is also usually very obvious if it is pure or not.

as a "pragmatic" example, the command-query segregation principle is a nice one: if a function returns a value it must be pure, if it returns Unit, it means it has side effects. This is of course another way of "tagging".

loom [dot] 20111010T123018-200 [at] post [dot] gmane [dot] org" type="cite">

That people in the Java world manage to do proper testing is a huge
overstatement. I would rather say that the opposite is true.

    True, but this can cause the reverse effect, like large memory
    consumption, thrashing of the cpu etc. In practice, I'd rather have
    code that works plainly and then optimize the bottlenecks I find,
    controlling how I do it. Of course it is easier to optimize pure
    code (e.g., scala's parallel collections). Btw, when the code is not
    pure I might need to optimize it manually anyway, so I need to use a
    profiler anyway.

So, your argument is that because you can't optimize some parts of the code 
you might as well not do it for any part? Or maybe you're totally against
compiler optimizations? The JVM is not the right platform for you then.

It means I rather optimize myself, with the help of a library, than let the compiler do it for me. of course its not black and white here either, small optimizations: sure, huge ones, like memoization and parallelism: no.

Anyways, this is getting into what I call a theological argument, with fuzzy arguments, so lets stop here. If you can show me a piece of code

loom [dot] 20111010T123018-200 [at] post [dot] gmane [dot] org" type="cite">

/Jesper Nordenberg

On Mon, Oct 10, 2011 at 12:56 PM, Jesper Nordenberg wrote:
> martin odersky epfl.ch> writes:
>> So the only thing IO provides is that code *not* using it is certified
>> to be pure, whereas in Scala everything is in the IO monad.
>
> That's quite a huge thing, don't you agree?

Sure. But I think there will at some point be better ways to track
effects than monads. Monads are too
rigid. Adding a single println somewhere means you have to rewrite
your whole program.

This thread is epic.

On 10/10/2011 9:31 PM, "martin odersky" <martin.odersky@epfl.ch> wrote:

On Mon, Oct 10, 2011 at 12:56 PM, Jesper Nordenberg <megagurka@yahoo.com> wrote:
> martin odersky <martin.odersky <at> epfl.ch> writes:
>> So the only thing IO provides is that code *not* using it is certified
>> to be pure, whereas in Scala everything is in the IO monad.
>
> That's quite a huge thing, don't you agree?

Sure. But I think there will at some point be better ways to track
effects than monads. Monads are too
rigid. Adding a single println somewhere means you have to rewrite
your whole program.

Tony Morris tmorris.net> writes:
> This thread is epic.

Tony, I was just wondering when you would chime in. Care to enlighten us with
your views on the epicness of this thread and the IO monad?

/Jesper Nordenberg

> var bar = foo(3)
> println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
> // an ahha moment, realizing foo is dangerous. check what foo actually does and
> then, after making sure all is well:
> println(bar.value * 2)
>
> So I need to jump through a hoop to use foo, but it is a very small one. I don't
> need to use bind, or propogate Dangerous up the call trace or anything like that.

Same argument would apply to `Option`. As a careful programmer, you
would always check `x.isDefined` before calling `x.get`, wouldn't you?
Actually, that would be the same as using `null`. Instead, we have that
higher level abstraction called "catamorphism" (or use the pattern
matching instead if you wish) which allows the compiler to check what
you are doing. Same thing for side effects.

On Mon, Oct 10, 2011 at 1:49 AM, Jesper Nordenberg wrote:
> Ittay Dror skrev 2011-10-10 07:16:
>>
>> ...
> That's basically the same question as "what does a type system buy me?". It
> enables automatic detection of programming errors at compile time rather
> than at runtime.
>
> It also opens up opportunities for performance optimizations and makes your
> code a whole lot easier to test.
>
> …

Types also document aspects of the program for the reader.

> Types also document aspects of the program for the reader.

Documentation which cannot get out of sync with the code and can be
reasoned about. The latter might seem like a totally academic idea, but
is extremely useful.

> You can do the same trick for every language: You can say *all* Scala
> functions just "assemble" a computation which then gets executed only
> by calling main. So instead of Monadic bind, Scala has
>
> val x = a; b
>
> On the type level, every type T would be interpreted as IO[T]. Every
> function application is a bind. And voila! No side effects!

But you agree that this interpretation is not useful at all? Having no
possibility to choose whether to use IO or not does not make an effect
system.

On Mon, Oct 10, 2011 at 3:18 PM, Lars Hupel wrote:
>> You can do the same trick for every language: You can say *all* Scala
>> functions just "assemble" a computation which then gets executed only
>> by calling main. So instead of Monadic bind, Scala has
>>
>>   val x = a; b
>>
>> On the type level, every type T would be interpreted as IO[T]. Every
>> function application is a bind. And voila!  No side effects!
>
> But you agree that this interpretation is not useful at all? Having no
> possibility to choose whether to use IO or not does not make an effect
> system.

Of course.

On Mon, Oct 10, 2011 at 07:35, Lars Hupel wrote:
>> This is true in theory, but in practice, I usually know what a function does
>> when I use it. People don't name their functions 'foo', they name them
>> 'formatHD', 'readFromDB' etc. I can't imagine a situation where I'd write a
>> function named 'factorial' and inside have code to format the HD.
>
> Sorry, but that doesn't work. I have stumbled upon so many seemingly non
> side-effecting methods with "innocent" names which did some really
> fundamental state changes that I came to the conclusion that anything
> which is not "documented" in the types harms development.

Word.

def isScalaLang(url: java.net.URL): Boolean = url == new
java.net.URL("http://www.scala-lang.org")

body p { margin-bottom: 0cm; margin-top: 0pt; }

Lars Hupel wrote:

1 [at] dough [dot] gmane [dot] org" type="cite">

var bar = foo(3)
println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
// an ahha moment, realizing foo is dangerous. check what foo actually does and 
then, after making sure all is well:
println(bar.value * 2)

So I need to jump through a hoop to use foo, but it is a very small one. I don't 
need to use bind, or propogate Dangerous up the call trace or anything like that.

Same argument would apply to `Option`. As a careful programmer, you
would always check `x.isDefined` before calling `x.get`, wouldn't you?
Actually, that would be the same as using `null`. Instead, we have that
higher level abstraction called "catamorphism" (or use the pattern
matching instead if you wish) which allows the compiler to check what
you are doing. Same thing for side effects.

With Option, I have a way out. I can write 'someOption getOrElse default'. So if some low level function returns Option, it doesn't mean every client of it needs to also return Option.

With IO, I must propagate it up. Not only is this inconvenient, it also hurts abstraction. I know formatting the HD is a horrific thought, but I never encounter this. I do encounter reading from files and maybe creating them maybe even deletion. When I do that, I don't want this implementation detail to propagate up my whole program. e.g., If I have a fetchUser(userName) method, I wish for it to return User, regardless if it is a constant function (e.g. for testing), a partially applied one with a list of Users, or one that reads from the DB or from the filesystem.  I would be happy if the compiler warned me before reading from the DB, so I'd make sure I'm doing it right, but then get out of my way once I've made sure my code is sound.

1 [at] dough [dot] gmane [dot] org" type="cite">

> With Option, I have a way out. I can write 'someOption getOrElse default'. So if
> some low level function returns Option, it doesn't mean every client of it needs
> to also return Option.

That's only true because `Option` is not "opaque" in its definition. If
`IO` had a catamorphism you also had a way out.

> With IO, I must propagate it up. Not only is this inconvenient, it also hurts
> abstraction. I know formatting the HD is a horrific thought, but I never
> encounter this. I do encounter reading from files and maybe creating them maybe
> even deletion. When I do that, I don't want this implementation detail to
> propagate up my whole program. e.g., If I have a fetchUser(userName) method, I
> wish for it to return User, regardless if it is a constant function (e.g. for
> testing), a partially applied one with a list of Users, or one that reads from
> the DB or from the filesystem. I would be happy if the compiler warned me before
> reading from the DB, so I'd make sure I'm doing it right, but then get out of my
> way once I've made sure my code is sound.

That's not entirely true. If you want to both side-effecting and not
side-effecting functions uniformly, you could simply lift the latter to
the former.

However, I see a point (also what Martin said) that Monads are too
"rigid", i. e. you have to propagate the use, but as long as there are
no other formalisms which doesn't suffer from that "problem" (if you
wish to call it so), `IO` is a reasonable solution.

As I said, the usual approach is that you strictly separate both types
of functions and transform the side effects with your pure functions.
`bind` and `fmap` are there for that.

body p { margin-bottom: 0cm; margin-top: 0pt; }

Lars Hupel wrote:

1 [at] dough [dot] gmane [dot] org" type="cite">

With Option, I have a way out. I can write 'someOption getOrElse default'. So if 
some low level function returns Option, it doesn't mean every client of it needs 
to also return Option.

That's only true because `Option` is not "opaque" in its definition. If
`IO` had a catamorphism you also had a way out.

But then I'd start lying about my types, right? "foo: Int => Int" can mean it is using "bar: Int => IO[Int]" and reducing its result

1 [at] dough [dot] gmane [dot] org" type="cite">

With IO, I must propagate it up. Not only is this inconvenient, it also hurts 
abstraction. I know formatting the HD is a horrific thought, but I never 
encounter this. I do encounter reading from files and maybe creating them maybe 
even deletion. When I do that, I don't want this implementation detail to 
propagate up my whole program. e.g., If I have a fetchUser(userName) method, I 
wish for it to return User, regardless if it is a constant function (e.g. for 
testing), a partially applied one with a list of Users, or one that reads from 
the DB or from the filesystem. I would be happy if the compiler warned me before 
reading from the DB, so I'd make sure I'm doing it right, but then get out of my 
way once I've made sure my code is sound.

That's not entirely true. If you want to both side-effecting and not
side-effecting functions uniformly, you could simply lift the latter to
the former.

However, I see a point (also what Martin said) that Monads are too
"rigid", i. e. you have to propagate the use, but as long as there are
no other formalisms which doesn't suffer from that "problem" (if you
wish to call it so), `IO` is a reasonable solution.

Again, the thread started by me trying to find a killer argument for using IO monad for everyday OO developers. I don't think they'd be convinced to use IO at high levels of their code just because somewhere deep down there's a call to file.getLastModified.

1 [at] dough [dot] gmane [dot] org" type="cite">

As I said, the usual approach is that you strictly separate both types
of functions and transform the side effects with your pure functions.
`bind` and `fmap` are there for that.

On Mon, Oct 10, 2011 at 10:30, Ittay Dror wrote:
>
>
> Lars Hupel wrote:
>
> var bar = foo(3)
> println(bar * 2) // doesn't compile, bar is of type Dangerous[Int]
> // an ahha moment, realizing foo is dangerous. check what foo actually does
> and
> then, after making sure all is well:
> println(bar.value * 2)
>
> So I need to jump through a hoop to use foo, but it is a very small one. I
> don't
> need to use bind, or propogate Dangerous up the call trace or anything like
> that.
>
> Same argument would apply to `Option`. As a careful programmer, you
> would always check `x.isDefined` before calling `x.get`, wouldn't you?
> Actually, that would be the same as using `null`. Instead, we have that
> higher level abstraction called "catamorphism" (or use the pattern
> matching instead if you wish) which allows the compiler to check what
> you are doing. Same thing for side effects.
>
> With Option, I have a way out. I can write 'someOption getOrElse default'.
> So if some low level function returns Option, it doesn't mean every client
> of it needs to also return Option.
>
> With IO, I must propagate it up. Not only is this inconvenient, it also
> hurts abstraction. I know formatting the HD is a horrific thought, but I
> never encounter this. I do encounter reading from files and maybe creating
> them maybe even deletion. When I do that, I don't want this implementation
> detail to propagate up my whole program. e.g., If I have a
> fetchUser(userName) method, I wish for it to return User, regardless if it
> is a constant function (e.g. for testing), a partially applied one with a
> list of Users, or one that reads from the DB or from the filesystem.  I
> would be happy if the compiler warned me before reading from the DB, so I'd
> make sure I'm doing it right, but then get out of my way once I've made sure
> my code is sound.

Better to propagate up than down.

To see the difference, imagine that all IO methods in Java had
something like a @throws clause (let's call it @io), except that it
could not be caught. One can easily see that it propagates up. Now
consider a Function class -- how would the apply method be annotated?
If it doesn't have an @io annotation, it can call anything annotated
with @io. If it has @io, then everything calling it will have to be
annotated with @io. Consider what this would do to a collections
library... What is happening is that this tagging is propagating
*down* to methods that, otherwise, would not need to know anything
about it.

With an IO Monad, I can pass the whole monad to any method, and it
won't care less. So while it has to propagate "up" to the "main", it
does not need to propagate "down" to all methods.

2011/10/10 Daniel Sobral <dcsobral@gmail.com>

Better to propagate up than down.

To see the difference, imagine that all IO methods in Java had
something like a @throws clause (let's call it @io), except that it
could not be caught. One can easily see that it propagates up. Now
consider a Function class -- how would the apply method be annotated?
If it doesn't have an @io annotation, it can call anything annotated
with @io. If it has @io, then everything calling it will have to be
annotated with @io. Consider what this would do to a collections
library... What is happening is that this tagging is propagating
*down* to methods that, otherwise, would not need to know anything
about it.

For that reason you want effect polymorphism. 

With an IO Monad, I can pass the whole monad to any method, and it
won't care less. So while it has to propagate "up" to the "main", it
does not need to propagate "down" to all methods.

But you need to duplicate methods to work in both pure and monadic world,see `map` and `mapM` in haskell (http://stackoverflow.com/questions/932639/haskell-cant-use-map-putstrln)

> But you need to duplicate methods to work in both pure and monadic world,
> see `map` and `mapM` in haskell

See documentation: `mapM` is just `sequence . map`. `sequence` exists
precisely for that.

On Mon, Oct 10, 2011 at 11:52, Lukas Rytz wrote:
>
>> With an IO Monad, I can pass the whole monad to any method, and it
>> won't care less. So while it has to propagate "up" to the "main", it
>> does not need to propagate "down" to all methods.
>
> But you need to duplicate methods to work in both pure and monadic world,
> see `map` and `mapM` in haskell
> (http://stackoverflow.com/questions/932639/haskell-cant-use-map-putstrln)

Let the Haskellites correct me if needed, but all I saw there was a
simple type mismatch.

Ittay,

You gain two things.

First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type. Think of this in terms of a more familiar data type: List. When you construct the list List(1,2), observe what you're not doing. You're not mutating a variable to first be 1 and then be 2, with some intervening logic. No, you have separated concerns. You're saying that there may be a computation later that will receive first the value 1 and then the value 2, and it will combine them in some yet-to-be-defined way. In exactly the same sense, a value of type IO[String] is just data that will be consumed by some computation (e.g. unsafePerformIO) later. That computation might write to files and launch missiles, but then again it might not. It all depends on the implementation of the IO data type, which you don't need to care about.

Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another. To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)

Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.

This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:

def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
  xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)

Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?

"Abstract ideas are conceptual integrations which subsume an incalculable number of concretes—and without abstract ideas you would not be able to deal with concrete, particular, real-life problems. You would be in the position of a newborn infant, to whom every object is a unique, unprecedented phenomenon."

The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).

+1

On Mon, Oct 10, 2011 at 10:50 AM, Runar Bjarnason <runarorama@gmail.com> wrote:

The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).

--
L.G. Meredith
Managing Partner
Biosimilarity LLC
7329 39th Ave SWSeattle, WA 98136

+1 206.650.3740

http://biosimilarity.blogspot.com

On Mon, Oct 10, 2011 at 6:50 PM, Runar Bjarnason <runarorama@gmail.com> wrote:

The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).

You can also measure how often you want to circuvemt a type system. | think that the main problem is the coarsity of IO.

On Mon, Oct 10, 2011 at 1:04 PM, Runar Bjarnason <runarorama@gmail.com> wrote:

Thanks for the detailed reply--I don't know how Ittay feels about the response, but it leaves me just as profoundly unconvinced that IO monads per se are worthwhile in almost any typical case (see below).

First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type.

But there isn't one thing which is "an IO data type".  You might have text output or binary output or Base64 encoded output or some other weird thing.  It might be for a log file or a blog post or a database record and these might be the same data type but formatted differently.  The key is transforming from your data type to/from text or binary or Base64 or whatnot (and not mixing your output types/formats up); the IO container adds nothing beyond a flag that states that in this particular case, you intend the data for IO instead of something else.  You could mark any part of your program with types in this way, whether it's eventually going to be side-effecting or not, with equal (dis)utility.  Foldable data type transformers are useful, but that has nothing to do with IO specifically.  Thus, this position for separation does not seem to me to be at an overwhelmingly useful place.

Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another.

You know, you also don't need to have this understanding if you add an extra println.  You _do_ need to know in what order things happen, but you _also_ need to know in what order you've built your IO operations.  You gain nothing in simple cases.  In complex cases, such as with reading and overwriting records on disk using seeks, you _could_ use the extra level of abstraction to sneak in a layer of caching that avoids too much disk churn...but you could just as well if you gave people a println-style interface and a flush().  Only if all IO computations are lazy do you have potential for benefit: you can reorder large amounts of data without having to cache large amounts of data in memory.  But again, this is a pretty specialized use-case.
 

To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)

Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.

This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:

def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
  xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)

Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?

You should take advantage of it, but it has nothing to do with IO.  You can take advantage of this for data transformation operations without any IO-specific anything, and even with having inverse-sequence be full of side-effecting IO operations.

  --Rex

body p { margin-bottom: 0cm; margin-top: 0pt; }

+1

Rex Kerr wrote:

zA [at] mail [dot] gmail [dot] com" type="cite"> On Mon, Oct 10, 2011 at 1:04 PM, Runar Bjarnason <runarorama [at] gmail [dot] com" rel="nofollow">runarorama@gmail.com> wrote:

Thanks for the detailed reply--I don't know how Ittay feels about the response, but it leaves me just as profoundly unconvinced that IO monads per se are worthwhile in almost any typical case (see below).

First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type.

But there isn't one thing which is "an IO data type".  You might have text output or binary output or Base64 encoded output or some other weird thing.  It might be for a log file or a blog post or a database record and these might be the same data type but formatted differently.  The key is transforming from your data type to/from text or binary or Base64 or whatnot (and not mixing your output types/formats up); the IO container adds nothing beyond a flag that states that in this particular case, you intend the data for IO instead of something else.  You could mark any part of your program with types in this way, whether it's eventually going to be side-effecting or not, with equal (dis)utility.  Foldable data type transformers are useful, but that has nothing to do with IO specifically.  Thus, this position for separation does not seem to me to be at an overwhelmingly useful place.

Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another.

You know, you also don't need to have this understanding if you add an extra println.  You _do_ need to know in what order things happen, but you _also_ need to know in what order you've built your IO operations.  You gain nothing in simple cases.  In complex cases, such as with reading and overwriting records on disk using seeks, you _could_ use the extra level of abstraction to sneak in a layer of caching that avoids too much disk churn...but you could just as well if you gave people a println-style interface and a flush().  Only if all IO computations are lazy do you have potential for benefit: you can reorder large amounts of data without having to cache large amounts of data in memory.  But again, this is a pretty specialized use-case.
 

To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)

Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.

This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:

def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
  xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)

Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?

You should take advantage of it, but it has nothing to do with IO.  You can take advantage of this for data transformation operations without any IO-specific anything, and even with having inverse-sequence be full of side-effecting IO operations.

  --Rex

On Mon, Oct 10, 2011 at 7:04 PM, Runar Bjarnason wrote:
> Ittay,
>
> You gain two things.
>
> First, you gain modularity by separating concerns. In the case of the IO
> data type, you are separating the concern of what to do from how to do it
> and when to do it. When you say service.fetchData, you're simply
> constructing a value of the IO data type. Think of this in terms of a more
> familiar data type: List. When you construct the list List(1,2), observe
> what you're not doing. You're not mutating a variable to first be 1 and then
> be 2, with some intervening logic. No, you have separated concerns. You're
> saying that there may be a computation later that will receive first the
> value 1 and then the value 2, and it will combine them in some
> yet-to-be-defined way. In exactly the same sense, a value of type IO[String]
> is just data that will be consumed by some computation (e.g.
> unsafePerformIO) later. That computation might write to files and launch
> missiles, but then again it might not. It all depends on the implementation
> of the IO data type, which you don't need to care about.
>
> Secondly, you gain compositionality because your IO actions (which are just
> ordinary data) can be composed with other actions in a completely
> predictable way. And I don't mean that you can copy and paste. I mean that
> since your IO actions are just first-class values, they can be composed
> arbitrarily to form more complex programs. What's more, such composition can
> be understood by local reasoning alone. You don't need to have an
> understanding of your entire program in order to understand what happens
> when you compose one IO action with another. To illustrate, this let's go
> back to the familiar List. A value of type List[IO[String]] can be turned
> into a value of type IO[List[String]] by composing all the actions in the
> list:
> ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)
>
> Note that we only needed to know how to compose two IO actions, and that was
> enough to compose the entire list.
>
> This brings us back to modularity. There is nothing particularly "IO" about
> this code. So we can abstract that part out and re-use it for, not just IO,
> but all monads:
>
> def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
>   xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)
>
> Of course, you could argue that it's somehow "simpler" to reinvent this
> functionality, in different ways, all over your code. It's a prevalent
> fallacy that it is simpler to not abstract. But you are in possession of a
> language, Scala, that makes abstraction easy. Why not take advantage of it?
>
> "Abstract ideas are conceptual integrations which subsume an incalculable
> number of concretes—and without abstract ideas you would not be able to deal
> with concrete, particular, real-life problems. You would be in the position
> of a newborn infant, to whom every object is a unique, unprecedented
> phenomenon."
>

The argument boils down to: It's very useful to have a way to compose
actions. But it's not specific to IO in any way. You can just as well
compose closures in a language without the IO monad. What's specific
about IO is that it marks the underlying action as having a particular
kind of effect.
I argue there are better, more polymorphic, ways to do this than monads

Cheers

body p { margin-bottom: 0cm; margin-top: 0pt; }

Runar Bjarnason wrote:

Ittay,

You gain two things.

First, you gain modularity by separating concerns. In the case of the IO data type, you are separating the concern of what to do from how to do it and when to do it. When you say service.fetchData, you're simply constructing a value of the IO data type. Think of this in terms of a more familiar data type: List. When you construct the list List(1,2), observe what you're not doing. You're not mutating a variable to first be 1 and then be 2, with some intervening logic. No, you have separated concerns. You're saying that there may be a computation later that will receive first the value 1 and then the value 2, and it will combine them in some yet-to-be-defined way. In exactly the same sense, a value of type IO[String] is just data that will be consumed by some computation (e.g. unsafePerformIO) later. That computation might write to files and launch missiles, but then again it might not. It all depends on the implementation of the IO data type, which you don't need to care about.

Secondly, you gain compositionality because your IO actions (which are just ordinary data) can be composed with other actions in a completely predictable way. And I don't mean that you can copy and paste. I mean that since your IO actions are just first-class values, they can be composed arbitrarily to form more complex programs. What's more, such composition can be understood by local reasoning alone. You don't need to have an understanding of your entire program in order to understand what happens when you compose one IO action with another. To illustrate, this let's go back to the familiar List. A value of type List[IO[String]] can be turned into a value of type IO[List[String]] by composing all the actions in the list:
ios.foldRight(io(Nil))((a, b) => for { t <- b; h <- a } yield h :: t)

Note that we only needed to know how to compose two IO actions, and that was enough to compose the entire list.

This brings us back to modularity. There is nothing particularly "IO" about this code. So we can abstract that part out and re-use it for, not just IO, but all monads:

def sequence[M[_]:Monad, A](xs: List[M[A]]): M[List[A]] =
  xs.foldRight(Nil.pure[M])((a, b) => for { t <- b; h <- a } yield h :: t)

Of course, you could argue that it's somehow "simpler" to reinvent this functionality, in different ways, all over your code. It's a prevalent fallacy that it is simpler to not abstract. But you are in possession of a language, Scala, that makes abstraction easy. Why not take advantage of it?

"Abstract ideas are conceptual integrations which subsume an incalculable number of concretes—and without abstract ideas you would not be able to deal with concrete, particular, real-life problems. You would be in the position of a newborn infant, to whom every object is a unique, unprecedented phenomenon."

Thank you for the detailed response.

To add to what Rex had to say:
* monads are useful. they provide common abstraction for use case of a value in a context (btw, in that respect, sequence is my favorite example since without the use of applicative, it is hard to imagine how one can turn a List[X[A]] to X[List[A]]). IO is a special case in that it is pervasive.
* unlike other monads, it is hard for me to find what additional important information IO adds. Option[_] for example documents the fact that an equivalent of null may be returned. IO[_] documents that some side effect can happen, so what? The CPU may heat up and more IO instances take more memory, do we document these effects?
* say I have two programs, one touches some configuration file, deep inside its logic, another opens a socket to read some data, again, deep inside. Both return IO of something. Now what does this IO tell me? nothing. I can't tell what side effect they have made. I only know they have made something. But won't any sufficiently large app do some kind of side effect? to me, IO is like those Java programs that deal with a function that throws an exception, then catch this exception and throw an Exception instance (otherwise, they break encapsulation), so now i have 2 functions that document they may throw Exception. What can I do with this? nothing, just call them one after the other, hope for the best and maybe wrap in a 'catch'.
* I can compose functions that hide the side effect by just composing them
* I don't mind having IO to tag that a low-level function is doing side-effect. Just as long as I can escape it when I want to (like I can use catamorphism on Option or List). But then is it so useful?

Ittay

Gents, isn't the STM monad a better fit for this discussion than the IO? There might be less questions as to it's intinsic value and more focus on the monadic constructs and alternatives. 
Martin, you stated that possibility a few times already - I'm sure we're all really curious now about what you have in mind :)
Also, since monads don't compose etc, why hasn't the discussion mutate to Applicative?
Thanks,Razvan
On 2011-10-10, at 2:32 PM, "nicolas.oury@gmail.com" <nicolas.oury@gmail.com> wrote:

On Mon, Oct 10, 2011 at 6:50 PM, Runar Bjarnason < (runarorama [at] gmail [dot] com> wrote:

The fact that it's possible to defeat an effect system (for example, in the presence of unsafePerformIO) does not mean that it's not useful. Just like the type system remains useful even though it's possible to defeat that (e.g. in the presence of asInstanceOf).

You can also measure how often you want to circuvemt a type system. | think that the main problem is the coarsity of IO.

On Mon, Oct 10, 2011 at 3:26 PM, Ittay Dror <ittay.dror@gmail.com> wrote:

* unlike other monads, it is hard for me to find what additional important information IO adds. Option[_] for example documents the fact that an equivalent of null may be returned. IO[_] documents that some side effect can happen, so what? The CPU may heat up and more IO instances take more memory, do we document these effects?

Maybe, maybe not. A value of type IO[A] is just a program that produces a value of type A, given some IO subsystem that knows how to run it. The type doesn't tell you what the program does, only what it is.

That's not to say that you couldn't inspect an IO program. You could have an IO data type with a finite set of instructions, and a function that tells you what such a program does without running it. There's nothing about the IO type that tells you how it's implemented, and this is a good thing. That is an implementation detail over which we want to abstract.

 

* say I have two programs, one touches some configuration file, deep inside its logic, another opens a socket to read some data, again, deep inside. Both return IO of something. Now what does this IO tell me? nothing. I can't tell what side effect they have made.

That might be important if you want to be able to replace one with the other in your program. Also note that when you're in IO, you have effects but not side-effects. This is an important distinction.

List[Int] does not tell you which numbers are in your list. It only tells you that you have zero or more integers. But the fact that List doesn't give us all the information we might want doesn't mean that we should never use List.
 

* I can compose functions that hide the side effect by just composing them

For very trivial cases, maybe. But there will come a time when you want to use a function and you cannot, because it has a hidden dependency on JDBC, or a logging framework, or some other I/O having occurred before it. Your reasoning is basically: "I can write code that isn't modular but works just fine". Sure. But if you can easily separate concerns, why wouldn't you?
 

* I don't mind having IO to tag that a low-level function is doing side-effect. Just as long as I can escape it when I want to (like I can use catamorphism on Option or List). But then is it so useful?

The catamorphism for IO should be called only once in your program, at the very end. I think it's a common misunderstanding that you want to "get things out" of IO, or "escape it" somehow. The cases where you would want that are exceedingly rare, even nonexistent. The idea with monadic IO is that you only write the effectful stuff in the monad. Where you want to work outside of IO (which is most of your code), you should do that with pure functions as usual. You can then lift any such pure function into the monad with map and flatMap.