Back to Questions
7.1k
views12
comments
An F#-style iteratee
f#iteratee

I believe I've found a more F#-like way of creating an Iteratee. I was checking out the AsyncSeq.fold and thus the AsyncSeq.scan, which led me to the Seq.scan. This appears to achieve nearly the goal as Iteratee. Once you get the enumerator from the resulting seq<_>. The one thing I'd like to be able to do that doesn't appear immediately obvious is how to kill values early. I think the easiest would be to do something like:

1
2

let data = [1..10]
let result = Seq.scan (+) 0 data |> Seq.filter ((<) 30)

That's not a direct translation of Iteratee -- it's really closer to an Enumerator : Iteratee -> Iteratee -- but does the same job. For specific instances of Iteratee/Enumerator, one only needs to wrap this to embed the initial state:

1
2

let ``sums less than 30`` : seq<int> -> seq<int> =
    Seq.scan (+) 0 >> Seq.filter ((>) 30)

Binding to a subsequent enumerator could then be achieved using either Seq.collect. Of course, this also readily lends itself to AsyncSeq.scan and AsyncSeq.bind for asynchronous data streams.

Thoughts?

.

Using the approach of IterIO, I think I've found a way to make a more F#-style iteratee. The big question remains: should the chunk type be bound to byte or should it be generic? For most applications, a generic would be ideal. However, since iteratee is really meant for working with I/O, is there really any reason to use anything other than byte as a chunk data type?

I'm primarily asking b/c I will be creating a structurally equatable ArraySegment<_> type if I need the generic. I've already created BS/ByteString for the same purpose. Also, removing the generic has certain performance benefits (, though I can't find the reference just now).

Interested in your opinions.

By on 11/24/2011 8:10 AM ()

Tomas Petricek pointed out the pickler that is build into F#. This addresses nearly the same problem iteratee exists to solve. For now, I'd recommend people look there.

That said, I was able to add the `fix` operator to FSharpx, which appears to have resolved some of the stack tension in my original Iteratee implementation. I'm working on resolving remaining issues.

By on 10/19/2011 6:02 PM ()

In the end, this doesn't stack up with how iteratee works. :( In addition, further analysis of the enumerator package shows that it uses a few tricks in Haskell to compile into a CPS-friendly approach.

I'm beginning to doubt iteratee is possible in F#.

By on 10/17/2011 4:52 PM ()

Iteratees are definitely possible to encode in F#, the question is whether they are useful at all if so, for what applications.

By on 11/1/2011 12:14 PM ()

Indeed. Excellent question. And further, how best to encode them. As you've mentioned on Twitter, side-effecting functions can wreak havoc on an iteratee. My current attempt is using a Stream<'a> -> IterResult<'a,'b> in a Seq.scan hidden under another function. Stream<'a> uses an ArraySegment<'a> to keep mutability to a minimum, but that doesn't enforce side-effect free functions within iteratees themselves. It'll be interesting to determine how well or badly these behave under real application use.

By on 11/29/2011 1:31 PM ()

Using the above, iteratee becomes:

1

'State -> 'a -> 'State

with the initial state embedded within the function definition.

It would be simple to then compose iteratees using computation expressions by creating an IterateeBuilder based on that function definition.

By on 10/10/2011 5:58 PM ()

I believe Oleg's original iteratee was derived from this (left-fold) by hiding state and allowing early termination.

By on 11/1/2011 12:13 PM ()

Correct. He essentially embedded the State monad (initially), also setting the initial state internally. I believe the later developments allowed swapping out or transforming the State with other variations (IO, etc).

By on 11/29/2011 1:28 PM ()

I just realized that this is effectively the Continuation monad:

1

('a -> 'State) -> 'State

Embed the state, wrap Seq.scan in a new function (maybe Seq.enumerate) and call it a day?

By on 10/11/2011 5:40 PM ()

Did you do anything with pipelets?

By on 10/7/2011 11:13 PM ()

Not yet. I need to vet this idea first. Pipelets are coming.

By on 10/7/2011 11:46 PM ()

I have some tweaks to do to them at some point...

By on 10/8/2011 1:47 AM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper