Back to Questions
7.5k
views12
comments
+4
score		+
-
An F#-style iteratee
f#iteratee

I believe I've found a more F#-like way of creating an Iteratee. I was checking out the AsyncSeq.fold and thus the AsyncSeq.scan, which led me to the Seq.scan. This appears to achieve nearly the goal as Iteratee. Once you get the enumerator from the resulting seq<_>. The one thing I'd like to be able to do that doesn't appear immediately obvious is how to kill values early. I think the easiest would be to do something like:

1
2

let data = [1..10]
let result = Seq.scan (+) 0 data |> Seq.filter ((<) 30)

That's not a direct translation of Iteratee -- it's really closer to an Enumerator : Iteratee -> Iteratee -- but does the same job. For specific instances of Iteratee/Enumerator, one only needs to wrap this to embed the initial state:

1
2

let ``sums less than 30`` : seq<int> -> seq<int> =
    Seq.scan (+) 0 >> Seq.filter ((>) 30)

Binding to a subsequent enumerator could then be achieved using either Seq.collect. Of course, this also readily lends itself to AsyncSeq.scan and AsyncSeq.bind for asynchronous data streams.

Thoughts?

.Reply
Hide this comment

Using the approach of IterIO, I think I've found a way to make a more F#-style iteratee. The big question remains: should the chunk type be bound to byte or should it be generic? For most applications, a generic would be ideal. However, since iteratee is really meant for working with I/O, is there really any reason to use anything other than byte as a chunk data type?

I'm primarily asking b/c I will be creating a structurally equatable ArraySegment<_> type if I need the generic. I've already created BS/ByteString for the same purpose. Also, removing the generic has certain performance benefits (, though I can't find the reference just now).

Interested in your opinions.

By on 11/24/2011 8:10 AM ()Reply
Hide this comment

Tomas Petricek pointed out the pickler that is build into F#. This addresses nearly the same problem iteratee exists to solve. For now, I'd recommend people look there.

That said, I was able to add the `fix` operator to FSharpx, which appears to have resolved some of the stack tension in my original Iteratee implementation. I'm working on resolving remaining issues.

By on 10/19/2011 6:02 PM ()Reply
Hide this comment

In the end, this doesn't stack up with how iteratee works. :( In addition, further analysis of the enumerator package shows that it uses a few tricks in Haskell to compile into a CPS-friendly approach.

I'm beginning to doubt iteratee is possible in F#.

By on 10/17/2011 4:52 PM ()Reply
Hide this comment

Iteratees are definitely possible to encode in F#, the question is whether they are useful at all if so, for what applications.

By on 11/1/2011 12:14 PM ()Reply
Hide this comment

Indeed. Excellent question. And further, how best to encode them. As you've mentioned on Twitter, side-effecting functions can wreak havoc on an iteratee. My current attempt is using a Stream<'a> -> IterResult<'a,'b> in a Seq.scan hidden under another function. Stream<'a> uses an ArraySegment<'a> to keep mutability to a minimum, but that doesn't enforce side-effect free functions within iteratees themselves. It'll be interesting to determine how well or badly these behave under real application use.

By on 11/29/2011 1:31 PM ()Reply
Hide this comment

Using the above, iteratee becomes:

1

'State -> 'a -> 'State

with the initial state embedded within the function definition.

It would be simple to then compose iteratees using computation expressions by creating an IterateeBuilder based on that function definition.

By on 10/10/2011 5:58 PM ()Reply
Hide this comment

I believe Oleg's original iteratee was derived from this (left-fold) by hiding state and allowing early termination.

By on 11/1/2011 12:13 PM ()Reply
Hide this comment

Correct. He essentially embedded the State monad (initially), also setting the initial state internally. I believe the later developments allowed swapping out or transforming the State with other variations (IO, etc).

By on 11/29/2011 1:28 PM ()Reply
Hide this comment

I just realized that this is effectively the Continuation monad:

1

('a -> 'State) -> 'State

Embed the state, wrap Seq.scan in a new function (maybe Seq.enumerate) and call it a day?

By on 10/11/2011 5:40 PM ()Reply
Hide this comment

Did you do anything with pipelets?

By on 10/7/2011 11:13 PM ()Reply
Hide this comment

Not yet. I need to vet this idea first. Pipelets are coming.

By on 10/7/2011 11:46 PM ()Reply
Hide this comment

I have some tweaks to do to them at some point...

By on 10/8/2011 1:47 AM ()Reply

    Topic tags

    See all tags
    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper

    Logging in...