Here's another choice - convert the list into a Set then convert it back:

1
 let uniq a = a |> Set.of_list |> List.of_seq 

You can even leave the result as a Set, then treat it as a sequence (seq<'a>) for an even shorter solution:

1
2
3
4
5
let uniq = Set.of_list


let a = [ 1; 1; 1; 1; 3; 3; 4; 4; 8; 9; 11; 11; 11 ];
let b = a |> uniq;

> b;;
val it : Set<int> = seq [1; 3; 4; 8; ...]
>

By on 12/18/2007 6:47 PM ()

Hello J,

I was looking for something like this - thank you.

Only note: syntax has changed (Nov 09). So, of_list is now ofList, of_seq is now ofSeq.

BRN..

By on 11/25/2009 12:30 PM ()

For information, F# now has a Seq.distinct and Seq.distinctBy functions (they don't require the input to be sorted - they use a dictionary to remember the previous items).

By on 11/26/2009 1:32 AM ()

Nice! Thanks tomasp for that explanation. I'm still trying to wrap my brain around it, but I'm getting there. Thanks. Just to clarify, an expression name can have a single quote in it, correct? There's really nothing different about the uniq', right? Or is the ' a special operator?

jhugard, that is an _extremely_ elegant solution. I like it. What's the difference between a Set and a hashtable? I hadn't discovered the Set -- that's going to make my project go soooo much faster. Thanks!

By on 12/19/2007 8:54 AM ()

Set and Map are both immutable structures, and both are implemented using binary trees. When inserting a value, structural comparison (or IComparable) is used to find where the key belongs. Therefore, these are efficient for small keys. Set contains keys only, and Map contains keys, each with an associated value. Each unique key can only appear once in Set or Map, and only one value can be bound to each key in Map. When enumerated, the resulting Set or Map is sorted (because they are implemented with binary trees).

HashSet and HashMultiMap are mutable structures, and both are implemented using hash comparison. When inserting a value, structural hash (or a user-supplied hash function) and (=) are used to find if a key already exists. Therefore, these are a good choice when the key is a complex or recursive data structure. HashSet contains keys only, and HashMultiMap contains keys and an associated value. Each key can only appear once in a HashSet or HashMultiMap, but multiple values can be bound to a single key in a HashMultiMap. When enumerated, both appear to be unsorted (sort order is not documented, but appears to be in insertion order - though I wouldn't count on it).

By on 12/19/2007 2:57 PM ()

Hello J,

Your explanation, and the code in your example, gave me a clue as to where to look for more information on sets and F#.

I hope finding such references will be easier when the language becomes a fully supported, main stream, product.

For now, this is the best i found: [link:msdn.microsoft.com] Moving up stream, or drilling down puts the material in context. I did notice it uses the old syntax, though.

Odds are, I would not have found that if I had not see this thread on unique values in lists.

Thanks again,

BRN..

By on 11/25/2009 12:56 PM ()

If you want to learn the F# library, it is useful to start at

[link:msdn.microsoft.com]

and spend a few minutes going one or two levels deep from all the links there (at least Core and Collections), just to be familiar with the overall organization of the F# core library, and to at least get a sense of some of the names of types/modules/functions you may eventually want to come back to (to look at in more detail later, on demand).

Regarding the old-style names, thanks, I'm filing a doc bug, with luck it will be fixed soon.

By on 11/25/2009 1:06 PM ()

The single quote is a valid part of expression names. The only restriction is that your name cannot start with it. So, uniq' is okay, whereas 'uniq won't work (and will give you some weird error messages, if I remember correctly).

<removed data structures explanation as it was woefully incomplete and, somehow, I totally forgot about Map -- Refer to jhugard's explanation below instead>

Regards,

z.

By on 12/19/2007 9:36 AM ()

Hi,
implementing this by sorting the data and then using something like "uniq" is definitely possible. As far as I know there is nothing like "uniq" available in F#, but it is not so difficult to implement.

So, the first thing is to create a list and to sort it:

1
2
3
 
let a = [ 1; 11; 9; 1; 1; 3; 4; 8; 11; ]
a |> List.sort compare

The "|>" operator in F# does pipelining, so it calls the thing on the right side with the thing on the left side as an argument. The first argument of the sort function is a comparison function, which compares the elements - here we can use generic "compare" function which works on most of the standard F# types.

Implementing the "uniq" will be slightly more difficult - since we have the list sorted already, we can walk through the list and remember the last processed element - we will then check whether the current element is different and skip the element if it was already added to the resulting list. You can implement this (using pattern matching :-) ) as a primitive recursive function:

1
2
3
4
5
6
7
let rec uniq' lst res last =
  match lst with
  | [] -> res
  | h::tl when h=last -> uniq tl res last
  | h::tl -> uniq tl (h::res) h

let uniq lst = (uniq' lst [] -1) |> List.rev

The function takes the input list as a first argument, currently collected resulting list as a second one and last processed value as a third argument. Since we're adding the elements to the beginning of the "res" list, it will reverse the list, so it calls "List.rev" when it completes to reverse it once again (to the original ordering).

It is also possible to implement the same thing using "fold_left" function, which "remembers" the state while walking through the list and allows you to modify the state when processing each element. Here the state will be a tuple which contains last element ("st") and the result created so far ("lst") and when processing the element it either adds it to the resulting list or returns the original. The initial state that we'll give to the fold_left is -1 (some special value) and empty list (the result collected so far).

1
2
3
4
5
 
sorted 
  |> List.fold_left (fun (st,lst) a -> if st = a then (st,lst) else (a,a::lst)) (-1, []) 
  |> snd
  |> List.rev

The call to "snd" at the end drops the first element from the tuple, so we get just the resulting list and finally, "List.rev" reverses the list - similarly to the previous implementation.

Hope this helps!

By on 12/18/2007 5:32 PM ()
IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy
Built with WebSharper