Back to Questions
3.7k
views4
comments
Stuck with sockets - please help
f#testingserverocamlsockets

Hello,

I am porting an ocaml game ([link:freetennis.sf.net)] to F# and .NET. I have been completely stuck for two days with what seems a socket problem, and I am in desperate need of help. :(

At each game frame (i.e. each iteration of the main loop), the client and the server must exchange, via sockets and TCP, an information packet (the state of the simulated world: the position of the players, etc). The number of frames per seconds (FPS) therefore depends on the round-trip speed (the time it takes for a packet to travel from client to server and back). The higher the FPS, the smoother the game is.

Porting the game to F#, and testing the game on my LAN, I discovered a huge latency which wasn't in the ocaml version, resulting in a very jerky motion and an unplayable game. A round trip of the packet was taking as much as 0.33 seconds, resulting in just 3 frames per second. On a LAN! (But only when client and server process are on two different machines. If they are on the same machine, the fps is much higher.)

I then created a test application in both ocaml and F#, which is attached below. The application simply serializes and transfers a pair of integers from client to server and back, at each frame. The client and server processes are on two different machines on a LAN. Here is the result:

With ocaml, the test application does 4700 frames per second.

With F#, the test application does about 3 frames per second.

The serialization is done with Marshal.to_channel in ocaml, BinaryFormatter.Serialize() in F#.

I also tried to rewrite everything so as not to use BinaryFormatter and Serialize/Deserialize, but only BinaryWriter and BinaryReader. This was a huge effort, involving serializing dozens of discriminated unions by hand, but in the end there was no gain in speed.

I then tried writing the test application in Visual Basic, and found it to have the same problem. So this may not be a F# problem, but a .NET problem with sockets. However I don't know what else to try. I am considering switching back to ocaml. Could somebody please tell me what to do or try? :)

Thanks a lot for any help.

Here is the test application in both ocaml and F#:

========================
F# version
========================

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

open System.Runtime.Serialization.Formatters.Binary
open System.Net.Sockets

let rec mainLoopServer ns time0 frameCount = 
    let time1 =
        Sdl.SDL_GetTicks ()   
    in
    let timePassedSinceStartup = (* in seconds *)
        (float_of_int time1 - time0) / 1000.0  
    in
    let fps = float_of_int frameCount / timePassedSinceStartup in
    print_endline ("frames per second = " ^ (string_of_float fps));

    let formatter = new BinaryFormatter() in
     let ( pack: int * int)  = 
        unbox (formatter.Deserialize(ns)) 
    in

    formatter.Serialize(ns,  box pack);
    mainLoopServer ns time0 (frameCount + 1)

let initServer port = 
     let tcpL = new TcpListener(System.Net.IPAddress.Any, port) in 
    tcpL.Start();
    print_endline ("server: started listening to port " ^ (string_of_int port) ^ ".");

    let  ns = 
        let clientSocDn = 
            tcpL.AcceptSocket ()  (* blocking *) 
        in
        print_endline "server: client accepted";
        new NetworkStream(clientSocDn) 
    in
    mainLoopServer ns (float_of_int (Sdl.SDL_GetTicks ())) 0 

let rec mainLoopClient ns = 
    let formatter = new BinaryFormatter() in

    let pack = 
        (3,4)
    in
    formatter.Serialize(ns,  box pack); 

     let ( pack: int * int )  =
        unbox (formatter.Deserialize(ns))
    in
    
    mainLoopClient ns


let initClient serveraddr port = 
    let socketForServer = new TcpClient(serveraddr, port) in
        
    print_endline ("client: connected to server at address " ^ serveraddr ^ " and port " ^ (string_of_int port) ^ "."); 
    let ns = socketForServer.GetStream() in

    mainLoopClient ns
      
    
let _ = 

    let server = ref false in
    let serveraddr = ref "localhost" in
    let port = ref 5000 in
    let arguments = [ "-server", Arg.Set server, "Whether to call this program as a server or a client.";
              "-serveraddr", Arg.String (fun x -> serveraddr := x), "The address of the server. Examples: 192.168.1.2, localhost.";
              "-port", Arg.Int (fun x -> port := x), "The TCP port to use. Default is 5000."] in

    let anonFun s =
        print_endline "No anonymous arguments allowed." 
    in
    Arg.parse arguments anonFun "Usage:";

    if !server then
        initServer (!port)
    else
        initClient (!serveraddr) (!port)

=========================
OCAML VERSION
=========================

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

open Unix

let rec mainLoopServer time0 inChan outChan frameCount = 
    let time1 = Unix.gettimeofday () (* in seconds *) in

    let timePassedSinceStartup = time1 -. time0 in
    let fps = (float_of_int frameCount) /. timePassedSinceStartup in
    print_endline ("Frames per second = " ^ (string_of_float fps));

     let pack  = 
        (Marshal.from_channel inChan : int * int)
    in

    Marshal.to_channel outChan pack [];
    flush outChan ;

    mainLoopServer time0 inChan outChan (frameCount +1)

let rec mainLoopClient  inChan outChan = 
    let pack = 
        (3,4)
    in
    Marshal.to_channel outChan pack [];
    flush outChan ;

     let _pack  =
        (Marshal.from_channel inChan : int * int)
    in
    
    mainLoopClient  inChan outChan

let initServer port = 

    let soc = Unix.socket Unix.PF_INET Unix.SOCK_STREAM 0 in
    let clientSocket, _  = 
    ( 
        Unix.setsockopt soc SO_REUSEADDR true;
        (try
            Unix.bind soc (Unix.ADDR_INET (Unix.inet_addr_any, port))
        with Unix.Unix_error (err, _, _) ->
            (print_endline ("Error: " ^(Unix.error_message err) ^ ".");
             exit 0 ));
        Unix.listen soc 5;
        print_endline "server: waiting for client to connect...";
        Unix.accept soc 
    )
    in
    let inChan =  Unix.in_channel_of_descr clientSocket
    and outChan = Unix.out_channel_of_descr clientSocket in
    mainLoopServer  (Unix.gettimeofday ()) inChan outChan 0

let initClient ( serveraddr : string) port = 
    
    print_endline "client: connecting to server";
    let soc = Unix.socket Unix.PF_INET Unix.SOCK_STREAM 0 in
    Unix.connect  soc (Unix.ADDR_INET  (Unix.inet_addr_of_string serveraddr, port) );
    print_endline "client: connected to server";

    let inChan = Unix.in_channel_of_descr soc
    and outChan = Unix.out_channel_of_descr soc in

    mainLoopClient inChan outChan
    
let _ = 

    let server = ref false in
    let serveraddr = ref "localhost" in
    let port = ref 5000 in
    let arguments = [ "-server", Arg.Set server, "Whether to call this program as a server or a client.";
              "-serveraddr", Arg.String (fun x -> serveraddr := x), "The address of the server. Examples: 192.168.1.2, localhost.";
              "-port", Arg.Int (fun x -> port := x), "The TCP port to use. Default is 5000."] in

    let anonFun s =
        print_endline "No anonymous arguments allowed." 
    in
    Arg.parse arguments anonFun "Usage:";

    if !server then
        initServer (!port)
    else
        initClient (!serveraddr) (!port)

Maurizio

.

I did it! It was indeed a bug in the .net framework (not in F#). I tried serializing to a MemoryStream instead of a NetworkStream, and then writing the bytes to the NetworkStream. It worked. It is fast.

This works:

1
2
3
4
5
6

 let bf = new BinaryFormatter() in
  let ms = new MemoryStream () in
  bf.Serialize(ms, box pacchettoIniziale);
  let bytes = ms.GetBuffer () in
  ns.Write(bytes, 0, System.Convert.ToInt32(ms.Length));
  ns.Flush()

This does not:

1
2
3

 let bf = new BinaryFormatter() in
  bf.Serialize(ns, box pacchettoIniziale);
  ns.Flush()

(also note that, in my benchmark application, the flush command is missing)

Now only one question remains: why does my game crash on Deserialize if compiled with optimizations enabled?

However thank you very much for your support. Without your second test application, I would not have discovered the solution. :)

Regards
Maurizio

By on 8/7/2007 10:01 AM ()

Good question!

I just tried the following code on my laptop and got 430 communications/sec. It's a #light version of the above with the SDL.getTicks call replaced.

I'll look into this a bit more and encourage other people to do the same, given the huge discrepancies we're seeing.

don

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

 

open System.Runtime.Serialization.Formatters.Binary
open System.Net.Sockets


let rec mainLoopServer ns (time0:System.DateTime) frameCount = 
    let time1 = System.DateTime.Now
    let diff = time1 - time0
    let fps = float frameCount / float diff.TotalSeconds
    printfn "frames per second = %f" fps;

    let formatter = new BinaryFormatter() 
    let pack  = unbox<int*int>(formatter.Deserialize(ns)) 
    formatter.Serialize(ns,  box pack);
    mainLoopServer ns time0 (frameCount + 1)


let initServer port = 
    let tcpL = new TcpListener(System.Net.IPAddress.Any, port) 
    tcpL.Start()
    printfn "server: started listening to port %d." port;

    let  ns = 
        let clientSocDn = tcpL.AcceptSocket ()  (* blocking *) 
        printfn "server: client accepted";
        new NetworkStream(clientSocDn) 

    mainLoopServer ns System.DateTime.Now 0 

let rec mainLoopClient ns = 

    let formatter = new BinaryFormatter() 

    let pack = (3,4)
    formatter.Serialize(ns,  box pack); 
    let pack = unbox<int*int> (formatter.Deserialize(ns))
    
    mainLoopClient ns


let initClient serveraddr port = 
    
    let socketForServer = new TcpClient(serveraddr, port) 
        
    printfn "client: connected to server at address %s and port %d." serveraddr port; 
    let ns = socketForServer.GetStream() 

    mainLoopClient ns
      
    
let _ = 

    let server = ref false 
    let serveraddr = ref "localhost" 
    let port = ref 5000 
    let arguments = [ "-server", Arg.Set server, "Whether to call this program as a server or a client.";
              "-serveraddr", Arg.String (fun x -> serveraddr := x), "The address of the server. Examples: 192.168.1.2, localhost.";
              "-port", Arg.Int (fun x -> port := x), "The TCP port to use. Default is 5000."] 

    let anonFun s =
        print_endline "No anonymous arguments allowed." 
    Arg.parse arguments anonFun "Usage:"

    if !server then initServer (!port)
    else initClient (!serveraddr) (!port)
By on 8/6/2007 1:31 PM ()

This one gave me 9000/s when "bytes" is set to "true" (i.e. transacting single bytes). Of course OCaml will do better here too - I haven't tried that. It also gave me 1650/s when "bytes" is set to false. Here we only report the result every 10000 iterations, indicating there is some small overhead to the DateTime.Now calls etc. BTW my machine is a dual-core Intel laptop, Dell Latitude 420.

Of course this doesn't explain why you're getting 3 communications/sec on your version of the micro benchmark, nor poor performance on the overall program.

It's possible binary serialization is at fault for the poor performance on your larger program. It would be interesting to know the overall number of bytes being sent to see how much greater it is in the case of .NET. The .NET binary serialization format is considerably fatter than the OCaml binary serialization format, because it sends type information, and rechecks that type information upon reading - the OCaml one does not. There can be a difference of 10x in the sizes of the formats.

If that's the problem then replacing binary serialization is a good way to go, though you will still have to make sure the actual number of bytes being sent is low enough. Also, manual serialization may not necessarily improve performance unless you preserve the "graph" of data - both .NET and OCaml binary serialization preserve the underlying object graph should there be any internal sharing in that graph.

regards

don

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

 

#light

open System.Runtime.Serialization.Formatters.Binary
open System.Net.Sockets
open System.IO

let bytes = false

let rec mainLoopServer (ns: #Stream) (time0:System.DateTime) frameCount = 
    if frameCount % 10000 = 0 then 
      let time1 = System.DateTime.Now
      let diff = time1 - time0
      let fps = float frameCount / float diff.TotalSeconds
      printfn "frames per second = %f" fps;

    if bytes then 
        ns.WriteByte 0uy
        ns.ReadByte() |> ignore
    else
        let formatter = new BinaryFormatter() 
        let pack  = unbox<int*int>(formatter.Deserialize(ns)) 
        formatter.Serialize(ns,  box pack);
    mainLoopServer ns time0 (frameCount + 1)


let initServer port = 
    let tcpL = new TcpListener(System.Net.IPAddress.Any, port) 
    tcpL.Start()
    printfn "server: started listening to port %d." port;

    let  ns = 
        let clientSocDn = tcpL.AcceptSocket ()  (* blocking *) 
        printfn "server: client accepted";
        new NetworkStream(clientSocDn) 

    mainLoopServer ns System.DateTime.Now 0 

let rec mainLoopClient (ns: #Stream) = 

    let formatter = new BinaryFormatter() 

    if bytes then 
        ns.ReadByte() |> ignore
        ns.WriteByte 0uy
    else 
        let pack = (3,4)
        formatter.Serialize(ns,  box pack); 
        let pack = unbox<int*int> (formatter.Deserialize(ns))
        ()
    
    mainLoopClient ns


let initClient serveraddr port = 
    
    let socketForServer = new TcpClient(serveraddr, port) 
        
    printfn "client: connected to server at address %s and port %d." serveraddr port; 
    let ns = socketForServer.GetStream() 

    mainLoopClient ns
      
    
let _ = 

    let server = ref false 
    let serveraddr = ref "localhost" 
    let port = ref 5000 
    let arguments = [ "-server", Arg.Set server, "Whether to call this program as a server or a client.";
              "-serveraddr", Arg.String (fun x -> serveraddr := x), "The address of the server. Examples: 192.168.1.2, localhost.";
              "-port", Arg.Int (fun x -> port := x), "The TCP port to use. Default is 5000."] 

    let anonFun s =
        print_endline "No anonymous arguments allowed." 
    Arg.parse arguments anonFun "Usage:"

    if !server then initServer (!port)
    else initClient (!serveraddr) (!port)


// Writing single bytes: 7400    
// Writing serialized bytes: 1400    
// Writing serialized bytes, with time test every iteration: 400
By on 8/6/2007 2:06 PM ()

Don,

Thank you for your attempt. Unfortunately it seems I need more help :).

You say you had both processes running on the same machine? But that is fast for me too. What is slow is running server and client on different machines. In this case I get 3 frames per second. Sorry for not mentioning this in my first post. (By the way, this does not seem to be a problem in my LAN, because it does not happen with ocaml, and because I tried two different LANS.)

Now to the result of your benchmarks:

When I run your _first_ benchmark application on two different machines in my LAN, I get 3.1 round-trips per second, which is no good news.

When I run your _second_ benchmark application on two different machines:

with bytes = true, I get 4100 frames per second, which is fast. That might be a clue.

with bytes = false, I get fps = 0 and then the program seems to hang for a minute. This means that frame #10000 never arrives. Which probably means we are actually getting 3 fps, as before.

As for the size of packet in my real game: it is small. I don't know how to measure it, however it contains two dozen floats, a couple of strings of about 20 characters, and a dozen discriminated unions. But that does not seem important since the problem is happening in the benchmark too, which only exchanges 2 floats.

Another thing that mght be a clue is that I have to compile my game with optimizations disabled (-Ooff). If I try to run it with any other optimization level, it crashes when I call Deserialize. I did not try the benchmark application with optimizations enabled, to see if it too crashes. Maybe I should.

What do you suggest to try next? I was wondering... is it possible to serialize the packet onto a string (as opposed to a networkStream), and then get the bytes of the string, and send them with NetworkStrem.WriteBytes? thanks again.

Maurizio

By on 8/6/2007 11:25 PM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper