FMIS restarting and memory issue

Question

I have Flash Media Interactive Server 3.5.5 on Windows 2008 server. I developed video chat application (many to many), it's only 1 FMS application with about 50-100 room instances and 1 lobby instance to which all users are connected. There are up to 600 - 700 users at a time which produce about 2000-2500 connections to FMS.
Problem I have is with steady raise of memory usage, which if goes little above 2GB will make FMS unresponsive and it would restart itself after about 1-2hours. To avoid that Im restarting FMS manualy every 3-4 days. Interesting thing I noticed is that if I restart FMS from admin console, used RAM memory won't fall at levels as if it would right after whole Windows server is restarted.
In my Application.xml there are following configs:
<Scope></Scope>
<Distribute numprocs="3">Instances</Distribute>
These are things from logs during FMS unresponsive period or right before it:
master.log
2011-05-21    22:01:11    996    (i)2581223    Core (3592) is no longer active.    -
2011-05-21    22:01:16    996    (i)2581223    Core (1868) is no longer active.    -
admin.log
2011-05-21    22:01:06    2832    (i)2581242    Core (3592) disconnected : _defaultRoot_:_defaultVHost_:::_0.    -
2011-05-21    22:01:09    2832    (i)2581242    Core (1868) disconnected : _defaultRoot_:_defaultVHost_:::_1.    -
edge.log
2011-05-21    14:22:23    840    (e)2661034    Connect failed ( , -1 ) : Connect failed: (10061)    -
2011-05-21    14:23:05    840    (e)2661034    Connect failed ( , -1 ) : Connect failed: (10061)    -
2011-05-21    22:01:06    840    (i)2581250    Edge disconnected from core (3592).    -
2011-05-21    22:01:09    840    (i)2581250    Edge disconnected from core (1868).    -
access.log
session    disconnect    2011-05-21    21:29:06    6632    68.191.113.251    29476    39743    -    -    -    -    200    -
session    disconnect    2011-05-21    21:29:06    6632    72.198.193.1    4759    6837    -    -    -    -    200    -
session    disconnect    2011-05-21    21:29:06    6632    83.70.249.91    11996    14797    -    -    -    -    200    -
session    disconnect    2011-05-21    21:29:06    6632    108.8.209.2    57227    76111    -    -    -    -    200    -
session    disconnect    2011-05-21    21:29:06    6632    212.183.140.17    3073    3595    -    -    -    -    200    -
Please help me with this issue.

Answer 1

Thanks for your answers, I'm replying with delay because I wanted to check these things some more.
When I said 'memory is not less', I meant that total memory in Windows after FMS restart doesn't return to memory levels when Windows is restarted, but I guess that's because of some Windows processes and not connected to FMS.
As for FMS processes, these are the things are notice, few hours after FMS restart I get 1 FMSCore.exe that is around 200-220MB, and 3-4 FMSCore's that are around 40-70MB. Size of that largest FMSCore.exe doesn't go down even when there are much less users connected to FMS. And like I said before, after 3 or more days of running FMS becomes unresponsive, and restarts itself. Right after FMS restart largest FMSCore.exe weights around 70MB but 15mins after it's already 120MB as users connect to video chat application.
As for logs, my core.log files have several lines like:
2011-07-04    03:59:24    3308    (e)2611392    Exception while processing message: Unexpected virtual message from client 6446421578071032403: 93 { # 92 " : t 00 00 00 02     -
2011-07-04    03:59:24    3308    (e)2611029    Bad network data; terminating connection : chunkstream error:message length 12086012 is longerthan max rtmp packet length    -
2011-07-04    03:59:24    3308    (e)2631029    Bad network data; terminating connection : (Adaptor: _defaultRoot_, VHost: _defaultVHost_, IP: 94.4.57.121, App: chatcity/stream, Protocol: rtmp, Client: 6446421578071032403, Handle: 2497053121) : 88 1E & D7 FB / 8D 7 99 CB X ...
2011-07-02    14:09:57    1228    (e)2611363    Server detected a backward timestamp from 7990 to 7985 in file: C:\Program Files (x86)\Adobe\Flash Media Server 3.5\applications\chatcity\streams\_definst_\captured_16627033    -
While edge.log has:
2011-07-04    04:58:56    1384    (e)2661034    Connect failed ( , 8134 ) : Connect failed: (10061)    -
2011-07-02    21:09:05    6864    (e)2661034    Connect failed ( , -1 ) : Connect failed: (10061)    -
And master.log has:
2011-07-04    00:30:27    7128    (i)2581223    Core (6280) is no longer active.    -
2011-07-04    00:30:27    7128    (i)2581223    Core (4352) is no longer active.    -