Massive CPU usage FMS 3.5 live streaming

Hi,
Hope this is the right group to post a question.
We have a problem working with FMS 3.5 (installed on a IIS server).
Configuration of the servers all based on IIS:
webserver (authentification, xml data loaded by Flash app, data (on IIS) sended to and recieve from FMS).
Seperate FMS 3.5 (latest update) server. Only running FMS (no other apps are running on this machine).
Starting this project we tested the amount of connections (data flow, CPU, memory) by multiplying it with the aimed number of users (at least 500
simultaneously). Now is seams that this is not the way to do this.
Clients         |             CPU       |    Memory
    10                           2%            12,5 %
    50                           5%            12,5 %
    100                        12%            17,0%
    200                        17%            25,0%
    300                       100%            30,0%
Does anyone have the same problem (high usage of CPU) with a higher amount of connections?
I hoped/figured that the FMS sollution would provide a wider/higer range of connections without the trouble of server capacity.
We use servers with 2 processors (seperate), and 4 gig internal memory, run on 2003 server standard edition (32 bit version).
We are still testing and trying to figure out how to solve this problem. I hope we did something wrong and be able to fix this.
I read a lot of articles on the adobe site (including the optimize FMS for streaming media).
Martijn

You should check out this blog post by Kevin Towes, the FMS Product Manager:
http://blogs.adobe.com/ktowes/2008/08/breaking_1gbps_on_flash_media.html
"Generally network administrators will run a server network adaptor anywhere from 60-80% capacity (average about 700Mbps). This is not the limit of what FMS is capable of. Limiting utilization is useful to allow burst headroom, and to reduce the CPU load. There is a direct relationship between the network adaptor and the server’s CPU. As a network adaptor utilization nears 100%, the server’s CPU utilization significantly increases."
Do you know what percentage of your network card you're using?
Jody

Similar Messages

Trying to find lowest latency setting for FMS 3.0 live streaming app

Hi,
I work for a technology company in the agricultural industry,
and we recently bought Adobe FMS 3.0 for live streaming of an
online auction application. Initially the tests were ok, but the
stream latency is becoming a real issue. It starts at around 1s
delay, which is barely acceptable, but an hour into the sale it
stretches to 2-3s, and more. This is becoming a real deal breaker
for the Flash solution that looked very promising at the beginning.
We need to find the absolute lowest latency setting for the
live streaming application supplied with FMS3.0.
Low latency (1s max) and stable latency has highest priority,
more important than picture quality or uninterrupted streaming.
How can we achieve this? Are there any settings where I can
tweak the stream latency?
I am not a Flash expert, but I can find and modify the
configuration files on the FMS Windows 2003 server.
Any help would be immensely appreciated.

On server side:
FMS installs itself with default configurations optimized for
Max CPU efficiency. Reducing/shutting-off FMS publisher queues
using Application.xml config should help in removing the delay that
you experienced.
<Application>
<StreamManager>
<Live>
<Queue enabled="true">
</Queue>
</Live>
</StreamManager>
</Application>
Changing default setting to <Queue enabled="false">
disables live publisher queues & aggregate messages ad
therefore "the delay". (the delay should be reduced to <100 ms
for a perfect kind of network or LAN)
--> On client side: Do NOT use NetStream.setBufferTime API
at all ! i.e. do not set any buffer on client side.; Flash player
(upto 10, the current version) would try its best to keep the
buffer around 0.1 seconds, which is the default value.
OmicronPersei8,
Which publisher are you using to publish the live stream?
Sometimes , it is possible that a some fault at stream publisher
may make buffer at flash player grow.
I hope this would help !

Fms force record live streams

Hi there,
Just wondering if it is possible to force recording of all streams in a given application ...
So the scenario s that the client is sending live stream, but as well as this we need to recrd that stream . I know I can do sass and create a stream object. But I ned red if there was a better way which would be a simple config option
Many thanks
Matt

Hi Matt,
I don't think you can accomplish DVR without writing a server side script.
You can, however, go through this- the DVR reference guide:
http://help.adobe.com/en_US/flashmediaserver/devguide/WS236AE81A-5319-4327-9E44-310A93CA09 C6Dev.html
Thanks.

We need to optimize FMS for smooth live streaming can anyone help?

So we are coming across the same problem over and over again. I called Adobe FMS tech support and they assisted us to achieve what we need, but we are having two main issues.
First, we need to allow live, clear, and smooth video broadcasting with FMS so we implemented dynamic bandwidth detection so the system can automatically detect a users specifications and adjust video accordingly. This has worked better for us but we are coming across two main issues; first, the video is now very pixilated, and second, we are still get a pause or delay in movement from the stream every 4 seconds or so.
We are looking to get similar video quality as seen on the justin.tv live broadcasting site. I know this video detection and quality is achievable, but not sure what values we need to put in or are missing?
Any help would be greatly appreciated.
Thanks,
Steve

I'm interested in doing something similar - bridging FMS to a
an Asterisk conference. I know breeze has this capability, does
anyone know if this can be done using FMS?

[SOLVED] Xorg & Firefox lock Gnome desktop, causing massive CPU usage

Hi there,
since I have installed ArchLinux x64 on this laptop (Core 2 Duo, 9600M GT with Nvidia's binary driver 195.36.15-2, Gnome 2.30) Xorg spins out of control every so often - more often repeatedly in an hour, sometimes not for a prolonged time.
This appears to occur (so far only) when Firefox is running (with any site, independent of Flash, but looks as if its main window has to be open) with which it then consumes an entire CPU core, making the mouse pointer and any terminal cursor blink rapidly, as well as windows being selectable, but not e. g. resizable or closable. Any keyboard input to a terminal has to be hammered in repeatedly to be accepted.
It also seems as if it is especially prone to happen if another program (say, gThumb) is ordered to switch to full-screen by F11 with FF's window open in the background, which frequently for whatever reason then also wants to change into full-screen, causing "conflict".
Switching to a console and killing Firefox or starting it via terminal and canceling it there ends each occurrence of the issue. Xorg.0.log does not report any errors.
An upgrade to Xorg 1.8 does not appear to have affected it.
I know this bug (or ones like it) has also occurred to others for years now, but I have so far been unable to find any solutions in reports concerning it.
Any help, please?
My xorg.conf (many options in "Device" have been inserted as attempts to solve the issue):
# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig: version 1.0 (buildmeister@builder58) Wed Dec 9 16:34:26 PST 2009
Section "ServerLayout"
Identifier "X.org Configured"
Screen 0 "Screen0" 0 0
InputDevice "Mouse0" "CorePointer"
InputDevice "Keyboard0" "CoreKeyboard"
Option "ignoreABI" "True"
EndSection
Section "Files"
ModulePath "/usr/lib/xorg/modules"
FontPath "/usr/share/fonts/misc"
FontPath "/usr/share/fonts/100dpi:unscaled"
FontPath "/usr/share/fonts/75dpi:unscaled"
FontPath "/usr/share/fonts/TTF"
FontPath "/usr/share/fonts/Type1"
EndSection
Section "Module"
Load "dri2"
Load "extmod"
Load "dbe"
Load "record"
Load "glx"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
Section "InputClass"
Identifier "touchpad catchall"
MatchIsTouchpad "on"
Driver "synaptics"
Option "TapButton1" "1"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Monitor Vendor"
ModelName "Monitor Model"
Option "DPMS" "True"
EndSection
Section "Device"
Identifier "Card0"
Driver "nvidia"
VendorName "nVidia Corporation"
BoardName "G96 [GeForce 9600M GT]"
Option "AllowIndirectPixmaps" "True"
Option "AllowSHMPixmaps" "0"
Option "AddARGBGLXVisuals" "True"
Option "BackingStore" "True"
Option "DPI" "96 x 96"
Option "DynamicTwinView" "1"
Option "NoLogo" "True"
Option "OnDemandVBlankInterrupts" "True"
Option "PixmapCacheSize" "2000000"
Option "RegistryDwords" "PowerMizerEnable=0x1; PerfLevelSrc=0x3333; PowerMizerDefault=0x3; PowerMizerDefaultAC=0x1"
Option "RenderAccel" "True"
Option "TripleBuffer" "True"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
DefaultDepth 24
SubSection "Display"
Depth 24
EndSubSection
EndSection
edit: I cannot say exactly what apparently ended this issue, however (of the changes I explicitly remember) after the following it does no longer occur:
1. Had to delete the extensions.* (cache, ini, rdf) files as they became corrupted.
2. Removed the Fox Splitter (aka Split Browser) extension.
Last edited by misc (2010-06-23 12:14:09)

Daemonjax wrote:Try another browser and see what happens... I like chromium.
Well yes, I though about this too, but that certainly wouldn't solve the problem (esp. were it to appear with other apps); besides, there are just way too many features (extensions, scripts etc.) that Chrome a) does not have or b) would be a royal pain to properly add.
Last edited by misc (2010-04-16 10:01:31)

1 day old MBP, MASSIVE cpu usage when any USB device is plugged in?!?

I just purchased a brand new MBP 2.2
If I plug anything into any of the usb ports my CPU starts clocking and within 10 minutes my system is running it's fans at full speed trying to keep it cool. This can be with no activity AT ALL. I can simply plug a USB thumb drive in and watch as the system gets hotter and hotter!
Has anyone heard of this and is there a solution. I had planned on booting and external USB drive to run a separate OSX install but this makes that impossible. Any help is greatly appreciated.
Kalel

Sorry, it was just taking FOREVER to index my USB drives. One drive is only 120GB and after almost 6 hours it still has 4 hours remaining! For reference it indexed my Firewire drive in about 15 minutes.

Does FMS support live stream switching

I read a blog article (
http://www.peachpit.com/articles/article.aspx?p=665127)
talking about switching between different bit rate video streams of
the same video content on fly. But I can not find such ability in
RTMP's specification. I am wondering if such function has to be
implemented by users with ActionScript? Thanks!
Yue

Yes FMS does support live stream switching but its not at protocol level hence you are not finding it in RTMP specification.
please read details about it here:
http://help.adobe.com/en_US/FlashMediaServer/3.5_Deving/WS5b3ccc516d4fbf351e63e3d11a0773d5 6e-7fea.html
Hope you find it useful.

System Interrupts and High CPU usage.

Hi there, I have a new AcerAspire V15, which worked great until a system update last weekend.
Now I have massive CPU usage spikes for no obvious reason and the highest usage seems to by the System Interrupts. I have scanned and rescanned my laptop and have found no viruses whatsoever.
The issue is so bad that I can not even stream a video from youtube without issue, let alone get any work done, or gaming as was the intention of me buying this laptop.
Any solutions?

Hi MrvV ,
We
may uninstall the update to check if it causes the issue .
Open
Programs and Features in Control Panel ,click
View installed updates and uninstall the update .
If the problem is solved ,we can update the driver of all the hardware components and reinstall the system update again .
After reinstallation ,if the problem still exists ,we can ask Acer support team whether the update will conflict with your model of laptop .
Here is the link for Acer support :
http://www.acer.com/worldwide/support/
Besides ,I suppose your laptop is not installed with Winserver OS .You can get better help in the
following forums :
Window 7 :
https://social.technet.microsoft.com/Forums/windows/en-US/home?category=w7itpro
Window 8.1 :<o:p></o:p>
https://social.technet.microsoft.com/Forums/windows/en-US/home?category=w8itpro
Best Regards,
Leo
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

Lightroom Mobile Sync - Extremely High CPU Usage/Sync Process Causes LR To Lag

Since my other thread doesn't seem to be getting any responses, I'm pasting what I've found here. Please keep in mind I am not a beginner with Lightroom and consider myself very familiar with Lightroom's features excluding the new mobile sync.
1st message:
I'm on Lr 5.5 and using the 30 day trial of Adobe CC to try syncing one collection of slight more than 1000 images. Despite already having generated the Smart Previews, I can see my CPU crunching through image after image (the rolling hills pattern in the task manager) while doing the sync. I was assuming, since I already created the Smart Previews, that the sync of this collection would begin immediately and be done by simply uploading all of the existing Smart Previews. The Smart Previews folder of the catalog is 871MB and has stayed the same despite the CPU obviously doing *something*. As it is now, the sync progress is incredibly slow, almost at a pace like it's actually exporting full-res JPGs from the RAW images (as a comparison only, I know this should not be what it's actually doing).
Another side effect of this is that I'm basically unable to use my computer for other tasks due to the high CPU utilization.
Win 7 x64 / Lightroom 5.5
Intel i5 2500k OC'd 4.5GHz
16GB RAM
SSD for OS, separate SSD for working catalog and files
2nd message:
As a follow up, now Lightroom thinks all 1026 photos are synced (as shown in "All Sync Photographs" portion of the Catalog though all images after the 832nd image show the sync icon per image stuck at "Building Previews for Lightroom Mobile" and the status at the top left corner has been stuck at "Syncing 194 photos" for over 12 hours. Is there no option to force another sync via Lightroom Desktop and also force the iOS app to manually refresh (perhaps by pulling down on the collections view, like refreshing via the Mail app)?
3rd message:
One more update, I went into Preferences and deleted all mobile data, which automatically signed me out of Adobe CC and then I signed back in. Please keep in mind the Smart Previews were long generated before even starting the trial, and I also manually generated them again (it ran through quickly since it found they were already generated) many times. Now that I'm re-syncing my collection of 1026 images, I can clearly see Lightroom using the CPU to regenerate the Smart Previews which already exist. I have no idea why it's doing this except that it's making the process of uploading the Smart Previews extremely slow. I hope this time around it will at least sync all 1026 images to the cloud.
4th message:
All 1026 images synced just fine and I could run through my culling workflow on the iPad/iPhone perfectly. Now I'm on a new catalog (my current workflow unfortunately uses one catalog per event) and I see the same problem: Smart Previews already generated but when syncing, Lightroom seems to re-generate them again anyway (or take up a lot of CPU simply to upload the existing Smart Previews). Can anyone else chime in on what their CPU utilization is like during the sync process when Smart Previews are already created?
New information:
Now I'm editing a catalog of images that is synced to Lightroom Mobile and notice that my workflow has gotten even slower between photos (relative to what it was before, this is not a discussion about how fast/slow LR should perform). Obviously Lightroom is syncing the edited settings to the cloud, but I can see my CPU running intensively (all 4 cores) on every image I edit and the CPU utilization graph looks different than before I started using LR mobile sync. It still feels like every change isn't simply syncing an SQLite database change but re-generating a Smart Preview to go with it (I'm not saying this is definitely what's happening, but something is intensively using the CPU that wasn't prior to using LR Mobile).
For example: I only update the tint +5 on an image. I see the CPU spike up to around 30-40%, then falls back down, then back up to 100%, then back down to another smaller spike while Lightroom says "Syncing 1 photo". I've attached a screenshot of my CPU graph when doing this edit on just one image. During this entire time, if I try to move onto edit another image, the program is noticeably slower to respond than it was prior to using LR mobile, due to the fact that there appear to be much more CPU intensive tasks running to sync the previous edit. This is proven by un-syncing the collection and immediately the lag goes away.
I'd be happy to test/try anything you have in mind, because it's my understanding that re-syncing photos that are edited that are already in the cloud should be simply updating the database file rather than require regenerating any Smart Previews or other image data. If indeed that's what it should be doing, then some other portion of LR is causing massive CPU usage. If this continues, I will probably not choose to proceed with a subscription despite the fact that i think LR mobile adds a lot of value and boosts my workflow significantly if it wasn't causing the program to lag so badly in the process.
I know this message was incredibly long and probably tedious to read through so thanks in advance to anyone who gets through it
-Jeff

Thanks for reporting. Just passed along your info to some of our devs. One of the things that needs to be created (beside smart previews) during an initial sync are thumbnails + previews for the LrM app - Guido
Hi Guido,
Thanks for pointing this out. I realized the same thing when I tried syncing a collection for offline mode and found out the required space sounded more like Previews + Smart Previews rather than just the Smart Previews.
greule wrote:
Hi Jeff, are your images particularly large or do you make a lot of changes which you save to the original file as part of your workflow?
The CPU usage is almost certainly from us uploading JPEG previews not the Smart Previews - particularly during develop edits as these force new JPEG previews to be sent from Lightroom desktop, but would not force new Smart Previews (unless the develop edits are modifying the original file making us think the Smart Preview is out of date) to be sent.
Guido
My images are full-resolution ~22mp Canon 5D Mark III RAW files so they're fairly large. Even if I only make one basic change such as exposure changes, I saw the issue. By "save to the original file" I'm assuming metadata changes such as timestamps, otherwise edits to the images aren't actually written to the original file. I'm only doing develop module edits so I shouldn't be touching the original file at all at this point in my workflow.
I think it makes sense now that you mention that new JPEG previews need to be generated and sent to the cloud due to updated develop edits. My concern is that this seems to be done in real-time as opposed to how Lightroom Desktop works (which is to render a new Standard Preview or 1:1 Preview on demand, which means only one is being rendered at any given time while viewing it in Loupe View or possibly 2 in Compare View). If I edit, for example, 10 images quickly in a row, once the sync kicks in a few seconds later, editing the 11th image is severely hindered due to the previous 10 images' JPEG previews being rendered and sync'd to the cloud (I'm assuming the upload portion doesn't take much CPU, but the JPEG render will utilize CPU resources to the fullest if it can). Rendering Standard/1:1 Previews locally and being able to walk away while the process finishes works because it is at the start of my workflow, but having to deal with on-the-fly preview rendering while I'm editing greatly impacts my ability to edit. Perhaps there can be a way to limit max CPU utilization for background sync tasks?
It may help to know that I'm running a dual-monitor setup, with Lightroom on a 27" 2560x1440 display maximized to fit the display (2nd display not running LR's 2nd monitor). Since I'm using a retina iPad, the optimal Standard Previews resolution should be the same at 2880 pixels.
Thanks again for the help!

High CPU Usage With 2.6.32

Hi
Ever since the kernel update my CPU usage has its own will. Would it normally stay under 10%, does it now ad 20% while only moving a window around. (nothing else open) Scrolling up and down got me to 95%! (with firefox 1 tab open/nothing else open) And as you can imagine this is getting annoying. Is there a solution? Someone who could help me?
I have Openbox as a WM. No DE. /usr/bin/X -nolisten tcp uses almost always around 5% with tops to 40%.
The problem occurs with my Acer Extensa 5230.
Processor
Intel® Celeron® Processor 900
(1MB L2 cache, 2.20GHz, 800MHz FSB)
Chipset
Mobile Intel® GL40 Express
Any more information? Logs? Errorfiles?

Hi Klaus1
I have several plug-ins by cfx and the slick volumes 4, 6 8 and 10. I can post a list of all of them if you like. Up to now they never made any trouble, but you'll never know...
The edit in the timeline is about 25 minutes long. But the project file contains captured dv-footage of about 2 hours, so that alone adds up to something like 26 GB. Then there are a few rendered effect-clips done with some of the plug-ins like stabilizing or color correction which also add to the 30 GB, as well as some audio-files (some music, my voice-over commentary). I emptied the trash several times while working on the movie. At this moment there are only a few MB of erased audio in it.
It all sounds a bit like this is an insanely huge and complex project, but as i said, i have done projects like this in imovie without the massive cpu-usage. I'd like to continue to use imovie, but if this problem keeps coming up with future projects, i might as well take the step and switch to Final Cut Express.
For now I will try your idea of exporting the movie as it is now and open it in a new project file.
Thanks for your help and input so far!

Unusually high cpu usage with iMovie 6.0.3

I'm having trouble with very high cpu-usage. Just the open imovie app is taking up about 50%, during playback it sometimes peaks over 100%! The project file is pretty big (about 30 gig), but that is mainly because of the amount of footage imported into it. The movie itself is about 25 minutes in length. There is some audio editing going on: some soundeffects, musicfiles (no drm-protection), audio commentary that i recorded within imovie. I know that imovie has trouble when there is a lot of audio/audio editing involved and this project is probably a little more complex than other stuff i did in imovie, but i never had such problems. i can hardly use playback anymore. I recently updated Quicktime to 7.6.4 and Perian to version 1.2.1. Any ideas what's going on or how to work around the problems? Since the movie is about 80% finished i don't really want to start all over again. Thanks in advance for your input!

Hi Klaus1
I have several plug-ins by cfx and the slick volumes 4, 6 8 and 10. I can post a list of all of them if you like. Up to now they never made any trouble, but you'll never know...
The edit in the timeline is about 25 minutes long. But the project file contains captured dv-footage of about 2 hours, so that alone adds up to something like 26 GB. Then there are a few rendered effect-clips done with some of the plug-ins like stabilizing or color correction which also add to the 30 GB, as well as some audio-files (some music, my voice-over commentary). I emptied the trash several times while working on the movie. At this moment there are only a few MB of erased audio in it.
It all sounds a bit like this is an insanely huge and complex project, but as i said, i have done projects like this in imovie without the massive cpu-usage. I'd like to continue to use imovie, but if this problem keeps coming up with future projects, i might as well take the step and switch to Final Cut Express.
For now I will try your idea of exporting the movie as it is now and open it in a new project file.
Thanks for your help and input so far!

Time shifting and record live stream

Hellow everyone,
I would like to stream videos for my viewers and them have a abiliti to timeshifting live stream.
Can I use live application for timeshifting? Or I should use livepkgr to do that.
For archive and play on the VOD mode I like to stream live and record at same time. in the livepkgr I see in the stream folder create a stream file with the f4f extention automaticaly whoever in the live application I should call application.record on the FMS to save live stream on the server.
Please advise me the better solution to archive and time shifting.
Many Thanks.

livepkgr application for for HTTP streaming...
f youjust want to stream over rtmp use live application and code to record the stream...recording is pretty simple in AMS just call Stream.get() s.record(...) s.play() where s is the stream which you are recording...
application.onPublish = function(clientObj, streamObj) { trace("recording started..."); var strName = "recorded_" + streamObj.name; var s = Stream.get(strName); if (s == undefined ) return; s.onStatus = function(info) { trace(info.code); } if (!s.record("record")) { s.trace("record failed."); } s.play(streamObj.name, -1, -1, true); }

FMS 4 configuration / High CPU usage problem

I am seeing extremly high CPU usage issues with the FMS 4 server. It seems to be a problem with the FMS Edge process? If you start and stop the service, the processor goes back to normal operations.
Looking in the Windows Event logs I am seeing these errors:
Bad network data; terminating connection : bad chunk version 104 on input stream 00000000077A2E80
Also:
Bad network data; terminating connection : (Adaptor: _defaultRoot_, VHost: Unknown, IP: , App: , Protocol: tunnel) : h o s t = % 2 2 % 2 0 % 2 D F O O B A R % 7 C e c h o % 2 0 % 2 2 % 2 0 S h a r e n a m e % 2 2 % 0 A e c h o % 0 A e c h o % 2 0 % 2 2 % 2 0 % 2 0 S o m e S h a r e % 2 0 % 2 0 D i s k % 2 0 % 2 2 % 6 0 i d % 6 0 % 2 0 % 2 3 % 2 2
Can someone shed some light on what might be causing these issues?
Running on a virtual server/ Win2k8R2
Thanks, Tandrist

Hi
Can you please provide us some little more environment information on this issue, like:
- what were you trying to do when you saw this issue, if there was a live telecast or if that was a simple vod workflow?
- What publisher application were you using to send livestream to FMS?
- were you also doing some kind of recording while doing live streaming??
- In case of live streaming what audio/video encoding and bitrates was there in use?
- how many number of connections were there?
- What amount of cpu and mem usage was there (when you say very high cpu usage).. you can see it using task manager
It will help if you can also share your fms logs (inside fms_install_folder/logs) so that we can see if something abnormal was going on there..

Live stream problem with FMS 4.5 + FMLE

Hi,
I've got a live stream (and DVR) setup using FMS 4.5 and FMLE.
The hardware for the FMS server we use is 2x Xeon E5645 2.4GHz, 32GB RAM, OCZ RevoDrive R3 X2 PCI-e SSD. Installed OS is CentOS 5.8.
We have 6 workstation PCs, each equiped with 4 BlackMagic Intensity PRO capture cards and FMLE 3.2. Using these PCs we stream Live video to FMS server.
Video stream is grabbed from HDMI cable, to BlackMagic Cards, which are encoding by FMLE using H.264 format, 700kbps bitrate per stream.
Video stream is published to server through RTMP stream, which is later saved for DVR in RAW format. We only keep 3 days of recording and delete the old ones.
Server and Encoder PCs are in the same network, connected by gigabit managed switch.
The problem I'm having is that after 10-15 hours FMLE starts to drop frames, because video buffer is increased. What I observed is that this happens immediately when Server CPU load increases above 60%.
Based on the above observation I decreased the number of channels streamed by the server to 10, which reduced CPU load. But the problem still persists.
Whenever I restart FMS and delete all DVR data, the CPU load (when streaming 10 live channels) is only 1%, but after 2 days CPU load increases to 50-60%.
Whenever I restart FMS and don't delete DVR data, the CPU load is 5-10%, and after 2 days it still increases to 60-70%.
Another thing I observed is that there is only single fmscore process running, but it has lots of threads which are switched on and off in split seconds. These threads are launched on different CPU cores, but at any given point in time the distribution of the load isn't equal among CPU cores. This leads certain CPU cores being loaded by more than 60% and frame drops start to occur.
For the moment there are just 10 users watching this service, so I don't think this load accounts for the problem.
Has anybody had similar problem, or know how can I optimize or finetune the system to run without problem? I would appreciate any suggestions.
Another thing I noted through last couple of days:
When I was restarting FMS the CPU load was reduced to 30%, but after 1 week past when I restart it CPU load only goes down to 75%. Everything is the same, nothing has been changed and there is no disk IO issues involved.
P.S. I've modified application.xml using these values:
<Scope>vhost</Scope>
<Distribute numprocs="5">app</Distribute>
<LifeTime>
<RollOver></RollOver>
<MaxCores></MaxCores>
</LifeTime>

Hi
How many channels are you publishing?
If there are too many channels, it is recommended to have one FMSCore process start for each of them. To do so, you will have to change the scope to app.
---snippet---
<Application>
<Process>
<Scope>app</Scope>
<Distribute numprocs="3">inst</Distribute>
</Process>
</Application>
Also, to delete older content, you will have to enable disk management. Refer http://help.adobe.com/en_US/flashmediaserver/devguide/WSeb6b7485f9649bf23d103e5512e08f3a33 8-8000.html#WSec225f632fa00875-23954b6f1300b641158-8000 for more info.

Live stream on FMS issues when reaching 1600 concurrent users

Hey all,
We have an FMS 4.5.2 installation with 1x Origin and 2x Edge setup only for Live streaming. We are using FMLE latest version to stream to the Origin server and Edge servers are connecting to the Origin server. We are running four applications stream1, stream2, stream3 and stream4 all for live streaming (copied live) application. All servers have default configuration except edge/origin setup.
All servers have 4Gbit connections in bonding and network traffic is nicely distributed across all bonds and uplink is 10Gbit to the ISP.
Server is a 2 CPU / Quad Core HP DL 380 with 64Gbit of memory running Ubuntu 10.04 LTS
Now to the problem.
We are streaming live stream from FMLE with 700kbit/s to for example stream1 application and when we hit around 1600 concurrent users with 50% on edge1 and 50% on edge2 (800 concurrent users on each edge) in the player buffer drops and all people experience buffering issues at approx 30secs - 2minutes intervals. (network is not congested because if at that time do an SCP from the server outside network it copies 1Gbyte file within seconds.
Funny thing is that If I start a second FMLE and stream live stream to for example stream2 application at same time and open up second player on client the video runs great without any buffer issues from the same server at the same time.
edge1 server:
application: stream1 with 800 concurrent users, player has buffer issues
application: stream2 with 4 concurrent users, player has no issues
edge2 server
same as above
So my current conclusion is that it has to be something per application since other application does not have this issues when running simultaneously? We tried changing fmscore number settings and some buffer settings but nothing helped.
at the time when we get buffering issues this are the only problematic things that get in the log and its in edge.00.log on both EDGE servers but not at same times:
Edge1:
2012-04-23    12:50:21    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    12:55:30    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    12:56:42    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    12:56:42    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    13:14:40    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    13:20:30    29270    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
Edge2:
2012-04-23    12:56:32    9625    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    13:02:23    9625    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
2012-04-23    13:08:03    9625    (e)2661034    Connect failed ( , 8134 ) : Connect failed: Connection refused (111)    -
there is no packet loss between edge and origin servers and latency is at 0.2ms and nothing in the logs of the origin server
We even tried to deploy Wowza Medis Servers with edge / origin setup and were able to handle around 4000 (2000 on one and 2000 on second edge) concurrent users without any issues.
Anyone has any ideas or at least what are our next options to do and what settings to change on the FMS? or how to debug and what to check when buffering issue appears? any more debugging we can enable on the FMSs? too se if we hit some kind of limit somewhere?
thanks

hey, i got the similar problem, logging like this
2012-11-12
18:50:12
23434
(e)2661034
Connect failed ( , 5779212 ) : Connect failed: Connection refused (111)
2012-11-12
18:50:54
23434
(e)2661034
Connect failed ( , 5779212 ) : Connect failed: Connection refused (111)
2012-11-12
18:51:36
23434
(e)2661034
Connect failed ( , 1166880400 ) : Connect failed: Connection refused (111)
2012-11-12
18:54:14
23434
(e)2661034
Connect failed ( , 1175301776 ) : Connect failed: Connection refused (111)
2012-11-12
18:54:55
23434
(e)2661034
Connect failed ( , 1164775056 ) : Connect failed: Connection refused (111)
2012-11-12
18:55:37
23434
(e)2661034
Connect failed ( , 16 ) : Connect failed: Connection refused (111)
2012-11-12
19:13:08
23434
(e)2661034
Connect failed ( , 1158459024 ) : Connect failed: Connection refused (111)
it seems that the port number is invalid, but we never use such ports.

Massive CPU usage FMS 3.5 live streaming

Similar Messages

Maybe you are looking for