Poor *First Launch* Performance with AppV 5

Question

I'm having an issue where the first time I launch a rather large AppV 5 application the performance I get is very slow.  Subsequent launches, however, are near native.  We reboot our Citrix server's daily so this is causing some issues for us.
 I have noticed that the applications that launch slower tend to be the larger applications we sequence.  Performance is slow regardless whether the package is fully mounted or streamed.
I've detailed it here with some animated gifs showing the performance slowness.
http://trentent.blogspot.ca/2014/11/appv-5-first-launch-application.html
The short of it:
When AppV is "launching" the application for the first time after a net stop/net start it starts consuming memory and CPU for the 196 seconds that it's launching, peaking at nearly 600MB RAM and 50% CPU (though most of the time it's peaked at 25%
CPU).
The AppV Debug logs do not give a whole lot of info as to what AppVClient.exe is doing during this time.  Most of the logs show the application "start" as they setup their components, and when the application has launched.  Almost all
the logs show the first second or two of application launch and the last second or two before the GUI.
When the application is launched after a reboot complete with add-appvpackage/publish-appvpackage (global) the first launch is ~40s.
Subsequent launches of the application are consistently between 11-15s (near native).
Does anyone have any thoughts about what I can do to make it so the application will launch at (near) native speed for the first launch?  I am fresh out of ideas.

Answer 1

Let me share my test results with your provided test package:
Test Case 1
Default Package (without any modifications):
- Add-Package: ~23.5s
- Publish-Package: ~2.0s
- First start registry.exe: Nearly instant, but Registry staging takes 35s with high CPU usage (>=30%)
- Second start registry.exe: Instant
Test Case 2
Removed all ActiveX exts., COM Object exts. and Filetype assosciations in AppxManifest.xml & renamed HKLM\Classes to HKLM\Classes_TEST
(I developed a tool that allows me to modify the .appv package directly including all files in it):
- Add-Package: ~3s
- Publish-Package: ~1.6s
- First start registry.exe: Instant, but registry staging takes 33s with high CPU usage (30%)
- Second start registry.exe: Instant
Test Case 3
Removed all COM object extensions in AppxManifest.xml:
- Add-Package: ~2.6s
- Publish-Package: ~1s
- First start registry.exe: Not completely instant, takes 6s to start and Registry staging takes 35s with high CPU usage (>=30%)
- Second start registry.exe: Instant
These are my findings:
1) The long time needed to add the package is due to the amount of COM object extensions. The size of the manifest file is 71MB, so that's really big. When I remove the COM objects from the manifest, the file size is 209KB.
2) The high CPU usage is caused by registry staging. The registry file is ~58MB. With the first start, registry staging starts, the .dat hive gets loaded and the content will be copied to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\AppV\Client\Packages\<Package-GUID>\Versions\<Version-GUID>\REGISTRY.
In the same tree as the REGISTRY, you will find a key named "RegistryStagingFinished" when the staging has been finished. I observed that the high CPU load decreases as soon as registry staging has been finished (this is the case when RegistryStagingFinished
is there). So the staging took ~35s on my test System to do the registry staging for a 58MB registry hive.
And now the intersting part. If you close the virtualized app / the app you start in the VE (in your case registry.exe) before registry staging is finished (so before RegistryStagingFinished is set), it will re-start the registry staging from the
beginning the next time you start the app. To me it looks like it doesn't continue from the last point it stopped the registry staging. This might be something that could be improved in a next release so that it continues registy staging from the point it
previously stopped.
What does it mean for you:
If you need all integration points like COM, ActiveX, etc., you won't be able to reduce the add-package time. Otherwise if you remove them from the package, the add-package time can be reduced to <5s.
Your registry is as well quite big (58MB). Loading that into registry needs time. There is no way around, but when deploying the package globally, you could start a process within the VE of your application right after publishing it so that the registry
staging executes once and is already finished when the first user starts the app.
Due to the big registry size and the amount of integration points of your package, I believe there won't be a lot of possibilities to improve the experienced performance issues except reducing the size and amount of integration points.
Regards, Michael - http://blog.notmyfault.ch -