Powerbuilder application crashes after upgrade to 12.5.2 (build 5006)

Question

I have already seen few similar posts on this forum for similar issue. However, I feel mine is different from those and could be some basic steps am missing. Thanks in advance for any comments.
We are evaluating the effort and benefits of migrating our PB 10.2.1 (build 9731) application to  PB 12.5 Classic. After downloading the trial version from the SAP site, I could migrate the application easily with only a handful of warnings. However, while running the application either from build (EXE) or from IDE, it crashes (windows application failure) frequently. In addition to that, few of our application screens opens as blank screen (doesn’t display anything in the DW attached).
My operating system is Windows 7 (32 bit) Service Pack 1. The application has many calls external DLLs (both windows standard and custom), as samples at the bottom. I believe that the correct (from PB 12.5) DLLs are copied to Powerbuilder home directory (C:\Software\Sybase\PowerBuilder 12.5 & C:\Software\Sybase\Shared\PowerBuilder) and the source code directory in my attempt to ensure that there are no issues with DLLs. However, initially I had installed powerbuilder 12.5 also under the same root as Power builder 10.0, but as I read few posts that DLLs from different version residing in same place can cause issues, uninstalled and installed PB 12.5 on different location.
Our PB application also interact with the .Net modules of the application by communicating through a custom DLL. However, that seems still working and I am able to open the .Net screens from the migrated application.
Below are the main scenarios in which the application crashes and every time I get a DLL error as given below.
1. While running the application from IDE and selecting ‘Exit’ from the main window. This will close any instantiated objects and close the mutex by calling the windows API.
function ulong CreateMutexA(ulong lpMutexAttributes, boolean bInitialOwner, ref string lpName) library "kernel32.dll" alias for "CreateMutexW" function ulong GetLastError() library "kernel32.dll" alias for "GetLastError;Ansi"
function ulong CloseHandle(ulong hMutex) library  "kernel32.dll" alias for "CloseHandle;Ansi"
This will result in the below crash.
Faulting application name: PB125.EXE, version: 12.5.2.5006, time stamp: 0x50f93412
Faulting module name: kernel32.dll, version: 6.1.7601.17932, time stamp: 0x503275b9
Exception code: 0xc0000005 Fault offset: 0x0004c372
Faulting process id: 0x2bdc
Faulting application start time: 0x01cf3e72371227c9
Faulting application path: C:\Software\Sybase\PowerBuilder 12.5\PB125.EXE
Faulting module path: C:\Windows\system32\kernel32.dll
Report Id: 8feef8fc-aa65-11e3-8049-f500274aecdc
2. Opening of certain windows results in the below crash. If I try to debug the code and move mouse over the line of code being executed, then also I get the same crash. I don’t do the mouse move then I can proceed with the same line, however it will crash at a later line calling ‘classname’.
Problem Event Name: APPCRASH
Application Name: PB125.EXE
Application Version: 12.5.2.5006
Application Timestamp: 50f93412
Fault Module Name: PBVM125.dll
Fault Module Version: 12.5.2.5006
Fault Module Timestamp: 50f93437
Exception Code: c0000005
Exception Offset: 00175357
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 3081
Additional Information 1: 38ca
Additional Information 2: 38ca6ad0537dbd0555785bd862b8e94f
Additional Information 3: 0cd5
Additional Information 4: 0cd54a5b6672f15bf2ca366539d89c12
3. Few of the application screen opens but doesn’t display the datawindow attached to it.

Answer 1

... many calls external DLLs (both windows standard and custom),
You have a very complex application and it is unlikely that you will receive much help in this type of setting.  As Chris has already pointed out, previous efforts appear to have been done in a sloppy manner; something indicated by the ANSI qualifiers on API functions that are not needed or relevant.  In addition, the PB alias of CreateMutexA for the API function CreateMutexW is also suggestive.
I believe that the correct (from PB 12.5) DLLs are copied to Powerbuilder home directory (C:\Software\Sybase\PowerBuilder 12.5 & C:\Software\Sybase\Shared\PowerBuilder)
You should be very very careful about doing things like this.  One should not simply scatter DLLs across directories without very good reasons as it often leads to strange problems.  On a machine with the IDE installed, you should allow PB to install everything where it wants them.  Copying or moving files is something done as a last resort - it is certainly something that you need to document in case you need to undo it or redo it at a later time.
Lastly, you might reconsider attempting this with the trial version.  I don't how "current" the trial version is but I suspect it might be a bit buggier than the most current maintenance build.  I'm guessing that you will have to do a bit more than recompiling to get the functionality related to "opening .Net screens" working.  It might help to try creating a new application and put in some basic functionality to interact with all of these external processes, screens, dlls, etc since I suspect this is where the bulk of your problems exist.