Project Server 2010 PWA Provisioning Error - System.UnauthorizedAccessException: Access is denied.

Hi,
When migrating from a Prod Environment to a Test Environment using the Project Server 2010 5 Database backup and restore process (actually 6 DB's with the Project Sites DB), upon encountering a failure during provisioning the PWA Instance, we get the errors listed
at the base of this posting in our Event Logs.
To clarify what we have done already, we have:
1. Backed up the 6 databases from the Source Environment.
2. Removed the existing PWA_Content and PWS_Content (Project Sites) databases from the Target Environment Farm.
3. Restored the 6 databases in the Target Environment.
4. Given the SP Farm Service Account 'db_owner' rights to each of the 6 restored databases.
5. Added the PWA_Content and PWS_Content databases  (exact same names) to the Target Environment.
6. Locked down (offline/stopped), all of the SharePoint Content databases in the Farm except for the PWA_Content database.
7. Run the PWA Provisioning process from the 'Project Server' SharePoint Service in Central Admin.
It is during the PWA provisioning that we encounter these errors.
As the Target Environment is for Test/Dev purposes, the SP Farm Service Account is setup to support all of the SharePoint Farm Services, is in the Local Admin Group of each of the Servers (App, WFE and DB), and is actually in the sysadmin
role on the DB Server.
Any suggestions on what is causing this denied access situation would be most appreciated.
Cheers,
Wayne
Event Log Entries:
============================================
Log Name:      Application
Source:        Microsoft-SharePoint Products-Project Server
Date:          1/22/2014 2:47:34 PM
Event ID:      6971
Task Category: Provisioning
Level:         Error
Keywords:     
User:          ACCOUNTS\svc_psfarm
Computer:      SERVER.accounts.domain.com
Description:
Failed to provision site PWA with error: Microsoft.Office.Project.Server.Administration.ProvisionException: Membership synchronization failed. ---> System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
   at Microsoft.SharePoint.Library.SPRequest.AddRoleDef(String bstrUrl, String bstrName, String bstrDescription, Boolean bHidden, Int32 lRoleOrder, UInt64 iPermMask, Byte iType, Int32 lRoleDefID)
   at Microsoft.SharePoint.SPRoleDefinitionCollection.AddCore(SPRoleDefinition role)
etc....
Failed to create the Report Center web (Exception: PSI Entry Point:
Project User: DOMAIN\svc_psfarm Correlation Id: c0a30b18-46e4-4447-b469-f8fd3bb1490a
PWA Site URL: http://pstest2010/PWA
SSP Name: Project Server
PSError: NoError (0))
Failed to grant 'DOMAIN\svc_psfarm' access to web application 'SPWebApplication Name=Portal'.
Error: System.Security.SecurityException: Access denied.  Only machine administrators are allowed to create administration service job definitions of type: Microsoft.SharePoint.Administration.SPUpdateWorkerProcessGroup, Microsoft.SharePoint, Version=14.0.0.0,
Culture=neutral, PublicKeyToken=71e9bce111e9429c.
   at Microsoft.SharePoint.Administration.SPAdministrationServiceJobDefinition..ctor(String name, SPService service, SPServer server, SPJobLockType lockType)
   at Microsoft.SharePoint.Administration.SPUpdateWorkerProcessGroup..ctor(SPTimerService timerService, String[] loginsToAdd, String[] loginsToRemove)
   at Microsoft.SharePoint.Administration.SPWebApplication.GrantAccessToProcessIdentity(String username, SPPolicyRoleType policyRole)
   at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.GrantAccessToWebApp(List`1 webApps, String userName) The Zone of the assembly that failed was: MyComputer

Is the Test and Production in the same domain. If they are not, then the SIDs associated with the SQL accounts may look the same but they are not.  At times, I have had to recreated accounts when in different domains.
I would also use PowerShell cmdlets and see if it finds any issues, such as Test-SPContentDatase.. Validate the SharePoint content databases, because if not setup properly the Provisioning will fail.
Also check these properties.
$web=get-spweb http://prodproj01/pwa
$Web.AllProperties[“PWAURL”] ### see what the value is.. you may have to blank it out, specially if it is pointing to a different URL
Here is how to fix.
$Web.AllProperties[“PWAURL”]=””
$web.Update()
Michael Wharton, MVP, MBA, PMP, MCT, MCTS, MCSD, MCSE+I, MCDBA
Website http://www.WhartonComputer.com
Blog http://MyProjectExpert.com contains my field notes and SQL queries

Similar Messages

  • Project Server 2010 (PWA Timesheet) Error

    Hello,
    I am an administrator for Project Server and had a resource submit a timesheet in error. The timesheet was approved at the time and recalled by the resource at the same time. When the resource had tried to recall, they received back an error. I delegated
    as the resource and tried to recall it as the user but it gets stuck with the green loading picture and never moves past that. It seems the status update was accepted when it got approved and now the timesheet is "stuck". From the user's standpoint,
    it shows only as "Submitted" in their Timesheet status, but somehow it does not let me delete the timesheet. The error received is saying Invalid object, timesheet failed/blocking correlation. Anyone have any idea how to resolve this? Can't seem
    to fix this error.
    This is the error i get in return:
    Timesheet:
    GeneralInvalidObject (20009). Details: id='20009' name='GeneralInvalidObject' uid='fc00fed3-1cb1-48b2-83b6-800b03b33f1d' parameter='nextApproverUID'.
    Queue:
    GeneralQueueJobFailed (26000) - TimesheetReview.ReviewTimesheetMessage. Details: id='26000' name='GeneralQueueJobFailed' uid='fbebdb81-29b5-416b-818d-a085a4f1d808' JobUID='e9104f34-f416-4b89-9d3e-8574a868e065' ComputerName='WEGSFES29090'
    GroupType='TimesheetReview' MessageType='ReviewTimesheetMessage' MessageId='1' Stage=''. For more details, check the ULS logs on machine
    WEGSFES29090 for entries with JobUID e9104f34-f416-4b89-9d3e-8574a868e065. 
    Thanks!

    Is the Test and Production in the same domain. If they are not, then the SIDs associated with the SQL accounts may look the same but they are not.  At times, I have had to recreated accounts when in different domains.
    I would also use PowerShell cmdlets and see if it finds any issues, such as Test-SPContentDatase.. Validate the SharePoint content databases, because if not setup properly the Provisioning will fail.
    Also check these properties.
    $web=get-spweb http://prodproj01/pwa
    $Web.AllProperties[“PWAURL”] ### see what the value is.. you may have to blank it out, specially if it is pointing to a different URL
    Here is how to fix.
    $Web.AllProperties[“PWAURL”]=””
    $web.Update()
    Michael Wharton, MVP, MBA, PMP, MCT, MCTS, MCSD, MCSE+I, MCDBA
    Website http://www.WhartonComputer.com
    Blog http://MyProjectExpert.com contains my field notes and SQL queries

  • Project Server 2010: PWA Removing Default Project Site Security Groups When Creating a New Project

    I looked for this specific issue with Project Server 2010/PWA/SharePoint and could not find an exact answer... hopefully someone can help.
    We are currently using Project Server 2010 and have a number of project site templates that are used dependent upon the enterprise project type selected. Each of these project site templates have unique permissions which should create the default security
    groups on the project site upon publishing/syncing:
    <Project Name> Members
    <Project Name> Owners
    <Project Name> Visitors
    <Project Name> Project Managers (Project Web App Synchronized)
    <Project Name> Team Members (Project Web App Synchronized)
    Web Administrators (Project Web App Synchronized)
    Whether a user creates a project through PWA or Project Pro 2010 and imports the project into PWA, we get a weird result in the Site Permissions of the newly created project site. PWA will remove all default security groups from the project site template
    and add a whole list of users in the Site Permissions list without groups. 
    Once the project is published and the project site is created, we can then go back and add those default security groups back in the project Site Permissions and even add a couple of custom groups without them being removed on all subsequent project syncs
    or publishing. 
    How do we get PWA to not overwrite the project site templates' security groups and place each user in the proper default security groups? At the same time, how is PWA adding a number of users into the Project Site Permissions?
    Thanks in advance.

    Paul,
    Thanks for that information. Right now we are using the Test environment to turn the Auto-sync feature back on. I suspect that the reason this is happening is due to PWA groups/categories/security templates. There may be more than one PWA group that is "overwriting"
    the default project site groups upon initial creation of the project. We will look further into the security settings to tighten up the policies. 

  • Project Server 2010 PWA Site corrupted? default.aspx Corrupted, contains jibberish?

    Hey Everyone,
    Running Project Server 2010 Server w/ SQL Server 2008 R2 back end
    Sometime after 5:30 pm Friday , our main PWA site url which is
    http://servername/PWA  now contains the below mess of characters and text... seems the default.aspx file located in
    \\servername/PWA share got corrupted, messed with? ANY ideas how to fix this? or how why this happened.  really need a fix...... Thanks
    Jack Galante Sr Systems Engineer

    Jack,
    Are you the Project Server Administrator?Does this happen for all users or just one? If just one, please see if the users system is the issue.
    If it is all users, then Do you know if anybody else with approproate permissions added any custom code?
    I am not really sure of the issue here. Here are a few things you could try:
    1) Reboot all the servers
    2) Restart the Project Server related services like Events Service and Queue Service
    3) Navigate to Central Admin, and see if you restart the Project Application Service. 
    If none of the above work, then I would recommend contacting Microsoft Support before making any further changes.
    Prasanna Adavi,PMP,MCTS,MCITP,MCT http://thinkepm.blogspot.com

  • Project Server 2010 SP2 Timesheet Errors

    Hi,
    We use Project Server 2010, SP2 in SEM mode. Users use IE9. Some of our users complaining that while entering timesheet, sometimes "h" for hours is auto populated next to the hours entered in the cell... and sometimes dont appear and then timesheet
    freezes. Anyone have any suggestions when would this happen and any resolutions?
    And some users also get the following error...
    An error occurred when communicating with the server. Check connectivity with your administrator to determine whether further action is necessary", then attempt to approve TimeSheet"
    Please suggest.
    Kishore Dodda

    Let us know what turns up. It certainly looks like you are losing connectivity. The other possibility is that your databases are in need of attention. Ask your SQL DBAs whether they are following best practices for Database management and have a SQL maintenance
    plan running on the Project Server databases.
    Gary Chefetz, MCITP, MCP, MVP msProjectExperts
    Project and Project ServerFAQs
    Project Server Help BLOG

  • Project Server 2010 PWA - attempting to move folders from root to document library subfolder does not work

    Hello all,
    We have an issue whereby attempting to move documents from the top level folder to a subfolder in a PWA site appears to work initially, but within a few minutes the documents are moved back to the top level folder with no error message.
    Uploading a document has the same effect.
    Permissions are granted at all levels in the document folder.
    Only some users seem to be affected.
    Only some project sites are affected.
    As an admin user I can successfully upload and move documents without issue.
    Any thoughts on what might cause this ?   I've actually never seen this before (and I have worked on a lot of SharePoint/Project Server deployments) so interested to know if anyone out there has a bright idea on how this might get fixed ?
    Many Thanks

    Hi,
    I understand you encountered the issue when save document from Office client application to SharePoint 2010 site.
    Is there any Proxy settings in networking side? Please bring up IE > Internet options > Connections > Lan settings. If so, please uncheck options in Proxy server and test the issue again.
    If you have deploy GPO to add SharePoint site into Trusted sites, please check if there is any other GP associated with trusted sites that might affect the issue. You could also remove it from Trusted sites via IE options and test the issue again.
    In addition, please provide more error message to help narrow down the issue, you could locate error information in Event log at least.
    Regards,
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact
    [email protected] .
    Rebecca Tu
    TechNet Community Support

  • Need to send a email alert to Projectmanger when publishing a project in project server 2010 PWA site

    Hi All
    When creating and publishing a project in PWA(server 2010) need to send a mail alert to the concerned project manager.
    How can i achieve this..
    I have configured the outgoing mail sending in sharepoint and its working fine now.
    Please help me..
    Thanks Advance
    SharePoint Module Lead Assyst International Private Ltd Cochin India

    Nothing is built in Project Server to do this for you automatically.
    They way I have done this in the past is to create a SQL job using SSIS.  It runs every hour and when it sees a project schedule that has a recent PUBLISHED date then this is a trigger to send an email to project manager.
    Cheers!
    Michael Wharton, MVP, MBA, PMP, MCT, MCTS, MCSD, MCSE+I, MCDBA
    Website http://www.WhartonComputer.com
    Blog http://MyProjectExpert.com contains my field notes and SQL queries

  • Project Server 2010 PWA Printing Issue

    we are having  issues when printing from the Project Web App schedule web part where if they went over a certain number of rows (around 42 – spooky!)  the printed page was blank (although the pop-up with just the grid displayed for printing
    looked just fine).  This was after seeing this warning – which is expected – when you are printing a grid of more than 30 rows: Print Warning - There are more than 30 records in the current grid. 
    In my testing internally I saw the same, I clicked OK on the warning, the new page opened with everything looking good and I could choose my printer – but the print was blank (I did try a large plan and in this case the final 40 or so tasks printed…) 
    However, if I turned off the compatibility setting under Tools, Compatibility View Settings, so that these were not applied to Intranet sites then the project page does not open and it says it requires IE 7 or above for it to open.
    Regards,
    Ankit Manchanda

    Ankit,
    Make sure that you have updated printer drivers, also try using different printer model
    Hrishi Deshpande Senior Consultant

  • Issue in Custom Filter in PWA of Ms Project Server 2010

    Hi,
    i am facing an issue on setting a cutom filter in the Project Center view of PWA. I am setting a custom filter and it works fine.
    Projects are filtered as per the filter. However, on selecting the filter for editing, I get a message viz. "Failure. There was an error processing the request".
    I am looking for pointers and guidance to resolve this issue. Any helpin this regard will be appreciated.

    Hi,
    Evenif you already checked this out, it is likely to be a security issue. This functionality is managed by permissions in PWA server settings and not in the Central Admin.
    Check this article :
    http://msprojectnow.com/Blog/tabid/142/entryid/556/Enabling-Task-Sheet-functionality-to-Add-Yourself-to-a-Task-in-Project-Server-2010-PWA.aspx
    Check if you're not belonging to multiple categories that might have the given permission disabled.
    Hope this helps,
    Guillaume Rouyre, MBA, MCP, MCTS |

  • An Unknown Error Has Occurred - Project Server 2010 - Tasks Page

    Greetings!   Some of my resources are recieving the error "An Unknown Error has occurred" (in red)  when trying to access their TASKS Page in Project Server 2010.  
    I've researched this error quite a bit and am finding suggestions and answers pretty much all over the board.    Before I have my server people start installing a slew of hot fixes - I want to know if anyone has an idea of the underlying cause
    for this error.    Since this issue doesn't affect everyone (i.e. I can access my task page), is it possibly related to the actual schedule in which they have assignments?    
    Any help is appreciated.
    Thanks!
    ~Randy 

    i have found the solution for this issue, you need to do this:
    in the logs you must search the "exception" and the logs give you a ID, this ID is about the task assignment in the specific resource , so this assignment is corrupt.
    this ID, you need to merge in the SQL server in the project server database, you need to merge ID task with ID Project with ID Resource, so when you identify the ID project and ID resource you can
    DELETE this assignment in the project from project profesional, because this task has a corrupt assignment, you must delete
    ONLY FROM project professional.
    This solutión solve this specific problem for this assignment  that
    IS DIFERENT TO THIS SOLUTION  post of Brian Smith:
    http://blogs.msdn.com/b/brismith/archive/2010/07/02/project-server-2010-an-unknown-error-has-occurred-in-project-center-resource-center-or-tasks.aspx?Redirected=true
    Erick Gutiérrez PMI Membership #ID 2089740 MTCS - Microsoft Project Server Managing Projects

  • Understanding Project Server 2010 Error ID 12015

    Are there any known bugs with Project Server 2010 producing the error ID 12015.
    I have a user who has the most up to date version of Project Pro 2010 who cannot open projects from Project Server 2010.
    Below is there screenshot showing their Project Pro version:
    We have the oldest version of Project Pro in Project Server set to 14.0.6134.5000
    Below is a screenshot of the Error ID they are receiving with a completely new version I have never seen before:
    I have been researching for days trying to figure out where the 14.0.4751.1000 version number is coming from. We have all our users on the same versions that are displayed in the first screenshot. 
    Has anyone ran into this issue before?
    Thanks,
    Brent

    14.0.4751.1000 is of that users MPP. 
    and minimum 14.0.6134.5000
    version is specified in project server instance. But your user is having 14.0.4751.1000
    version 
    So resolution is either user update mpp with latest Cu or you specified 14.0.4751.1000
    version in project server .
    As every after Cu update version is getting changed of MPP
    kirtesh

  • Project Server 2010 Migratet from one farm to another farm

    Hi ,
    I need to migrate existing Project server 2010 farm one server to another server.
    Kindly suggest me what is the best way we need to fallow.
    Existing Environment two server:
    1.  App server(SharePoint 2010 +Project Server 2010)- Existing PWA site is running
    2.Database server(with DNS instance name)
    New Environment:
    1.App server (fresh installed SharePoint + project server 2010)
    2.DB server fresh server with sql 2008
    Hasan Jamal Siddiqui(MCTS,MCPD,ITIL@V3),Sharepoint and EPM Consultant,TCS
    |
    | Twitter

    Event viewer Error:
    Failed to provision site PWA with error: Microsoft.Office.Project.Server.Administration.ProvisionException: Membership synchronization failed. ---> System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
       at Microsoft.SharePoint.Library.SPRequest.RemoveRoleDef(String bstrUrl, Int32 lRoleID)
       at Microsoft.SharePoint.SPRoleDefinitionCollection.Delete(SPRoleDefinition roleDefinition)
       at Microsoft.Office.Project.Server.BusinessLayer.SharePointSecurityHelper.DeleteRoleByName(SPWeb web, String roleName)
       at Microsoft.Office.Project.Server.BusinessLayer.SharePointSecurityHelper.DeleteRoles(SPWeb web)
       at Microsoft.Office.Project.Server.BusinessLayer.SharePointSecurityHelper.ConfigureDefaultPWAWSSSecurityModel(SPSite topSite, SPWeb web)
       at Microsoft.Office.Project.Server.BusinessLayer.Admin.<>c__DisplayClass18.<QueueUpdateUsersAddRemoveStatusOnPwaRootsAndWorkspaces>b__17()
       at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass4.<RunWithElevatedPrivileges>b__2()
       at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
       at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param)
       at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode)
       at Microsoft.Office.Project.Server.BusinessLayer.Admin.QueueUpdateUsersAddRemoveStatusOnPwaRootsAndWorkspaces(Dictionary`2 deletedUserUidsLoginNames, Guid[] addedUserUids, Boolean isFullSync, Boolean syncWorkspaces)
       at Microsoft.Office.Project.Server.BusinessLayer.Admin.SynchronizeMembershipForPwaAppRootSite()
       at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.SynchronizePwaMembership(ProjectProvisionSettings provset, ProjectSite projectSite)
       --- End of inner exception stack trace ---
       at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.SynchronizePwaMembership(ProjectProvisionSettings provset, ProjectSite projectSite)
       at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.CreateSite(ProjectProvisionSettings provset
    Hasan Jamal Siddiqui(MCTS,MCPD,ITIL@V3),Sharepoint and EPM Consultant,TCS
    |
    | Twitter

  • Project Professional 2010 Not Publishing to Project Server 2010

    I am having trouble publishing a project from Project Professional 2010 to the Project Server 2010 (PWA). What I am trying to publish is a master project which is contains many enterprise projects as subprojects. When I press publish, it prompts me
    to save and if I click yes then it goes through and saves all the subporjects and then just stops there and doesn't publish the changes to the master project. If I press publish and click no to the save prompt then it gives me the status "sychronizing
    data to server" for around 15 seconds and then just does nothing after the sychonization finishes.
    It usually should be giving a "publishing job done 0%" or "publishing job completed" when it publishes, but it gives nothing like that. I also check the project through the PWA and it has not updated yet, but when I open it for editing in the client it has
    everything updated and up to date, meaning that the save is working properly and just not the publishing.
    All my subprojects are able to publish correctly, it's just this master project that is not working.
    Does anyone know why this is happening? Is the publishing process timing out for some reason? Or is it something else?
    Please help.
    EDIT: The progress notification at the bottom of Project Professional just stops at "save completed sucessfully" whenever I try to publish

    To check the queue, navigate to Server Settings > Manage Queue and view the jobs that are present. If there is an error when publishing you should see it in the queue.
    Alex Burton
    www.epmsource.com |
    Twitter
    Project Server TechCenter |
    Project Developer Center |
    Project Server Help | Project Product Page

  • Reports permission in Project server 2010

    Hi,
    I have reports created in SSRS, i need to bring those reports to Project Server PWA. How to do that? i need the steps.
    Reports like Number of active resources working in Project, Number of projects in that PWA instance.
    My requirement is, i have a structure like Executives - Program Managers1- 1.Project Manager 2. Project Manager. 
    Program Manager2 - 1. Project Manager 2. Project Manager.
    Each program manger has to see only their projects(the project which comes under him) in PWA not the other Program Manager's project and Project Manager has to see only his project not the other project manager's project in the same program.
    I have Executive who should see all the projects in the organization.
    Can someone assist me how to execute in Project Server 2010 PWA?
    K.Gokulraj

    Hello,
    Sounds like you might need to use the project server security to filter the data in the reports, take a look at this example:
    http://msdn.microsoft.com/en-us/library/office/ee862408(v=office.12).aspx
    To get the reports in side PWA there are several options. As you are querying this I guess SSRS in not SharePoint Integrated mode? If in SSRS Native mode you can just add page viewer web parts on the pages in the PWA site then reference the report server
    report URL.
    Paul
    Paul Mather | Twitter |
    http://pwmather.wordpress.com | CPS

  • The efficiency value in the dashboard is more than 100% in Project server 2010

    The efficiency value in the dashboard is more than 100% in Project server 2010. As I have admin access trying to find out what could be the issue. Please reply if there is any suggestion on this.

    Hi Chaishashi,
    Efficiency isn't a value that is standard in Project Server 2010, and so it is likely that what you are viewing is a custom report.  I suggest you contact the report author to resolve this issue.
    Ben Howard [MVP] | <a href="http://www.applepark.co.uk">web</a> | <a href="http://www.applepark.co.uk/ben-howard-blog"> blog</a> | <a href="http://shop.oreilly.com/product/0790145369154.do">book</a>
    | <a href="http://www.applepark.co.uk/p2o">Export Project to Outlook</a>

Maybe you are looking for

  • LSMW for Outbound Delivery

    Hi, I am not able to find an LSMW for creating GR for outbound delivery. I tried using recording in MIGO but it did not work. Also MB1C is not working for outbound delivery. Please help. Regards, Anubhav

  • Need help!! Compaq V3500 No audio driver available in website

    Need Help Compaq V3510TU P/N GG201PA#AKL Windows XP Professional 32 bit, After I re installed the new OS.. The windows doesn't have the audio driver.  I tried everything download many of Conexant HD  many many version... they don't work at all.  Plea

  • Import JavaBean Model - Invalid class

    Hi, I wanted to import a JavaBean Model as described in the Tutorial "Using EJBs in Web Dynpro Applications" (see: https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/1f5f3366-0401-0010-d6b0-e85a49e93a5c). Problem: i always get a

  • Purches could not be made. Help please.

    Hey, when I want to buy something to a game it says "Your purchase could not be carried out if you want help, contact support for itunes on http://www.apple.com/support/itunes/ww/." The problem is that the site isn't helping. I can't find the solutio

  • 2tier or 3tier landscape for AS/400 ?

    Dear all, My company is considering upgrade options for current AS/400 server. There are 2 options we are considering. 1.) Upgrade the existing box to a bigger capacity to have higher SAPS. Landscape remains 2tier. 2.) Change 2 tier to 3 tier and spl