Power BI - Office 365 data storage for Excel (Power Pivot & Power Query) architecture

Hi, I trying to find a comprehensive view of the Power BI architecture, specifically where the data for Power Pivot & Power Query files are held. I've seen reference to the Vertipaq Engine and the data being stored in the Excel File, however this is
in reference to 2010.
Other references I've seen (SharePoint on premise with Power Pivot) have shown me that the data held in the Excel files on the SharePoint sites are actually held in SQL Server instances.
Could anyone provide clarity on the architecture for Power BI please...
Tim
Thank you for you time folks!

Hi Tim,
let my try to explain this:
When you load data into Power Pivot it is actually loaded into Vertipaq (in-memory column-store) and it resides there until you close Excel. Technically speaking Power Pivot is a single-user SQL Server Analysis Services instance hosted within Excel. When you
close Excel a backup of the Power Pivot "database" is created and stored within the resulting .xlsx-file .
Any user that has access to the Excel file also has access to the Power Pivot model, so there is actually no security
This is very similar to any other excel file
Once the Excel is stored in SharePoint / SharePointOnline and is access the first time the backup that is stored within the Excel is restored into SQL Server Analysis Services instance running in SharePoint Integrate mode
all connections to the Excel-File are redirected to that SSAS instance. After some time of inactivity the model is unloaded from that SSAS instance again to free up the memory
the data itself so basically always resides inside Power Pivot or respectively the Excel file
Power Query only stores the actual query (= M-script) but no data
hth,
gerhard
Gerhard Brueckl
blogging @ http://blog.gbrueckl.at
working @ http://www.pmOne.com

Similar Messages

  • How to get sharepoint online (office 365) data in cloud/windows azure (provider) hosted app using Javascript?

    How to get sharepoint online (office 365) data in cloud/windows azure (provider) hosted app using Javascript?
    I wish to retrieve sharepoint online data in html page (hosted in windows azure) using javascript and then need to play with AngularJS

    Hi,
    According to your description, you might want to get data from Office 365 SharePoint Online(also known as host web) and pass to the Provider Hosted App which is hosted
    in Windows Azure site.
    I would suggest you take a look at the links below about accessing data from the host web
     for a quick start:
    https://msdn.microsoft.com/en-us/library/office/fp179927(v=office.15).aspx#SP15Accessdatafromremoteapp_Codeexample
    http://dannyjessee.com/blog/index.php/2014/07/accessing-sharepoint-data-from-provider-hosted-apps-use-the-right-context/
    Aother sample solution for your reference:
    https://code.msdn.microsoft.com/SharePoint-2013-Get-the-0ec36bb6
    Best regards
    Patrick Liang
    TechNet Community Support

  • SR830 data storage for rs232

    Hi,
    I found a LabVIEW code here which can storage data from input signal (e.g.A/I)in SR830 and transfer it by the GPIB.
    I used function generator to generate a sine wave with frequency:30Hz& Vp-p:100mVolt , conencting to SR830.
    Clearly ,I got a correct result when i used the code by GPIB communication interface(left graph).
    Than,I tried to change the communication interface RS232 to accept the same signal,but it has mistakes(right graph).
     Here is original code "data storage for GPIB"
    Here is rewrite code for RS232 by myself,but what's wrong i did?
    Attachments:
    SR830 DATA STORAGE EXAMPLE.VI ‏54 KB
    scan test1.vi ‏43 KB

    The SR830 expects three integer arguments (separated by commas) for the TRCA? query:
    * i = 1 or 2 specifies which buffer to download from
    * j >= 0 specifies the starting point
    * k>=1 specifies how many points to download
    If you know k, you can calculate the exact number of bytes in the response. For your code, which downloads 4000 points at a time, that will be something like 60 kB (if memory serves, the response in ASCII transfer mode is 15 bytes/value). Make sure that you're not hitting any timeout or serial buffer size limits with a transfer of that size.  
    Edit: You have your bitrate set to 9600 baud (1200 bytes/second) and a 10 second timeout. That will read 12 kB before timing out, or 1/5th of your transfer. The 830 supports baud rates up to 19,200, which will help, but you'll also need either a longer timeout or to transfer your data in smaller chunks. 

  • Do I need to purchase Office 365 A3 plan for faculty to get free Office 365 ProPlus for students in Office 365?

    Hi there
    Our school has A2 plan for both students and faculty. We recently contacted our reseller and placed an order for free  Office 365 ProPlus A for our students. We received an acknowledgement from Microsoft regarding this order saying that it was successfull
    and I can see the license added when I log into the VLSC website. Problem is, I can't see this licence listed when I log into Office 365 admin. It is not listed under subscriptions or licenses. Apologize for my ignorance, but I don't know how can I get this
    license added in Office 365 so that our students can start using editable versions of office apps on their IPADs. Do we need to purchase Office 365 A3 plan for our staff to get this student's license enabled in Office 365? Can somebody please help?

    I have similar questions concerning getting started with the Student Advantage program offering.
    I've read the blog posts and made as many as 5 calls to Office 365 and still do not understand how to get this going.
    We have a subscription to Education A2 for both faculty and students that we started in January 2014.  We receive this at no charge.  So I'm starting with a subscription at no cost.  I do not have an Education Reseller because we started our
    subscription through the website with a trial and had that converted to the no cost A2 plan.
    When I am in the portal and look under 'Purchase Services' I can see the option to add Office ProPlus for Faculty at $2.00/user/month and for Office ProPlus for Students at $1.50/user/month.  My understanding of Student Advantage is that if I license
    all my faculty/staff accounts for Office ProPlus I would then get the Student access at no charge.  So I've licensed the Office ProPlus for Faculty for all of our faculty accounts yet I cannot get a clear answer about how to get the Student accounts added
    at no cost.
    I most recently was told by Office 365 support that I needed to speak to a reseller, which I did today.  He told me that I would need to do one of two things.
    1. Upgrade my no cost A2 faculty accounts to A3.  Which for me and what I see in the portal would take them from $0 to $4.50/user/month.  With that A3 for faculty I would get the Office ProPlus for the Students accounts on A2 at no charge.  
    2. I would need an Open Value license covering all school FTEs for Office at a cost of about $29/user/year.
    Both of these options are higher than licensing all Faculty accounts for Office ProPlus via the portal at $2/user/month which I have already done.
    So I don't understand why that doesn't qualify us for Student Advantage and now have to license all FTE, most of whom don't even touch a computer, for Office.

  • Office 365 Outlook prompts for password whenever outlook is restarted.

    Hello Folks,
    We have recently deployed Office 365 (We had Kerio email system previously, But we did not do a migration).
    The problem is 
    Office 365 Outlook prompts for password whenever outlook is restarted
    Tried the following workarounds, 
    1) Logon network security under More
    Settings on the Microsoft Exchange Security tab
    is set to a value other than Anonymous Authentication
    2) Tried removing the Generic Credentials from Credential manager.
    The is issue is only for some users (Majority). 
    (OS Platform: Windows 7/Windows 8).
    Thanks in Advance
    Murugupandian Pandi

    The first thing I would recommend to do is to make sure that your install missing Outlook updates and try again.
    I would also recommend asking them in this forum: https://social.technet.microsoft.com/Forums/en-US/home?forum=outlook
    This posting is provided AS IS with no warranties or guarantees , and confers no rights.
    Ahmed MALEK
    My Website Link
    My Linkedin Profile
    My MVP Profile

  • Office 365 Developer site for Share point is not working.

    hi,
    i created office 365 developer site for share point, it is not working for share point, i am getting error when click on Share point  from ADMIN.We will getting am Error:

    Hi,
    I understand that you stay in the Office 365 admin Center, when you click the SharePoint option, you get the error page.
    I recommend to enter the URL as below in the Brower to check whether the account can access SharePoint admin Center.
     https://sitename /_layouts/15/online/SiteCollections.aspx
            Here is a similar thread for
    your reference:
    http://community.office365.com/en-us/f/148/t/192116.aspx
    Regarding SharePoint Online, for quick and accurate answers to your questions, it is recommended that you initial a new thread in Office 365 forum.
    Office 365 forum
    http://community.office365.com/en-us/forums/default.aspx
    Best Regards,
    Linda Li
    Linda Li
    TechNet Community Support

  • Can we use SQL Server as a primary data storage for SharePoint 2013?

    Is it possible to use external SQL Server as a primary data storage for SharePoint 2013?
    Having this implemented we can generate reports, query data and it would become very very powerful to use.
    Or
    Is there a way to keep on using existing content databases and using external SQL Server as a secondary data storage at the same time. So I want it to be like redundant data in SQL Server.
    Thanks 

    Hi，
    Not sure if I understand your question correctly, SharePoint has its own content database storing SharePoint list data, we can use SSRS/SSAS integated with SharePoint mode to use sharepoint list as datasource to generate reports.
    http://technet.microsoft.com/en-us/library/bb326358(v=sql.105).aspx
    http://sqlmag.com/business-intelligence/delivering-bi-through-sql-server-and-sharepoint
    http://www.mssqltips.com/sqlservertip/2068/using-a-sharepoint-list-as-a-data-source-in-sql-server-reporting-services-2008-r2/
    Thanks
    Daniel Yang
    TechNet Community Support

  • Brand/ Remove Office 365 Login Page for SharePoint Online

    Hi  there,
    HAPPY NEW YEAR!
    Thank you very much for setting up this great forum. I have learned a lot of good tips from here.
    We have a challenge which we would like to find a solution to.
    We are using Project Online with SharePoint Online with Office 365.
    We are trying to find a solution to brand/ (if branding is not possible) get rid of the Office 365 log in page.
    What we are trying to achieve is:
    1. Users click on a link, say:
    https://abc.sharepoint.com/project1/
    2. Instead of seeing Office 365 login page, we would like them to be able to see the login page with our own brand or a login box to enter username and password.
    Thank you very much in advance for the time you set aside to share your knowledge with us.
    Sincerely, we really appreciate your kind help.
    Charlotte Tang
    Charlotte Tang

    Hi Charlotte,
    According to your post, my understanding is that you want to customize the Office 365 login page for SharePoint Online.
    We can implement ADFS for the Office 365 subscription, then customize the ADFS forms based on login page.
    Per my knowledge, If no ADFS you will need to use the Microsoft login page.
    Here are two similar threads for you to take a look at:
    http://community.office365.com/en-us/forums/148/t/194202.aspx
    http://community.spiceworks.com/topic/380634-o365-custom-login-page
    Best Regards
    Dennis Guo
    TechNet Community Support

  • Powerpivot 2013 doesn't work on Excel 2013 / Office 365: data model issue

    Hi guys
    I recently upgraded my Office 2010 to Office 365, with Excel 2013. Before installing Office 365 I uninstalled Office 2010. I'm running Windows 8 64 bits.
    I have the Powerpivot 2013 add-in, I can activate it without any issue. However when I click on "manage" on any spreadsheet (including new empty ones) I have an error "We
    couldn't load the Data Model.  This may be because the Data Model in this workbook is damaged.", followed by "PowerPivot is unable to load the Data Model".
    I tried to repair the installation to no avail. I tried to launch excel in admin mode, nothing better.
    This issue seems to be shared by some other people on this forum but
    with no satisfactory answer so far.
    I had a SQL server 2012 plugin installed previously which I uninstalled as well, but no change.
    Would you think of something else?
    Thanks

    Hi lourson,
    Someone work out this issue to stop the Tabular Model instance. Please take a look at the following blog:
    PowerPivot 2013 Error: Data Model Damaged:
    http://www.mssqlgirl.com/powerpivot-2013-error-data-model-damaged.html
    For more information, please see the same issue at Connect site:
    https://connect.microsoft.com/SQLServer/feedback/details/780949/powerpivot-couldnt-load-data-model-when-ssas-tabular-instance-is-running#details
    Hope you can solve this issue on your new work environment.
    Regards,
    We
    are trying to better understand customer views on social support experience, so your participation in this
    interview project would be greatly appreciated if you have time.
    Thanks for helping make community forums a great place.

  • What is the best data structure for loading an enterprise Power BI site?

    Hi folks, I'd sure appreciate some help here!
    I'm a kinda old-fashioned gal and a bit of a traditionalist, building enterprise data warehouses out of Analysis Service hypercubes with a whole raft of MDX for analytics.  Those puppies would sit up and beg when you asked them to deliver up goodies
    to SSRS or PowerView.
    But Power BI is a whole new game for me.  
    Should I be exposing each dimension and fact table in the relational data warehouse as a single Odata feed?  
    Should I be running Data Management Gateway and exposing each table in my RDW individually?
    Should I be flattening my stars and snowflakes and creating a very wide First Normal Form dataset with everything relating to each fact? 
    I guess my real question, folks, is what's the optimum way of exposing data to the Power BI cloud?  
    And my subsidiary question is this:  am I right in saying that all the data management, validation, cleansing, and regular ETTL processes are still required
    before the data is suitable to expose to Power BI?  
    Or, to put it another way, is it not the case that you need to have a clean and properly structured data warehouse
    before the data is ready to be massaged and presented by Power BI? 
    I'd sure value your thoughts and opinions,
    Cheers, Donna
    Donna Kelly

    Dear All,
    My original question was: 
    what's the optimum way of exposing data to the Power BI cloud?
    Having spent the last month faffing about with Power BI – and reading about many people’s experiences using it – I think I can offer a few preliminary conclusions.
    Before I do that, though, let me summarise a few points:
    Melissa said “My initial thoughts:  I would expose each dim & fact as a separate OData feed” and went on to say “one of the hardest things . . . is
    the data modeling piece . . . I think we should try to expose the data in a way that'll help usability . . . which wouldn't be a wide, flat table ”.
    Greg said “data modeling is not a good thing to expose end users to . . . we've had better luck with is building out the data model, and teaching the users
    how to combine pre-built elements”
    I had commented “. . . end users and data modelling don't mix . . . self-service so
    far has been mostly a bust”.
    Here at Redwing, we give out a short White Paper on Business Intelligence Reporting.  It goes to clients and anyone else who wants one.  The heart
    of the Paper is the Reporting Pyramid, which states:  Business intelligence is all about the creation and delivery of actionable intelligence to the right audience at the right time
    For most of the audience, that means Corporate BI: pre-built reports delivered on a schedule.
    For most of the remaining audience, that means parameterised, drillable, and sliceable reporting available via the web, running the gamut from the dashboard to the details, available on
    demand.
    For the relatively few business analysts, that means the ability for business users to create their own semi-customised visual reports when required, to serve
    their audiences.
    For the very few high-power users, that means the ability to interrogate the data warehouse directly, extract the required data, and construct data mining models, spreadsheets and other
    intricate analyses as needed.
    On the subject of self-service, the Redwing view says:  Although many vendors want tot sell self-service reporting tools to the enterprise, the facts of the matter are these:
    v
    80%+ of all enterprise reporting requirement is satisfied by corporate BI . . . if it’s done right.
    v Very few staff members have the time, skills, or inclination to learn and employ self-service business intelligence in the course of their activities.
    I cannot just expose raw data and tell everyone to get on with it.  That way lies madness!
    I think that clean and well-structured data is a prerequisite for delivering business intelligence. 
    Assuming that data is properly integrated, historically accurate and non-volatile as well, then I've just described
    a data warehouse, which is the physical expression of the dimensional model.
    Therefore, exposing the presentation layer of the data warehouse is – in my opinion – the appropriate interface for self-service business intelligence.
    Of course, we can choose to expose perspectives as well, which is functionally identical to building and exposing subject data marts.
    That way, all calculations, KPIs, definitions, and even field names, and all consistent because they all come from the single source of the truth, and not from spreadmart hell.
    So my conclusion is that exposing the presentation layer of the properly modelled data warehouse is – in general - the way to expose data for self-service.
    That’s fine for the general case, but what about Power BI?  Well, it’s important to distinguish between new capabilities in Excel, and the ones in Office 365.
    I think that to all intents and purposes, we’re talking about exposing data through the Data Management Gateway and reading it via Power Query.
    The question boils down to what data structures should go down that pipe. 
    According to
    Create a Data Source and Enable OData Feed in Power BI Admin Center, the possibilities are tables and views.  I guess I could have repeating data in there, so it could be a flattened structure of the kind Melissa doesn’t like (and neither do I). 
    I could expose all the dims and all the facts . . . but that would mean essentially re-building the DW in the PowerPivot DM, and that would be just plain stoopid.  I mean, not a toy system, but a real one with scores of facts and maybe hundreds of dimensions?
    Fact is, I cannot for the life of me see what advantages DMG/PQ
    has over just telling corporate users to go directly to the Cube Perspective they want, that has already all the right calcs, KPIs, security, analytics, field names . . . and most importantly, is already modelled correctly!
    If I’m a real Power User, then I can use PQ on my desktop to pull mashup data from the world, along with all my on-prem data through my exposed Cube presentation layer, and PowerPivot the
    heck out of that to produce all the reporting I’d ever want.  It'd be a zillion times faster reading the data directly from the Cube instead of via the DMG, as well (I think Power BI performance sucks, actually).
    Of course, your enterprise might not
    have a DW, just a heterogeneous mass of dirty unstructured data.  If that’s the case,
    choosing Power BI data structures is the least of your problems!  :-)
    Cheers, Donna
    Donna Kelly

  • Best practices for office 365 SHARED CALENDAR for whole school / organization

    hi
    we need guidance on best practice for setting up SHARED CALENDAR on Office365 exchange server for entire organization (school)of150 staff.
    Requirements
    + all staff should have read only / reviewer permissions on calendar
    +handful staff should have editor permissions on calendar
    + the calendar should synchronise custom categories and colors
    Current Solution
    at the moment we have found that a shared mailbox is the best solution because;
    - allusers can add the shared mailbox on outlook 2010as additional mailbox as readonly
    - all the categories & colors for the calendarare automatically synchronised because the color categories are stored within this mailbox.
    - you can edit calendar permissions in outlook to allow some users as "editor" of the calendar.Problem with Current Solution
    the problem however is that the users also need to access this...
    This topic first appeared in the Spiceworks Community

    Hi Aleksei,
    I think Inactive mailboxes in Exchange Online is the feature that you want. This feature makes it possible for you to preserve (store and archive) the contents of deleted mailboxes indefinitely.
    A mailbox becomes inactive when an In-Place Hold or a
    Litigation Hold is placed on the mailbox before the corresponding Office 365 user account is deleted.
    But I'm afraid that it might be impossible to "easily share certain folders or even whole mailbox with people in the company". As can been seen from below articles, this only allows administrators, compliance officers, or records managers
    to use the In-Place eDiscovery feature in Exchange Online to access and search the contents of an inactive mailbox:
    http://technet.microsoft.com/en-us/library/dn144876(v=exchg.150).aspx
    http://blogs.technet.com/b/exchange/archive/2013/03/21/preserve-mailbox-data-for-ediscovery-using-inactive-mailboxes-in-exchange-online.aspx
    Anyway, this is the forum to discuss questions and feedback for Microsoft Office client. For more details about your question, I would suggest you post in the dedicated forum of
    Exchange Online, where you can get more experienced responses:
    https://social.technet.microsoft.com/Forums/msonline/en-US/home?forum=onlineservicesexchange
    The reason why we recommend posting appropriately is you will get the most qualified pool of respondents, and other partners who read the forums regularly can either share their knowledge or learn from your interaction with us. Thank you for your understanding.
    Regards,
    Ethan Hua
    TechNet Community Support
    It's recommended to download and install
    Configuration Analyzer Tool (OffCAT), which is developed by Microsoft Support teams. Once the tool is installed, you can run it at any time to scan for hundreds of known issues in Office
    programs.

  • Announcing Office 365 compliance toolset for public folders

    Announcing compliance toolset for public foldersWe are pleased to announce that the Office 365 archiving and eDiscovery toolset is being extended to include public folders. Customers can now place public folder content on hold to preserve it for legal or regulatory requirements. They can perform eDiscovery functions on public folder data, including search, preview and export. Entire public folders or specific content (based on keywords) can be preserved immutably. They can delete aged items out of public folders. And all administrative activities on public folders are fully audited. Throughout all these operations, users can continue to interact with public folder data without any change in their experienceTo perform an eDiscovery search on public folders, you simply specify “public folders” as the source. If you choose to put public...
    This topic first appeared in the Spiceworks Community

    Hi,
    Please see the step 1 in the following article:
    Configure Legacy On-Premises Public Folders for a Hybrid Deployment
    http://technet.microsoft.com/en-us/library/dn249373(v=exchg.150).aspx
    Thanks,
    Simon Wu
    TechNet Community Support

  • Bulk Data Storage for Travelling & Charging

    Good day readers
    I shall be travelling later this year and don't want to take lots of SDHC cards. Does anyone have any advice on what to take that stores lotss of Video files and Stills please. I want to keep bulk and weight down.
    I have looked at the Vosonic range but these seem expensive
    I have considered a small Netbook - a lot less than a Vosonic unit - 160gig to 250gig will be sufficient
    I want to be able to view the video files and stills
    I also want to be able to charge the units, there will be no mains electricicity where I am going, so solar charging ideas needed to the Camcorder batteries and the data storage device
    All Ideas welcome, what solutions have people used
    Thanks....

    cookie2402 wrote:
    Harm
    Thanks, but how do I upload the SDHC data to the Hard Drives. I want to upload the SDHC cards and store the data and re-use the SDHC card.
    Sorry if I wasn't clear
    John
    John, many netbooks have a slot for installing your SDHC card and can read it. Then use Harm's suggested USB port hard drive to off load the data.  Possibly even better use an external SSD drive as they take appreciably less power than a hard drive.
    Bill

  • Would i be able to buy microsoft office 365 unlimited access for iphone use in kenya? in kenya

    i need microsoft office 365  for iphone with unlimitted acess is it possible  on free simcard

    You would need to contact Microsoft.  It's their product.

  • "Office 365 Mailbox" missing for users that are member of Ricipent Management role

    Hi,
    I have a hybrid setup with Office 365 and one exchange 2013 standard server on-premises.
    I currently have an issue with that I have a button after pressing the + under recipient to create a Office 365 mailbox from the ECP, but users that are members of the Recipient Management role don't have that button visible.
    What extra permissions are required to be able to create an Office 365 mailbox from the on-premises Exchange?

    Hi SeidKrv,
    Thanks for your update.
    Following article introuduces the permissions that need to assigned before running "New-Mailbox" command.
    Please focus on "Recipient Provisioning Permissions" session.
    Recipients Permissions
    http://technet.microsoft.com/en-us/library/dd638132(v=exchg.150).aspx
    Based on the article, it seems both Recipient Management role and Organization Management role are required.
    More detailed information on both management role as below:
    1. Administrators who are members of the Recipient Management role group have administrative access to
    create or modify Exchange 2013 recipients within the Exchange 2013 organization.
    2. Administrators that are members of the Organization Management role group have administrative access to the entire Exchange 2013 organization and
    can perform almost any task against any Exchange 2013 object, with some exceptions. By default, members of this role group can't perform mailbox searches and management of unscoped top-level management
    roles.
    Thanks
    Mavis Huang
    TechNet Community Support

Maybe you are looking for