Extending Power BI with Python and R - Second Edition

9

Loading Large Datasets Beyond the Available RAM in Power BI

In the previous chapter, you learned how to read from and write to a CSV file using both Python and R. When it comes to reading a file, whether you use Power BI’s standard data import feature or the techniques shown in the previous chapter, the main limitation on the file size is the amount of RAM available on the machine where Power BI Desktop is installed.

In a data enrichment phase, it may be necessary to extract information needed for ongoing analysis from very large files (terabytes in size). In these cases, it is almost always necessary to implement big data solutions to be able to handle such large amounts of data. Very often, however, it is necessary to import files that are ...