Incremental Refresh is only available on certain Power BI accounts and requires the use of Power BI Service
Power BI account types that support Incremental Refresh
Incremental Refresh is only available on certain Power BI accounts and requires the use of Power BI Service (powerbi.com):
- Power BI Premium
- Premium Per User (PPU)
- Power BI Pro
- Power BI Embedded Models
Important Note:
- Incremental Refresh only works in Power BI Service. If you test in Power BI Desktop, the refresh will only bring in data within the RangeStart and RangeEnd parameters for endpoints that have Incremental Refresh set up.
- Once a dataset with Incremental Refresh is published to Power BI Service, it cannot be downloaded again as a PBIX with data included. This is different from a normal report that allows full refreshes and downloads.
How Power BI Detects Data Changes
Incremental Refresh relies on a DateTime column to identify which rows fall into the archive or refresh window.
In the HammerTech template, the refresh window is set to 1 year (a relatively large window). Detect Data Changes is enabled using the LastModifiedDate field.
This means that during each refresh:
- Only new records are added
- Updated records within the refresh window are reprocessed
- Power BI does not reprocess the entire year’s worth of data on every refresh.
This approach makes the refresh even more efficient, as only the relevant partitions are updated.
Best practices and considerations for Power BI Incremental Refresh
First Refresh: The initial refresh after publishing can take longer as both archive and incremental data are processed. If the initial refresh fails, consider reducing the archive window.
Testing in Desktop: Remember the incremental refresh logic will not be applied until published to Power BI Service.
Adjusting Windows: Use smaller refresh windows if refresh times are too long, or extend archive periods if clients need access to older history.
Monitoring: Using the Refresh History in Power BI Service to track performance and identify bottlenecks.
Troubleshooting
Endpoint doesn’t have a Created Date field
Some endpoints might not contain a Created Date. In these cases, the LastModifiedDate field can be used as the partition date. However may cause issues such as
- Duplicate rows when edits occur outside the archive window
- Reduced performance benefits if more rows than expected are reprocessed
- Configuring Incremental Refresh is still recommended in these cases, but these limitations should be kept in mind.
Edits are coming through as duplicated rows
This usually occurs when the LastModifiedDate is used as the partition date and edits are made outside the archive window. To realign the data, reupload the PBIX file to trigger a full refresh.
Historical Data is edited but not reflected
If edits are made to records that fall within the archive window, the will not appear in refreshes. To capture these edits, reupload the PBIX file to trigger a full refresh.
Incremental Refresh option is disabled (greyed out)
This happens when the table does not have filters applied for RangeStart and RangeEnd. Ensure you have applied parameters to a valid DateTime field in Power Query before enabling Incremental Refresh.