What is Incremental Refresh?

Incremental Refresh is a data processing approach where only new or updated data is refreshed, instead of reloading the entire dataset every time.

Why would you use Incremental Refresh?

Incremental Refresh is most useful in large databases, reports or systems where refreshing all the data repeatedly is inefficient, especially when most of the data doesn’t change (e.g. previous years’ records remain unchanged). Incremental Refresh updates only recent data or modified data. This means that it reduces refresh times, uses less resources and improves reliability.

Why use Incremental Refresh with the HammerTech Reporting API?

Within the HammerTech reporting API, there are multiple endpoints which will have a lot of data over time. For example, the employerDiaryEntries endpoint creates daily records for each employer, and the observations endpoint stores every inspection question and answer in the system. For such endpoints, it is best practice to incrementally refresh these endpoints to reduce refresh times and minimize resource allocation, e.g. Hours worked recorded in the employerDiaryEntries from 8 months ago wouldn’t change.

Incremental Refresh is also useful for endpoints where data rarely changes. For example, reportable questions answered by workers during induction often remain unchanged for the duration of their employment.

HammerTech Reporting API endpoints suggested for Incremental Refresh

Endpoints that store large amounts of data, or are not frequently edited, are also good candidates for Incremental Refresh. These include (but are not limited to):

Large datasets -

Issues
Observations
Employer Diary Entries
Meetings
Incidents
Injuries
Worker Hours Worked
PTPs

Not frequently edited -

Worker Profile Reportable Answers
Worker Reportable Answers
Permits
PTP Tasks