Skip to main content

Costs of running data reporting

Problem statement​

Let's assume we are running a field services business and we have to identify revenue generated per branch. Our financial information is managed in Netsuite and we use ServiceTitan to manage operations.

Reporting requirements​

In order to accomplish the above:

  • We need to fetch the revenue data (Invoices) from Netsuite. Next identify what all APIs in Netsuite constitute Invoices data.

  • We need to fetch the operations data per branch (jobs) from ServiceTitan. Next identify what all APIs in ServiceTitan constitute Jobs data.

  • Find the common link between Netsuite Invoices and Service Titan Jobs to be able to establish a link between revenue and branch operations.

  • Clearly identify the definition of "revenue". Do we exclude a few invoices or skip a few branches.

  • Clearly identify how you want to report on the data - daily, weekly or multiple times a day.

  • Finalize the report design (look and feel) and identify whom all should the report be shared with. Executives should be able to see all the branches while the individual branch managers should be restricted to their branches.

Teams involved in implementation​

The above activities can be broadly split across the following four team members:

  • Business User - The user driving the reporting reqiurements

  • Product Manager - Bridge between the technology and business user.

  • Data Engineers - Responsible for fetching the data from Netsuite, ServiceTitan and for joining the data from the two.

  • BI Developer - Responsible for developing the BI reports and sharing it with the correct audience.

  • Optional - You may have to interact with Netsuite and Service Titan experts to map the correct APIs.

Tools required to implement the above​

  • Product Manager - Project Management tools

  • Data Engineers -

    • ETL Tools for fetching the data (Airbyte, FiveTran, ELT Data, ADF, AWS Glue, etc.) and to make sure the data is not duplicated.
    • Transformation tools to join the Netsuite and ServiceTitan data. Commonly used tools - Spark, ADF Dataflows, etc.
    • Orchestration tools - To schedule, manage and coordinate the data ingestion and transformation processes. Commonly used tools - Airflow, Prefect, Dagster, etc.

  • BI Developer - BI Tools for data visualization. Commonly used tools - Power BI, Tableau, Qlik, etc.

Total costs​

The total cost to run the above reporting requirements will be inclusive of the people + technology costs + cloud costs. The annual costs to keep reporting up can easily exceed $50K.