The cheapest warehouse was the one we already had

Ara ran the free version of Google Analytics 4. It answers most day-to-day questions well, but it keeps detailed event data for at most 14 months. That is barely enough to compare one year with the last, and nowhere near enough to see longer trends. The marketing team and the data analysts wanted both.

The plan #

Working with Ara’s data engineer, I set out a route that would keep the raw data for as long as we needed it and put it in front of the people who use it:

1. GA4 to BigQuery. GA4 can export every event to Google BigQuery each day, even on the free version.
2. BigQuery to an Azure data warehouse. BigQuery had a retention limit of its own, so the raw events would be copied to Azure for safekeeping.
3. Azure to Power BI. Analysts and marketing would explore long-term and year-on-year figures in dashboards, rather than learning GA4’s own reports.

The 60-day catch #

The limit in step two came from where we started. The BigQuery sandbox needs no billing account, and in the sandbox every table expires after 60 days. The export was keeping less history than GA4 itself.

Turning on billing removes the expiry. That changed what the warehouse was for: once BigQuery could keep the data indefinitely, Azure was no longer needed to keep it safe, only to get it into Power BI.

Costing the warehouse #

So we costed both. Running an Azure data warehouse would have cost more than keeping the data on Google Cloud, and keeping it on Google Cloud cost almost nothing: once we passed BigQuery’s free storage allowance, the bill came to around 30 to 40 cents a month.

We kept the data in BigQuery. The cheapest warehouse was the one we already had.

Where it stopped #

This was a side project. I led it across the web and data teams because I wanted marketing to have better data, and it was never anyone’s main priority, so we worked on it when the main work eased off.

The last stage, connecting the data to Power BI, is where it stalled. Then the people behind it moved on. Our data engineer left, then one of the two data analysts on the performance team, and within a year I had left too. With nobody left to carry it, the project was abandoned, even though I still think it would have been very valuable to Ara.

What I’d do differently #

The technical decisions held up. What the project lacked was ownership that didn’t depend on the people doing the work. I would ask a project leader to help set the goals, hold regular check-ins, and keep us accountable to it. A side project needs that more than a main one, because nothing else keeps it moving, and because it has to survive people leaving.g