Data Science

Your job is at the tip of the spear.

Many years ago, I used to do this job — just not under the same name.

It wasn’t called Data Science back then. Around 2018 I started noticing new titles appearing: Data Scientist, Data Engineer, Data Architect.

Before that? It was Data Analyst, Systems Architect, Batch Team, DBA.

The data world became more sophisticated than just managing customer information for businesses. The arrival of Python allowed analysts to experiment much more quickly — testing algorithms and formulas on datasets pulled from the warehouse.

Previously, they might not have had the access — or the appetite from IT — to build applications directly inside the warehouse just to experiment. Pulling data down onto a workstation and running transformations for a few hours didn’t raise too many governance alarms.

And that’s really one of the core roles of the Data Scientist:

Try stuff.

These days I’m actually amazed at how far the role has come from the nickname “SQL’er”.

Facts, Hunches & Proving Things

You’ll hear a lot of people getting worked up about politics and debates these days. They say:

“I believe in facts. I don’t do hunches.”

Well — your job is to have hunches.

Then prove them right or wrong.

If they’re right? Hot dog.

Next step: document it properly and get a colleague to double-check it. If your hunch holds more water than your belly button, then you start showing its value to the business.

That “thing” you created might be:

A report
A graph
A predictive model
Something dead cool that drops into the main app and makes customers and colleagues very happy

That’s impact.

The Dependency Trap

But here’s what happens next.

That “thing” becomes part of design plans. It could take months to get onto the development pipeline. Meanwhile, you’re expected to keep producing it manually.

Every Monday you roll into work and bang out three reports.

Now you can’t get back to your next hunch.

You start developing what I call dependency memory.
The business depends on you remembering to run things.

In the old days, I was lucky. IT security was looser (Windows XP era…), and with some PowerShell knowledge I could automate parts of that process and free up time for new insights.

Then came the lockdown years of tighter security — execution rights restricted, automation frowned upon.

But technology moved on.

SQL Server Agent gave structured job control.
Modern platforms like Databricks let you spin up compute specifically for experiments and scheduled jobs.
Pipelines became safer, auditable, and repeatable.

The mature Data Scientist doesn’t just prove a hunch — they industrialise it.

And If Your Hunch Is Wrong?

That’s good news too.

If you document your method and outcome, nobody else has to chase the same dead end.

That’s progress.

Failure, properly recorded, is efficiency.

Data Science isn’t just statistics.

It’s curiosity. It’s controlled experimentation. It’s proving things. It’s automating value. It’s turning “I think…” into “Here’s the evidence.”

And if you do it well —
you’re not just at the tip of the spear.

You’re deciding where it points.