There's Gold in Your Data, But Also a Lot of Dirt.
The AI arms race is in full swing. Across industries, there’s a mad scramble to integrate AI, as if it were some kind of magical machine that turns corporate data into wisdom.
- “We need AI!”
- “We need GPT-driven insights!”
- “AI will revolutionise our business!”
There’s gold in those hills, they say. And maybe there is. But here’s the question nobody’s asking: What if the data you’re feeding it is rubbish?
A close friend summed it up perfectly: “Feeding GPT with bad data just enables you to make poorly informed decisions quicker.” (Thanks, Steve.) And as it turns out, most corporate data isn’t gold—it’s a 44-petabyte mountain of unstructured chaos.
The Data Mountain Problem: This Isn’t a Gold Mine, It’s a Mess
We recently saw a real-world scan across 44PB of data. The results were horrifying:
- Max folder depth: 106 (Yes, 106 layers deep. Imagine finding a document in that.)
- Empty directories: 98.6 million (Folders that serve no purpose. Just sitting there.)
- Subfolders in a single directory: 3.8 million (A directory so deep it’s practically a cave system.)
- Files & directories older than Jan 2022: 178.4 million (Nobody’s touched them. Nobody knows what’s in them.)
- Dark Data: 93TB (Data stored but never used—just rotting away.)
- ROT Data (Redundant, Obsolete, Trivial): 0.89PB (That’s nearly a petabyte of useless information.)
And yet, this is the data estate that companies want to train AI on. It’s like handing a prospector a shovel, pointing at an entire mountain range, and saying, “Find the gold.”
Good luck.
AI Needs Clean, Structured, Relevant Data—Not Digital Landfill
Here’s the thing: AI doesn’t fix bad data. It magnifies it.
- Bias In, Bias Out – If your corporate data is messy, incomplete, or outdated, AI will bake those errors into every insight it generates.
- False Confidence – AI models don’t “know” things; they predict patterns based on inputs. If those inputs are nonsense, AI will generate nonsense with authority.
- Compliance & Security Risks – If sensitive or non-compliant data is buried in unstructured storage, AI won’t know to exclude it—until it’s too late.
- Storage & Compute Waste – AI models cost a fortune to train and run. If you’re processing terabytes of ROT data, you’re literally burning money for no reason.
It’s not enough to have AI. You need the right data—or you’re just making bad decisions at machine speed.
How Do You Extract the Gold Without Hauling the Dirt?
Successful gold miners didn’t just grab a shovel and start digging. They used precision tools to separate gold from waste.
The same goes for AI.
- Data Discovery & Classification
- Before feeding AI, you need to identify, classify, and clean your data—or risk training models on duplicated, corrupted, or outdated records.
- Data Governance & Quality Controls
- AI models don’t ask: “Is this data accurate?” That’s your job. Without governance, AI becomes an amplifier of bad decisions.
- Real-Time Data Validation
- AI is most powerful when it’s trained on fresh, relevant data. Outdated inputs lead to outdated insights.
- AI-Specific Risk Management
- If your AI is making business-critical or compliance-related decisions, you need audit trails, explainability, and oversight—because regulators are watching.
Final Thought: Not Every Prospector Struck Gold
During the original Gold Rush, many hopefuls never found a single nugget. Others wasted their life savings digging in the wrong place.
The ones who got rich? They knew how to find, refine, and extract value properly.
The same applies to AI. If you don’t audit, clean, and refine your data before plugging it into AI, then you’re just another prospector swinging wildly at a mountain of rock, hoping for the best.
If you’d rather not leave it to chance, have a look at Lightning IQ—because gold mining is easier when you have the right tools.
Nick Pollard leads EMEA consulting for One Discovery. He is a seasoned leader with more than 20 years of experience working in real-time investigation, legal and compliance workflows across highly regulated environments like banking, energy and healthcare as well as national security organizations. You can contact at nick.pollard AT onediscovery.com
Nick Pollard : Mar 12, 2025 9:27:24 AM
