Default HubSpot Blog

Digging Into Big Data: There's Gotta Be a Pony in Here Somewhere

Posted by Mark Bunger on Aug 10, 2015 12:10:45 PM

There’s an old joke about an optimistic young boy who wanted nothing more than a pony. He begged his parents for years, and one sunny morning he woke up to see a huge pile of manure in the farmyard. Immediately, he dashed out and started digging wildly in it. His mother rushed out crying, “What on earth are you doing in that muck?” “With this much manure,” he beamed, “there’s got to be a pony in here somewhere!”

As we found in our recent report about Big Data in Agriculture, many precision agriculture vendors are piling up data in the farmyard, and many growers are digging for ponies – and hoping that even more data will produce an even bigger pony, or maybe two or three ponies. For instance, this article quotes Monsanto COO Brett Begemann saying, “It’s a story I always hear from farmers: I’ve got piles and piles of yield data. But I don’t know what to do with it.” A farmer in the same article used Monsanto’s FieldScripts, but even though the prescribed planting regimen didn’t outperform the control region, he’s “optimistic about the potential – even though his first run didn’t show any real benefit.” He believes the pony is in there, somewhere.

Obviously this strategy is as backwards as the joke is ridiculous. And yet, as our research into Big Data in all kinds of industries shows, it’s the rule, not the exception. While the joke's particularly apt in Ag, every industry struggles with its own form of manure. How did we get into this mess?

  • Digging in big data is easy for information-based industries like telecom, banking, and retail, since they already capture the data, just to run their business. They can literally start analyzing it for free, instantaneously – they just upload a pile of existing enterprise data to a vendor’s system and see what the vendor can dig out. If they find something interesting – a pony! – then they can spend money to optimize the system, and find more ponies faster.
  • In contrast, in material-based industries like farming (and manufacturing, automotive, oil), data is not captured consistently or at all. So before users have anything to analyze, they have to spend thousands or millions of dollars to deploy new sensors, and make data connections to ancient machines. Even after all that, they might find that the data tells them nothing actionable – there’s no pony in the pile.

This conundrum explains why, even though material-based industries have been envisioning a digital future (Industry 4.0, intelligent oilfield, personalized medicine…) for years or decades, 87% of enterprises believe Big Data analytics will redefine the competitive landscape of their industries within the next three years, and yet most rate their progress in Big Data as extremely poor.

(click image for individual links)

Look For Ponies, Not Piles

How do you move away from digging in piles of data of uncertain relevance? Simply start looking for ponies, not piles. In other words, start with profits: prioritize profit-driving strategic goals and business cases, and then figure out how to get the data to achieve them. This strategy turns the pile-first process on its head:

  1. What opportunities and threats do we face? Higher yield at lower cost to the grower and the environment.
  2. What use cases will help us attack them? Reducing crop losses due to fungal infestation.
  3. What is the business case for each use case? Applying an improved fungicide costs $5 more per acre, but it improves yield by $7.50 per acre.
  4. What data will each use case need? Where is moisture retained most on the farm, and how much fungicide is effective?

For industries’ Big Data initiatives (like those listed above) to get moving, measurably, the framework above provides a good (re)starting point. Over time, we expect to see adoption accelerate, as the business-first approach becomes more widespread; industry-specific digital maturity models (like ISO9000 and Six Sigma) arise; or the success of new entrants (equivalents to Amazon, AirBNB, and Uber in information industries) grabs C-suite attention in agriculture, manufacturing, oil and gas, and other companies at the information meets matter intersection.


For more on Lux’s research on Big Data, contact Mark at