The rise of big data has transformed decision-making across industries, offering unprecedented access to large-scale, quantifiable insights. Yet, as data volumes grow, a paradox is emerging: the more we quantify, the more we need qualitative insights to make sense of it all. This is where thick data – ethnographic, contextual, and human-centred information becomes indispensable, says Rennie Naidoo, professor in Information Systems (IS) and research director at the Wits School of Business Sciences.
The tension between big data, large-scale, quantitative data used to identify patterns and trends, and thick data, deep, qualitative insights that capture human context and meaning, highlights a core challenge in modern analytics: a quantification bias.
Organisations often assume that if something cannot be measured numerically, it is not worth considering.
But what happens when key insights such as customer motivations, cultural dynamics, or emotional responses fall outside the realm of quantifiable metrics?
The power and limits of big data
Businesses rely on algorithms to optimise operations, detect patterns, and automate decision-making. In marketing, big data helps companies personalise content, predict consumer behaviour, and fine-tune advertising strategies.
In finance, it powers risk assessment models that inform investment decisions. In healthcare, it enables predictive analytics that improve patient outcomes.
Despite these advantages, big data has inherent limitations. It excels at identifying correlations but often struggles with causation. Algorithms can tell us what is happening, but not always why it is happening.
A recommendation engine may know that users in a certain demographic prefer a specific product, but it does not understand their underlying motivations. A predictive policing system may highlight high-crime areas based on historical data, but it does not account for the socio-economic conditions shaping those patterns.
The rise of quantification bias
Quantification bias emerges when organisations prioritise measurable data over qualitative insights, assuming that only what can be counted holds value. This mindset leads to a reliance on algorithms and analytics, while overlooking the complexities of human experience that cannot be reduced to numbers.
This bias is particularly evident in corporate decision-making, where key performance indicators (KPIs) and dashboards dominate strategy sessions. Organisations often optimise for the metrics they can track, shaping business strategies around numerical targets.
Yet, when data lacks context, decisions based purely on quantifiable trends can lead to unintended consequences.
Consider AI-driven recruitment platforms. Many companies use automated screening tools to rank candidates based on quantifiable factors such as education level, years of experience, or previous job titles.
However, these systems often fail to account for personal integrity, drive, or leadership potential – qualities that are difficult to quantify but crucial for long-term success. As a result, organisations may inadvertently filter out candidates who could have excelled in ways that traditional metrics fail to capture.
Thick data as a corrective lens
Thick data provides the human context that big data lacks. Rooted in ethnographic research, it offers deep, qualitative insights into behaviour, culture, and emotions.
Unlike big data, which looks at patterns across large datasets, thick data focuses on small, meaningful observations that reveal the reasoning behind those patterns.
Research intensive universities are increasingly recognising the importance of integrating both quantitative and qualitative methods in data science and graduate research programmes.
Mixed-methods approaches, which combine statistical analysis with ethnographic insights, are being introduced into curricula to address the limitations of purely numerical data.
Tech companies increasingly recognise the need for thick data to refine their algorithms and humanise their AI-driven insights. For example, user experience (UX) research relies on thick data to understand how people interact with digital platforms beyond simple engagement metrics.
It is not enough to track click-through rates. Companies must also understand why users abandon a webpage or why they feel frustrated with an interface.
Similarly, in healthcare, predictive models will benefit from ethnographic research that explores how patients experience illness, trust medical professionals, or adhere to treatment plans.
A model predicting high rates of hospital readmission based on quantifiable factors such as age, previous visits, or chronic conditions may miss a crucial qualitative insight.
For example, patients who lack social support or feel misunderstood by healthcare providers are more likely to return, regardless of their clinical data.
When organisations prioritise big data without integrating thick data, the result is often misinterpretation, bias reinforcement, and flawed decision-making.
A clear example in South Africa is how crime prevention strategies rely heavily on crime heat maps generated from reported incidents.
While these maps help allocate law enforcement resources, they also risk reinforcing blind spots. In many townships and informal settlements, crime often goes unreported due to deep-rooted distrust of the police and the courts.
As a result, big data skews attention toward areas where crime is formally recorded rather than where it is most deeply felt.
This creates a cycle where well-policed suburbs receive more proactive interventions while high-crime, but underreported areas remain underserved. Thick data gathered through community engagement and ethnographic research could help break this pattern.
By understanding why certain crimes go unreported, how residents perceive law enforcement and the courts, and what social conditions drive criminal activity, decision-makers could develop more nuanced and effective crime prevention strategies.
Solutions would then go beyond surveillance and response to focus on trust-building, economic upliftment, and local partnerships, a far more sustainable approach to public safety in South Africa.
In corporate environments, over-reliance on big data can lead to misaligned marketing strategies. A brand might analyse millions of consumer interactions and conclude that younger customers prefer fast, flashy ad content.
However, thick data might reveal that while younger consumers engage with these ads, they actually build brand loyalty through long-form storytelling and authenticity.
Relying solely on numerical engagement metrics could lead to marketing campaigns that miss deeper emotional connections.
Data synergy, not data supremacy
The future of decision-making does not lie in choosing between big data and thick data, but in integrating the two. Organisations must recognise that numbers tell only part of the story.
A data-driven culture should not mean a data-exclusive culture. Instead, it should balance quantitative breadth with qualitative depth.
True data intelligence requires interdisciplinary collaboration. Data scientists and ethnographers must work together, ensuring that algorithms are informed by lived experiences and cultural nuances.
Businesses should invest in human-centred research alongside AI development, ensuring that models reflect not just measurable patterns but meaningful narratives.
Technology itself is evolving to bridge the gap between big and thick data. AI systems are increasingly incorporating natural language processing, sentiment analysis, and context-aware computing to capture human experiences beyond traditional numerical inputs.
Digital platforms are exploring new ways to collect qualitative insights at scale, blending machine learning with user feedback loops.
The future is not about choosing between big data and thick data. It is about integrating the two. The more we quantify, the more we must complement our analysis with context, meaning, and human insight.
Organisations that embrace this dual approach will be better equipped to make informed, ethical, and effective decisions in an increasingly complex world.
Ultimately, the future belongs less to those who can count and more to those who will listen closely to the stories behind the numbers.
