The Knowledge Problem in AI

In 1945, Friedrich Hayek wrote an essay that most technology companies building AI have not read. Its title was "The Use of Knowledge in Society," and its central argument was that the knowledge relevant to economic coordination is dispersed across millions of individuals, mostly tacit, and cannot be centralised without being destroyed.

Hayek was writing about price systems. The argument applies equally well to AI systems.

The technology we are building captures what was written down — explicit knowledge, documented processes, recorded decisions. It does this extraordinarily well. But the most valuable knowledge in any organisation, market, or society was never written down. It exists in judgment, relationship, experience, and context. And AI systems, at present, have no native way to access it.

This is the knowledge problem in AI. It is not a bug that will be patched. It is the central challenge of deploying AI that actually produces results rather than plausible-sounding outputs.

What AI actually knows

A large language model knows what existed in its training data. It knows what was published, posted, documented, indexed. It does not know what was understood, felt, negotiated, or passed between people in conversations that were never transcribed.

eGain's research found that 35% of critical operational knowledge in organisations isn't documented anywhere — but is repeatedly requested. That is the fraction AI cannot access. It is also, in most organisations, the fraction that matters most.

35% of critical operational knowledge is never documented anywhere. It is also the most-requested knowledge in most organisations. AI has no access to it.

The hallucination problem is a symptom of this gap, not a separate phenomenon. Enterprise benchmarks report 15-52% hallucination rates across commercial LLMs. In legal contexts, that figure rises to 69-88%. In medical contexts, 43-64%. What is the AI doing when it hallucinates? It is filling a knowledge gap with something that sounds right, because the actual answer isn't in the training data. The hallucination is the knowledge problem made visible.

47% of enterprise AI users made at least one major business decision based on hallucinated AI content in 2024.

Retrieval-Augmented Generation reduces hallucination rates by up to 71% when properly implemented — because it gives the model access to specific, current, organisational knowledge rather than forcing it to rely solely on training data. That improvement is not a model improvement. It is a knowledge access improvement.

What this looks like in hiring

I have been building Majhi OS — autonomous hiring operations infrastructure — for long enough to know exactly where AI fails in recruiting, and why.

The explicit layer of a hiring system is well-suited to AI. Identifying active mandates. Tracking pipeline stages. Measuring response rates. Detecting where communication has decayed. Flagging mandates approaching failure. These are pattern-recognition problems over recorded data, and AI handles them well.

The tacit layer is where AI hits a wall.

What makes a specific VP of Sales candidate actually right for a company at this moment — given the CEO's leadership style, the board's risk tolerance, the market the company is entering, the team dynamics that haven't been resolved — is knowledge that exists in the minds of maybe five people. None of it is in any database. It cannot be inferred from a LinkedIn profile or a resume. It is the product of relationship, observation, and judgment built over years.

This is where Majhi Group operates. The value of retained executive search is not access to candidates — that problem is largely solved. The value is the tacit knowledge: who is actually right for this specific seat, why, and what will make them succeed or fail in this particular company at this particular moment. We've placed 25+ C-suite and VP leaders with a 90%+ offer acceptance rate and a 30-45 day average close. That outcome is not produced by better database access. It is produced by better judgment — accumulated, experiential, and tacit.

An AI system that does not understand this distinction will automate the wrong things, confidently.

India and the undocumented knowledge layer

India leads global enterprise AI deployment at 57%, with 47% of Indian enterprises now running multiple AI use cases in production. Indian businesses invested an average of $31 million in AI in 2025 — above the global average.

But the knowledge problem is particularly acute in the Indian context. How business actually gets done in India — the relationship networks, the regional trust dynamics, the institutional knowledge of which cities produce which kinds of talent for which roles — is almost entirely tacit. It lives in communities, families, professional networks, and years of operating in specific markets. Almost none of it is in the training data of any AI system built primarily on English-language internet content.

AI training data has documented bias toward overrepresented topics, with hallucination rates rising by 25% or more on underrepresented subjects. Indian regional markets — Bhubaneswar, Cuttack, Vizag, Coimbatore, Indore — are underrepresented in global AI training data in ways that matter enormously for any organisation trying to use AI to make decisions about those markets.

In Kalahandi — the district in western Odisha where I am from — this gap is even wider. The knowledge of local talent pipelines, institutional quality, economic mobility patterns, and what opportunity actually looks like for a graduate from Bhawanipatna versus one from Bhubaneswar: none of this is documented. It exists in local networks. It will not be captured by any AI system trained on global internet data. And it is exactly the knowledge that matters for anyone trying to build economic infrastructure in the region.

The AI adoption conversation in India tends to focus on model access and infrastructure. The harder conversation — what knowledge needs to be made explicit before AI can be useful — happens less often. That is the conversation that will determine which Indian organisations actually benefit from their AI investments.

What this means practically

California Management Review published research in 2026 making the argument that tacit knowledge is the next competitive moat precisely because AI commoditises explicit knowledge. If AI can access it, anyone with API access can access it. What remains as competitive advantage is what AI cannot reach: judgment, relationship, operational intelligence built from experience.

This reframes the AI adoption question. The question is not only "what can AI do for us?" It is "what do we know that AI doesn't — and how do we build systems that extend that knowledge rather than substitute for it?"

The Majhi OS architecture takes this seriously. The system is designed to accumulate operational learning — which mandate recovery actions actually work, which recruiter patterns correlate with shortlist acceptance, which candidate signals predict offer acceptance or early attrition. That accumulated intelligence is not in any AI model's training data. It is built from operational observation over time, in specific markets, for specific hiring profiles. That is the moat. Not the model — the proprietary knowledge the model is trained against.

For organisations thinking about AI adoption, the implication is this: AI amplifies what you know. It does not generate what you haven't captured. The organisations that benefit most are those that have made their operational knowledge explicit — documented, structured, and accessible — before deploying AI on top of it. Those that skip that step are deploying AI on a weak foundation, and the 85% of AI projects that fail due to data quality problems are the predictable result.

The knowledge problem is not a reason to avoid AI. It is the reason to think carefully about what you know, what is written down, and what needs to be captured before you hand the system a problem and expect a reliable answer.

Sources: eGain — Tacit Knowledge Research · SQ Magazine — LLM Hallucination Statistics 2026 · Suprmind — AI Hallucination Rates · Getmaxim — AI Hallucinations 2025 · EY-CII — India AI Report 2025 · California Management Review — Tacit Knowledge as Competitive Moat