Bring the Model to the Data: Private Enterprise AI Inside Your Firewall

A whitepaper on data sovereignty, integration over replacement and making the systems you already run searchable.

Executive summary

The obstacle to private enterprise AI is rarely the model. It is the data and specifically what happens to the data when the AI runs. Public AI tools ask you to send corporate records to infrastructure you do not control, in regions you did not choose, under a privacy regime you cannot verify. For any organisation processing personal data in India, that tension is now a legal liability, not a philosophical one. The Digital Personal Data Protection Act, 2023 (DPDP Act) puts purpose limitation, data minimisation and reasonable security safeguards on the statute book, with penalties that scale to ₹250 crore.

This paper describes two systems Kappal deploys for enterprise clients and the design principles behind them.

• The problem. Enterprise data is distributed across SAP, SharePoint and legacy systems that do not talk to each other. Employees cannot find what they need, so they fall back on public AI tools and every paste becomes a compliance liability under the DPDP Act.

• Solution 1: the AI Knowledge Directory. A locally hosted, DPDP-compliant search and discovery layer deployed inside your own VPC. It indexes siloed SAP, SharePoint and legacy data into a centralised vector database and gives your teams semantic search with strict role-based access control. A junior engineer can ask the same question as the CFO and receive a different, correctly scoped answer, because the index enforces the same permissions as the source systems.

• Solution 2: the Private AI Vendor Directory. The same architecture pointed at procurement data. It ingests historical vendor records, delivery timelines and market pricing into a dynamic intelligence graph, so that when a supply chain breaks, procurement officers can discover in seconds which Tier-2 vendors have the capacity, compliance and reliability to step in today.

Both systems run on Azure OpenAI instances locked to the Central India region, inside the client's own virtual private cloud. No corporate data is used for training. No data leaves the firewall. We do not replace your ERP. We make it searchable.

1. The problem is not the model, it is the data

Every enterprise AI conversation we walk into collapses into the same two questions and they are never the questions the vendor wants to answer.

The first is: where does my data go? The second is: what happens to the systems I already paid for? In our experience, the AI budget dies on the second question more often than on the first.

Consider the average mid-sized Indian manufacturer or financial services firm. The customer master lives in SAP ECC. The contracts live in SharePoint. The last fifteen years of vendor correspondence sit in a legacy document system that nobody remembers how to migrate. None of these systems talk to each other in any useful way. The data is not missing. It is present, complete and locked in place.

So what do employees actually do when they need to find something? They ask a colleague. Then they call around. Then, increasingly, they paste the documents into a public AI tool and hope for the best.

That last habit is the one that should worry a compliance officer. The DPDP Act does not distinguish between an accidental leak and a careless one. Processing personal data without a lawful basis or beyond the purpose for which it was collected, is an offence under the statute. The same tool that makes your sales team productive is quietly turning every employee into someone who processes data without knowing the obligations attached to it.

The deeper problem is structural. Enterprise data is not sitting in a lake waiting to be queried. It is distributed, duplicative, permissioned and almost entirely unstructured from the perspective of the people who need it. The value is there. The access is not.

This is the gap we kept running into and it is the reason we stopped proposing yet another model and started building the thing that sits between the model and the data.

2. Design principle one: data sovereignty is the product

We built the first version of the Knowledge Directory the way most teams would, with the embedding pipeline in the same tenant as everything else. It worked. It was also undeliverable, because the client's CISO could not sign off on it and we could not argue with his reasons.

Three questions came up in every security review, in the same order:

1. Which region is my data processed in?
2. Who can see it, including your staff?
3. Is my data used to train anything?

We decided to make those three questions the specification rather than the objection. Every subsequent design decision starts from the answer.

Region. All model inference runs on Azure OpenAI instances locked to the Central India region. If your data is Indian, it stays in Indian infrastructure. This is a deliberate choice: it aligns with how the DPDP Act treats cross-border transfer and it gives the client a single, auditable statement of where processing happens. We do not route anything through another region for convenience, because convenience is exactly how sovereignty commitments evaporate.

Tenancy and training. Corporate data is never used to train or fine-tune models, not in our infrastructure and not in the foundation model. We run inference against the deployed model with your data in context and we do not persist prompt or document content outside your VPC beyond what the retrieval pipeline itself requires. This is not a marketing claim. It is a contractual term we put in writing.

Residency. The deployment is a locally hosted stack inside your own VPC. The vector store, the index, the orchestration layer and the model endpoints all live inside your network boundary. Zero data leaves your firewall. Not for a batch job, not for a report, not for a feature demo.

There is a common objection here and it deserves a straight answer.

Yes, a locally hosted, in-region stack costs more to operate than a shared SaaS tenant. The difference is that it converts an unquantifiable compliance risk into a fixed, known infrastructure cost. For a significant data fiduciary, that trade is not even close.

3. Design principle two: integration over replacement

The second question, about the systems you already paid for, deserves the same bluntness. Big companies do not replace their ERP because someone builds a nicer one. They do not replace it because the ERP is not a product they chose. It is a set of business processes that have been accumulating for twenty years. The cost of replacement is not the licence fee. It is the risk of breaking order-to-cash in a company where order-to-cash is the company.

We do not position ourselves against the ERP. We sit on top of it.

The Knowledge Directory connects to SAP ECC and Oracle through their native interfaces: RFC and BAPI on the SAP side, REST and OData where the system exposes them. The same holds for SharePoint and for the legacy document stores and for whatever the client is actually running. We pull the metadata and the content that needs to be searchable, we index it and we return the user to the source system for the authoritative record. Writes, where they exist, go back through the same governed APIs, so the source of truth stays the source of truth.

The positioning is simple and we use it in every conversation: we are not here to replace your ERP. We are here to make it searchable.

This reframe matters more than it sounds. It changes the buying conversation from a migration project to an augmentation project. The risk profile collapses. The CTO is no longer defending a multi-year, multi-crore replacement plan against a sceptical board. They are approving a pilot that connects to a system already running, with read-only access and measurable outcomes in weeks.

4. The AI Knowledge Directory: how it actually works

The Directory is a retrieval pipeline, not a chatbot bolted onto SharePoint. The pieces are worth naming, because the failure modes live in the joints between them.

• Connectors. Each source system has a connector that understands its permission model. The SAP connector reads the same authorisation objects your users already have. The SharePoint connector reads the same sharing permissions. The legacy connector handles scanned documents with OCR, because most of the institutional knowledge in an Indian enterprise is still paper that was scanned and then forgotten.

• Ingestion and chunking. Documents are split into chunks sized for retrieval, with structural context preserved. This sounds mundane and it is where most retrieval quality is won or lost. A chunk without its section heading is a chunk that cannot answer a question.

• Embedding and storage. Chunks are embedded using Azure OpenAI embeddings, again in the Central India region and stored in a vector store inside the VPC. We keep the keyword index too. Hybrid retrieval, keyword plus semantic, consistently outperforms pure vector search on the messy, acronym-heavy language of real enterprise documents.

• Query time. A user's query is embedded, both retrieval paths run, results are re-ranked and then the access control layer runs. The RBAC check is not an afterthought on top of search. It is the search. Every result is filtered against the permissions materialised from the source systems, so the index can only return what the specific user could open in the source system anyway.

This is the part we are most asked about, because it is the part that usually gets skipped. Permission models in SAP and SharePoint are per-user and per-object. You cannot index once and let everyone search everything, because that defeats the entire purpose of having permissions. We materialise the effective permissions at index time, for each document and each group and we enforce them at query time. The practical consequence: a junior engineer can search the same corpus as the CFO and see a smaller, correct, authorised slice of it. The financial projections are in the index. They are simply not in the junior engineer's answer.

We will be honest about the cost of doing this properly. Materialising permissions adds ingestion overhead and a little query latency. That is the difference between a demo and a deployment and we build the real one.

5. The Private AI Vendor Directory: procurement gets the same treatment

The second deployment is the same architecture pointed at a different, equally siloed dataset: your supply base.

A mid-sized manufacturer we know keeps vendor master data in the ERP and purchase order history in the ERP too. But delivery performance lives in a spreadsheet that one manager maintains. Compliance certificates live in email attachments. Market pricing lives in whatever a sourcing analyst last negotiated. When a supplier fails, the knowledge of who could step in is distributed across roughly twenty people, none of whom are in the same meeting.

The Vendor Directory ingests all of it: vendor master records, historical delivery timelines, quality and compliance records and market pricing. It builds a dynamic, searchable intelligence graph, not a static table. The graph knows that Vendor A delivers on time in normal conditions but fails under surge demand. It knows that Vendor B, a Tier-2 supplier who has never held a prime contract, has the capacity, the certifications and the delivery record to step in.

The procurement scenario is the one that sells this to operations teams. When the supply chain breaks and it will break, your procurement officers should not have to call twenty people and reconstruct what the data already knows. They type a question in plain language: who can supply this component, in this region, within this timeline, with this compliance certificate. The graph answers in seconds, ranked by capacity, compliance and historical reliability. The officer still makes the call. But the call is informed by everything the company knows, not by whoever happened to pick up the phone first.

Nothing about the vendor data leaves the firewall either. Your supplier relationships, your negotiated prices and your exposure when a supplier fails are commercial intelligence you do not want in a shared cloud tenant and the architecture does not require you to put them there.

6. Security and compliance posture

For anyone evaluating this against the DPDP Act, here is how the pieces line up.

The enterprise remains the data fiduciary under the Act. We act as a data processor on its instructions and under contract, which is the relationship the Act is built around. Purpose limitation is enforced by design: the index stores only what the deployment is scoped to index and nothing is repurposed. Data minimisation is enforced at ingestion; we do not hoard source systems wholesale. Reasonable security safeguards are implemented as a default architecture: encryption in transit and at rest, tenant isolation inside the VPC, RBAC at every layer and audit logs of who queried what. And because inference runs in the Central India region, the organisation can state, in one sentence, where its data is processed.

We also keep the human layer honest. When access is revoked in the source system, it is revoked in the index at the next refresh. We do not maintain a parallel permission set that drifts from the one your security team already manages.

7. What a deployment actually looks like

The pattern we recommend and the one that has worked, is deliberately unglamorous.

Start with one connector and one use case. A pilot should index a single system, usually the one with the most trapped knowledge and serve a single team. Define the success metric before the pilot starts and we recommend retrieval hit rate: of the questions that team actually asks, what fraction returns a useful, authorised answer. Four to six weeks is enough to know whether the deployment earns its keep. Then add connectors, expand the RBAC mapping and widen the user base one team at a time.

The sequencing exists for a reason. Every connector you add is a new permission model to materialise and a new set of users to protect. It is better to do one system perfectly, with the security review signed off, than to index everything and then explain to a CISO why a test environment's documents are showing up in production search.

8. What we believe

The next decade of enterprise software will not be won by whoever builds the better model. It will be won by whoever makes the systems companies already run searchable, without asking those companies to surrender their data to do it.

The model is a commodity that gets cheaper every quarter. The data is not. It is unique, it is permissioned and it is protected by law. The companies that treat their data that way and bring the model to the data instead of the other way around, are the ones that get the benefit of AI without inheriting its liabilities.

We built the Knowledge Directory and the Vendor Directory on that bet. The first makes your existing systems answerable. The second makes your supply base legible at the exact moment you need it. Both run inside your firewall, in your region, under your control.

Your ERP does not need replacing. It needs to be asked the right questions. We built the system that lets it answer.

About Kappal Software

Kappal Software Private Limited (kappal.in) is a software company building custom applications, AI/ML solutions and cloud infrastructure for enterprises across AgriTech, FinTech, e-commerce and manufacturing. Our tagline is "Engineered for Efficiency" and we mean it literally: we build systems that remove friction from operations that already work, rather than asking clients to rebuild what they have. For more on this work, visit kappal.in or write to us through the contact page.