Open Source vs Self-Hosted vs API Key: Where Your Client's Data Actually Goes
Three phrases get used interchangeably in every conversation we have with CA firms about AI, and they mean completely different things:
"We'll use an open-source model, so the data stays with us."
That sentence contains a mistake. Open-source has nothing to do with where the data goes.
This post separates the three ideas, because the confusion between them is what leads firms to either paste client ledgers into a consumer chat product believing it is safe, or spend on GPUs they did not need believing it was the only compliant option.
The Two Questions That Are Actually Being Asked
There is one axis people think they are choosing on, and there are really two.
Question 1 — Who controls the model weights?
Open weights (Llama, DeepSeek, Mistral, Qwen) means the model file is published and anyone can download and run it. Closed weights (GPT, Claude, Gemini) means only the vendor can run it.
Question 2 — Whose machine does the inference run on?
Your own hardware or your own cloud tenancy — or somebody else's.
These are independent. Crossing them gives four real configurations, not three:
| Vendor's machine | Your machine / your tenancy | |
|---|---|---|
| Closed weights | ChatGPT, Claude, Gemini — consumer subscription or paid API key | Rarely available; some enterprise deals only |
| Open weights | Llama or DeepSeek served by a hosted provider (Bedrock, Together, Groq, Fireworks) | Self-hosting — your GPU, or your own cloud VPC |
The top-left and bottom-right cells are what most people mean by "API key" and "self-hosted". The bottom-left cell is the one that catches firms out: you can use a fully open-source model and still send every byte of client data to a third party. Open weights bought you nothing on confidentiality in that configuration. What it bought you is the freedom to move — which matters, but it is a different benefit.
The Distinction That Matters Most, and Nobody Checks
Within "vendor's machine", there is a split that carries more practical weight than the open-versus-closed question:
A consumer subscription and a paid API key are not the same product, and they do not carry the same data terms.
Consumer chat products are built for individuals. Historically, several have used conversation content to improve their models unless the user turns that off, or unless the account sits on a business or enterprise tier. Developer API tiers are generally built the other way round: inputs are not used for training by default, and retention windows are shorter and documented.
So the firm that says "we pay ₹2,000 a month for the Plus plan, so we're a paying customer, it must be fine" has drawn a conclusion from the wrong fact. Paying does not mean the data terms changed. The tier does.
We are deliberately not printing a table of specific vendors' current retention policies here, because they change — sometimes several times a year — and a table like that is wrong within months. What does not change is the list of things to check.
What to Actually Check Before Client Data Goes Anywhere
Ask these of any AI vendor, in writing, and keep the answer in your file:
- Is our input used to train or improve your models? Get the answer for the specific tier you are buying, not the company's general privacy page.
- How long is input retained, and where? "Zero retention" and "30-day abuse-monitoring retention" are different answers, and both may be acceptable — but you need to know which one you have.
- In which country does inference happen, and where is data at rest? Not where the company is headquartered. Where the compute runs.
- Do you offer a data processing agreement? If you are handling client data, you need a contract, not terms of service you clicked past.
- Who at the vendor can read our inputs, and under what circumstances? Human review for safety or abuse monitoring is common and not automatically disqualifying — undisclosed human review is.
- Is there a sub-processor list, and will we be told when it changes?
If a vendor cannot answer 1 through 4 in writing, that is itself the answer.
Why This Is a Professional Obligation, Not Just an IT Preference
Two things sit on top of a CA firm here.
Confidentiality under the ICAI Code of Ethics. A member must not disclose information acquired in the course of professional work without the client's consent or a legal duty to do so. Sending a client's trial balance to a third-party service is a disclosure question that needs a considered answer, not an assumption. The answer may well be that the arrangement is fine — with the right contract and the right tier — but the firm has to have reached that conclusion deliberately.
The DPDP Act, 2023. Where the firm processes personal data on a client's behalf, the client is ordinarily the Data Fiduciary and the firm is a Data Processor. The Act makes the Data Fiduciary responsible for processing carried out by its processors, and requires that the processing be under a valid contract. Practically, that means your client's obligations flow down to you, and yours flow down to your AI vendor. A chain is only as good as the weakest contract in it.
Payroll files, employee-level ledgers, director details, customer master data — these carry personal data. A depreciation schedule generally does not. Not every audit artefact raises the question, and treating all of them as equally sensitive is its own failure: it stops firms from using AI on the eighty percent where there was never an issue.
So Which Configuration Should a Firm Choose?
| Firm situation | Sensible configuration | Why |
|---|---|---|
| Solo practitioner or small firm, using AI for research, drafting, summarising standards | Consumer subscription on a business tier, no client data pasted in | The work that benefits most — understanding a standard, drafting a note, restructuring text you wrote — involves no client data at all |
| Any firm putting client data through AI | Paid API tier with a signed DPA, or an audit-native product with contractual terms | The contract is the control, not the model |
| Firm with a large client that mandates in-country processing | Open-weight model in your own cloud region, or a vendor that can evidence in-country inference | Contractual and technical answer to the same question |
| Firm with genuine regulatory pressure and the scale to absorb the cost | Self-hosted open-weight model | You control the whole chain — and you now own the operational burden too |
| Firm that has not yet decided | Start with the no-client-data uses | You get most of the benefit while the governance question is being settled properly |
Notice what is not in that table: a recommendation to self-host by default. Self-hosting solves a confidentiality problem by converting it into an operations problem — GPU capacity, model updates, uptime, evaluation, and someone in the firm who owns all of it. For the cost side of that decision, we have run the India numbers in detail in hosting your own open-source LLM: the India cost and ROI math. For most firms the honest answer is that a contract achieves what the GPU was going to achieve, for less money.
The Third Option Firms Forget
There is a configuration that gets skipped: the model never sees the client data at all.
A large share of what auditors want from AI is not "read this ledger and tell me what's wrong". It is: explain this clause, draft this observation from these facts I typed, restructure this note, tell me which standard governs this procedure, check my reasoning. None of that requires uploading a client file.
Firms that separate their AI usage into "work on client data" and "work on my own thinking" find that the second bucket is large, immediately useful, and raises no confidentiality question whatsoever. It is also where the fastest wins are, because there is no procurement conversation in front of it.
For where each tool actually fits by task, see which AI model should CAs use for which task, and for the subscription-level comparison, ChatGPT vs Claude vs Perplexity vs Grok for Indian CAs.
Frequently Asked Questions
Does using an open-source AI model mean my client data stays private?
No. Open source describes whether the model weights are published, not where the data goes. An open-source model served by a third-party API sends your data to that third party exactly as a closed model would. Privacy is determined by whose machine runs the inference and what the contract says — not by the model's licence.
Is a paid ChatGPT or Claude subscription safe for client data?
Paying for a consumer subscription does not by itself change how your inputs are handled. Consumer tiers and developer API tiers carry different data terms, and business or enterprise tiers differ again. Check the terms for the exact tier you hold, in writing, and get a data processing agreement before client data is involved.
Do I need to self-host to comply with the DPDP Act?
Generally no. The Act is concerned with lawful processing, accountability, and having a valid contract with your processor — not with owning the hardware. A vendor with in-country processing and a signed DPA can meet the requirement. Self-hosting is one way to get there, not the only way.
What is the difference between open weights and open source?
In practice most "open-source" models release the weights under a licence that permits use and modification but does not include the training data or full training code. For a CA firm the distinction rarely matters; what matters is that the weights can be downloaded and run on infrastructure you choose.
Where does CORAA sit on this?
CORAA runs open-source LLMs hosted in India, wrapped in our own patent-pending deterministic execution layer — the wrapper is the intellectual property, not the base model. Customer data is hosted on AWS Mumbai (ap-south-1) and is not replicated outside Indian regions. Client data never enters a foundation model's training. ISO/IEC 27001:2022 is certified; SOC 2 Type II is in progress. Full detail sits on our Trust Centre.
The Short Version
Open source is a licensing fact. Self-hosted is an infrastructure fact. An API key is a commercial relationship. Only the last two tell you where client data goes, and the one that decides your professional position is the contract attached to whichever you pick.
Firms that get this right do not necessarily spend more. They just stop guessing.