In the digital transformation era, Large Language Models (LLMs) like ChatGPT are increasingly being integrated into organizational workflows to boost efficiency and enrich customer experience. Self hosting these LLMs appears to be a viable solution to address privacy concerns, but it’s not entirely without its challenges. This article explores the residual privacy issues that persist with self hosted LLMs, the implications of server degradation, computing costs, and the resources required to build your own LLM.
Summary of the Pros and Cons of Self Hosting LLMs:
Pros | Cons |
Full control over training data | Residual privacy issues: DPIAs and the right to be forgotten, information and consent obligations, data minimization |
Potentially less data exposure | Server degradation over time |
No third-party involvement | High computing costs |
No need for third-party agreements | Significant time and expertise required to train and deploy an LLM at scale |
No data transfer to third-party services | Difficulty of obtaining quality, legal training data |
Sole responsibility for data security | |
Potential for irresponsible use of outputs by employees | |
May not match the security standards and output safety measures of companies like Microsoft Azure | |
Effort required to keep model up to date |
Data Protection Impact Assessments and the Right to be Forgotten
Self hosting an LLM implies that your organization retains full control over the training data, which can significantly alleviate privacy concerns. However, certain issues persist. For instance, Data Protection Impact Assessments are likely still necessary to evaluate the potential risks associated with such data processing. Furthermore, the right to be forgotten, which involves deleting personal data upon request, can be challenging to implement. While LLMs don’t save their training data, they don’t exactly “forget” the data they’re trained on either. While there is a lack of clarity around whether the right to be forgotten applies to machine learning models, these are indications that the law could lean in that direction with examples like Weight Watchers being required by the FTC to delete algorithms trained on minors’ data.
How Private AI helps
Organizations must first understand the types of personal data they are using to train and fine-tune their LLMs. Given the complex nature of conversational data, including typos, code switching between languages, and unexpected file structures, creating a privacy assessment becomes a gargantuan task. Private AI makes this easy by being able to identify over 50 different types of personal information across text, audio, images, and documents in 50 different languages.
Moreover, removing unnecessary personal information from data prior to training or fine-tuning their models, avoiding the models’ “memorization” of that personal information in the first place. See our redaction capabilities in real-time.
Information Obligations and Consent
Even when an organization opts for a self hosted LLM, information obligations under various privacy laws, such as the General Data Protection Regulation (GDPR), remain applicable. Information obligations require transparency about an organization’s data processing activities and organizations are required to provide clear and comprehensive information about the processing of personal data. This includes detailing the purposes of processing, the categories of data being processed, and the rights of the data subjects.
Carefully abiding by the information obligations becomes even more important when consent for the processing of personal data is required. Under the GDPR, that is the case if the prompts sent to the LLM include “special categories of personal data” as defined by Article 9 of the GDPR, such as health data. The processing of this data is prohibited with very limited exceptions, one of which is explicit, valid consent. Thus, if you want to use customer data to train or fine-tune your self hosted LLM, you need to ensure you disclose this purpose at the time of collecting the data and, if special categories of personal data are concerned, you must obtain explicit consent, which can also be withdrawn at any time.
Therefore, even with a self hosted LLM, organizations must ensure they are fully compliant with these consent and information obligation requirements to avoid potential legal and reputational risks. A challenge may arise to internally distinguish between data you receive to train or fine-tune the LLM, particularly if consent is withdrawn at a later time.
How Private AI helps
Often, organizations are unaware of the types of information they are processing in the first place. Private AI can flag whether special categories of personal information or other types of personal data are being processed by your LLM based on jurisdiction-specific requirements. See the list of entities Private AI can detect and redact.
Data Minimization
The principle of data minimization, which mandates that data processing should be limited to what is strictly necessary for the intended purpose, can pose a significant challenge even in a self hosted environment. While self hosting an LLM does provide an organization with greater control over the data being processed, it does not completely eliminate the risk of data overexposure.
In a self hosted setup, the organization has the responsibility to ensure that only the minimum required personal data is transferred to different parts of its organization, which includes being processed by an LLM. However, given the complexity and the broad learning capabilities of these models, there’s a risk that they might inadvertently process more data than necessary, leading to potential overexposure. This could happen, for instance, if an LLM is trained on a large dataset and starts generating outputs based on sensitive information that was not intended to be part of the processing.
How Private AI helps
Private AI can help ensure data minimization is adhered to, removing unnecessary personal information automatically from both training data and prompts across text, audio, images, and documents. See here for a list of file types Private AI supports.
Irresponsible Use of Output by Employees
Another concern with self hosted LLMs is the potential for irresponsible use of output by employees. Even though the prompts or training data are not transferred to third-party services like OpenAI , employees might still mishandle and/or share sensitive information inadvertently by sending the output to persons with different levels of access control as themselves, leading to potential data breaches.
How Private AI helps
The easiest way to prevent sensitive information from being output by an LLM is to remove it directly at the time of training and fine-tuning. Private AI can identify and remove unnecessary sensitive information before the LLMs are even trained. Download our Whitepaper here to learn more about what Private AI can do.
Server Degradation, Resources, and Data Quality
Beyond privacy concerns, self hosting LLMs also come with their own set of challenges. Server degradation, for instance, refers to the gradual decline in the server’s performance over time due to continuous use, which can affect the performance of the LLM.
Moreover, self hosting LLMs can be resource intensive. The computing costs associated with running these models can be substantial, especially for larger models. Additionally, building your own LLM requires significant time and expertise, which might not be feasible for all organizations.
Obtaining quality training data is another significant challenge. The data used to train LLMs must be diverse, representative, and free from biases. Ensuring this can be a daunting task. Additionally, the use of such data must not violate copyright and privacy laws. This means the data should not contain any copyrighted or personal information, which adds another layer of complexity to the process.
Security
Lastly, security is a crucial aspect to consider. While self hosting provides more control over data, it also means that the organization is solely responsible for securing that data. Given the substantial investments made by companies like Microsoft Azure in security measures, it might be challenging for organizations with limited budgets to match these standards. Some examples of relevant security solutions are Azure Active Directory to manage access, Microsoft Defender for threat management, Azure Monitor to maximize the availability and performance of applications and services, and Azure Firewall to protect against intrusions. Furthermore, Microsoft Azure has also invested heavily in rendering the output safe by implementing content filters and abuse monitoring to prevent biased outputs.
Conclusion
While self hosting LLMs can offer enhanced control over data and potentially address some privacy concerns, it’s not a panacea. Organizations need to carefully consider the residual privacy issues, the implications of server degradation, the costs involved, the challenges of obtaining quality training data, and the security measures required before deciding to self host an LLM.
An alternative for businesses who want to unlock the power of LLMs without compromising data privacy within their organization is PrivateGPT, the privacy-preserving solution for LLMs. Built off of Private AI’s established de-identification technology, PrivateGPT removes 50+ entities of PII, PCI, and PHI from user prompts before they get shared with OpenAI’s language model APIs. The scrubbed messages then get shared with ChatGPT, and then the reply is rehydrated to add the personal information back in for a seamless user experience.
PrivateGPT deploys as a container so no data is ever shared with a third party, including Private AI. This allows users to interact with various LLMs via their own internal apps in a safe and secure way and also acts as a VPN. It can be deployed as a privacy-preserving chat interface for end users, or as an API for developers to use in existing workflows. Try it free today or request an API key to get started in your own environment.
Get started with PrivateGPT today:
