Generative AI and Personal Data: What Businesses Should Check Before Uploading Customer Information or Contracts
Businesses are increasingly using generative AI to review contracts, summarize meetings, respond to customer inquiries, and draft documents.
However, when contracts or other materials entered into an AI service contain personal data relating to customers or employees, the issue extends beyond operational efficiency. It may raise concerns regarding personal data processing, confidentiality, and the management of internal business information.
Businesses should therefore understand how an AI service stores and uses input data and establish internal rules that reflect how generative AI is actually used within the organization.
How Is Information Entered into Generative AI Processed?
From the user’s perspective, generative AI appears to involve simply entering a question and receiving a response.
In practice, however, prompts and uploaded files are transmitted to the service provider’s systems. Depending on the service policy and account type, they may be retained as conversation records or system logs and may also be used to improve the service.
Before entering business materials, companies should check:
▪️ Whether input data may be used to train the AI model
▪️ How long conversations and uploaded files are retained
▪️ Whether conversations and uploaded materials can be deleted
▪️ In which country and on which servers the data is processed
▪️ What access permissions are granted when the AI is connected to external applications or internal systems
Using a paid or enterprise account does not automatically resolve every data protection issue. The relevant contractual terms and actual data-processing arrangements must also be reviewed.
Is Information No Longer Personal Data Once the Name Is Removed?
Removing a customer’s name or contact information from a contract does not necessarily mean that the remaining information is no longer personal data.
Personal data includes not only information that directly identifies an individual, such as a name, but also information that may identify a person when combined with other readily available information.
Even after a name has been removed, an individual may still be identifiable through a combination of information such as:
▪️ The company name and a specific job title
▪️ The contract date and transaction amount
▪️ The employee’s department and performance evaluation
▪️ Detailed consultation history and family relationships
▪️ A case number and the location of a dispute
A company name alone is generally information about a legal entity. However, when combined with a specific position, transaction details, or other contextual information, it may make an individual employee or representative identifiable.
Businesses should therefore remove information that is unnecessary for the intended task rather than deleting names alone. Actual names, contact details, company names, and similar identifiers may also need to be replaced with fictional or generic information.
Does Entering Customer Information Automatically Violate Korea’s Personal Information Protection Act?
Entering customer information into a generative AI service does not automatically constitute a violation of Korea’s Personal Information Protection Act, commonly referred to as the PIPA.
However, a business may generally use personal data only within the scope of the original purpose of collection and an appropriate legal basis. It is therefore necessary to assess whether the intended AI use is related to the original processing purpose and whether additional consent or another legal basis is required.
The relationship between the business and the AI service provider must also be reviewed.
The applicable legal framework may differ depending on whether the provider processes data solely on the company’s instructions or also uses the input data for its own purposes, such as model training or service improvement. Depending on the actual arrangement, the use of the service may involve outsourced processing, third-party provision, or other forms of data processing.
If personal data is transferred to servers or service providers located outside Korea, the requirements governing overseas transfers must also be considered.
The applicable rules cannot be determined solely from the fact that customer information was entered into an AI service. Businesses should review the actual contractual structure and data flow, including:
▪️ The legal basis for using personal data in the relevant AI task
▪️ The purposes for which the service provider uses input data
▪️ Whether the data is used for training or service improvement
▪️ The data-retention period and deletion procedures
▪️ The server locations and countries to which data is transferred
▪️ The involvement of subprocessors or downstream service providers
▪️ How input data is handled after the service contract ends
Contracts and Consultation Records Raise More Than Personal Data Issues
Contracts and customer consultation records may contain confidential business information in addition to personal data.
Examples include:
▪️ Transaction terms and supply prices
▪️ Sales strategies and business plans
▪️ Technical materials and development information
▪️ Confidentiality obligations agreed with customers or business partners
▪️ Strategies for litigation, investigations, or other disputes
▪️ Non-public financial information and internal decision-making materials
Entering such information into an external generative AI service may raise issues under the PIPA, but it may also create risks involving contractual confidentiality obligations, trade secret protection, customer security agreements, and internal company policies.
A corporate generative AI policy should therefore cover not only personal data but also trade secrets, contractual information, technical materials, and dispute-related documents.
Internal Rules Businesses Should Establish for Generative AI Use
When employees are left to decide individually how generative AI should be used, the company may be unable to identify what information has been entered into which external service.
Rather than issuing a general instruction not to enter personal data, businesses should establish practical and specific rules that employees can follow in their day-to-day work.
1. Approved Services and Accounts
The company should designate the generative AI services and account types that employees are permitted to use.
Employees should also be prohibited from entering business materials through personal accounts or unapproved services.
2. Prohibited Input Data
The company should clearly define the information that must not be entered into generative AI services.
This may include resident registration numbers, bank account information, health information, other sensitive or high-risk personal data, trade secrets, non-public contractual terms, and materials relating to litigation or criminal investigations.
3. Approval Procedures for High-Risk Documents
High-risk materials, such as contracts, human resources documents, and customer consultation records, may be made subject to prior review or approval by the responsible department.
4. Standards for Removing or Replacing Identifying Information
Businesses should establish standards for removing or replacing information that could identify an individual or a transaction party, including names, contact details, company names, and transaction amounts.
Rather than uploading an entire contract, employees should extract only the clauses required for the relevant task. Actual customer information may also be replaced with fictional names or sample data.
5. Conversation History and Model-Training Settings
The company should determine whether the service allows input data to be excluded from model training, whether conversation history can be disabled, and how deletion features operate.
6. Internal Reporting and Incident Response
Employees should be required to report promptly if personal data or confidential business information is entered into an AI service by mistake.
The company should also establish procedures for deleting records, disabling external integrations, requesting deletion from the service provider, and taking any other necessary measures.
What Should a Business Do If Customer Information Has Already Been Entered?
If an employee mistakenly enters customer information or an internal document into a generative AI service, the company should first stop any further sharing or use of the resulting output.
The following matters should then be reviewed:
▪️ Which account and AI service were used
▪️ What personal data or confidential information was entered
▪️ Whether the conversation and uploaded files can be deleted
▪️ Whether the service was connected to any external applications
▪️ Whether the information may have been used for training or service improvement
▪️ Whether a third party may have gained access to the information
Where necessary, the company should delete the conversation history and uploaded files, revoke external access permissions, and request that the service provider stop processing or delete the relevant information.
Whether the incident requires a personal data breach report or notification to affected individuals should be determined by considering the type and volume of information involved, the possibility of third-party access, whether the data has been deleted, and the potential harm to the individuals concerned.
Effective Generative AI Use Requires Governance, Not Just Prohibition
Generative AI can improve efficiency in contract review, document drafting, and customer communications.
However, when its use is left entirely to individual employees, a business may have little control over where customer information and internal documents are transmitted or how they are subsequently used.
Businesses should establish specific rules regarding approved services, prohibited data, the removal or replacement of identifying information, approval procedures for high-risk documents, and incident response.
Decent Law Firm advises businesses on the legal use of generative AI, including compliance with Korean personal data protection requirements, review of service terms and data-processing agreements, overseas data transfers, trade secret protection, and the preparation of internal AI-use policies.
This content is provided for general informational purposes only and does not constitute legal advice regarding any specific matter or business.
