Using Large Language Models Responsibly in Healthcare

Large language models (LLMs) are a category of generative AI focused on text generation. EMA and the Heads of Medicines Agencies (HMA) published guiding principles in September 2024 to promote safe, responsible, and effective use among medicines regulatory staff. These principles are highly relevant for Hong Kong healthcare professionals who use tools such as ChatGPT, Copilot, or hospital-approved AI assistants.

What LLMs can and cannot do

LLMs can assist with drafting emails, summarising literature, translating text, coding support, and sifting through large volumes of information. However, they can produce hallucinations — plausible but inaccurate responses — and pose data security risks if sensitive information is entered into prompts.

Principle 1: Ensure safe input of data

The first interaction with an LLM is critical. Users must understand whether the model is hosted internally (under organisational control) or is an open online service that may retain prompt data.

Key practices:

• Never enter patient identifiers, clinical records, trade secrets, or contract-restricted content into public LLM tools
• Draft prompts carefully and double-check text before copy-pasting — hidden content in copied documents can alter LLM behaviour
• Adapt your approach to the level of control your organisation has over the LLM application

Principle 2: Apply critical thinking and cross-check outputs

LLMs generate text from statistical correlations, not verified facts. The greater the risk of potential harm, the greater the degree of scrutiny required.

Key practices:

• Avoid automation bias — review outputs for accuracy, relevance, fairness, and legal compliance
• Redraft new information rather than copying LLM text verbatim
• Ask for sources and quote exact sentences when summarising documents
• Always review and test code generated by an LLM before use in clinical systems

Principle 3: Continuously learn

LLMs evolve quickly and their risk profile shifts. Responsible use requires ongoing education on prompt engineering, tool settings, and emerging limitations.

Principle 4: Know whom to consult

When facing data protection, security, or severely biased outputs, know your organisation's contacts — typically information security, privacy officers, or clinical governance leads. Report incidents so they can be investigated and addressed.

Relevance for Hong Kong healthcare professionals

Under Hong Kong's Personal Data (Privacy) Ordinance (PDPO), patient data must not be entered into unapproved external AI services. Hospital Authority and private provider policies may further restrict permitted use cases. Before using any LLM in clinical, pharmacy, or administrative work, confirm your institution's approved tools and governance requirements — the EMA principles provide a practical checklist either way.

Source: EMA/HMA — Guiding principles on the use of large language models (August 2024)