SecureMed-LLM: A Privacy-Preserving Framework for Safeguarding Clinical Language Models

Abstract

The rapid advancement of Large Language Models (LLMs) has transformed applications of artificial intelligence (AI), particularly in critical domains such as healthcare. However, their widespread adoption introduces significant challenges, including tampering, malicious interference, and data privacy violations. Traditional safeguards often fail to address these risks comprehensively, leaving LLMs vulnerable to adversarial attacks, prompt injections, and privacy breaches. In this work, we propose SecureMed-LLM using BioMedCLIP, a robust framework designed to protect LLMs in clinical settings through a multi-tiered defense strategy. Our approach integrates local data anonymization via the Med-Guard module, differential privacy training (DP-SGD), medical compliance validation, and encrypted inference using the Elliptic Curve Integrated Encryption Scheme (ECIES) with Curve25519. The framework is evaluated on the OPEN-I Chest X-ray dataset, demonstrating resilience against adversarial attacks (e.g., FGSM, PGD) with minimal performance degradation (BLEU score > 0.63 under perturbation), Image anonymization with controlled noise (σ = 15) preserves diagnostic utility (BLEU score = 0.70) while enhancing privacy, and differential privacy (ϵ = 3.0) reduces membership inference attack success rates by 45%. The results emphasize the balance between security and the utility of SecureMed-LLM in generating clinical reports with 78.3% accuracy while safeguarding sensitive patient data. This work provides an extensible solution for privacy-preserving AI in healthcare, addressing both technical and regulatory challenges in LLM deployment.