Beyond One-Model-Fits-All: A Survey of Domain Specialization for Large Language Models

by   Chen Ling, et al.
NEC Laboratories America
George Mason University
Emory University
Rutgers University
University of Pittsburgh

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP), providing a highly useful, task-agnostic foundation for a wide range of applications. The great promise of LLMs as general task solvers motivated people to extend their functionality largely beyond just a “chatbot”, and use it as an assistant or even replacement for domain experts and tools in specific domains such as healthcare, finance, and education. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles, caused by the heterogeneity of domain data, the sophistication of domain knowledge, the uniqueness of domain objectives, and the diversity of the constraints (e.g., various social norms, cultural conformity, religious beliefs, and ethical standards in the domain applications). To fill such a gap, explosively-increase research, and practices have been conducted in very recent years on the domain specialization of LLMs, which, however, calls for a comprehensive and systematic review to better summarizes and guide this promising domain. In this survey paper, first, we propose a systematic taxonomy that categorizes the LLM domain-specialization techniques based on the accessibility to LLMs and summarizes the framework for all the subcategories as well as their relations and differences to each other. We also present a comprehensive taxonomy of critical application domains that can benefit from specialized LLMs, discussing their practical significance and open challenges. Furthermore, we offer insights into the current research status and future trends in this area.


A Systematic Review of Natural Language Processing for Knowledge Management in Healthcare

Driven by the visions of Data Science, recent years have seen a paradigm...

Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond

This paper presents a comprehensive and practical guide for practitioner...

DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains

In recent years, pre-trained language models (PLMs) achieve the best per...

Knowledge-augmented Deep Learning and Its Applications: A Survey

Deep learning models, though having achieved great success in many diffe...

A survey on extremism analysis using Natural Language Processing

Extremism research has grown as an open problem for several countries du...

Event Prediction in the Big Data Era: A Systematic Survey

Events are occurrences in specific locations, time, and semantics that n...

GLUECons: A Generic Benchmark for Learning Under Constraints

Recent research has shown that integrating domain knowledge into deep le...

Please sign up or login with your details

Forgot password? Click here to reset