Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches

ArXi:2512.12677v2 Announce Type: replace-cross We explore efficient strategies to fine-tune decoder-only Large Language Models (LLMs) for downstream text classification under resource constraints. Two approaches are investigated: (1) attaching a classification head to a pre-trained causal LLM and fine-tuning on the task using the LLM's final-token embedding as a sequence representation, and (2) instruction-tuning the LLM in a prompt-to-response format for classification.