Large Language Models

Recent Trends

---

Accessibility and democratization

Training large models from scratch has required substantial resources, even when model weights and software were publicly available for others to use and adapt.

Today, the field has become significantly more democratized through multiple channels:

• Open-source and open-weight models have become increasingly competitive with proprietary alternatives. For example, DeepSeek-V3 was released on December 26, 2024. Its release evaluation reported results competitive with GPT-4o on several benchmarks, while making its weights available. Such comparisons depend on the benchmark and the particular model versions evaluated.

• Both proprietary providers (e.g. OpenAI, Anthropic, Google) and open-source platforms offer API access with straightforward pricing and documentation, making it feasible to build AI-powered applications without machine learning expertise. The chatbot integrated into this course platform, for instance, is built using such APIs.

• Community platforms like HuggingFace have become a norm for AI development, providing model hosting and tools for training, fine-tuning, evaluation, and deployment of models.

This democratization doesn’t eliminate all barriers as training frontier models still requires resources available only to well-funded organizations. However, it has significantly expanded who can use, adapt, and build applications with powerful language models.

---

Reasoning models

One of the most significant recent developments is the emergence of reasoning models — language models specifically designed to solve complex problems through explicit step-by-step thinking rather than generating answers directly.

OpenAI’s o1-preview, released in September 2024, demonstrated improvements in mathematics, science, and coding tasks by generating internal reasoning before arriving at answers. This approach uses additional computation during inference (when generating responses) to improve performance on some difficult tasks, with increased response time and cost.

In January 2025, DeepSeek released DeepSeek-R1, an open-weight reasoning model that reported performance comparable to OpenAI’s o1 on selected mathematical and coding benchmarks. DeepSeek published a technical report describing its architecture and training approach, allowing researchers to study the released model and methods.

These reasoning models operate differently from standard language models:

• They may generate internal reasoning before answering. Some models expose reasoning text, while others hide it or show a summary. OpenAI’s o1 release, for example, did not expose raw chains of thought. Visible steps can be checked for errors, but they are not guaranteed to faithfully explain what caused the answer; experiments have demonstrated unfaithful chain-of-thought explanations.

• They spend more computational resources during response generation — sometimes generating thousands of internal reasoning tokens before producing a final answer — trading speed for improved accuracy on complex problems.

• They can check their own work, catching and correcting errors before providing final answers, similar to how humans might review their reasoning before submitting an answer.

By mid-2025, DeepSeek released R1-0528, which showed significant improvements in reasoning quality and reportedly reduced hallucination rates by 45-50% compared to earlier versions. Google’s Gemini 2.5 introduced a “Deep Think” mode for step-by-step problem-solving, and other major providers have followed with their own reasoning-focused models.

This shift represents a move beyond simply scaling model size toward optimizing how models use their capabilities during inference — essentially making models “think harder” about difficult problems rather than just making them larger.

---

Efficiency: Doing more with less

As the costs of training and running large models have become increasingly apparent, the field has shifted focus toward efficiency rather than pure scale. Several approaches are driving this trend:

• Mixture of Experts (MoE) architectures that use multiple smaller specialized sub-models that activate selectively based on the input. Rather than processing every input through all one huge model, MoE models route inputs to relevant experts.

• In 2024 and 2025, developers increasingly focused on making models smaller and more efficient, with models like Microsoft’s Phi series demonstrating that carefully trained smaller models on high-quality data can approach the performance of much larger ones trained on broader datasets. These compact models can run on laptops, mobile devices, or embedded systems rather than requiring data center infrastructure.

• Advanced fine-tuning techniques make it possible to adapt models more efficiently without full retraining. Methods like LoRA (Low-Rank Adaptation), which updates only a small number of additional parameters rather than the entire model, Direct Preference Optimization (DPO), which simplifies the RLHF training process, and various prompt tuning approaches allow developers to customize models for specific applications while reducing both computational cost and time requirements.

• Distillation transfers capabilities from a “teacher” model to a “student” model, often a smaller one. DeepSeek’s report describes training smaller models on responses generated by DeepSeek-R1. Its 32 billion parameter distilled model outperformed o1-mini on several reported benchmarks; this is a performance comparison, not a comparison of their parameter counts.

• Alternative architectures like RecurrentGemma and Mamba explore alternatives to the standard transformer design, seeking better efficiency for certain tasks through different approaches to processing sequential data.

These efficiency-focused developments reflect growing recognition that practical deployment often values computational efficiency, speed, and deployability as much as raw capability.

---

Multimodal capabilities

Multimodal models process or generate more than one type of data. During 2024 and 2025, these capabilities became prominent in widely used AI products.

Providers offer different combinations of text, image, audio, and video capabilities. A model that accepts images does not necessarily generate them, and real-time interaction depends on the model and application. These capabilities enable applications such as:

• Models can process documents containing text, images, charts, tables, and diagrams without requiring separate processing pipelines for each element type, making them practical for analyzing complex real-world documents.

• Users can ask questions about images, diagrams, photographs, or videos and receive detailed explanations that reference visual content, enabling applications from educational assistance to accessibility tools.

• Models can generate images from text descriptions, edit existing images based on natural language instructions, or even create video content, expanding creative possibilities beyond purely text-based interaction.

• Multimodal capabilities improve accessibility by enabling systems that can describe visual content for visually impaired users, transcribe and caption audio for hearing-impaired users, or translate between modalities in other ways.

The integration of multiple modalities represents a move toward more natural, human-like interaction with AI systems, matching how humans actually process information through multiple senses rather than purely through text.

---

Systems integration and agentic AI

Language models are increasingly deployed as components within larger systems rather than as standalone tools. Several approaches characterize this trend toward what’s sometimes called “agentic AI”:

• Retrieval-Augmented Generation (RAG) combines language models with external knowledge sources. A retrieval system finds relevant information from databases, document collections, or the internet and supplies it as context. This can improve grounding and provide information beyond a model’s training cutoff, but it does not guarantee that the sources or generated answers are correct.

For example, a company chatbot might search internal documentation before generating an answer about company policies. The retrieved documents need to be current, and the answer still needs to be checked against them. RAGTruth documents unsupported and contradictory statements in answers generated even with retrieved context.

• Tool use and function calling enable models to interact with external systems. Models can decide when to call APIs, query databases, perform calculations using code execution environments, or interact with other software tools to accomplish tasks. This transforms models from pure text generators into interactive agents capable of taking actions beyond generating text.

• Agent frameworks orchestrate complex workflows where models plan sequences of actions, use tools, evaluate results, and adjust their approach based on feedback. Developer tools increasingly integrate language models directly into software development workflows — for example, AI assistants that can browse codebases, understand project structure, suggest changes, run tests, and help commit code.

This systems-level integration represents a conceptual shift from viewing language models as standalone text generators to seeing them as reasoning engines that can coordinate with other computational resources to accomplish complex tasks.