Trend One: A Reality Check

This year marks a pivotal moment for realistic expectations regarding AI. When generative AI first gained widespread attention, it was surrounded by sensational media coverage, with many experimenting with tools like ChatGPT and DALL-E. Now, as the initial excitement settles, we are beginning to develop a clearer understanding of the capabilities of AI-powered solutions. Many generative AI applications are being integrated into existing workflows rather than functioning as standalone tools. For instance, features like co-pilot in Microsoft Office and generative fill in Adobe Photoshop demonstrate how AI can enhance everyday tasks, allowing us to grasp its current strengths and limitations.

Trend Two: Multi-Modal AI

The emergence of multi-modal AI models allows for the processing of diverse data types. Current interdisciplinary models, such as OpenAI's GPT-4V and Google's Gemini, can seamlessly transition between natural language processing and computer vision tasks. For example, users can inquire about an image and receive a natural language response or request verbal instructions for a task and receive visual aids alongside written guidance. The integration of video data further broadens the scope of training and inference, enabling models to learn from a variety of inputs.

Trend Three: Smaller Models

While large models have propelled the generative AI movement, they come with significant drawbacks. Training a model the size of GPT-3 consumes as much energy as 3,000 households annually. In contrast, smaller models are less resource-intensive and can achieve impressive results with fewer parameters. Innovations in large language models (LLMs) are focused on maximizing output with reduced parameters. For instance, Mistral's Mixtral, a mixture of experts model, integrates eight neural networks, each with seven billion parameters, outperforming larger counterparts in efficiency.

Trend Four: GPU and Cloud Costs

The trend toward smaller models is driven by both necessity and entrepreneurial spirit. Larger models require more GPUs for both training and inference, putting pressure on cloud costs as providers strive to meet increasing demands. The race to acquire GPUs adds to the urgency of optimizing model efficiency.

Trend Five: Model Optimization

This past year has seen a rise in techniques for optimizing pre-trained models, such as quantization, which reduces model size and speeds up inference. Additionally, techniques like Low-Rank Adaptation (LoRA) allow for the freezing of model weights while injecting trainable layers, resulting in faster fine-tuning and reduced memory requirements.

Trend Six: Custom Local Models

Open-source models present opportunities for organizations to create tailored AI solutions, trained on proprietary data to meet specific needs. Keeping AI training local mitigates risks related to sensitive information being exposed or misused, and approaches like Retrieval-Augmented Generation (RAG) help manage data without bloating model size.

Trend Seven: Virtual Agents

The concept of virtual agents goes beyond conventional chatbots, focusing on task automation. These agents are designed to perform various functions, from making reservations to connecting with other services, indicating a significant shift in how we interact with AI.

Trend Eight: Regulation

Recent developments, such as the European Union's provisional agreement on the Artificial Intelligence Act, highlight the growing importance of regulation. Issues surrounding the use of copyrighted material in training AI models remain contentious, and we anticipate further developments in this area.

Trend Nine: Shadow AI

The use of AI by employees without IT oversight, known as Shadow AI, poses risks related to security and compliance. Research indicates that a significant percentage of employees utilize AI tools without corporate policies in place, which could lead to unintended consequences, such as exposing sensitive information or violating copyright laws. As the capabilities of generative AI grow, so too does the responsibility to manage its use effectively.

In conclusion, these nine AI trends are poised to impact the technology landscape throughout 2024. However, we invite you to consider what other trends might emerge. What do you think is the missing 10th trend? Share your thoughts in the comments!