How to run and fine-tune IBM Granite AI models for your projects

Using IBM Granite for AI-powered coding, time series forecasting, and more.

BJ Hargrave from IBM introduced the IBM Granite family of AI models, emphasizing their open source availability and enterprise-ready features. Unlike proprietary models like ChatGPT, Granite is designed for businesses needing control over their data, regulatory compliance, and efficient deployment. The models are available under the Apache 2.0 license, with carefully curated training data to ensure trust and compliance.

Granite includes a variety of models tailored for different tasks, such as large language models (LLMs), vision models, time series forecasting, and geospatial analysis. BJ highlighted the importance of flexibility, efficiency, and customization, allowing businesses to fine-tune models for their specific needs. The workshop also covered IBM’s approach to ensuring safety and security through Granite Guardian models, which help mitigate risks by monitoring inputs and outputs.

The session featured practical demonstrations, starting with document summarization using Granite’s language models. BJ walked through a workflow that included “chunking” large documents, using Docling for intelligent text processing, and leveraging hierarchical chunking to fit content into the model’s 128k token context window. He also discussed retrieval augmented generation (RAG) techniques, demonstrating how Granite can enhance enterprise AI applications by integrating with vector databases like Milvus for knowledge retrieval.

Read more: How to build a multiagent RAG system with Granite

Additional demonstrations included using the Granite time series model for energy demand forecasting and deploying Granite’s code model for AI-assisted programming. BJ showed how developers can run these models locally using tools like Ollama or deploy them via cloud services like Replicate. He emphasized IBM’s commitment to open source AI, making Granite models freely accessible on Hugging Face and GitHub, ensuring businesses have reliable, adaptable AI solutions.

Key takeaways

1. Granite is built for enterprise AI – Unlike closed models, Granite provides businesses with control over their AI, ensuring compliance, security, and customization.
2. Diverse model capabilities – Granite supports various AI tasks, including text generation, vision processing, time series forecasting, and geospatial analysis.
3. Flexible deployment options – Developers can run Granite locally with Ollama or deploy it in the cloud via services like Replicate, making it accessible for different use cases.

Conclusion

BJ reinforced that IBM Granite is not a frontier model but a practical, enterprise-focused AI solution. With its open source foundation, modular design, and business-ready capabilities, Granite enables organizations to deploy AI with confidence. The workshop provided hands-on insights into running, fine-tuning, and integrating Granite models, making it a valuable resource for developers looking to explore AI in real-world applications.

More from We Love Open Source

• Why AI won’t replace developers
• Comparing GitHub Copilot and Codeium
• How to build a multiagent RAG system with Granite
• Developing a genAI application using IBM Granite Code
• Build a local AI co-pilot using IBM Granite Code, Ollama, and Continue