article thumbnail

DIY LLM Evaluation, a Case Study of Rhyming in ABBA Schema

Xebia

DIY LLM Evaluation, a Case Study of Rhyming in ABBA Schema It’s becoming common knowledge: You should not choose your LLMs based on static benchmarks. My hypothesis is that AABB rhyming is so common, that the instructions of ABBA are not strong enough to overcome the weight of the pull of the training data.

article thumbnail

Healthcare organizations must create a strong data foundation to fully benefit from generative AI

CIO

Since the introduction of ChatGPT, the healthcare industry has been fascinated by the potential of AI models to generate new content. While the average person might be awed by how AI can create new images or re-imagine voices, healthcare is focused on how large language models can be used in their organizations.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Rocket Mortgage lays foundation for generative AI success

CIO

That’s why Rocket Mortgage has been a vigorous implementor of machine learning and AI technologies — and why CIO Brian Woodring emphasizes a “human in the loop” AI strategy that will not be pinned down to any one generative AI model. We want to be able to plug in the right model at the right time.

article thumbnail

CIOs in financial services embrace gen AI — but with caution

CIO

Highly regulated, customer-centric, and dependent on layers of human involvement and manual processes, financial services are ripe for automation through artificial intelligence (AI). So business technology leaders in financial services are carefully navigating a path toward AI. Banks have to get ready to take the step forward.”

article thumbnail

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Speaker: Maher Hanafi, VP of Engineering at Betterworks & Tony Karrer, CTO at Aggregage

Executive leaders and board members are pushing their teams to adopt Generative AI to gain a competitive edge, save money, and otherwise take advantage of the promise of this new era of artificial intelligence.

article thumbnail

What is AI’s current impact on cybersecurity?

CIO

In our inaugural episode, Michael “Siko” Sikorski, CTO and VP of Engineering and Threat Intelligence at Unit 42 answers that question and speaks to the profound influence of artificial intelligence in an interview with David Moulton, Director of thought leadership for Unit 42. What’s Sikorski’s critical concern?

article thumbnail

How to Use Generative AI and LLMs to Improve Search

TechEmpower CTO

Artificial Intelligence (AI), and particularly Large Language Models (LLMs), have significantly transformed the search engine as we’ve known it. With Generative AI and LLMs, new avenues for improving operational efficiency and user satisfaction are emerging every day.

article thumbnail

LLMOps for Your Data: Best Practices to Ensure Safety, Quality, and Cost

Speaker: Shreya Rajpal, Co-Founder and CEO at Guardrails AI & Travis Addair, Co-Founder and CTO at Predibase

Large Language Models (LLMs) such as ChatGPT offer unprecedented potential for complex enterprise applications. However, productionizing LLMs comes with a unique set of challenges such as model brittleness, total cost of ownership, data governance and privacy, and the need for consistent, accurate outputs.