Name
Case Study: Using Monte Carlo Simulations and Large Language Models to Evaluate AI Vendors for Roy Hill Mining Operations
Date & Time
Tuesday, November 26, 2024, 11:20 AM - 11:50 AM
Oliver Chikumbo
Description

•    Understanding how Monte Carlo simulations and Large Language Models can be employed to assess AI vendor capabilities, focusing on aligning with Roy Hill's strategic goals.
•    Exploring the comprehensive vendor evaluation framework is developed, combining survey results with independent LLM assessments for multi-dimensional analysis.
•    Looking at the Multi-Persona LLM framework is tested for problem-solving capabilities, showing improvement with prompt engineering and curated personas.
•    Vendor feedback and enhanced evaluation metrics are incorporated to mitigate information asymmetry and confirmation bias.
•    Debating how future research will refine methodologies, explore knowledge graphs, and develop a diverse library of personas to enhance AI capability assessments.