Abstract: Large Language Models have emerged as the top-notch tool in the software engineering field, from requirement gathering and analysis to code generation. Several approaches have been developed ...

Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...

Cognitive load refers to the amount of mental effort required to perform a task, including everything a tester must keep in mind while testing, such as requirements, system behaviour, test data, ...

Try the demo mode to see how it works, or connect a backend to run actual k6 tests. See web/ for local development or WEB_DEPLOYMENT.md for deployment instructions.

In this tutorial, we show how we treat prompts as first-class, versioned artifacts and apply rigorous regression testing to large language model behavior using MLflow. We design an evaluation pipeline ...

Single binary MCP server with ZERO dependencies! A native MCP (Model Context Protocol) server built with Go that provides access to developer conference CFPs from developers.events.