Mar 24, 2024

a tool for multimodal model evaluations

Announcing https://nonfinito.xyz — a tool that makes it easy to run and share evaluations of multimodal models.

We support hundreds of models via OpenRouter:

Run the same evaluation on multiple models to compare their outputs:

Vision-Language Models (VLMs) are supported:

Write your own evaluator code or use our built-in evaluators:

You can try it out at https://nonfinito.xyz.

If you are curious, it's built with OpenRouter, Remix, Cloudflare R2, PostgreSQL, Clerk, inngest, Radix & Fly.io.

Photo Credit: Naoya Hatakeyama: Excavating the Future City