Rendered at 12:45:40 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
maxzhdev 8 hours ago [-]
I can’t imagine how to correctly assess the ability to humor, because this is a very subjective and relative phenomenon. The work ahead is serious
dlcarrier 12 hours ago [-]
Can you also measure how often the LLM response makes people laugh? Sometimes the responses that aren't attempting a joke are the funniest, and I'd be more interested in stats of which LLM succeed in that metric.
killiandunne1 12 hours ago [-]
Interesting point - thoughts on how to do this? Honestly most models are not-to-kinda funny so I'd be surprised if there were many lol moments. Oral delivery is something I think is v interesting though
rafaepta 13 hours ago [-]
measuring humor might be halfway to measuring taste. congrats on this... really original contribution. wonder if you're planning to evolve the benchmark to incorporate a multi-language dimension. would love to see how Mistral and models built outside the US would perform.
killiandunne1 13 hours ago [-]
Nice a good way to 10x inference costs... worth seeing though ahaha