Selected work / Project profile
LLM evaluation studies
Which language model is best for a particular job? My LLM evaluation work uses structured comparisons of output quality to inform that choice. LLM means large language model, the kind of model behind many text and coding assistants.
Compare the work
A model’s reputation is a starting point for a shortlist. Its output is what someone will actually use. Evaluation brings the attention back to whether that output meets the requirements of the task.
These are model-selection studies. My role here is evaluation, rather than authorship of third-party evaluation software.
What makes a comparison useful?
Start by specifying the task and what a satisfactory answer would look like. Keep inputs and constraints comparable, and record the model versions and settings so the comparison has a clear context.
Correctness, following instructions, and the amount of checking still required deserve separate attention. For example, an answer can arrive in the requested format and still contain a factual error. Treating those as distinct criteria makes it easier to see what needs improvement.
A useful result should also leave the reader able to examine how it was reached. Tasks, outputs, and scoring criteria give a performance claim its meaning; a score by itself leaves too much unexplained.
Choosing within a workflow
Model selection is part of system design. A comparison can inform which model to use for a job, while the resulting work still needs checks of its own. That connects these studies with Constellation, where coordination and verification sit alongside model dispatch.