LG AI Research Institute researcher wins best paper award at international conference for AI model performance evaluation (Korea)

According to the LG AI Research Institute, Kim Seung Woo-won, a doctoral student at Carnegie Mellon University in the U.S., presented the results of a joint research project with the North American Association of Artificial Intelligence (NAACL), an international conference on computational linguistics and other fields.
American Chapter of the Association for Computational
Kim won the Best Paper Award at the LG Linguistics Conference.
While working as an intern at the AI Research Institute's Super Intelligence Lab, he worked with Lee Mun-tae, the head of the Super Intelligence Lab, and other researchers on generative AI (artificial intelligence).
The research team led by Kim and others included researchers from Yonsei University and Massachusetts Institute of Technology.
Researchers from multiple universities, including MIT, participated in the project. Existing evaluation methods for generative AI models rely on abstract indices such as flexibility and harmlessness, and the results are different from those obtained when evaluated by humans.
On the other hand, BigGen BenchMARK classifies the capabilities that an AI model should have into nine types, including the ability to carry out instructions, make logical inferences, and understand diverse languages and cultural contexts.
It consists of 765 items that evaluate the performance of a person's abilities. LG AI Research Institute used BigGen benchmarking to evaluate 103 generative AI models, and cross-validated them with a group of experts.
The evidence showed a high level of validity.