Conference System of NULES of Ukraine, GLOBAL AND REGIONAL PROBLEMS OF INFORMATIZATION IN SOCIETY AND NATURE USING 2021

Font Size: 
Method of semantic comparison of sentences for grading systems with open questions
Дмитро Сергійович Гарбар, Кирило Костянтинович Кадомський

Last modified: 12-05-2021

Abstract


In modern recruitment systems, one of the key tasks is to automate the evaluation of competencies using free-answer questions. This task involves the process of semantic matching of respondent’s answer against the etalon, which is challenging to formalize. To address this problem, a method for estimating the semantic similarity of sentences is proposed, specifically designed for evaluating the correctness of free answers. The method is based on capturing triplets consisting of subject, relation, and object, and reduces the task of assessing the similarity of sentences to the evaluation of the set of exclusive combinations of triplets. A weighted matching technique is proposed for triplets, considering the differences in the role of the subject and the object in determining the correctness of the answer. The effectiveness of the proposed method is demonstrated by experimental comparison with the existing method of assessing semantic similarity through greedy pairing and word semantics.

Keywords


semantic comparison; evaluation automation; natural language processing; OpenIE