Key-Sun Choi
2020
Effective Crowdsourcing of Multiple Tasks for Comprehensive Knowledge Extraction
Sangha Nam
|
Minho Lee
|
Donghwan Kim
|
Kijong Han
|
Kuntae Kim
|
Sooji Yoon
|
Eun-kyung Kim
|
Key-Sun Choi
Proceedings of The 12th Language Resources and Evaluation Conference
Information extraction from unstructured texts plays a vital role in the field of natural language processing. Although there has been extensive research into each information extraction task (i.e., entity linking, coreference resolution, and relation extraction), data are not available for a continuous and coherent evaluation of all information extraction tasks in a comprehensive framework. Given that each task is performed and evaluated with a different dataset, analyzing the effect of the previous task on the next task with a single dataset throughout the information extraction process is impossible. This paper aims to propose a Korean information extraction initiative point and promote research in this field by presenting crowdsourcing data collected for four information extraction tasks from the same corpus and the training and evaluation results for each task of a state-of-the-art model. These machine learning data for Korean information extraction are the first of their kind, and there are plans to continuously increase the data volume. The test results will serve as an initiative result for each Korean information extraction task and are expected to serve as a comparison target for various studies on Korean information extraction using the data collected in this study.
Crowdsourcing in the Development of a Multilingual FrameNet: A Case Study of Korean FrameNet
Younggyun Hahm
|
Youngbin Noh
|
Ji Yoon Han
|
Tae Hwan Oh
|
Hyonsu Choe
|
Hansaem Kim
|
Key-Sun Choi
Proceedings of The 12th Language Resources and Evaluation Conference
Using current methods, the construction of multilingual resources in FrameNet is an expensive and complex task. While crowdsourcing is a viable alternative, it is difficult to include non-native English speakers in such efforts as they often have difficulty with English-based FrameNet tools. In this work, we investigated cross-lingual issues in crowdsourcing approaches for multilingual FrameNets, specifically in the context of the newly constructed Korean FrameNet. To accomplish this, we evaluated the effectiveness of various crowdsourcing settings whereby certain types of information are provided to workers, such as English definitions in FrameNet or translated definitions. We then evaluated whether the crowdsourced results accurately captured the meaning of frames both cross-culturally and cross-linguistically, and found that by allowing the crowd workers to make intuitive choices, they achieved a quality comparable to that of trained FrameNet experts (F1 > 0.75). The outcomes of this work are now publicly available as a new release of Korean FrameNet 1.1.
Search
Co-authors
- Sangha Nam 1
- Minho Lee 1
- Donghwan Kim 1
- Kijong Han 1
- Kuntae Kim 1
- show all...
Venues
- LREC2