Ajou University repository

Are We Training with The Right Data? Evaluating Collective Confidence in Training Data using Dempster Shafer Theory
Citations

SCOPUS

0

Citation Export

Publication Year
2022-01-01
Journal
Proceedings - International Conference on Software Engineering
Publisher
IEEE Computer Society
Citation
Proceedings - International Conference on Software Engineering, pp.11-15
Keyword
data uncertaintyDempster Shafer theorymachine learningsafety
Mesh Keyword
Data centricData uncertaintyDempster-Shafer theoryHigh risk environmentMachine learning modelsMachine-learningQuality assurance practicesSoftware intensive systemsTraining dataTraining dataset
All Science Classification Codes (ASJC)
Software
Abstract
The latest trend of incorporating various data-centric machine learning (ML) models in software-intensive systems has posed new challenges in the quality assurance practice of software engineering, especially in a high-risk environment. ML experts are now focusing on explaining ML models to assure the safe behavior of ML-based systems. However, not enough attention has been paid to explain the inherent uncertainty of the training data. The current practice of ML-based system engineering lacks transparency in the systematic fitness assessment process of the training data before engaging in the rigorous ML model training. We propose a method of assessing the collective confidence in the quality of a training dataset by using Dempster Shafer theory and its modified combination rule (Yager's rule). With the example of training datasets for pedestrian detection of autonomous vehicles, we demonstrate how the proposed approach can be used by the stakeholders with diverse expertise to combine their beliefs in the quality arguments and evidences about the data. Our results open up a scope of future research on data requirements engineering that can facilitate evidence-based data assurance for ML-based safety-critical systems.
ISSN
0270-5257
Language
eng
URI
https://aurora.ajou.ac.kr/handle/2018.oak/36811
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85132965104&origin=inward
DOI
https://doi.org/10.1109/icse-nier55298.2022.9793521
Type
Conference
Funding
This work was supported by the BK21 FOUR program of the National Research Foundation (NRF) of Korea funded by the Ministry of Education (NRF5199991014091) and the Basic Science Research Program through the NRF funded by the Ministry of Science and ICT (NRF-2020R1F1A1075605).
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Lee, Seok-Won Image
Lee, Seok-Won이석원
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.