Active PAR

P3168

Standard for Robustness Evaluation Test Methods for a Natural Language Processing Service that uses Machine Learning

This standard specifies test methods for evaluating the robustness of a Natural Language Processing (NLP) service that uses machine learning. Models of NLP generally feature an input space being discrete and an output space being almost infinite in some tasks. The robustness of the NLP service is affected by various perturbations including adversarial attacks. A methodology to categorize the perturbations, and test cases for evaluating the robustness of an NLP service against different perturbation categories is specified. Metrics for robustness evaluation of an NLP service are defined. NLP use cases and corresponding applicable test methods are also described.

Sponsor Committee
C/AISC - Artificial Intelligence Standards Committee
Status
Active PAR
PAR Approval
2022-05-13

Working Group Details

Society
IEEE Computer Society
Learn More
Sponsor Committee
C/AISC - Artificial Intelligence Standards Committee
Working Group
RAIS_WG - Robustness of Artificial Intelligence based Service Working Group
Learn More
IEEE Program Manager
Christy Bahn
Contact
Working Group Chair
Qing An

P3129

IEEE Draft Standard for Robustness Testing and Evaluation of Artificial Intelligence (AI)-based Image Recognition Service

This standard provides test specifications with a set of indicators for interference and adversarial attacks, which can be used to evaluate the robustness of Artificial Intelligence-based Image Recognition services. This standard specifies robustness requirements and establishes an assessment framework to evaluate the robustness of Artificial Intelligence-based Image Recognition service under various settings.

Learn More

No Active Standards
No Superseded Standards
No Inactive-Withdrawn Standards
No Inactive-Reserved Standards
Newswire

Sign up for our monthly newsletter to learn about new developments, including resources, insights and more.