Assessment of Mean Teacher and Prominent Adversarial Unlabeled Data for Language Classification
M.Sc. thesis evaluating Mean Teacher semi-supervised fine-tuning with augmented unlabeled text to improve adversarial robustness in BERT and DistilBERT classifiers.
Bhupender Kumar Saini (2022). Assessment of Mean Teacher and Prominent Adversarial Unlabeled Data for Language Classification. M.Sc. thesis, University of Koblenz and Landau.
Thesis record
Author: Bhupender Kumar Saini
Institution: University of Koblenz and Landau
Research question
Can Mean Teacher semi-supervised fine-tuning, combined with augmented unlabeled text, improve accuracy under adversarial attack without sacrificing clean-data accuracy?
Method
Two identical transformer classifiers were trained as a student and teacher. The student optimized binary cross-entropy on labeled data and consistency loss on augmented unlabeled data. The teacher received exponential-moving-average weights from the student:
The overall objective combined classification and consistency costs:
Unlabeled examples were created through synonym substitution, context-based augmentation, and back translation. The proposed MTBERT and MTDistilBERT variants were compared with conventionally fine-tuned BERT and DistilBERT.
Evaluation
- Datasets: IMDB movie reviews and the COVID-19 fake-tweets dataset.
- Attacks: BAE, PWWS, TextBugger, and TextFooler under black-box settings.
- Measures: clean accuracy, accuracy under attack, attack-success rate, word perturbation, perturbation confidence, and required model queries.
Findings
Across both datasets, the Mean Teacher variants improved clean accuracy by approximately 0-2 percentage points and accuracy under attack by approximately 20-30 percentage points relative to conventional fine-tuning.
MTBERT was the strongest model overall. For example, on IMDB it retained 30.92% accuracy under TextFooler attack, compared with 0.10% for BERT; on COVID-19 fake tweets it retained 54.05%, compared with 7.62% for BERT. The proposed models also generally required more attacker queries and reduced attack success rates.
Boundaries
The approach used two model instances and was therefore more time- and memory-intensive than conventional fine-tuning. Results demonstrate improved resilience against the evaluated attacks, not universal robustness against every adversarial strategy.