RNN Decomposition | Fraol Batole

Co-author ICSE 2023 Modularity

This project studies whether trained recurrent neural networks can be decomposed into modules after training. The goal is to make learned behavior reusable and replaceable without retraining the entire model from scratch.

Problem

Trained RNNs are typically treated as monolithic artifacts, so reusing or replacing a learned behavior often requires retraining the full network.

Approach

The method identifies module boundaries inside a trained RNN and decomposes the model into units that capture distinct learned behaviors for reuse or replacement.

Results

The evaluation used 5 canonical datasets and 4 model variants per dataset. Decomposition changed accuracy by -0.6% and BLEU by +0.10%; reuse changed accuracy by -2.38% and BLEU by +4.40%; replacement changed accuracy by -7.16% and BLEU by +0.98%.

Why it matters

Post-training modularity gives neural systems a maintainability path closer to software systems, where localized components can be reused or replaced independently.

Full approach and results are in the paper (Imtiaz et al., 2023).

References

2023

ICSE’23

Decomposing a recurrent neural network into modules for enabling reusability and replacement

Sayem Mohammad Imtiaz, Fraol Batole, Astha Singh, and 3 more authors

In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), Jan 2023

arXiv Bib PDF

@inproceedings{imtiaz2023decomposing,
  title = {Decomposing a recurrent neural network into modules for enabling reusability and replacement},
  author = {Imtiaz, Sayem Mohammad and Batole, Fraol and Singh, Astha and Pan, Rangeet and Cruz, Breno Dantas and Rajan, Hridesh},
  booktitle = {2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)},
  pages = {1020--1032},
  year = {2023},
  publisher = {ICSE},
  organization = {IEEE}
}