Towards accurate low-bitwidth BERT
Le, Phuoc-Hoan Charles · eScholarship@McGill (McGill) · 2023
Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.
eScholarship@McGill (McGill) · McGill University · dissertation · 2023
Abstract
Transformer-based models or BERT (Bidirectional Encoder Representations from Transformers) models have seen a lot of success in natural language processing tasks in recent years. However, these networks are very difficult to deploy on edge devices where memory and compute resources are limited. To lower their computational and memory cost, pruning and quantization can be applied. However, naively applying these techniques to compress transformers, can cause a significant reduction in the accuracy of these models.
This thesis will focus on improving the performance of quantization-aware training or improving the final performance or accuracy of a quantized transformer-based model without increasing the computational complexity of the forward pass of the network or slightly changing the network's architecture to increase the model capacity.In this thesis, we give one example of such and propose a more efficient quantization-aware training pipeline, called Efficient Two Stage Progressive Quantization in which the model with quantized weights is fine-tuned first, and at the same time, the bitwidth of the weights is lowered. Then, the model with quantized weights and activations will be fine-tuned.
At the same time, we strategically choose which bitwidth to fine-tune on and to initialize from, and which bitwidth to fine-tune under augmented data. We are able to ternarize the weights of BERT, compressing it by 14.9x while outperforming the full-precision model performance across nearly all GLUE datasets, and without data augmentation, it can outperform existing BERT ternarization methods. Also, we outperform the state-of-the-art BERT binarization in performance without adding an extra module, saving 18% more memory than BinaryBERT
Open the resolved locator · 10.82308/17519
Metadata licence: OpenAlex metadata (CC0)