Over the past decade, the financial sector has developed in tandem with the globalization trends through Financial Technology (Fintech), Decentralized Finance (DeFi), and Central Bank Digital Currencies (CBDC), indicating the potential to enhance financial inclusion and system efficiency. This research analyzes public discourses on DeFi and CBDC adoption using BERT and RoBERTa methods emphasizing multilabel classification for text understanding and word relationships. The research findings indicate that RoBERTa's multilabel classification has proven to be more effective than BERT's multilabel classification in analyzing public discourse on CBDC and DeFi because RoBERTa, with epoch four, learning rate 2e-5, and batch size 32, exhibits higher levels of accuracy, precision, recall, and F1 compared to BERT. Specifically, RoBERTa demonstrates an accuracy of 0.9636, with MacroP 0.9798, MicroP 0.9811, MacroR 0.9770, MicroR 0.9775, MacroF1 0.9793, and MicroF1 0.9793. This study recognizes the challenges and variations in model performance that need to be focused on in future research. Additionally, this research has the potential to provide a better understanding of public discourse on financial innovation topics, considering various perspectives.