Malware is malicious software that is designed to damage or control computer systems. Malware also includes various types such as computer viruses, Trojan horses, spyware, adware, crimeware and other software that has harmful intentions. This research conducts malware detection, especially on Remote Access Trojan using system calls (syscall) on Android operating system. Malware analysis Remote Access Trojan on the Android operating system using dynamic analysis support by running each malware and application on the Android operating system to get system call (syscall) information that is running. The results of the system call (syscall) are then selected by feature selection using the ensemble learning classification method. This study aims to determine the characteristics of malware based on system call (syscall) obtained from dynamic analysis and find a comparison of the results of accuracy, precision, recall and f1-score values from the gradient boost classifier, bagging classifier, voting classifier and stacking classifier methods. And the results obtained accuracy of 95%, precision of 92%, recall of 100% and f1-score of 95.85% by using voting classifier which is the most effective method among all the methods. The obtained accuracy, precision, recall and f1-score results are quite good and can be applied in remote access Trojan malware detection in future research.
Keywords—Malicious Software, Remote Access Trojan, Android Operation System, System Call (Syscall), Ensemble Learning