*UPDATE*
After much investigation, I discovered that the recently updated WP Cerber is the source of all this. If someone can investigate why this is happening and how to prevent/fix it would be highly appreciated.
For now, I just use wordfence to secure my site.