and ever since then we never look at a kitty the same way again. meow! — ou also know of ways to prevent them from going wrong, ! The beauty of optimism is when you know of things that can go wrong, and y yet — you choose to gamble it all off and hope that nothing will go wrong Confusing? Edgy? Silly? — yes all of it and more. Somewhere in Aug 2020, I & my team were working in frenzy on expanding our RedHat Ansible Playbooks repository. ( if you are uninitiated into the world of Ansible — start here ) We had built & tested neat playbooks ( ) to install & configure MySQL, PostgreSQL & MongoDB onto (multi-flavor) Linux VMs. These could also create databases & users. human-readable code which allows us to define the desired states of our components In order to test these — Two Classic Infrastructure Virtual Server — Public Instances on IBM Cloud were setup— one with CentOS Minimal install and the other with Ubuntu. Opted for default automatic private VLAN network interface. Felt too optimistic and skipped adding SSH keys to the server during creation ( ). vulnerability-1 introduced! Next come security groups — An IBM™ Cloud Security Group is a set of IP filter rules that define how to handle incoming (ingress) and outgoing (egress) traffic to both the public and private interfaces of a virtual server instance. While selecting from the default provided IBM security groups for the public & private interface of the provisioned devices — dabbled around as shown below and ended up with . Terminology across different cloud providers always intrigues me :) , “all” does not imply “ingress+egress” rather it means all-ingress-only. vulnerability-2 Quite a wordy affair! Our requirement comprised: Assigning these VMs as target hosts in Ansible Tower Inventory onto which multiple users from within company can execute the DB setup ( ) playbooks and on their local machine. ( ) & other connect to the created databases over internet via any IDE Internet facing Database instances! Use these machines to test the playbooks remotely during the authoring phase. And opted for the typical super user privileges and open access, wherever we were tempted to! Do not do that! :) Let’s call it an experimental niche Lab setup— different from usual Dev or Prod environments. Life was hunky & dory, till one fine morning — a cybersecurity red alert popped in, yikes! We had left insecure internet facing MongoDB databases like sitting ducks. The had happened! Though shocking, it was all a little amusing as none of our databases had any relevant or confidential data. Bazinga! ( ). infamous ‘meow’ attack that moment when you find your wallet stolen, but it was your empty wallet Meow is a scripted bot attack that scrambles the data & indexes of victim Mongo databases by overriding it with numbers and appending the string ‘meow’. Attackers have an ironic sense of humor! Find out more about meow here. Awakened by the jolt, it was time to secure the setup for our internet facing Mongo databases. Though, there are multiple options & levels of security, read ; but based on our unique need, cost considerations & smart recovery options — following were the steps to security. MongoDB’s official security checklist here 1. Recreated the IBM Cloud Classic Virtual Server, this time . Well, not really. We were working on Ansible, remember? added a pair of SSH keys and disabled root login Wondering if choosing to recreate was a tough call? So, we already have playbooks that automated all our subsequent configurations and setup. One-click redo. Recreation was preferable to avoid running into malware traces later on. 2. Back to of IBM Cloud Classic Virtual Server. on all ports. Security Groups removed allow_all ingress Custom security groups with incoming ip-whitelisting though more secure & limits network exposure; was not preferable as our users would be accessing from local machines across globe. It would be unmanageable to whitelist user IPs on demand. So, how would our in-house end users experimenting — access database instances from their local machine — IDE ? The core requirement! A hassle free shortcut was SSH Tunnel with port forwarding. Users could connect via encrypted ssh tunnel to the database hosting Virtual Server and forward traffic from local machine db port to Server side DB port. For example: . traffic from 27017 port of local machine would be forwarded to Server side mongo dB 27017 port, within a secure & encrypted SSH tunnel— thus allow IDEs to connect SSH tunnel is not a fool proof solution, . But this approach in combination with prior tightly managed SSH keys — did increase our setup’s security. it does have potential backdoor threats 3. Enable Access Control & Enforce Authentication on the Mongo DB instances from the Ansible Playbook. Mongo DB does not enable access control/authorization by default ( ). dated Oct 2020, current latest version — MongoDB 4.4 So, in the mongo DB installation — Ansible playbook, the step to was added. enable security.authorization This would ensure a user can access only the database resources and actions for which they have been granted privileges. Prior to it —a user could access any database and perform any action, yikes! 4. However, after enabling this — connections made from Robo 3T (or RoboMongo), an open source MongoDB GUI were successful but MongoDB Compass GUI started behaving erratically. — as defined here. Configure TLS/SSL Transport layer encryption to Mongo DB So, decided to skip this configuration from final assembly. And I did not want our users to be forced to pick one GUI over other for a post-playbook verification step. - Other options like — ’ was not needed — as the databases were purely for use-and-tear-down. It was not necessary to secure the data at rest, however those with different business needs — may certainly do. ‘ encryption at rest - I did find interesting and it would have helped us secure our mostly transient Mongo databases better. But, this feature was only available in Enterprise Edition and Atlas (MongoDB-as-a-Service); and we were using free Community Edition. Auditing feature Alrighty then — this was the real threat received :) from Meow attackers. Never paid any bitcoin, phew! : Stay safe even whilst experimenting! Lesson learnt Also published at https://medium.com/wabi-sabi-techexpress/when-the-meow-attack-on-our-mongodb-shook-us-86230d79c509 Soul staring Cat & sitting cute ducks - g ; Image source: openclipart.or ssh tunneling - Image from ssh.com; rest images - first hand captures & self created explainers from the experience.

Assembly

Super

Target

2022 - HackerNoon Contributor of the Year - Google

2022 - HackerNoon Contributor of the Year - Google Cloud Platform

Nominated for 2022 - HackerNoon Contributor of the Year - Google Cloud Platform

Nominated for 2022 - HackerNoon Contributor of the Year - Google

Technical Solutions Consultant | Solutions Architect | Senior Manager

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

Learnings From The Meow Bot Attack on Our MongoDB Databases

Learnings From The Meow Bot Attack on Our MongoDB Databases

Too Long; Didn't Read

Companies Mentioned

Rani Pallavi Mohapatra

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Learnings From The Meow Bot Attack on Our MongoDB Databases

Too Long; Didn't Read

Companies Mentioned

Rani Pallavi Mohapatra

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES