5

So one of your servers is compromised. You determine this because you notice a weird process running (as root, unfortunately). No helpful information about this process from Googling around.

First things first, let's clean this known dirty server up. Nuke it from orbit, see ya later. How do I deal with a compromised server? is a good guide for dealing with that server.

Now, a process running as root on a box inside the internal network does not give me a lot of confidence that this attack is contained to just this single server, however, I can't immediately find any signatures on other boxes. It is going to be a massive task, likely with substantial data loss or downtime, to fully nuke the entire infrastructure and rebuild.

What steps should you take to feel confident an attack has not spread to other servers on the network?

Bonus: This is a broad question. I can't find any good resources for describing current best practices for dealing with incident response/forensic analysis. Do any resources exist?

Community
  • 1
Anthony Kraft
  • 1,149
  • 1
  • 9
  • 18
  • I think this question is too broad: Maybe you deal with a typical low budget network where no monitoring was ever done, no access control and the usual non-patch management. Or maybe you deal with a networking containing proper protections, breach detection, separation etc. And of course it depends on which kind of data might be in danger. There is simply no general response which encompasses all the different cases. – Steffen Ullrich Apr 18 '16 at 16:01
  • 1
    Figuring out how the malware got on that one server might shed some light onto whether other servers were vulnerable to that same exploit. Disassembling that malicious binary could also tell whether it has the capability to compromise other hosts on the network. – André Borie Apr 18 '16 at 16:11
  • @AndréBorie: your suggestions assume that there are still enough information on the compromised machine what the attacker did, i.e. that the attacker did not remove any tools after use. I would not assume this to be the case if the machine was simply used as the initial entry point into the network and the attacker moved on already. – Steffen Ullrich Apr 18 '16 at 16:22
  • @SteffenUllrich you are right, that's something to keep in mind. The issue here is to determine with enough confidence whether the attack was sophisticated enough to cover its tracks (for a targeted attack I would assume yes, but I'd say that's not the case for an automated attack that just dropped its crappy spam/DoS bot), and move on from there. – André Borie Apr 18 '16 at 16:28

2 Answers2

1

"What steps do you take to feel confident..." Restore from back up. Now you said running as root so I will assume this is a Unix variant (Linux/BSD/etc) here are some things you can do to give you a baseline of what may be going on. 1) Take the machine offline if possible:

mkdir /tmp/DFIR
netstat -a | grep -i "tcp\|udp\|icmp" >> /tmp/DFIR/connections.txt
last >> /tmp/DFIR/lastlogins.txt
lsof ID_OF_SUSPICIOUS_PROCESS >> /tmp/DFIR/suspicious-PID.txt

Now that you have an idea of baseline things to look at, this will likely be a waste of time. Depending on what was used to get in, whether or not there is a backdoor, logfiles were erased, it can be a time consuming task. But if you want to continue...

Once you have the PID of the suspicious process, you can determine (if timestomp or chage wasn't used) when it came to light/modifications via the MACE times. You can take a stab at trying Volatility but unless you are already familiar with the gist of forensics, memory analysis is time consuming.

Now I will stop since the initial question was broad, you stated root, and for all I know, you're using Windows. My advice, restore from a backup unless there is something where you intend on pursuing civil or criminal charges. In which case, you would need an expert.

ADDED (EDIT)

Before you nuke the system, I would create a YARA signature to look for whatever you saw. I would definitely do the netstat to monitor connections, on the next install of ANYTHING, I would create a hardened remote syslog server, minimize access to the principle of least privilege, for starters. But again, as mentioned the question is really broad from the onset.

munkeyoto
  • 8,682
  • 16
  • 31
0

Like @SteffenUllrich said, it depends on a lot of factors. Do you have any IDS? Firewall? Is your network a small one or a big one?

Anyway, even if a lot of elements are missing, here is some basics to have for a small network: - Isolate the computer that is compromised from the network

  • Read the logs (keep in mind there may be logs missing/erase) and search for informations coming from other compromised machine
  • If you have any IDS, some investigation need to be done
  • Try to find your ressources that has the higher risk (risk = threat x vulnerability x cost) and secure them (read there logs could be a nice idea to see if it is compromised)
  • Inform your coworker and ask for help. The faster the compromised machine are find, the better you will be effective to protect your network.

I try to answer you the best way I can with the little info, I hopes it help a little.

For further attack, Mozilla create a tool call MIG that can help you to search for information with a simple request: http://mig.mozilla.org/

RandomSecGuy
  • 95
  • 1
  • 8