Thursday, October 27, 2011

A Story From a System Administrator

I want to share a story with the hope that I learned from. It involves hacking, system administration and computer's forensic. My goal of sharing the story is to think deeply about the situation as an attacker and as a system administrator. In another word, think of what I would have done differently if I was the attacker. Also think of what the system administrator did and comment on it.

The story is real. However, I am not going to disclose the identity of any of the people involved. 

There was this system administrator whom responsibilities include managing a large and complex network. One of the VLAN in that network allows anyone to configure his machine manually to use any free IP in that VLAN.  All you need to do is to hook up your computer to one of the network's jacks and configure your machine's network setting.

The system administrator had arwaptch installed to monitor the mac addresses that join the network. Arpwatch is a good tool to detect changes on mac-to-ip mapping, detecting new mac addresses joining the network, and alerting system administrators.
System/Network administration note :
An alternative and a better way is to have a DHCP server in the network. In your DHCP configuration you allow only specific mac address to join the network. Also, ask your user to register their mac by communicating with you. When they do so, make sure you record that mac address, the name of the person, his contact details and also the network jack they are going to hook up their machine too. Also, make sure that you have a record of which switch's port maps to what network jack. Finally, write a script that query your switch for the MAC-port translation's table using SNMP protocol.
If you do all this, then you know exactly what mac addresses live in your network. Also, you know the mapping between the mac address, the network jack and the switch port. For example, 52:54:05:F3:95:01 --> Network Jack Port #2007-A --> Switch port # 18. Having all that information makes it easy for you to write a script that alert you if something does not feel right. For example, a mac address that was connected to another network jack.
Hacking/Security note:
If you happen to know a mac address that is already registered in the DHCP server, you can always fake your real mac address. However, if the network's administrator had the setup above in place, he will end up visiting the network jack location. You will be in a awkward situation if that happens.
Back to the story. Someone in that network connected his machine to one of the jacks and configured his machine to use one of the available IPS. The system administrator was notified so he started an investigating. Since he inherited all this work, there was no documentation of the network or records of anything. He knew that he needs to get this person but his hands were tights to his back. He decided to start small and build his knowledge base from the ground up.

The system administrator had the mac address from the arpwatch's alert message. However, he used arping to confirm it .. and there it was, the same mac address. The mac address was : 44:87:FC:XX:XX:XX.

The system administrator had the mac address but he needed to know more. So he ran nmap to figure out what OS the machine was running. Also he wanted to further confirm the mac address is legit. He looked online and found out that according to IEEE standards association, the prefix 44:87:FC is OUI, organizationally unique identifier, for ELITEGROUP COMPUTER SYSTEM CO., LTD.  However, Nmap was unable to guess which OS the machine is running but it was able to tell that that mac address is from Elitegroup Computer System Co too.
Computer's forensic note :
To really get the person and have a strong case against him. You need to collect as much information as possible. If you identify the machine by its mac address and also the network card's manufacture, then all you need is to get that machine's owner. He should be legally responsible.
Hacking note :
In general, you want to use virtual machine or even better someone else machine instead of your own machine. If you happen to use a VM, make sure to connect it directly to the network ( bridging mode ) instead of ( nat ) mode. However, remember that all your traffic goes through the host's machine network card. An experienced system administrator will be able to tell.  Also, if you are using VM, make sure that your VM hard disk is not stored in your machine hard disk. Computer forensic specialist will be able to restore that file unless you securely delete it. Another option would be to use external hard drive and throw it away when you are done. However, your hypervisor might write something in its log which is still in the disk.  This can still links back to you. 
At this point, the system administrator had more information about the machine, sort of. He was ready to know more about the person himself. Since the system administrator has access to the network firewall, he used tcpdump to sniff the traffic where the src or dest is that ip address. To the system administrator surprise, the person was using the university dns server on which the system administrator had full root access.

Immediately the system administrator knew that he needs to turn on the DNS query log. Now he had a record of the websites that the person was visiting. That just saved the system administrator some time since he was sniffing all the traffic anyway.
Hacking note :
For God's sake, do not use the DNS server for the person you are messing with. Please do not do that. It is just stupid. There is a lot of public DNS servers out there.
Computer forensic note:
Keep any information you get, you will need it to strengthen your case by showing the time stamp and the websites he was visiting at that time. Those websites can provide information about that person's identity if ordered by a court. Also, when you use tcpdump, make sure that it writes to a file in the desk. Forensic can prove that the file was written to the desk and never altered.
System administration note :
Do not forgot to turn the logrotate off so your log does not get deleted accidentally. You can always configure your dns server to send the query log to a different file which logrotate does not know about.

The system administrator figured out that the person is visiting websites like facebook, google, google plus and ebay among others. He figured out that it is now the time to run tcpdump in a deep aggressive mode by catching the entire packet and hex decoded it in the fly.
Hacking note :
Please do not visit social networking websites when you are doing something wrong.
Also, if you are messing with someone, make sure that you block all outgoing traffic. Allow only the ports that should have encrypted traffic pass through, i.e port 443, HTTPS, port 22, SSH. You never know what an application in your machine might connect to without you knowing. That can be used against you.  You also can create a SSH tunnel to another machine that can not be linked to you. However, please understand that even it is well known that port 443 is for HTTPS, it might not be the case. There is nothing that prevents system administrator from configuring his webserver to serve HTTP request over port 443.
So the system administrator was running tcpdump in one screen and it was writing to pcap file. In another screen , he was running another instant of tcpdump to decode that file. However, the person was using SSL all the time.

but he made a mistake ...

The system administrator saw the following :

0x0250:  696e 7669 7465 6420 796f 7520 746f 2074
0x0260:  6865 2065 7665 6e74 2054 6865 2041 6c65  he.event.The.Ale
That line is saying that someone in facebook has invited this person to an event. but wait, the system administrator did not see that person logging into facebook. Was he using SSL to login and then somehow switched to none SSL? is facebook using AJAX over HTTP instead of HTTPS? the system administrator could not tell. But who cares, he needed to get that person.

Doing a quick search in facebook, he was able to see the person's friend profile. a quick search in his friend list of friends, he was able to see only one guy lives in the city where the university is located. The profile of that person was open to public. It shows that he is a student at the university.

The system administrator was not convinced that that person is really whom he is after. so he waited. Another packet came in :

0x0390:  XXXX XXXX 6c73 6f20 636f 6d6d 656e XXXX  XXXX.also.commen
0x03a0:  7465 6420 XXXX    XXXX XXXX XXXX XXXX XXXX  ted.on.Person3.Name

That text reads : Person1 and Person2 also commented on Person3's link. What does that mean ? Person1 and Person2 are friends of the person he was after. Also, that means he also commented in person3's link. The system administrator went to all those profiles and the person name came up again. The system administrator now has the person identity confirmed from three different sources.
Computer forensic note :
collect your evidence, sign them, record every information about those files such as inode number, location in the desk, size, state information, everything. copy them off disc in a safe place. You will need those information to prove the files are legit and was not altered or faked.
The system administrator took an action by blocking all outgoing traffic from that IP. In the next morning he physically went to the machine's room. He was able to find the jack number and the room where that person was located. It was one of the students lab. No surprises.
Academic/Legal note :
If that system administrator reported the situation to his university security department, that person will be in a serious trouble. He will be suspended or at least be aggressively warned and banned from using any IT service. If the system administrator reported the situation to that person's academic advisor, he will lose all his respect ... It does not worth it, really.
Hacking note :
Never ever do something that you are not capable of. If you ever get caught, tell the truth and nothing but the truth. Be reasonable and understand that it is your failure that got you caught. Accept the fact in front of you. You might make it easy on yourself and gains others respect. Lying just make the person you messed with feel stupid which might lead to an unexpected behavior.


Jessa said...

Interesting story! So did the student get in trouble?

alialzabarah said...


Post a Comment