MSNM-Sensor for local testing

[daffi1238]

Good morning again,

I'm trying to use the project to get conclusions about a pcap file with "clean traffic" to calibrate dynamically each 60 minutes and after 2 hours starting to generate malicious traffic to check if the Q-st and D-st observations go out from the UCL's. Here I face with two main problems.

1. I'm using tcpreplay to generate the traffic in my computer, where I can choose the interface which will read this packets. Checking the activateNetflow.sh script, I not sure which interface is listening to generate the nfcapd files. The three interface avaliable are 'lo', 'enp0s3' and 'enp0s8'. I thought to apply the parameter -b
   "-b bindhost
   Specifies the hostname/IPv4/IPv6 address to bind for listening. This can be an IP
   address or a hostname, resolving to an IP address attached to an interface. Defaults
   to any available IPv4 interface, if not specified."
   but I wanted to take advice of you.

2. The other point is about the dashboard, it looks like just show (Represent) the Q-st and D-st value of the last 90 observations (1'5 hours) is there a way to expand that to several hours?

About the scenario, I'm just using routerR1 to analyze the results but in the context of me is enough to check the followed methodology.

Thank you and greetings,
Jesús.

[nesg-ugr]

HI Jesús,

nfcapd daemon gets traffic flows from every known NIC by default. I have never used -b option to specify an IP, but you can check it by yourself.

If you want to change the number of observations the monitoring graphs show, you have to modify the config.py file located at dashboard/mainboard/config.py Then, you have to restart the dashboard.

I hope it helps you!

Best,
Roberto.

[daffi1238]

Thank you Roberto,

I keep working on this and conclude that nfcapd has problem with tcpreplay or even with pcap files, I'm waiting to get a solution to post here as soon as possible.

In fact I post the problem I face and maybe you could help me.

What I'm trying to do is use the traffic in a pcap file to annalyze with the msnm-sensor and be able to reproduce any traffic or any environment in the virtual machine offered by you. For this I thought in two ways:

1. tcpreplay to read a pcap file in the NIC you want:
   sudo tcpreplay -i enp0s3 -K test.pcap
   After that using the activateNetflow script should get this packets like flow. wireshark at least read it but after some time working looking the file in ~/msnm-sensor/examples the csv's there have any information relative to the tcpreplayed traffic.

2. The other way I thought were don't use the activateNetflow script and generate a file in /tmp/netflow_captures using nfcapd -f, I already did a little script to automatize it. But the problem is that doing this in a pcap file:
   nfcapd -w -D -t 60 -l ./ -f 00.pcap -I localhost -p 12345
   and after that listing the contain in csv format:
   nfdump -r nfcapd.2017 -o csv
   The contain is empty.

I guess that the problem have to be in the timestamp o something like that, in any case, I comment the problem to trace the process that I'm following. I'll keep you informed.

Greeting and thank you all
Jesús.

Update 1: I didn't get the relation until just now. The problem is that the iptables rules are not apply to tcpreplay traffic. I'm checking for a solution.

[daffi1238]

Hi everybody,

I get that the traffic read from a pcap. For that you have to read it with tcpreplay in the host machine (or other virtual machine) changing the ip addresses and mac addresses to be congruent. In particular in my example the pcap add a server with 127.0.0.1 and several sensors with 127.0.0.x, and I used the next command to custom the pcap and all the macs (sources and destiny) were the 00:00:00:00:00:00, so I had to deploy the next commands:

1. Change the mac destiny to the mac that the virtual machine is using (We need to have a subnet that connect the host machine with the virtual one)
   tcprewrite --enet-dmac=08:00:27:35:3a:7a -i file1.pcap -o file2.pcap
2. Change the mac src to an arbitrary mac (I used the one that the machine who reads the pcap use
   tcprewrite --enet-smac=00:50:56:c0:00:08 -i file2.pcap -o file3.pcap
3. Change all the ip's (src and dest) to an arbitrary one to work properly in the next step
   tcprewrite --pnat=127.0.0.0/8:172.16.54.129/32 --infile=file3.pcap --outfile=file4.pcap --skipbroadcast
   (172.16.54.129 is the ip_address used by the virtual machine msnm, this step require some redundancy but for be more graphical in the proccess I decided to keep it)
4. Change the ip_src to an arbitrary ip in the network address used in the shared interface.
   tcprewrite --infile=file4.pcap --outfile=file5.pcap --srcipmap=[172.16.54.129]:[172.16.54.128]
5. This step is optional, I thought modify the 'activateNetflow.sh' just for create an iptables rule for the filter FORWARDING table and avoid some possible noise in the other tables. For that I activate ip_forward in the msnm-sensor machine and obligued to pass throught the redirect filter using like ip_dest some IP different to the one used by msnm-sensor
   tcprewrite --infile=out_nodiff.pcap --outfile=out_nodiff_noip.pcap --dstipmap=[172.16.54.129]:[172.16.54.130]

Until here we have the traffic read in the host machine passing through the REDIRECT iptables filter.

Some details:
To read the pcap with tcpreplay I used:
sudo tcpreplay -i vmnet8 -K out_nodiff_noip.pcap
being the vmnet8 a virtual interfaces shared with the msnm-sensor virtual machine.

I hope this will help you to be more flexible in future experiments.

Please don't close still the issue because probably with the next experiments I will some information to add here.

Greetings, Jesús.

[daffi1238]

Hi everybody again,

I was investing and discover a little deficiency. Using the context up described I get pass through the iptables of the msnm-sensor machine the traffic replayed using tcpreplay. The point is that I needed to limit the packets per minute to around 150. If I send more than that the csv returned in the examples/scenario_4/routerR1/data/sources/netflow/parsed appear like empty with 0 bytes contained. I did't investigate the reason of that but I guess that is not hard to extend this limitation. I'm planning to upload (in somewhere) the instructions followed to deploy this environment, I will text where when I finish my investment.

By the way, how were limited the number of packets was using --mbps 0.002:
sudo tcpreplay -i vmnet8 -K --mbps 0.002 05-full_year_trafico.pcap

Greetings, Jesús.

[daffi1238]

Hi again,
Just to comment that this problem persist but only with the traffic 'replayed' from a pcap. From a nmap scan or so there is no problem.

I will update soon

[daffi1238]

Hello again,

I think that I found a wall in my experiments related with this. I prepared a scenario to test the replaying of a traffic content in a pcap as I said before, and I found a limit in the bit rate taxe that nfcapd cap proccess (The problem comes from empty files in /tmp/netflow_captures). I even delete the payload of each packet (to be able to send more whit the same data rate) and the problem persist. And I thought that the problem was related with the pcap file but I tried a high data rate scan with nmap (sudo nmap -sS --min-rate 5000 -p- -Pn -n msnm-sensor) and the result was the same, empty files in /etc/netflow_captures

Did you had this problem during your experiments?