network

Fail2ban Configuration for NGINX anomalies

Fail2ban is a really cool log analyzer (mostly) that can block ips using several different methods (iptables, ipfw, ip route blackhole, etc.). The problem is that you have to define filters (regexes in fact) that will trigger the ban for each service, because each one has a different way to report anomalies. There are not so much given examples on the official wiki. On other websites I couldn’t find anything about nginx filters. Even worse, several websites report that you can use the filters defined  for Apache2, which is false, they will NOT work, the logs are very different.

For example, here is a trace for a non existent requested resource:

2011/12/29 16:13:33 [error] 3212#0: *241787 open() "/opt/foo/default/admin/phpmyadmin/index.php" failed (2: No such file or directory), client: 58.19.239.205, server: , request: "GET //admin/phpmyadmin/index.php HTTP/1.1", host: "88.191.135.71"

So, to be able to detect such hack tentative and block it, create a file named nginx-noscript.conf, and put:

[Definition]
failregex = open\(\) "/\S*(\.php|\.asp|\.exe|\.pl)\S*" failed.*client: <HOST>,.*
ignoreregex =

Then, add its definition in an entry in the /etc/fail2ban/jail.conf:

[nginx]
enabled = true
port = http,https
filter = nginx-noscript
logpath = /var/log/nginx*/*error.log
maxretry = 6

Here, if there are more than 6 occurences of a failed 404 request in less than 600 seconds (the default value, modifiable with the ‘findtime’ variable), the ip will be added to the ban list.

Default behaviour in implementation of STOMP protocol in RabbitMQ with python

Why STOMP?

Why STOMP, and not directly AMQP, as I’m using RabbitMQ. No real reason, but the fact that there are less dependencies on a STOMP client, as it’s just a socket with text sent.

Implementations

There are several implementations of the STOMP protocol for Python. The module I chose is python-stomp (version 0.2.9), from Benjamin W. Smith. It’s simple and easy to understand.

Simple Code Examples

sto_send.py:

1
2
3
4
5
6
from stompy.simple import Client
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='noneofyourbusiness')
stomp.put('Thomas est une b*te à Tetris...', destination='/queue/jeuvideo')
stomp.disconnect()

sto_receive.py:

1
2
3
4
5
6
7
8
9
10
11
12
from stompy.simple import Client
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='noneofyourbusiness')
stomp.subscribe('/queue/jeuvideo')
message = stomp.get()
 
print message.body
 
#stomp.ack(message)
stomp.unsubscribe('/queue/video')
stomp.disconnect()

Everything is working fine, when launching sto_receive.py, I receive the message. But when I launched several receivers, I noticed, that ONLY ONE programs received the message! After some research, I found the answer: As documented in the RabbitMQ wiki, the default exchange is ‘direct’:

[...]when messages leave a queue for a consumer, they are not duplicated. One message, sitting on a queue, is delivered to only one of the available consumers. [...] If there are multiple clients, all SUBSCRIBEing to the same queue, then there will be multiple consumers all on the same queue, leading to round-robin delivery to those clients.

There is an explanation on how to change the behaviour, by changing the exchange type, and some of particular bits (like the id). I even found an example of modification for use in the equivalent STOMP Ruby module.

Here are the modifications. The good news is that there is no need to patch the stompy module, as the author provided the possibility to pass arbitrary parameters to the headers by the use of the ‘conf’ variable.

The important points are:

  • You need to define an exchange of type amq.topic
  • You need to set an id, which is different for each client
  • As you’re using topics, you’ll have to specify a routing_key

sto_receive.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from stompy.simple import Client
import uuid
 
unique_id = uuid.uuid4()
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='nonononono')
 
stomp.subscribe('',
                conf={'exchange': 'amq.topic',
                      'routing_key':'x.#',
                      'id': unique_id,
                      })
 
# Wait for a message to appear
while 1:
    message = stomp.get()
    print message.body
 
#stomp.ack(message)
stomp.unsubscribe('',conf={'id': unique_id})
stomp.disconnect()

sto_send.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from stompy.simple import Client
 
 
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='nonononono')
 
 
for i in range(10000):
 stomp.put('Thomas est une b*te au Tetris...', destination='x.y',
          conf={'exchange':'amq.topic',
                #  'routing_key':'x.y'
                })
 
stomp.disconnect()

Puppet: Files found in modules without specifying ‘modules’ in file path will be deprecated in the next major release

DEPRECATION NOTICE: Files found in modules without specifying ‘modules’ in file path will be deprecated in the next major release.

If you get this warning in your puppet logs, you should take action (only if you don’t have any Puppet agent with a version <= 0.24) and modify all you references to file resources.
For example, if you have a module named 'ssh', normally, up to puppet 0.25 you would reference a file to it as:

source => ["puppet:///ssh/authorized_keys",]

But now, you need to insert a ‘module’ identifier in between like this:

source => ["puppet:///modules/ssh/authorized_keys",]

Just a small note: It seems that the templates do not need any modification.

How to understand the ARP queries and replies fields with pypcap

0

I had a hard time understanding the function of each field in an ARP packet. The problem is that the fields change of meaning, depending on the opcode field. The two useful ones are for ARP queries (what is the ethernet address of the ip address I’m giving now) and ARP replies (that ip address is located at this ethernet address).

So to fix this problem once for all, I decided to write a python script that shows the different field values when an ARP packet is captured.

There are several libraries available to the pythonista to manipulate network packets. The most known is certainly pylibpcap which is quite old now, and not really object oriented. It is more an adaptation one-to-one of the C libpcap library, which may be useful for some people.
Another library is pypcap, which is like pylibpcap, but much much more object oriented.
pypcap includes a huge quantity of protocols definitions, so it’s really cool to use, especially because it also includes a network packet capture method. There is no included method to send packets, but there are examples of how to do this in the test files.
Another possibility is scapy, which is an extremely complete program (more a program than a library, even though you can use it as a module). The fact that it’s not that easy to include scapy in my own program, even though there’s now a howto. My program doesn’t need all the bells and whistles given by scapy, so I settled on pypcap.

import dpkt, pcapfrom socket import inet_ntoa

def ether_decode(p):
return ':'.join(['%02x' % ord(x) for x in str(p)])

if __name__ == '__main__':
iface_name = 'eth1' # Here set your listening interface
pc = pcap.pcap(iface_name)
pc.setfilter('arp')

for ts,pkt in pc:
packet = dpkt.ethernet.Ethernet(pkt)

print "ARP packet received:"
print "op=%d" % packet.data.op
print "src=%s" % ether_decode(packet.src)
print "dst=%s" % ether_decode(packet.dst)
print "spa=%s" % inet_ntoa(packet.data.spa)
print "tpa=%s" % inet_ntoa(packet.data.tpa)
print "tha=%s" % ether_decode(packet.data.tha)
print "sha=%s" % ether_decode(packet.data.sha)
print

That’s it. Now, for example, 192.168.4.3 wants to get 192.168.4.254′s ethernet address (192.168.4.254 has 00:90:4c:49:00:2a address and 192.168.4.3 has 00:50:70:b4:19:0c), here is the output:


ARP packet received:op=1 src=00:50:70:b4:19:0c dst=ff:ff:ff:ff:ff:ff
spa=192.168.4.3 tpa=192.168.4.254 tha=00:00:00:00:00:00sha=00:50:70:b4:19:0c
ARP packet received:op=2 src=00:90:4c:49:00:2a dst=00:50:70:b4:19:0c
spa=192.168.4.254 tpa=192.168.4.3 tha=00:50:70:b4:19:0csha=00:90:4c:49:00:2a

Go to Top