For example, here is a trace for a non existent requested resource:

2011/12/29 16:13:33 [error] 3212#0: *241787 open() "/opt/foo/default/admin/phpmyadmin/index.php" failed (2: No such file or directory), client: 58.19.239.205, server: , request: "GET //admin/phpmyadmin/index.php HTTP/1.1", host: "88.191.135.71"

So, to be able to detect such hack tentative and block it, create a file named nginx-noscript.conf, and put:

[Definition]
failregex = open\(\) "/\S*(\.php|\.asp|\.exe|\.pl)\S*" failed.*client: <HOST>,.*
ignoreregex =

Then, add its definition in an entry in the /etc/fail2ban/jail.conf:

[nginx]
enabled = true
port = http,https
filter = nginx-noscript
logpath = /var/log/nginx*/*error.log
maxretry = 6

Here, if there are more than 6 occurences of a failed 404 request in less than 600 seconds (the default value, modifiable with the ‘findtime’ variable), the ip will be added to the ban list.

Why STOMP, and not directly AMQP, as I’m using RabbitMQ. No real reason, but the fact that there are less dependencies on a STOMP client, as it’s just a socket with text sent.

Implementations

There are several implementations of the STOMP protocol for Python. The module I chose is python-stomp (version 0.2.9), from Benjamin W. Smith. It’s simple and easy to understand.

Simple Code Examples

sto_send.py:

1
2
3
4
5
6

from stompy.simple import Client
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='noneofyourbusiness')
stomp.put('Thomas est une b*te à Tetris...', destination='/queue/jeuvideo')
stomp.disconnect()

sto_receive.py:

1
2
3
4
5
6
7
8
9
10
11
12

from stompy.simple import Client
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='noneofyourbusiness')
stomp.subscribe('/queue/jeuvideo')
message = stomp.get()
 
print message.body
 
#stomp.ack(message)
stomp.unsubscribe('/queue/video')
stomp.disconnect()

Everything is working fine, when launching sto_receive.py, I receive the message. But when I launched several receivers, I noticed, that ONLY ONE programs received the message! After some research, I found the answer: As documented in the RabbitMQ wiki, the default exchange is ‘direct’:

[...]when messages leave a queue for a consumer, they are not duplicated. One message, sitting on a queue, is delivered to only one of the available consumers. [...] If there are multiple clients, all SUBSCRIBEing to the same queue, then there will be multiple consumers all on the same queue, leading to round-robin delivery to those clients.

There is an explanation on how to change the behaviour, by changing the exchange type, and some of particular bits (like the id). I even found an example of modification for use in the equivalent STOMP Ruby module.

Here are the modifications. The good news is that there is no need to patch the stompy module, as the author provided the possibility to pass arbitrary parameters to the headers by the use of the ‘conf’ variable.

The important points are:

• You need to define an exchange of type amq.topic
• You need to set an id, which is different for each client
• As you’re using topics, you’ll have to specify a routing_key

sto_receive.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

from stompy.simple import Client
import uuid
 
unique_id = uuid.uuid4()
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='nonononono')
 
stomp.subscribe('',
                conf={'exchange': 'amq.topic',
                      'routing_key':'x.#',
                      'id': unique_id,
                      })
 
# Wait for a message to appear
while 1:
    message = stomp.get()
    print message.body
 
#stomp.ack(message)
stomp.unsubscribe('',conf={'id': unique_id})
stomp.disconnect()

sto_send.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

from stompy.simple import Client
 
 
 
stomp = Client(host='rabbitmq2')
stomp.connect(username='guest',password='nonononono')
 
 
for i in range(10000):
 stomp.put('Thomas est une b*te au Tetris...', destination='x.y',
          conf={'exchange':'amq.topic',
                #  'routing_key':'x.y'
                })
 
stomp.disconnect()

DEPRECATION NOTICE: Files found in modules without specifying ‘modules’ in file path will be deprecated in the next major release.

If you get this warning in your puppet logs, you should take action (only if you don’t have any Puppet agent with a version <= 0.24) and modify all you references to file resources.
For example, if you have a module named 'ssh', normally, up to puppet 0.25 you would reference a file to it as:

source => ["puppet:///ssh/authorized_keys",]

But now, you need to insert a ‘module’ identifier in between like this:

source => ["puppet:///modules/ssh/authorized_keys",]

Just a small note: It seems that the templates do not need any modification.

So to fix this problem once for all, I decided to write a python script that shows the different field values when an ARP packet is captured.

There are several libraries available to the pythonista to manipulate network packets. The most known is certainly pylibpcap which is quite old now, and not really object oriented. It is more an adaptation one-to-one of the C libpcap library, which may be useful for some people.
Another library is pypcap, which is like pylibpcap, but much much more object oriented.
pypcap includes a huge quantity of protocols definitions, so it’s really cool to use, especially because it also includes a network packet capture method. There is no included method to send packets, but there are examples of how to do this in the test files.
Another possibility is scapy, which is an extremely complete program (more a program than a library, even though you can use it as a module). The fact that it’s not that easy to include scapy in my own program, even though there’s now a howto. My program doesn’t need all the bells and whistles given by scapy, so I settled on pypcap.

import dpkt, pcapfrom socket import inet_ntoa

print "ARP packet received:"
print "op=%d" % packet.data.op
print "src=%s" % ether_decode(packet.src)
print "dst=%s" % ether_decode(packet.dst)
print "spa=%s" % inet_ntoa(packet.data.spa)
print "tpa=%s" % inet_ntoa(packet.data.tpa)
print "tha=%s" % ether_decode(packet.data.tha)
print "sha=%s" % ether_decode(packet.data.sha)
print

That’s it. Now, for example, 192.168.4.3 wants to get 192.168.4.254′s ethernet address (192.168.4.254 has 00:90:4c:49:00:2a address and 192.168.4.3 has 00:50:70:b4:19:0c), here is the output:


ARP packet received:op=1 src=00:50:70:b4:19:0c dst=ff:ff:ff:ff:ff:ff
spa=192.168.4.3 tpa=192.168.4.254 tha=00:00:00:00:00:00sha=00:50:70:b4:19:0c
ARP packet received:op=2 src=00:90:4c:49:00:2a dst=00:50:70:b4:19:0c
spa=192.168.4.254 tpa=192.168.4.3 tha=00:50:70:b4:19:0csha=00:90:4c:49:00:2a