instructions
This commit is contained in:
299
fail2ban-master/DEVELOP
Normal file
299
fail2ban-master/DEVELOP
Normal file
@@ -0,0 +1,299 @@
|
||||
.. __ _ _ ___ _
|
||||
/ _|__ _(_) |_ ) |__ __ _ _ _
|
||||
| _/ _` | | |/ /| '_ \/ _` | ' \
|
||||
|_| \__,_|_|_/___|_.__/\__,_|_||_|
|
||||
|
||||
================================================================================
|
||||
How to develop for Fail2Ban
|
||||
================================================================================
|
||||
|
||||
Fail2Ban uses GIT (http://git-scm.com/) distributed source control. This gives
|
||||
each developer their own complete copy of the entire repository. Developers can
|
||||
add and switch branches and commit changes when ever they want and then ask a
|
||||
maintainer to merge their changes.
|
||||
|
||||
Fail2Ban uses GitHub (https://github.com/fail2ban/fail2ban) to manage access to
|
||||
the Git repository. GitHub provides free hosting for open-source projects as
|
||||
well as a web-based Git repository browser and an issue tracker.
|
||||
|
||||
If you are familiar with Python and you have a bug fix or a feature that you
|
||||
would like to add to Fail2Ban, the best way to do so it to use the GitHub Pull
|
||||
Request feature. You can find more details on the Fail2Ban wiki
|
||||
(http://www.fail2ban.org/wiki/index.php/Get_Involved)
|
||||
|
||||
Pull Requests
|
||||
=============
|
||||
|
||||
When submitting pull requests on GitHub we ask you to:
|
||||
|
||||
* Clearly describe the problem you're solving;
|
||||
* Don't introduce regressions that will make it hard for systems administrators
|
||||
to update;
|
||||
* If adding a major feature rebase your changes on master and get to a single commit;
|
||||
* Include test cases (see below);
|
||||
* Include sample logs (if relevant);
|
||||
* Include a change to the relevant section of the ChangeLog; and
|
||||
* Include yourself in THANKS if not already there.
|
||||
|
||||
If you are developing filters see the FILTERS file for documentation.
|
||||
|
||||
Code Testing
|
||||
============
|
||||
|
||||
Existing tests can be run by executing `bin/fail2ban-testcases`. It has
|
||||
options like --log-level that will probably be useful. Run
|
||||
`bin/fail2ban-testcases --help` for the full list of options.
|
||||
|
||||
Test cases should cover all usual cases, all exception cases and all inside
|
||||
/ outside boundary conditions.
|
||||
|
||||
Test cases should cover all branches. The coverage tool will help identify
|
||||
missing branches. Also see http://nedbatchelder.com/code/coverage/branch.html
|
||||
for more details.
|
||||
|
||||
Install the package python-coverage to visualise your test coverage. Run the
|
||||
following (note: on Debian-based systems, the script is called
|
||||
`python-coverage`)::
|
||||
|
||||
coverage run bin/fail2ban-testcases
|
||||
coverage report
|
||||
|
||||
Optionally:
|
||||
coverage html
|
||||
|
||||
And then browse htmlcov/index.html and see how much coverage your test cases
|
||||
exert over the code base. Full coverage is a good thing however it may not be
|
||||
complete. Try to ensure tests cover as many independent paths through the
|
||||
code.
|
||||
|
||||
Manual Execution. To run in a development environment do::
|
||||
|
||||
./fail2ban-client -c config/ -s /tmp/f2b.sock -i start
|
||||
|
||||
some quick commands::
|
||||
|
||||
status
|
||||
add test pyinotify
|
||||
status test
|
||||
set test addaction iptables
|
||||
set test actionban iptables echo <ip> <cidr> >> /tmp/ban
|
||||
set test actionunban iptables echo <ip> <cidr> >> /tmp/unban
|
||||
get test actionban iptables
|
||||
get test actionunban iptables
|
||||
set test banip 192.168.2.2
|
||||
status test
|
||||
|
||||
|
||||
Testing with vagrant
|
||||
--------------------
|
||||
|
||||
Testing can now be done inside a vagrant VM. Vagrantfile provided in
|
||||
source code repository established two VMs:
|
||||
|
||||
- VM "secure" which can be used for testing fail2ban code.
|
||||
- VM "attacker" which can be used to perform attack against our "secure" VM.
|
||||
|
||||
Both VMs are sharing the 192.168.200/24 network. If you are using this network
|
||||
take a look into the Vagrantfile and change the IP.
|
||||
|
||||
|
||||
Coding Standards
|
||||
================
|
||||
|
||||
Style
|
||||
-----
|
||||
|
||||
Please use tabs for now. Keep to 80 columns, at least for readable text.
|
||||
|
||||
Tests
|
||||
-----
|
||||
|
||||
Add tests. They should test all the code you add in a meaning way.
|
||||
|
||||
Coverage
|
||||
--------
|
||||
|
||||
Test coverage should always increase as you add code.
|
||||
|
||||
You may use "# pragma: no cover" in the code for branches of code that support
|
||||
older versions on python. For all other uses of "pragma: no cover" or
|
||||
"pragma: no branch" document the reason why its not covered. "I haven't written
|
||||
a test case" isn't a sufficient reason.
|
||||
|
||||
pyflakes
|
||||
--------
|
||||
|
||||
pyflakes can be used to find unused imports, and unused, undefined and
|
||||
redefined variables. pyflakes should be run in any python code, including
|
||||
python based actions::
|
||||
|
||||
pyflakes bin/ config/ fail2ban/
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
Ensure this documentation is up to date after changes. Also ensure that the man
|
||||
pages still are accurate. Ensure that there is sufficient documentation for
|
||||
your new features to be used.
|
||||
|
||||
Bugs
|
||||
----
|
||||
|
||||
Remove them and don't add any more.
|
||||
|
||||
Git
|
||||
---
|
||||
|
||||
Use the following tags in your commit messages:
|
||||
|
||||
* 'BF:' for bug fixes
|
||||
* 'DOC:' for documentation fixes
|
||||
* 'ENH:' for enhancements
|
||||
* 'TST:' for commits concerning tests only (thus not touching the main code-base)
|
||||
|
||||
Multiple tags could be joined with +, e.g. "BF+TST:".
|
||||
|
||||
Use the text "closes #333"/"resolves #333 "/"fixes #333" where 333 represents
|
||||
an issue that is closed. Other text and details in link below.
|
||||
See: https://help.github.com/articles/closing-issues-via-commit-messages
|
||||
|
||||
If merge resulted in conflicts, clarify what changes were done to
|
||||
corresponding files in the 'Conflicts:' section of the merge commit
|
||||
message. See e.g. https://github.com/fail2ban/fail2ban/commit/f5a8a8ac
|
||||
|
||||
Adding Actions
|
||||
--------------
|
||||
|
||||
If you add an action.d/*.conf file also add a example in config/jail.conf
|
||||
with enabled=false and maxretry=5 for ssh.
|
||||
|
||||
|
||||
Design
|
||||
======
|
||||
|
||||
Fail2Ban was initially developed with Python 2.3 (IIRC). It should
|
||||
still be compatible with Python 2.4 and such compatibility assurance
|
||||
makes code ... old-fashioned in many places (RF-Note). In 0.7 the
|
||||
design went through major re-factoring into client/server,
|
||||
a-thread-per-jail design which made it a bit difficult to follow.
|
||||
Below you can find a sketchy description of the main components of the
|
||||
system to orient yourself better.
|
||||
|
||||
server/
|
||||
------
|
||||
|
||||
Core classes hierarchy (feel welcome to draw a better/more complete
|
||||
one)::
|
||||
|
||||
-> inheritance
|
||||
+ delegation
|
||||
* storage of multiple instances
|
||||
|
||||
RF-Note just a note which might be useful to address while doing RF
|
||||
|
||||
JailThread -> Filter -> FileFilter -> {FilterPoll, FilterPyinotify, ...}
|
||||
| * FileContainer
|
||||
+ FailManager
|
||||
+ DateDetector
|
||||
+ Jail (provided in __init__) which contains this Filter
|
||||
(used for passing tickets from FailManager to Jail's __queue)
|
||||
Server
|
||||
+ Jails
|
||||
* Jail
|
||||
+ Filter (in __filter)
|
||||
* tickets (in __queue)
|
||||
+ Actions (in __action)
|
||||
* Action
|
||||
+ BanManager
|
||||
|
||||
|
||||
failmanager.py
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
FailManager
|
||||
|
||||
Keeps track of failures, recorded as 'tickets'. All operations are
|
||||
done via acquiring a lock
|
||||
|
||||
FailManagerEmpty(Exception)
|
||||
|
||||
raised by FailManager.toBan after reaching the list of tickets
|
||||
(RF-Note: asks to become a generator ;) )
|
||||
|
||||
|
||||
filter.py
|
||||
~~~~~~~~~~
|
||||
|
||||
Filter(JailThread)
|
||||
|
||||
Wraps (non-threaded) FailManager (and proxies to it quite a bit),
|
||||
and provides all primary logic for processing new lines, what IPs to
|
||||
ignore, etc
|
||||
|
||||
.failManager [FailManager]
|
||||
.dateDetector [DateDetector]
|
||||
.__failRegex [list]
|
||||
.__ignoreRegex [list]
|
||||
Contains regular expressions for failures and ignores
|
||||
.__findTime [numeric]
|
||||
Used in `processLineAndAdd` to skip old lines
|
||||
|
||||
FileFilter(Filter):
|
||||
|
||||
Files-aware Filter
|
||||
|
||||
.__logPath [list]
|
||||
keeps the tracked files (added 1-by-1 using addLogPath)
|
||||
stored as FileContainer's
|
||||
.getFailures
|
||||
actually just returns
|
||||
True
|
||||
if managed to open and get lines (until empty)
|
||||
False
|
||||
if failed to open or absent container matching the filename
|
||||
|
||||
FileContainer
|
||||
|
||||
Adapter for a file to deal with log rotation.
|
||||
|
||||
.open,.close,.readline
|
||||
RF-Note: readline returns "" with handler absent... shouldn't it be None?
|
||||
.__pos
|
||||
Keeps the position pointer
|
||||
|
||||
|
||||
ipdns.py
|
||||
~~~~~~~~
|
||||
|
||||
DNSUtils
|
||||
|
||||
Utility class for DNS handling
|
||||
|
||||
IPAddr
|
||||
|
||||
Object-class for IP address handling
|
||||
|
||||
|
||||
filter*.py
|
||||
~~~~~~~~~~
|
||||
|
||||
Implementations of FileFilter's for specific backends. Derived
|
||||
classes should provide an implementation of `run` and usually
|
||||
override `addLogPath`, `delLogPath` methods. In run() method they all
|
||||
one way or another provide
|
||||
|
||||
try:
|
||||
while True:
|
||||
ticket = self.failManager.toBan()
|
||||
self.jail.putFailTicket(ticket)
|
||||
except FailManagerEmpty:
|
||||
self.failManager.cleanup(MyTime.time())
|
||||
|
||||
thus channelling "ban tickets" from their failManager to the
|
||||
corresponding jail.
|
||||
|
||||
action.py
|
||||
~~~~~~~~~
|
||||
|
||||
Takes care about executing start/check/ban/unban/stop commands
|
||||
|
||||
Reference in New Issue
Block a user