pytbull is an Intrusion Detection/Prevention System (IDS/IPS) Testing
Framework for Snort, Suricata and any IDS/IPS that generates an alert
file. It can be used to test the detection and blocking capabilities of
an IDS/IPS, to compare IDS/IPS, to compare configuration modifications
and to check/validate configurations.

The framework is shipped with about 300 tests grouped in 11 testing modules:

1. badTraffic: Non RFC compliant packets are sent to the server to test how packets are processed.
2. bruteForce: tests the ability of the server to track brute force attacks (e.g. FTP). Makes use of custom rules on Snort and Suricata.
3. clientSideAttacks: this module uses a
   reverse shell to provide the server with instructions to download remote
   malicious files. This module tests the ability of the IDS/IPS to
   protect against client-side attacks.
4. denialOfService: tests the ability of the IDS/IPS to protect against DoS attempts
5. evasionTechniques: various evasion techniques are used to check if the IDS/IPS can detect them.
6. fragmentedPackets: various fragmented payloads are sent to server to test its ability to recompose them and detect the attacks.
7. ipReputation: tests the ability of the server to detect traffic from/to low reputation servers.
8. normalUsage: Payloads that correspond to a normal usage.
9. pcapReplay: enables to replay pcap files
10. shellCodes: send various shellcodes to the server on port 21/tcp to test the ability of the server to detect/reject shellcodes.
11. testRules: basic rules testing. These attacks are supposed to be detected by the rules sets shipped with the IDS/IPS.

Source and Official document