Intrusion Detection and Prevention
Intrusion detection is the process by which an attempt is made to identify malicious behavior towards a computer system or systems attached to a network. Intrusion detection can be a manual process, for instance being done by a person reviewing an application's log files for suspicious behavior. When the process is automated and done by a computer, that computer is called an intrusion detection system. Intrusion detection systems can be network-based, host-based or a combination of the two.
With a network-based intrusion detection system, network traffic flows through a computer which analyzes the packets looking for signatures that match known malicious traffic. Heuristics can be applied to network traffic that indicate known bad behavior, such as one computer probing entire network ranges. If the system blocks this traffic by dropping the packet on its way to its destination, the system is called an intrusion prevention system.
Host-based intrusion detection systems can monitor file system activity looking for changes to files and folders that shouldn't change (such as core operating system components and utilities) and can examine log files looking for attempted break-ins, repeated login attempts and bad requests. They can also monitor portions of computer memory that should not be changing, such as system call tables, and will alert when a change occurs. An anti-virus program is, in a general sense, one form of a specialized host-based intrusion detection system.
If you're running a multi-user system with services running that are not locked down to allow limited host access, a host-based intrusion detection system is recommended. There are a number of open source host-based intrusion detection systems available, including OSSEC.