When we in security first approached our performance team with the bright idea to monitor all the syscalls, we were met with distinctly different views of the project. Whereas security was imagining hackers weeping outside the gates of our impenetrable systems; performance saw a specific dollar cost for monitoring that had better add value. At Netflix’s scale even a small performance improvement translates directly into large savings.
As we dug into the call flow with Brendan Gregg and his set of really useful performance tools, we found that AppArmor is already inspecting system calls by default in Ubuntu. We already use AppArmor to secure various executables and are expanding its use. Also it turns out to be hard to remove from Ubuntu, so at this point so we had a sunk cost from performance’s perspective working in our favor.
The challenge was could we replicate the functionality we got from the k/auditd framework with AppArmor? There are a few design choices working against us here: 1) AppArmor works on a whitelist principal, only allowing certain actions, 2) AppArmor is focused on enforcement of a policy based on a file path to a specific executable. With audit we want to ‘allow all’ (non-blocking) then audit certain things, and we want to do this globally not per executable file.
It seemed at first that the profile naming scheme would prevent a global profile, but Brendan pointed out the file globbing syntax and a profile for /** does match any and every file on the root file system. There may be path shenanigans that could get around this, put I haven’t thought any up yet.
I found a similar effort where an allow-all profile was suggested. I modified that a bit and then added audit around the areas I care about, here. If you want to test it out on an Ubuntu 14 VM give this a try:
If you logout and log back in the new sshd session will launch within this profile and you will see the problem with ‘audit network.’ The output below shows how each and every send/recv message for each keystroke and echo is logged as a separate entry.
There is a roadmap for finer grained permissions around network in AppArmor 3.0, at which point we can limit auditing to connect/accept/bind, but the AppArmor maintainer confirmedthis is not possible today.
So for process execution and file integrity monitoring this seems viable, but for network monitoring I do not think it will be performant to filter all these messages in user land.