One of the difficulties, that people new to auditing face, is getting the right balance between events that are interesting and the volume of events logged. A name for this problem is “the size of the haystack problem”. It comes from the expression “finding a needle in a haystack”. The bigger the haystack, the more time consuming it is to find the needle. The larger the volume of audited activities, the more challenging it is to find those that are interesting. When tuned properly, an auditing solution will be the first to tell you, when something untoward has happened. It’s better to find out that way, than it is to learn of the activity through other means, and only then to go back to the audited data to determine precisely what happened.
Making the process even more challenging is that if you are new to auditing, then it is likely, that you will not actually know which events indicate, that there is some activity occurring, which is worth investigating. Many IT professionals configure their auditing solutions to record everything. Soon after they do this, they are overwhelmed with data. By configuring the solution to record everything, they have made their haystack too big!
A reason this problem arises is that very few vendors provide detailed explanations, as to how they should interpret auditing events. For example, if you are learning to use the built-in auditing capabilities of Windows Server 2008, Windows Server 2008 R2, Windows Server 2012, or Windows Server 2012 R2, you’ll need to spend a substantial amount of time becoming familiar with specific event IDs and the messages associated with those events. This is because it isn’t necessarily clear from the outset, when configuring auditing on Windows Server operating systems, how specific activities correlate to specific recorded events.
The key to tending the size of the haystack when configuring auditing is to have an understanding of the outset of what type of activity you are interested in recording. For example, if you are looking for instances, where specific users have tried to access specific files, don’t start by turning on every auditing option and then searching through the log to find events related to file access. Instead, start by determining what you need to do to configure your auditing solution to record the activity, in which you are interested. Once you’ve determined, how you need to configure your auditing software, configure it appropriately and then test the solution by performing the activity you want recorded.
Testing is important, because it allows you to recognize the activity, that you are interested in recording in the event logs. Some auditing solutions help you out with this by having pre-configured filters, that will identify specific categories of activity. Even with solutions, that are helpful, when it comes to recognizing activity, you should test the solution in different ways. It might be that your auditing solution is great at detecting one variant of the activity, but doesn’t directly detect another variant of the same activity.
The trick to ensuring, that you can find the needle in the haystack, is to minimize the size of the haystack. The key to successful event log auditing is focusing on the activities, that you are interested in recording, rather than recording everything and hoping to put the pieces together after the activity has occurred.
