In this blogpost, we will take a close look at file access auditing on an EMC Isilon file system by leveraging native technologies. We will walk through the configuration process and explore the common challenges faced when working with the resultant audit logs.
Overview of File Activity Monitoring on EMC Isilon OneFS
EMC Isilon’s auditing capability was revamped with the release of OneFS 7.1. In particular, it now includes the ability to enable protocol access auditing cluster-wide.
Audited Actions
Once protocol auditing is enabled on an Isilon Access Zone, file access events are logged on the individual cluster nodes where the client initiated the access attempt. The table below displays all audited actions:
| Event Name | Example Activity | Exported via CEE? |
| Create | Create a file or directory | Yes |
| Close | Close a directory | Yes |
| Rename | Rename a file or directory | Yes |
| Delete | Delete a file or directory | Yes |
| Set_Security | Modify file or directory permissions | Yes |
| Read | The first read request on an open file handle | Yes |
| Write | The first write request on an open file handle | Yes |
| Get_security | Read security information for an open file handle | No |
| Logon | SMB session create | No |
| Logoff | SMB session logoff | No |
| Tree_connect | SMB first attempt to access a share | No |
Log File Management
Log files are rolled over to a new file once they reach 1 GB, at which point they are automatically compressed (as of OneFS 7.1.1).
One important thing to note is that due to regulatory requirements such as HIPAA, audit log files are never deleted from the cluster automatically, which can lead to issues with storage consumption and add overheard to the auditing process. But while manual cleanup of these files is recommended, this process can cause interruptions in SMB access, which forces administrators to perform cleanup during scheduled maintenance windows.
Event Forwarding
The OneFS 7.1.1 release also introduced the ability to forward events to either a syslog server or to third-party software via the Common Event Enabler (CEE). It is recommended that this feature is not enabled until the syslog endpoint or third-party software is configured, because if a backlog of events exists, it may take the third-party tool time to process the backlog, leaving the result set stale for a potentially extended period of time.
Configuring Protocol Auditing on EMC Isilon
Most audit configuration can be achieved through the OneFS console, but the CLI will need to be used when specific audit settings are required. The recommendation is to leverage a more scoped audit configuration to minimize the system impact.
To configure protocol auditing, take the following steps:
1. In the OneFS Web UI, select Cluster Management > Auditing. Select Enable Protocol Access Auditing and add the Zones to be audited.
2. For the purpose of this blog post, we will want to add the ability to audit successful and failed read and write attempts. Use the isi audit settings modify command to add these events:
Viewing the Protocol Access Audit Logs
The configurations applied thus far will store events on each of the individual cluster nodes, which can be viewed only by using the isi_audit_viewercommand unless you are leveraging syslog or CEE to send the events to third-party servers.
Available search parameters are displayed in the screenshot below:
A sample audit entry is displayed below:
Challenges with Interpreting Critical Access Events
Audit Entry Format
Unless the audit events are forwarded to a third-party server for analysis and consolidation, it is nearly impossible to parse them for anything meaningful. Unlike NetApp, these events cannot be viewed through a user-friendly interface such as Event Viewer, forcing users to know exactly what they are looking for before viewing the logs. If there is a specific investigation where things like date/time frame, target share or UserSID are known, admins can use tools such as grep to refine the scope of their query against the audit logs.
Let’s explore the scenario of a missing file. Let’s say an associate from accounting can’t find the text file that lists the billing codes required to process expense reports. They can tell you where the file used be (the Accounting shared folder), and maybe even the name of the file (Billing Codes), but not much more than that. With this information, the admin can run isi_audit_viewer –t protocol –v | grep delete | grep Billingto search the audit logs for a file deletion event.
Here you can see the UserSID responsible for deleting the file and the timestamp for the event. Depending on the amount of audit data stored, you may benefit from adding filters to limit the search to a likely timeframe.
UserSIDs
One huge shortcoming of the audit data provided by OneFS is in the way users are logged. For each event, a UserSID will be logged, along with an internal UserID. In order to determine who the actual user is, you can either run a reverse lookup against the UserSID returned, or you can run the isi auth usersview command. In the example above, the event returns UserID=1000038, so the command would be isi auth users view –uid 1000038.
Permission Changes
As with NetApp, the permission change data provided by OneFS leaves a bit to be desired. Let’s explore a scenario in which a user adds an ACE to the Accounting folder. We can filter our search of the audit log to the set-security event type by running the command isi_audit_viewer –t protocol –v | grep set-security.
As you can see, we can tell which resource was touched and who did it, but have no visibility into the permission that was added, removed or changed.
Denied Access Attempts
Monitoring failed access attempts is crucial to detecting early signs of compromise. Both Windows and NetApp events have an Audit Success or Audit Failure flag. However, failed access attempts are not as straight forward to interpret on the OneFS side.
Suppose a user attempts to delete the file Billing Codes.txt file but does not have that right. A series of events with eventtype=Create will be logged. In order to determine whether there were any failed access attempts, you would need to filter based on ntStatus value. The list of ntStatus values is shown in this Microsoft article. An ntStatus of 3221225506 indicates that access was denied.
There are several pitfalls to the way this data is represented:
- There is no way of understanding which operation was attempted. In the example above, the user attempted to delete a file, yet the event type that is returned is “create”.
- It is not clear whether access attempts have been successful or not without having intimate familiarity with ntStatus values.
How Netwrix Can Help
Data access governance software from Netwrix provides an effective and scalable approach to file activity monitoring that overcomes the challenges detailed above. Moreover, it can help you reduce the risk of cybersecurity incidents by enabling you to understand who has access to what and strictly limit access to sensitive data. You can:
- Audit activity across your IT ecosystem.
- Reduce access to sensitive data to the required minimum to reduce the risk of insider threats and minimize the damage from ransomware and other attacks.
- Streamline regular privilege attestations by data owners.
- Protect sensitive data whenever it goes with accurate and consistent tagging of content.
