Following up the recent post about removing confidential data from the log files,
that can be found over here:
that can be found over here:
I decided to post a new version of the script with several improvements comparing to the first version. In the initial release of the script was able to remove the confidential data from all the files in the same folder. Recently I was put against a bigger challenge though - I had to provide and sanitize the logs contained in multiple sub-folders in a complex folder structure. It would be a daunting challenge to copy and run the sanitizer script to every single location. Therefore I created a second version of the script with the following improvements:
1. The script digs now recursively through the whole sub-folders in a folder structure and removes confidential information from every single file
2. Previous version was creating a copy of the file where the confidential information has been found with a ".parsed" suffix next to the original file. In a multiple folder tree scenario fishing for those parsed files and manually removing the non-parsed files would be another time consuming task which I wanted to avoid. The new version of the script is creating a "_Senstive" folder at the top of the tree where it moves all the files with the confidential information. The sanitized version are replacing the original files in their respective locations
3. Minor bug fix - tests proved, that the previous version was not dealing well with the files containing other dots in the name, that the ones separating the file name and the file extension. In an extreme situation, when the files had similar names it would lead to overwriting ".parsed" versions of the files. Current version deals in a different way with renaming the files and this problem is resolved
Usage:
Save the code of the log
parser as a .sp1 file in the root of a directory containing all the files, that you want to sanitize. Run the log parser from
Administrator PowerShell and it will create new files for any log files,
that had the IP addresses detected
Log Sanitizer output
Additional Notes:
Be
careful with *.evtx files, as they store IP addresses in a way, that
there are spaces stored between each character (i.e. 1 0 . 1 . 1 2 2 . 1
3). This would not be detected by the log parser, so if you are
exporting Windows Event Viewer logs for parsing ensure they are exported
in the .csv format
Code:
New-Item
.\_Sensitive -Type Directory
$logfiles =
Get-ChildItem .\. -Recurse | ?{!$_.PSIsContainer}
Write-Host
forEach
($log in $logfiles){
Write-Host -f green "Parsing $log"
$IPsMatched = 0
Get-Content $log.VersionInfo.FileName | ?{$_ -match '(?<IP>(10|172|255|192|45|48|49)\.\d{1,3}\.\d{1,3}\.\d{1,3})'}
| ForEach-Object {$IPsMatched++}
if ($IPsMatched -gt 0){
Write-Host "Found $IPsMatched
IP addresses"
$parsedLogName =
$log.Name.SubString(0, $log.Name.lastIndexOf('.')) + "_parsed." +
$log.Name.Split('.')[-1]
$parsedLogFullName = $log.FullName | Split-Path
$parsedLogFullName += "\"
$parsedLogFullName += $parsedLogName
(Get-Content $log.VersionInfo.FileName) -replace
'(10|172|255|192|45|48|49)\.\d{1,3}\.\d{1,3}\.\d{1,3}','X.X.X.X' | Set-Content
$parsedLogFullName
Write-Host "Sanitized log file written into " -nonewline; Write-Host
-f yellow $parsedLogFullName
Write-Host $log.VersionInfo.FileName -nonewline; Write-Host -f yellow "
moved to .\_Sensitive folder"
Move-Item $log.VersionInfo.FileName .\_Sensitive
}
else {
Write-Host "No IP addresses
found"
}
Write-Host
}
