Convert Imperva Performance Report CSV from Epoch Date to Human Readable
Imperva SecureSphere lets you download Performance Reports, which are dumped into a CSV file. Timestamps are in 13-digit unix epoch format, which aren't much fun to read. Also, every value in the CSV file is quoted, so if you want to do anything interesting with the data in Perl, you first need to get rid of all the quotes.
The script below uses just two CPAN modules, Modern::Perl and Text::CSV, (note that it will not run without some additional re-work using Text::CSV_XS). Modern::Perl probably isn't strictly needed since I am not really using any newer Perl features but I always use it by default now.
It's pretty quick. It converts an 8500+ test CSV of mine in about 1.7 seconds. It takes one argument, which is the name of the CSV file you want to convert. You can then pipe the output to stdout or redirect it to a new file.
#!/usr/bin/env perl
use Modern::Perl;
use Text::CSV;
my $csv = Text::CSV->new({
binary => 1,
auto_diag => 1,
allow_loose_quotes => 1,
sep_char => ',',
});
my $file = $ARGV[0] or die "Need to supply CSV file as argument to $0\n";
my $time;
open (my $data, '<', $file) or die "Could not open file '$file' $!\n";
while (my $line = <$data>) {
chomp $line;
# Needed this next line because the Text::CSV->new options
# above were not removing all the quotes from the CSV
$line =~ s/"//gm;
if ($csv->parse($line)) {
my @fields = $csv->fields();
$time = $fields[0];
my @ts = localtime($time/1000);
$ts[4] += 1;
$ts[5] += 1900;
say sprintf('%d-%02d-%02d %02d:%02d:%02d', @ts[5,4,3,2,1,0]) . "," . $fields[1] . "," . $fields[2] . "," . $fields[3] . "," . $fields[4] . "," . $fields[5] . "," . $fields[6];
} else {
warn "Line could not be parsed: $line\n";
}
}
As an aside, chromatic has updated his Modern Perl
