« March 2014 | Main | May 2014 »
It expires today (April 30, 2014) but if you are looking to pick up some new web development skills, Apress has a 40% off code good at checkout on eBooks in their Web Development category. Use promo code WEBD40. According to the email I received today, all books in their Web Development category qualify.
Disclaimer: I do not have an affiliate relationship with Apress. I will not earn any commissions or referral fees.
Imperva SecureSphere lets you download Performance Reports, which are dumped into a CSV file. Timestamps are in 13-digit unix epoch format, which aren't much fun to read. Also, every value in the CSV file is quoted, so if you want to do anything interesting with the data in Perl, you first need to get rid of all the quotes.
The script below uses just two CPAN modules, Modern::Perl and Text::CSV, (note that it will not run without some additional re-work using Text::CSV_XS). Modern::Perl probably isn't strictly needed since I am not really using any newer Perl features but I always use it by default now.
It's pretty quick. It converts an 8500+ test CSV of mine in about 1.7 seconds. It takes one argument, which is the name of the CSV file you want to convert. You can then pipe the output to stdout or redirect it to a new file.
#!/usr/bin/env perl
use Modern::Perl;
use Text::CSV;
my $csv = Text::CSV->new({
binary => 1,
auto_diag => 1,
allow_loose_quotes => 1,
sep_char => ',',
});
my $file = $ARGV[0] or die "Need to supply CSV file as argument to $0\n";
my $time;
open (my $data, '<', $file) or die "Could not open file '$file' $!\n";
while (my $line = <$data>) {
chomp $line;
# Needed this next line because the Text::CSV->new options
# above were not removing all the quotes from the CSV
$line =~ s/"//gm;
if ($csv->parse($line)) {
my @fields = $csv->fields();
$time = $fields[0];
my @ts = localtime($time/1000);
$ts[4] += 1;
$ts[5] += 1900;
say sprintf('%d-%02d-%02d %02d:%02d:%02d', @ts[5,4,3,2,1,0]) . "," . $fields[1] . "," . $fields[2] . "," . $fields[3] . "," . $fields[4] . "," . $fields[5] . "," . $fields[6];
} else {
warn "Line could not be parsed: $line\n";
}
}
As an aside, chromatic has updated his Modern Perl
Sometimes, it seems like the worst term to come out the DevOps movement was "DevOps". After all these years, it still seems to have different meanings to different people. Case in point, an absolutely wonderful post recently from Jeff Knupp: How DevOps is Killing the Developer. If you haven't already done so, give it a read.
In the article, Mr. Knupp provides what I personally thought was a pretty good definition of what "DevOps" is (or was?):
"DevOps" is meant to denote a close collaboration and cross-pollination between what were previously purely development roles, purely operations roles, and purely QA roles.
One comment in particular that stood out for me was this one:
I think that this misses the point a bit. DevOps is more about automating the release/QA/ops process than it is about getting developers to actually spend most of their time *doing* release/QA/ops things.
Which was followed up with:
That's the "new" meaning. Author is correct in that the original aim was to bring both sides (Ops and Devs) together.
This one is from the Hacker News comments:
It's about more than just "unix duct tape". It's about 'Infrastructure as Code', treating servers like programming objects. It's about using configuration management tools like Chef and Puppet instead of writing bash scripts which only work on one system.
I've been following a lot of the folks initially involved in the DevOps movement and I must say that I missed the shift in focus of DevOps from a philosophy that all too often stressed that it was not so much about automation and tooling to a movement that seems to be more about automation and tooling and less about "dev" and "ops".
An interesting thing happened in the original article: A lot of good points were raised (which might not have done a good job drawing a line between DevOps and the perils of 'Full Stack Development') and then comes "The Totem Pole" section:
...but there is a hierarchy of usefulness of technology roles in an organization. Developer is at the top, followed by sysadmin and DBA. QA teams, "operations" people, release coordinators and the like are at the bottom of the totem pole.
A number of thoughts came to mind after reading this. First and foremost was my perception of the DBA reaction: "Aww...isn't that adorable? 'Developer is at the top...'!" This was then followed by my sympathy for the Security and Network folks at this organization who didn't seem to make it into the "Hierarchy of Usefulness", (unless, of course, they're being lumped into the '"operations" people' bucket). I don't think I'd have used a totem pole analogy because this hierarchy seems more pyramid shaped and, well, 'hierarchical' but I digress.
All in all, a fun post to read. I can't say I agree with everything posted in it but it certainly did a good job in starting a discussion.
