14 posts categorized "httpbuilder"

10/01/2010

Groovy News: HTTP Builder 0.5.1 Released

Thom Nichols released version 0.5.1 of HTTP Builder yesterday.  This version adds support for Open Authorization (OAuth).  OAuth support works with HttpBuilder, the RESTClient, and the HttpURLClient although the HTTP Builder News page indicates that the implementation is not perfect due to a limitation in the underlying Apache HTTPClient v4.0 libraries.  The HTTP Builder Download page includes links to get the jars as well as instructions for inclusion within Grape or Maven.

09/16/2010

PubSubHubbub Pings with Groovy!

Yes, the title does look a bit nonsensical but it isn't.  PubSubHubbub is an extension of the Atom and RSS standards, providing a server-to-server, publish/subscribe communications model.  The goal of the protocol is to provide near instantaneous notifications of change updates.  All the major blog platforms utilize pubsubhubbub to inform search engines, feed aggregators, etc., of updates made to sites.  

A server pulling your feed from the site can be configured as a hub.  That server can poll hundreds or thousands of different web site feeds in order to provide a centralized location where clients can pull feeds from.  Since the protocol is open, anyone can operate a hub so you have the theoretical benefit of a highly distributed series of hubs that people can point updates to or pull updates from.  There aren't too many open free hubs at this point--there are several private hubs currently in operation like Superfeedr and DotSpots.  The most well-known open free hub is the PubSubHubbub Reference Server.

The protocol provides a mechanism for registering a feed with a hub so that the hub does not need to waste bandwidth continually polling a site looking for updates.  You do this by declaring a hub inside your feed.  Typically, you would add something like the following to your atom template:

<link rel="hub" href="http://pubsubhubbub.appspot.com/" />

This registers the feed with the hub. With TypePad, all I need to do to ping the hub is publish some content like this article. With the ping, the hub will pull updates from my site and multi-cast them out to all its subscribers.

If you're interested more in what the ping looks like, below is a little script written in groovy, using httpbuilder, which issues an HTTP Post to the PubSubHubbub reference server. The post sends two parameters, "hub.mode" set to "publish" and "hub.url", which is the encoded url of your atom or rss feed. A successful ping is responded to by the hub with a 204 - No Content status code, (finally! a use for 204 status codes! :)

#!/usr/bin/env groovy

import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.ContentType.URLENC

def feedUrl = "http://blog.techstacks.com/atom.xml"
def hubUrl = "http://pubsubhubbub.appspot.com"

def  postBody = [ "hub.mode": 'publish', "hub.url": feedUrl ]

def http = new HTTPBuilder( hubUrl )

http.post( path: '/', body: postBody, requestContentType: URLENC ) { resp ->
  println "Server Response: ${resp.statusLine}"
}

07/26/2010

Scanning for Unsafe URLs - Update 2

This post is sort of a big deal for me because it is the 300th post to this blog.  I'm happy to have reached this particular milestone and want to thank everyone taking the time to visit it and especially thank all of you who have subscribed to my RSS feeds and who are following the site updates on twitter. Additional milestones reached over the past week include an all-time high 75 subscribers to the blog's RSS feed.  This week, the Techstacks Howto's site should reach it's 10,000th page view since the site launched in September 2009 and sometime in August, this blog should reach it's 100,000th page view since re-launching under TypePad roughly 13 months ago.

Saturday's post about updates to my Tools site briefly touched upon BadUrlChk, which I will now cover in more detail here.

PCI Scanners are now testing for unsafe URLs.  The original "Scanning for Unsafe URLs" post introduced a Perl script, which is still a work in progress, that tests many well known, unsafe URLs that we web site administrators don't necessarily want exposed to the Internet-at-large.  Although I think Perl is really cool and I want to get really good at it, a lot of my scripts make use of additional CPAN modules that many folks don't want to necessarily install on their own personal workstations.  The scripts that I've written in groovy also make use of third-party modules.  The Techstacks Tools site exists as a hosted site for people who just want to run the test without having to worry about installing perl or groovy or any of the third party modules that I like to use.

BadUrlChk is a port of the Perl script from the original article.  The output is the same right now but it is running on Google AppEngine and makes use of the Gaelyk framework and HTTPBuilder.  All you need to do is plug in a URL and the scanner will output success or failure messages.  If you've got a vulnerability reporting that the ColdFusion Administrator console is open and running the tool confirms this, re-run the tool after remediation of the vulnerability.  It should then report that ColdFusion console access fails.

Presently, the script and the hosted version running on the Tools site are not completely ready and should be considered early beta.  You can get a pretty good idea of whether or not a small sample of well known, unsafe URLs are open to the Internet but the tools do not presently handle redirection well nor do they handle Authentication responses from the web server.  I'm still working on that.  Sites that redirect can result in a lot of false positives so I'm working on some false positive handlers as well.  

The nice thing about command line scripts is output is easy to handle.  The trouble with them is that they don't always translate well on a web site.  I'm working to see if there is a better way to display this output.