14 posts categorized "httpbuilder"

12/22/2009

Bling v0.6 Released

A few weeks ago, Feedburner pings began throwing an exception:  "Request Throttled. Try again later" so this new release of Bling removes Feedburner from the list of XMLRPC services.

It also seemed that shortly after introducing BlogBuzzMachine into Bling's list, they decided to throw an exception that they should only be pinged using Feedburner's Pingshot service. So, if you are a user of Feedburner, like I am, and you enable the Pingshot service, you will be sending notifications out to BlogBuzzMachine. Therefore, BlogBuzzMachine has been removed.

So, the number of services currently receiving pings via Bling is at 19 along with the 5 search engines getting sitemap pings.

There is one new feature included in this release. A "debug" command-line option has been added. If you pass a -d or --debug (for POSIX compliance!) argument to bling, failed pings will include the reason for the failure from the xmlrpc service.

The main bling page has been updated and you can grab a copy from the downloads page. Please let me know of any problems in the comments. Thanks for taking a look and using it!

#!/usr/bin/env groovy

import groovy.net.xmlrpc.*
import groovy.util.slurpersupport.GPathResult
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.Method.GET
import static groovyx.net.http.ContentType.URLENC

def cli = new CliBuilder(usage: 'bling [-d,--debug]')
  cli.d(longOpt:'debug', required:false, type: GString, 'Run with verbose error messages')

def opt =  cli.parse(args)
  if (!opt) return

// You'll want to make this next section your own
def blogTitle = "YOUR_BLOG_TITLE_HERE"
def blogURL = "YOUR_BLOG_URL_HERE"
// def feedURL = "YOUR_BLOG_FEED_URL_HERE"
// New in Bling version 0.5:
// URL for Sitemap Pings
def sitemapURL = "YOUR_BLOG_SITEMAP_URL_HERE"

// Set up a map (hash) of popular rpc endpoints
// It is too bad for my syntax highlighter, but some of 
// the blogs containing periods in their names necessitated
// placing them in quotes.  

def trackbacks = [
  Google:'http://blogsearch.google.com/ping/RPC2',
  Weblogs:'http://rpc.weblogs.com/RPC2',
  Moreover:'http://api.moreover.com/RPC2',
  Syndic8:'http://ping.syndic8.com/xmlrpc.php' ,
  BlogRolling:'http://rpc.blogrolling.com/pinger/',
  NewsGator:'http://services.newsgator.com/ngws/xmlrpcping.aspx',
  Bloglines:'http://www.bloglines.com/ping',
  'Blo.gs':'http://ping.blo.gs/',
  BlogCatalog:'http://rpc.blogcatalog.com/',
  PubSub:'http://xping.pubsub.com/ping/',
  'MyBlog.jp':'http://ping.myblog.jp/',
  Goo:'http://blog.goo.ne.jp/XMLRPC',
  BlogPeople:'http://www.blogpeople.net/servlet/weblogUpdates',
  Twingly:'http://rpc.twingly.com/',
  Spinn3r:'http://rpc.spinn3r.com/open/RPC2',
  PostRank:'http://api.postrank.com/v2/ping',
  WasaLive:'http://www.wasalive.com/ping/',
  IceRocket:'http://rpc.icerocket.com:10080/',
  FeedBlitz:'http://www.feedblitz.com/f/f.fbz?XmlPing'
  ]

def searchengines = [
  Ask:'http://submissions.ask.com/ping?sitemap=',
  Bing:'http://www.bing.com/webmaster/ping.aspx?siteMap=',
  Google:'http://www.google.com/webmasters/tools/ping?sitemap=',
  Yahoo:'http://search.yahooapis.com/SiteExplorerService/V1/updateNotification?appid=Bling&url=',
  Moreover:'http://api.moreover.com/ping?u='
  ]

// Set up canned responses to make the outputted responses nicer.
// Previously, the output used the literal response from the endpoint
// which did not look all that nice in a terminal window.

def weal = "Thanks for the ping!"
def woe = "Ping Failed"

// Here is the section responsible for iterating through each ping
// url in the trackbacks map.

println "====XMLRPC PING RESULTS===="

trackbacks.each {
  try{
    def url = it.value
    def proxy = new XMLRPCServerProxy(url)
    response = proxy.weblogUpdates.ping(blogTitle, blogURL)
      response.data instanceof GPathResult

    if (!response.flerror)
        println "  ${it.key}".padRight(20) + "${weal}"
      else if (!opt.d)
        println "  ${it.key}".padRight(20) + "${woe}"
      else 
        println "  ${it.key}".padRight(20) + "${woe} - Reason: ${response.message}"

    }catch(ConnectException ex) {
      println "  ${it.key}".padRight(20) + "${woe}"     
    }catch(IOException ex){
      println "  ${it.key}".padRight(20) + "${woe}"
    }
}
println "==========================="
println "====SITEMAP PING RESULTS==="

searchengines.each {
    def url = it.value + sitemapURL
    def http = new HTTPBuilder( url )
      http.request( GET, URLENC ) { req ->
        headers.'User-Agent' = 'Bling/0.6'
        headers.'Referer' = 'http://blog.techstacks.com/bling.html'

      response.success = { resp, html ->
        println " ${it.key}".padRight(20) + "${weal}"
      }
      response.failure = { resp, html ->
        println " ${it.key}".padRight(20) + "${woe}"
      }
  }
}

println "==========================="

12/08/2009

Adding Timeouts in HTTPBuilder - Use Java

This is just a short post that's intended to let anyone discovering this site via search engine referral that in Groovy-module HTTPBuilder, at present, http timeouts need to be set in Java.  There is no groovy-fied way to set timeouts yet.

The following block illustrates how to set a connection timeout and a socket timeout on a basic httpbuilder GET request:

// create a new builder
def http = new HTTPBuilder( args[0] )
  http.request(GET,HTML) { req ->
    headers.'User-Agent' = 'GroovyHTTPBuilderTest/1.1'
    headers.'Referer' = 'http://blog.techstacks.com/'

    // Begin: java code to set HTTP Parameters/Properties
    // Groovy HTTPBuilder doesn't provide convenience methods
    // for many of these yet.
    req.getParams().setParameter("http.connection.timeout", new Integer(5000));
    req.getParams().setParameter("http.socket.timeout", new Integer(5000));
    // End java code to set HTTP Parameters/Properties

    response.success = { resp, html ->
      println "Server Response: ${resp.statusLine}"
      println "Server Type: ${resp.getFirstHeader('Server')}"
      println "Title: ${html.HEAD.TITLE.text()}"
    }
    response.failure = { resp ->
      println resp.statusLine
    }
  }

In the example above, I'm setting an http connection timeout to 5 seconds and a socket timeout to 5 seconds as well. (Not very practical but this is only for illustrative purposes any way) The script starts out in groovy, then switches over to a java syntax when it comes time to set the timeout parameters, then switches back to a groovy syntax for the response. I hope this helps!

12/03/2009

Configuring "Wire" Logging in Groovy HTTPBuilder

Enabling Debug mode in log4j within HTTPBuilder gives me the ability to, as a novice, figure out what the heck I was thinking when I constructed an http request the way I did.  Take this script for example, which makes use of the FeedBurner Awareness API and its intention is to display the number of subscribers to my blog (thank you all, by the way!) based on a specified date:

!/usr/bin/env groovy

import groovyx.net.http.RESTClient
import groovy.util.slurpersupport.GPathResult
import static groovyx.net.http.ContentType.URLENC

def stats = new RESTClient( 'https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=BloggingTechstacks' )
  
def resp = stats.get( dates: '2009-11-01' )
 resp.data instanceof GPathResult

println "${resp.data.feed.@uri}"
println "${resp.data.feed.entry.@circulation}"

Every time I execute this script and specify a date range of 11/01 through 11/30, instead of getting what I wanted, I get the number of subscribers from yesterday. Clearly, there is something wrong. I can't inspect the request and response with wireshark because it is ssl encrypted and I do not think that Google would provide me a copy of their key so I can decrypt that traffic. Log4J, though, can give me most of what I need to figure out what is wrong.

The documentation link makes it sound so simple yet with every new release of Groovy I install, I always forget to do something and it takes me a while to remember how to get it all working again.  The Groovy module HTTPBuilder supports log4j and even provides a sample configuration file to get started. This file should go to the ${groovy.home}/conf/ directory and this is the part that I always overlook. To enable debug logging, change the values of categories org.apache.http.headers and org.apache.http.wire from INFO to DEBUG. When I re-execute the script, I get the following:

./test.groovy
2009-12-03 09:39:23,142 DEBUG RESTClient - GET https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=BloggingTechstacks
2009-12-03 09:39:25,019 DEBUG wire - >> "GET /api/awareness/1.0/GetFeedData?uri=BloggingTechstacks HTTP/1.1[EOL]"
2009-12-03 09:39:25,021 DEBUG wire - >> "Accept: */*[EOL]"
2009-12-03 09:39:25,022 DEBUG wire - >> "Host: feedburner.google.com[EOL]"
2009-12-03 09:39:25,022 DEBUG wire - >> "Connection: Keep-Alive[EOL]"
2009-12-03 09:39:25,022 DEBUG wire - >> "User-Agent: Apache-HttpClient/4.0-beta1 (java 1.4)[EOL]"
2009-12-03 09:39:25,022 DEBUG wire - >> "Accept-Encoding: gzip,deflate[EOL]"
2009-12-03 09:39:25,023 DEBUG wire - >> "[EOL]"
2009-12-03 09:39:25,023 DEBUG headers - >> GET /api/awareness/1.0/GetFeedData?uri=BloggingTechstacks HTTP/1.1
2009-12-03 09:39:25,023 DEBUG headers - >> Accept: */*
2009-12-03 09:39:25,023 DEBUG headers - >> Host: feedburner.google.com
2009-12-03 09:39:25,024 DEBUG headers - >> Connection: Keep-Alive
2009-12-03 09:39:25,024 DEBUG headers - >> User-Agent: Apache-HttpClient/4.0-beta1 (java 1.4)
2009-12-03 09:39:25,024 DEBUG headers - >> Accept-Encoding: gzip,deflate
2009-12-03 09:39:25,084 DEBUG wire - << "HTTP/1.1 200 OK[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Content-Type: application/xml; charset=utf-8[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Date: Thu, 03 Dec 2009 14:39:24 GMT[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Expires: Thu, 03 Dec 2009 14:39:24 GMT[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Cache-Control: private, max-age=0[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "X-Content-Type-Options: nosniff[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "X-XSS-Protection: 0[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "X-Frame-Options: SAMEORIGIN[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Content-Length: 366[EOL]"
2009-12-03 09:39:25,087 DEBUG wire - << "Server: GFE/2.0[EOL]"
2009-12-03 09:39:25,088 DEBUG headers - << HTTP/1.1 200 OK
2009-12-03 09:39:25,088 DEBUG headers - << Content-Type: application/xml; charset=utf-8
2009-12-03 09:39:25,088 DEBUG headers - << Date: Thu, 03 Dec 2009 14:39:24 GMT
2009-12-03 09:39:25,088 DEBUG headers - << Expires: Thu, 03 Dec 2009 14:39:24 GMT
2009-12-03 09:39:25,088 DEBUG headers - << Cache-Control: private, max-age=0
2009-12-03 09:39:25,088 DEBUG headers - << X-Content-Type-Options: nosniff
2009-12-03 09:39:25,088 DEBUG headers - << X-XSS-Protection: 0
2009-12-03 09:39:25,089 DEBUG headers - << X-Frame-Options: SAMEORIGIN
2009-12-03 09:39:25,089 DEBUG headers - << Content-Length: 366
2009-12-03 09:39:25,089 DEBUG headers - << Server: GFE/2.0
2009-12-03 09:39:25,094 DEBUG RESTClient - Response code: 200; found handler: org.codehaus.groovy.runtime.MethodClosure@24bb6086
2009-12-03 09:39:25,096 DEBUG RESTClient - Parsing response as: application/xml
2009-12-03 09:39:25,104 DEBUG wire - << "[\r][\n]"
2009-12-03 09:39:25,104 DEBUG wire - << "[\r][\n]"
2009-12-03 09:39:25,105 DEBUG wire - << " [\r][\n]"
2009-12-03 09:39:25,105 DEBUG wire - << " [\r][\n]"
2009-12-03 09:39:25,105 DEBUG wire - << "  [\r][\n]"
2009-12-03 09:39:25,106 DEBUG wire - << " [\r][\n]"
2009-12-03 09:39:25,106 DEBUG wire - << "[\r][\n]"
2009-12-03 09:39:25,106 DEBUG wire - << "[\r][\n]"
2009-12-03 09:39:25,129 DEBUG RESTClient - Parsed data to instance of: class groovy.util.slurpersupport.NodeChild
BloggingTechstacks
43

My problem is right there in the first line of the log output. My dates argument is not getting passed in the request URI.

As an aside, to include SSL debug output within the console log, (in order to troubleshoot those "unrecognized certificate" issues, for example), add the -Djava.net.debug=ssl statement to the script's startup properties, right after the shebang.