Blogging Techstacks

BigIP: Integrating with WebSEAL

Any one with a Tivoli Access Manager for eBusiness installation that they've inherited knows two things:  It is a complex product and it can be quite difficult to get things working right.  Once you do get things working right, like most IBM products, it is bulletproof and runs pretty much always.

Integrating the WebSEAL component with an F5 BigIP though can be pretty interesting, to say the least. Recently I had to change the load-balancer sitting in front of WebSEAL to an F5 BigIP LTM based solution. This was primarily driven by me not wanting to manage two different front-end load-balancers. This post is meant to chronicle items that I could no longer do in WebSEAL when front-ending the WebSEAL proxies with an LTM.  Note:  I **could** have used Direct Server Return on the LTM but I didn't want to because of the BigIP functionality that is lost when using Direct Server Return. These have been documented at length in an F5 DevCentral blog post, which you can find at the DevCentral site .   Below are a few issues/items that arose and required some attention because of problems that they produced for the site.

ITEM 1: WebSEAL IP Address Based Authentication No Longer Works

The BigIP LTM, as I have noted quite frequently, is essentially a reverse proxy as far as your upstream load-balanced nodes are concerned. All requests from the LTM appear as if they are coming from your LTM, not from your end-user. Therefore, WebSEAL-based IP Address Authentication will no longer work. Interestingly, even though WebSEAL can pass an X-Forwarded-For header to an app server that it sits in front of, it won't do much with the X-Forwarded-For header that the BigIP LTM sends to it. So, if you utilize IP Address-based authentication in WebSEAL, you can disable it because you will need to move that functionality up into the BigIP. One of two things will happen if you don't do this. If you allow your internal IPs access to otherwise customer-IP-restricted junctions so that monitors or internal support people can get to it, chances are good that the IP address of your BigIP could be part of that list of internal IPs. This results in effectively allowing ALL access to that junction, (because all requests are seen as coming from the BigIP). If you don't allow internal network access to customer-IP-specific junctions, then moving to a BigIP effectively disables ALL access to that junction.

This is actually too bad because even though it takes some time getting the POP and ACL settings defined, no programming or scripting knowledge is necessary to set this kind of thing up. As far as I know, the only way to restrict access to resources based on IP Address on a BigIP is with the use of an iRule. iRules are a pretty flexible way to do all sorts of interesting things to traffic entering and exiting your web sites and the only real downside is that you need to know the Tcl scripting language in order to write them. To keep this article from becoming book-length, I've linked to another post on this site that provides a basic iRule that you can use as a skeleton to create your own IP Address restrictions for WebSEAL Junctions.

ITEM 2: Terminating SSL on the BigIP Requires SSL to WebSEAL Too

Under normal circumstances, when WebSEAL receives a request, it inspects the protocol used to access the site and rewrites the response back from the application servers to the browser using the correct protocol. So, if you terminate SSL at your WebSEAL tier but use HTTP between your webseal servers and your app servers, any URL that gets streamed back to the browser will contain the junction name and the correct protocol.

If you terminate SSL at the BigIP but do not also use SSL between the BigIP and your WebSEAL servers, WebSEAL sees all requests coming in over unencrypted http and will rewrite all responses as http accordingly. Depending upon browser settings and browser version, your users could see a lot of warnings in their browser regarding a mix of secure and insecure elements.

So, if you terminate SSL at the BigIP, you will also need to terminate SSL at WebSEAL. You can still utilize irules at the BigIP level and you don't have to license a CA signed certificate for your BigIP and all your WebSEAL servers. You can use a CA signed cert at the BigIP tier and use self-signed certificates at the webseal tier. You can even use host specific self-signed certificates at the WebSEAL tier if you desire--all we really care about here is that WebSEAL sees the request coming in over SSL so that it can rewrite responses back to the browser using https in the protocol portion of the URI. A future post will deal with what needs to be done to set up SSL between BigIP and your WebSEAL servers.

ITEM 3: WebSEAL Quality of Protection Levels and Protocol Redirects

Related to Item 2 above, if you now terminate SSL at your BigIP, you don't **necessarily** need to worry as much about protocol redirects or SSL Quality of Protection.

In WebSEAL, when requiring SSL on a particular junction, it's not uncommon to modify the default error page to redirect the request over SSL with a little embedded javascript. This is no longer necessary because you can redirect http to https using an irule on your BigIP. In fact, if you are terminating SSL at your BigIP and between the BigIP and WebSEAL, then that custom error will never actually get called and your http request may never actually get fulfilled.

Quality of Protection levels in WebSEAL for purposes such as removing SSL2 support or weak encryption ciphers will need to be duplicated (in another irule or simply by modifying the Client SSL profile for that site.) on your BigIP. You can leave them enabled on your webseal servers but the important thing to remember is that your end-users will no longer be interacting directly with WebSEAL over SSL connections. Those quality of protection levels will only impact ssl connections from your BigIP. Terminating SSL at your BigIP means that SSL connections from browsers will not pass-through to WebSEAL.


Deployment Guide Series: IBM Tivoli Access Manager for E-business 6.0
Deploying OpenLDAP

BigIP: IP Address Restrictions for WebSEAL

Switching your front-end load-balancer to a BigIP LTM to load-balance your WebSEAL servers breaks IP Address Authentication in WebSEAL. This is because the BigIP LTM acts as a reverse proxy, so your WebSEAL servers never see the IP Address of your end-user—only your BigIP's IP Address is seen by WebSEAL. If you are utilizing IP Address Authentication, the following irule could be used as a template to get you started.

Assumptions

• This example uses two WebSEAL junctions named /customerA-jct and /customerB-jct respectively.

• The Datagroup containing the Allowable IP addresses for Customer A is called "customerA_IPs" and the datagroup containing allowable IP addresses for Customer B is called "customerB_IPs".

• In the fine example used by 24, NCIS, MI-5, The Border and similar shows with nerds in the cast who often save the day, the "host" IP Addresses used in the datagroups in this template are completely invalid by-design. If your customer comes from multiple IP addresses, you can add a 'host' entry here for each IP.

• The 'network' addresses in this example represent your company's internal IP addresses, so this could be completely different.

• Even though the classes (datagroups) are being displayed in the iRule, I believe it is a best practice to not include the classes in the actual iRule but through the GUI, the iRule Editor, or using the bigpipe classes command line utility.

• Finally, although the BigIP is now going to be handling your IP Address restrictions, this rule assumes that WebSEAL will still manage the junction destinations. In other words, once the IP has been validated, BigIP will pass the request along to the junction and WebSEAL will take care of rewriting the URL and forwarding the request up to the upstream app servers.

iRule Template

# section 1: datagroups
class customerA_IPs {
 network 10.0.0.0/24
 host 512.434.356.278
}

class customerB_IPs {
 network 10.0.0.0/24
 host 912.834.756.678
}

# section 2: set default load-balanced pool
when CLIENT_ACCEPTED {
 set myPool [LB::server pool]
}

# section 3: http request handling 
when HTTP_REQUEST {
 switch -glob [HTTP::uri] { 
 "/customerA-jct*" { 
  if { not [matchclass [IP::client_addr] equals $::customerA_IPs] } { 
  HTTP::respond 403 content "403 - Forbidden"
  }
 }
 "/customerB-jct*" { 
  if { not [matchclass [IP::client_addr] equals $::customerB_IPs] } { 
  HTTP::respond 403 content "403 - Forbidden"
  }
 }
 default { 
  pool $myPool
  
 }
 }
}

Template Explained

The classes expressed in section 1 in the iRule are there for illustrative purposes only. You should create two datagroups, (which is what they are also known as) in the BigIP Admin GUI or using the iRule Editor.

[LB::server pool] represents the pool of servers that are presently assigned to the virtual IP. Section 2 simply sets a name to that pool.

Section 3 performs a regular expression match against the junction names. It should be read as, "when an http request is received let the request through unless the URI contains the name of one of your customer specific junctions and the client IP address is not in the allowable IP list. If a junction name is matched but the IP address is not in the allowable IP list, issue an HTTP 403 status code."

If you add more junctions but these junctions do not contain IP address restrictions, you don't need to do anything to this irule. However, if you add another restricted junction, you must create a new datagroup and then you can copy and paste the 5 lines of irule code starting at the quoted junction name and ending at the second curly brace prior to the default section. Rename the URI and the class name and you're good to go.


Deployment Guide Series: IBM Tivoli Access Manager for E-business 6.0

Apache: Error writing to log file. XXXXXXX messages lost

This particular error showed up recently over the past couple of weeks on one of the sites that I run. We have two apache web servers supporting this site that had been generating about 1.5GB worth of log file data on an average day, slowly growing over time but never appearing to climb anywhere above 1.8GB per day per server. Then one day recently while trying to troubleshoot a problem, doing a long directory listing on the directory showed that the log files were now between 300 and 500 MB for the most recent days. Opening the log, the first time-stamped request on one of these truncated daily logs showed that the first request occurred between 5:00PM and 6:00PM on that day but the first line in the log showed a message similar to the following:

Error writing to log file. XXXXXXX messages lost

Where XXXXXXX in our case was always some number in the 5.2 million to 5.3 million range. Disk space utilization on the volume was fine. In our case, the problem was load. Load increased on the site to the point where our daily logs files are now larger than 2GB per web server and rotatelogs on a Linux system will only write a 2GB log. Once the file hits 2GB, the log file is truncated. This only seems to occur on apache 1.3 and apache 2.0 running on Linux.

There are a couple of workarounds. The first is to rotate the log file more frequently. For example, set rotatelogs to rotate ever 43200 seconds (12 hours) instead of 86400. Alternatively, you can rotate based on filesize instead, like WebSEAL does by default. Another option is to take the rotatelogs executable from an apache 2.2 installation and use that instead. rotatelogs from apache 2.2 doesn't seem to have this problem.

I haven't used cronolog, so I can't say whether it has the same limitation or not.