Dynamically Updating Log Levels in JBoss
One of the nice things in JBoss that I find useful is that logging is all based around Log4J. One of the nice things about Log4J is that logging levels can be set dynamically without requiring a restart of the JBoss container. If you are having troubles with your app server and need to get some more information out of your logs while a problem is going on, you have a couple of options available to update your logging levels.
The "edit in vim" option
"Out of the box", the Log4J Service built into the JBoss app server periodically checks the log4j.xml file for changes. If you have access to this file, one way to update the logging level is to directly edit the file. By default, the jboss-log4j.xml is checked every 60 seconds so if you change the logging level for a particular logger to "DEBUG", you only have to wait, at most, 60 seconds for your logs to start dumping a ton of info. (Just don't forget to change it back when you're done or you risk running out of disk space). Just for laughs, tail your server.log and then open up jboss-log4j.xml on a test box from within your jboss server instances's conf directory. Move down to the org.apache logger, which is set to "INFO" by default. Change that to "DEBUG", save (but don't quit) jboss-log4j.xml, and watch what happens in your server.log. After some period of time, the following messages will show up in your server.log:
2009-03-22 10:13:32,193 DEBUG [org.jboss.logging.Log4jService$URLWatchTimerTask] (Timer-Log4jService) Configuring from URL: resource:jboss-log4j.xml
2009-03-22 10:13:32,193 DEBUG [org.jboss.logging.Log4jService] (Timer-Log4jService) Removed System.out adapter
2009-03-22 10:13:32,193 DEBUG [org.jboss.logging.Log4jService] (Timer-Log4jService) Removed System.err adapter
2009-03-22 10:13:32,202 DEBUG [org.jboss.logging.Log4jService] (Timer-Log4jService) Installed System.out adapter
2009-03-22 10:13:32,202 DEBUG [org.jboss.logging.Log4jService] (Timer-Log4jService) Installed System.err adapter
Send a request off to the root of your jboss app server and you'll see information related to set up and tearing down of sessions. Set your logging level for org.apache back to "INFO", send another request off to the jboss app server root, and nothing is logged.
Using the "edit in VIM" approach to setting log levels has one advantage over the jmx-console approach for new or intermediate level jboss administrators: You can see what loggers are already configured and you can create new loggers very easily—just add them. The disadvantage here is that you need to wait from 1 to 60 seconds for JBoss to reconfigure. If you're using a test or a development box, waiting isn't all that big a deal but if you are trying to troubleshoot a production problem during conference calls with your peers and management, 60 seconds can be an eternity.
This approach also has the disadvantage that you are actually modifying one of your production server's configuration files, which you then need to remember to back out unless you want all your changes to be permanent.
The "change it in the jmx-console" option
If you use the JMX-Console, which is also built into JBoss, you can also change the logging level at runtime by invoking the setLoggerLevel() operation, plug in the logger and level you want to modify, and you're off and running. You can reach it by accessing your jmx-console, then clicking jboss.system. The first entry in a default jboss installation is: service=Logging,type=Log4jService Click that link and you will be given a page that lists current settings for your log4j installation and methods you can invoke to get or set logger levels. getLoggerLevel will return the current level for the logger you are interested in. Enter 'org.apache' in the text box for the getLoggerLevel operation, click the Invoke button and you should see "INFO" returned in the browser. Click the "Back to MBean" button. In the text box for setLoggerLevel, enter 'org.apache' for the logger and enter DEBUG for the level and click Invoke. You are now in Debug logging mode for all org.apache.* components. Return to the MBean if you want to change the logging level back to INFO. If you do not want to wait up to 60 seconds for jboss to re-read the configuration, simply invoke the reconfigure operation and you will commit those changes right away.
The jmx-console approach's main disadvantage, in my opinion, is that you can't list the current loggers so if you don't know the name of the logger you want to modify, you have to open jboss-log4j.xml. Another disadvantage to the jmx-console approach is that if your app server is out of threads, memory, etc., then you aren't going to be able to invoke any jmx operations either. One advantage outside of 'script-ability' is that any changes you make in the jmx-console are retained only for as long as this jboss instance is running. If you stop jboss, then restart, you will begin fresh with your standard settings.
As I am just delving into this, if anyone else out there has additional information or other options that they would like to share, that would be great!
