Plugin Smithy Software Blog

During the upgrade of a machine from Debian 3.1 to Ubuntu 8.10 I did run into a pitfall that was quite surprising to me. One aspect of this was the upgrade from PostgreSQL 7.4 to PostgreSQL 8.3. The problem is that PostgreSQL backups have a dependency on the machine where the backup was created.

Because I wanted to re-partition the machine, I created a full backup of the machine and wiped out all the existing partitions. For two existing PostgreSQL databases I created backups with pg_dump. In the online PostgreSQL documentation I read the section 24.5. Migration Between Releases, and there it says: "It is recommended that you use the pg_dump and pg_dumpall programs from the newer version of PostgreSQL, to take advantage of any enhancements that might have been made in these programs." In order to comply with that recommendation, I downloaded a current 8.3 binary package from the EnterpriseDB download site, installed it on the machine and used the 8.3 pg_dump program to create backups of the two databases from the 7.4 database server.

Then the machine was installed from scratch with Ubuntu 8.10, and with that came the Ubuntu version of PostgreSQL 8.3. This is what happened when I tried to restore one of the databases with pg_restore:

pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 9; 1255 18035 FUNCTION
plpgsql_call_handler() postgres
pg_restore: [archiver (db)] could not execute query: ERROR: stat failed on 
file »/usr/lib/postgresql/lib/plpgsql.so«: No such file or directory

As it turns out, in the Ubuntu version of PostgreSQL 8.3 the file "plpgsql.so" is installed under the pathname "/usr/lib/postgresql/8.3/lib/plpgsql.so". This is the relevant section from the backup file created by pg_dump:

CREATE FUNCTION plpgsql_call_handler() RETURNS language_handler
    AS '/usr/lib/postgresql/lib/plpgsql.so', 'plpgsql_call_handler'
    LANGUAGE c;

The workaround was to replace '/usr/lib/postgresql/lib/plpgsql.so' with '/usr/lib/postgresql/8.3/lib/plpgsql.so' with vi, while preserving the offsets in the file because these obviously matter to pg_restore. This is the corrected section (note the four missing blanks in the line starting with "AS"):

CREATE FUNCTION plpgsql_call_handler() RETURNS language_handler
AS '/usr/lib/postgresql/8.3/lib/plpgsql.so', 'plpgsql_call_handler'
    LANGUAGE c;

When on my first attempt I just replaced one string with the other (which made the file four bytes longer), pg_restore crashed with a segmentation fault.

The lesson learned is that one should create the backups with pg_dump on the machine where the backups will potentially be restored, not necessarily on the machine where the PostgreSQL database server is running. As it is not always clear upfront on what machine with what PostgreSQL installation a backup will be restored, this problem can come as a surprise in the worst moment...

Disclaimer: I'm not a PostgreSQL expert. There might be ways I'm unaware of to prevent the creation of the machine dependency when creating the backup with pg_dump or to override the pathname of the library when doing the restore with pg_restore. At least there is no hint about the potential problem in the PostgreSQL  in the section about migrating between releases.

First the warning: The setup of IBM Rational Team Concert 1.0 Express-C to run under GlassFish with a SUN JDK is unsupported by IBM, so you will do this at your own risk. Having said that, up to now this works for me without problems.

I performed the installation with the following software:

• Ubuntu Linux 8.04
• GlassFish V2 UR2 (download page with installation information)
• IBM Rational Team Concert 1.0 Express-C (download page, documentation)
• Sun JDK 1.6.0_03-b05

How to install Team Concert under GlassFish can be derived to a great extent from the instructions how to install Team Concert under IBM WebSphere. The following is mostly an adoption of the Tomcat and WebSphere installation instructions to GlassFish. A few things are GlassFish-specific because of the directory layout I did choose, and because the authentication must be configured in a GlassFish-specific way.

Let me first talk about the directory layout that I did choose. We will install the Team Concert files and GlassFish under a common parent directory. I will call this "<root dir>" and use the concrete directory "/home/stm/tc" in the protocol. Under this directory we will have the "jazz" directory that is the unpacked Team Concert ZIP file, and "glassfish" with the GlassFish installation:

stm@holmium:~/tc$ cd /home/stm/tc
stm@holmium:~/tc$ ls -l
total 8
drwxr-xr-x 12 stm stmgroup 4096 Oct  3 14:47 glassfish
drwxr-xr-x  4 stm stmgroup 4096 Jun 18 23:51 jazz

Regarding GlassFish, I assume that you have set up GlassFish already, while leaving the GlassFish default ports 8080 and 8181 untouched. This will be important later because the pre-configured Team Concert Express-C repository contains different default ports for Tomcat.

First we will set up the users and groups. The bundled Tomcat installation already has an user "ADMIN" and several groups configured:

stm@holmium:~/tc$ cat jazz/server/tomcat/conf/tomcat-users.xml 
<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="JazzUsers"/>
  <role rolename="JazzAdmins"/>
  <role rolename="JazzDWAdmins"/>
  <role rolename="JazzGuests"/>
  <role rolename="manager"/>
  <user username="ADMIN" password="b521caa6e1db82e5a01c924a419870cb72b81635" 
                    roles="JazzUsers,JazzAdmins,manager"/>
</tomcat-users>

We will create this user and the groups in the GlassFish "file" realm. Log in to the GlassFish Admin Console and go to "Configuration" -> "Security" -> "Realms" -> "file":

Click on "Manage Users", and then "New...", and create the "ADMIN" user with the same group list as in the "tomcat-users.xml" file plus "admin", and enter a password of your choice:

The ADMIN user is supposed to be disabled after the Team Concert server has been set up, so create at least one additional user with the same groups. I did create an additional user "stm" like this. The result looks like this:

I don't want to have to setup a group-to-role mapping in the deployment descriptor for the Jazz web application, so I'm switching on the "Default Principal To Role Mapping" setting that maps the groups to J2EE roles. Click on "Security" and mark the corresponding check-box:

The Team Concert documentation section "Setting up a WebSphere Application Server" describes the system properties that must be set for the JVM. You can also get them from the "<root dir>/jazz/server/server.startup" file by looking at the "JAVA_OPTS" variable there. Three of the five system properties described in the documentation have to be adapted to the "<root dir>/jazz/server" directory:

• -Dcom.ibm.team.server.configURL=\
  file:/home/stm/tc/jazz/server/teamserver.properties
• -Dlog4j.configuration=file:/home/stm/tc/jazz/server/log4j.properties
• -Dcom.ibm.team.repository.provision.profile=\
  file:/home/stm/tc/jazz/server/provision_profiles
• -Dorg.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoadResourcesPlugin=true
• -Djava.awt.headless=true

In the GlassFish Admin Console, go to "Application Server", "JVM Settings", "JVM Options", enter the options and click "Save":

Do not forget to restart GlassFish to make the options effective.

Team Concert Express-C comes with a pre-configured Derby database. If you run the start-/stop-scripts from the directory "<root dir>/jazz/server", everything works out-of-the-box because all the configuration files use path names relative to the directory "<root dir>/jazz/server" in which the start-/stop-scripts are stored. As GlassFish will run with a different working directory and as I wanted to keep the Team Concert files outside of the GlassFish directory tree, we have to follow the instructions under Modifying the profile.ini files for using absolute pathnames.

Edit the files "<root dir>/jazz/server/provision_profiles/license-profile.ini" and "<root dir>/jazz/server/provision_profiles/profile.ini" and replace the relative pathnames with absolute ones. These are the files after modification:

stm@holmium:~/tc/jazz/server$ cat provision_profiles/license-profile.ini 
url=file:/home/stm/tc/jazz/server/license-update-site
featureid=com.ibm.team.rtc.server.license.express-c-release.feature
stm@holmium:~/tc/jazz/server$ cat provision_profiles/profile.ini 
url=file:/home/stm/tc/jazz/server/update-site
featureid=com.ibm.team.rtc.server.feature

There are also relative pathnames in the file "<root dir>/jazz/server/teamserver.properties" file. One is the pathname to the Derby repository database, and the other one is the pathname to the Lucene fulltext index. Change those entries from relative pathnames to full pathnames. These are the relevant parts of the file:

...
# JDBC DB location, specifying this property disables system-based
# selection of default location
com.ibm.team.repository.db.vendor=DERBY
com.ibm.team.repository.db.jdbc.location=/home/stm/tc/jazz/server/repositoryDB
...
# Fulltest index location
com.ibm.team.fulltext.indexLocation=/home/stm/tc/jazz/server/workitemindex
...

The Team Concert server also brings his own log4j configuration file "<root dir>/jazz/server/log4j.properties". I want to have the "jazz.log" file together with the normal GlassFish "server.log" file in the same directory, I change the corresponding entry in the "log4j.properties" file:

stm@holmium:~/tc/jazz/server$ grep jazz.log log4j.properties 
log4j.appender.file.File=/home/stm/tc/glassfish/domains/domain1/logs/jazz.log

Now we are ready to deploy the "jazz.war" file from "<root dir>/jazz/server/tomcat/webapps/jazz.war" through the GlassFish Admin Console. After that has happened, we can follow the steps "Running the setup wizard" to complete the Team Concert server installation. Because we left the GlassFish default ports untouched, the URL is https://localhost:8181/jazz/setup (not https://localhost:9443/jazz/setup as it would be the case for Tomcat). As GlassFish out-of-the-box uses a self-signed certificate, you will have to ignore the corresponding security warning. Then the Team Server login page appears:

As we will use the bundled pre-configured Derby repository database, we can choose the Fast Path Setup:

This takes us directly to step 4 of the setup wizard (two screenshots because of the long web page):

Team Concert has recognized automatically that this is not a Tomcat User Registry (step 1). In step 2 we create a user in the Team Concert server that corresponds to the user id that we have previously created in the GlassFish file realm ("stm" in our case).

In step 3 we cannot yet disable the default ADMIN access because we are logged in as ADMIN, so that has to happen later. In step 4 we assign one of the three free Team Concert developer licenses to the new user id.

Pressing "Next" takes us to the "Summary" page:

Clicking "Finish" takes us to the Jazz Team Server Status Summary page:

What is left is to adapt the configuration to the different HTTP ports. Click on "Advanced Properties" and scroll down to the section titled "com.ibm.team.repository.service.internal.webuiInitializer.ConfigPropertyInitializer". Then change the ports to the default values for GlassFish:

Then scroll down to the next section called "com.ibm.team.repository.servlet.internal.ServletConfigurationService", and change the ports in the same manner:

Finally, scroll back all the way to the top of the page and click "Save". You will get a message that this requires a server restart. Restart GlassFish, and then you are ready to start the Team Concert client to connect to the Team Concert server. The URL for connecting to the Team Concert server will be https://localhost:8181/jazz/.

It's been a week now that version 2 of the  Annotate Add-On for the IBM Rational ClearCase SCM Adapter has been released, and the first feedback has been positive. The major new feature of version 2 is the capability to display the ClearCase history for the annotated version, inside the Eclipse shell. This is a significant advantage over the the ClearCase history that is built into the IBM Rational ClearCase SCM Adapter, because the latter will open a separate window. It allows to view the event records for a ClearCase element, without having to leave the Eclipse shell.

Check out the online help for the version 2 of the Annotate Add-On for the IBM Rational ClearCase SCM Adapter. If this makes you curious,  you can get a fully functional time-limited evaluation version after registration, or you can get a functionally limited version anonymously here.

 

Up to now it was only possible to pay in Euro in the Plugin Smithy Software online shop. While it was possible from the beginning to pay online in Euro via PayPal from any region in the world,  I was told that this makes ordering more complicated for people in some countries, specifically in the Americas. For that reason the Plugin Smithy Software online shop now accepts US Dollars and Euros, depending on the home country of the purchaser. Customers from the Americas, Australia, New Zealand and Japan will be invoiced in US Dollars. Customers from all other regions will be invoiced in Euro as hitherto.

When I recently tried IBM Rational PurifyPlus 7.0 for Windows together with Microsoft Visual Studio 2005, to my surprise there was still no integration of the PurifyPlus tools into Visual Studio 2005, like it is available for Microsoft Visual Studio .NET 2003. This means that you cannot launch any of the PurifyPlus tools directly from within Visual Studio 2005. This is especially cumbersome with Purify. The most powerful combination is to use Purify directly together with the Visual Studio debugger, in order to be able to inspect the program state that triggered an error message in Purify.

Note that the PurifyPlus integration with Visual Studio .NET 2003 was not really exciting. Also with this integration it was not possible to debug programs that were launched through the integration (see here for an explanation and a workaround).

In order to be able to launch and debug a purified program directly from Visual Studio 2005, I integrated  Purify through a post-build event. The trick is to instrument the executable in-place through the post-build event. To prevent interference with normal Debug and Release builds this should be done in a separate build configuration.

The basic steps to set this up in Visual Studio 2005 are:

1. Create a new "Purify" build configuration as a copy of the "Debug" configuration in the Visual Studio "Configuration Manager".
2. Create a "Post-Build Event" with the following command:
   purify /Run=no /Replace "$(TargetPath)"
3. In the project settings under "Configuration Properties", "Debugging" extend the PATH environment variable to include the Purify product directory and the Purify cache directory (e.g. "C:\Program Files\Rational\Common" and "C:\Program Files\Rational\PurifyPlus\cache").

This is demonstrated in a Visual Studio 2005 sample project that can be downloaded here. After unzipping the project and opening it in Visual Studio 2005, step 3 must still be performed to reflect the actual installation directory of PurifyPlus on your machine. This screenshot shows how to set the PATH environment variable in the project settings under "Configuration Properties", "Debugging":

The sample project has the redistributable Purify API files integrated, so it is possible to directly call the Purify API functions from Visual Studio watchpoints. The following screenshot shows the result of calling the PurifyIsRunning() and PurifyWhatColors() in the Visual Studio watchpoint window, after having hit a breakpoint in the main program:

 

And this is the corresponding output in the Purify viewer:

With this "poor man's integration" I found the combination of Purify and Visual Studio 2005 much more useful.

This website is running on GlassFish, inside a Virtuozzo virtual server ("vServer"). The vServer is the medium-level vServer offering of the hosting company, with 400 MB of physical RAM guaranteed, and with a maximum RAM allocation of 2000 MB.

When I selected this vServer type, I thought there is no question that a low-volume website can be set up with GlassFish on it. I had selected a Debian image and installed a Sun 1.5 JDK, and then much to my surprise I couldn't even execute the Java virtual machine (JVM):

gtest@v32295:~$ java
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.


Restricting the heap size helped, and "-showversion" showed that the Server VM was selected:

gtest@v32295:~$ java -Xmx64m -showversion
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)
...
It appeared strange to me that the Server VM was selected, but after reading the JDK 6 documentation about the java command which references the description of the Server-Class Machine Detection
I examined the environment under Virtuozzo. And while the specs said that there are only 2000 MB maximum RAM available to the vServer, the operating system reports obviously numbers for the physical machine itself:
gtest@v32295:~$ cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 37
model name : AMD Opteron(tm) Processor 248
stepping : 1
cpu MHz : 548.443
cache size : 1024 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow
pni lahf_lm
bogomips : 4393.82
TLB size : 1088 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor : 1
vendor_id : AuthenticAMD
cpu family : 15
model : 37
model name : AMD Opteron(tm) Processor 248
stepping : 1
cpu MHz : 548.443
cache size : 1024 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow
pni lahf_lm
bogomips : 4386.88
TLB size : 1088 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp
gtest@v32295:~$ cat /proc/meminfo
MemTotal: 49909364 kB
MemFree: 49426572 kB
Buffers: 0 kB
Cached: 0 kB
SwapCached: 0 kB
Active: 0 kB
Inactive: 0 kB
HighTotal: 33551744 kB
HighFree: 33551744 kB
LowTotal: 16357620 kB
LowFree: 15874828 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
Mapped: 482792 kB
Slab: 0 kB
CommitLimit: 0 kB
Committed_AS: 1477616 kB
PageTables: 0 kB
VmallocTotal: 0 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB


As the operating system reports two processors and a lot more RAM than 2 GB, the JVM thinks that this is a "server-class" machine. What does that mean for the Java heap allocation? According to the Garbage Collector Ergonomics, without explicit specification the initial heap size will be "Larger of 1/64th of the machine's physical memory on the machine or some reasonable minimum", so in our case this would be  49909364/64 kB = approx. 779833 kB . The maximum heap size would be set to "Smaller of 1/4th of the physical memory or 1GB", so this should be 1 GB in our case.

These values are obviously too high for the given Virtuozzo environment. For the GlassFish installation this means that the maximum heap size must be specified explicitly in all places where the "java" command is executed.

Let's walk through the GlassFish installation process, along the guidelines in the GlassFish Project - V2 UR1 Final Build download web page:

1. Extracting the GlassFish download file went fine, as the installation instructions tell you anyway to specify the maximum heap size when invoking the java command.
   
2. The first problem came up when trying to run the Ant script that is supposed to set up GlassFish:
   gtest@v32295:~/glassfish$ lib/ant/bin/ant -f setup.xml
Buildfile: setup.xml

all:
    [mkdir] Created dir: /home/gtest/glassfish/bin

get.java.home:

setup.init:

check-java:

get.java.home:

setup.init:

validate-java: [echo] Current Java Version 1.6.0_3

BUILD FAILED
/home/gtest/glassfish/setup.xml:156: The following error occurred while
executing this line:
/home/gtest/glassfish/setup.xml:137: Execute failed: java.io.IOException:
java.io.IOException: Cannot allocate memory
   
   The solution was to modify the "lib/ant/bin/ant" script that launches Ant, adding a flag for the maximum heap size:
   
   ant_exec_command="exec \"$JAVACMD\" -Xmx64m $ANT_OPTS -classpath "$LOCALCLASSPATH" -Dant.home=\"$ANT_HOME\" -Dant.library.dir=\"$ANT_LIB\" $ant_sys_opts org.apache.tools.ant.launch.Launcher $ANT_ARGS -cp \"$CLASSPATH\" $ant_exec_args"
   
3. This change makes the script run a little bit longer. This time it will fail in a later step:
   
   gtest@v32295:~/glassfish$ lib/ant/bin/ant -f setup.xml

create.domain:
[exec] Using port 4848 for Admin.
[exec] Using port 8080 for HTTP Instance.
[exec] Using port 7676 for JMS.
[exec] Using port 3700 for IIOP.
[exec] Using port 8181 for HTTP_SSL.
[exec] Using default port 3820 for IIOP_SSL.
[exec] Using default port 3920 for IIOP_MUTUALAUTH.
[exec] Using default port 8686 for JMX_ADMIN.
[exec] Domain being created with profile:developer, as specified by
variable AS_ADMIN_PROFILE in configuration file.
[exec] Security Store uses: JKS
[exec] java.io.IOException: Cannot allocate memory
[exec] CLI130 Could not create domain, domain1

BUILD FAILED
/home/gtest/glassfish/setup.xml:172: The following error occurred while executing this line:
/home/gtest/glassfish/setup.xml:561: exec returned: 1

   This time the invocation of the "asadmin" command failed. The problem is that the "asadmin" command is generated during the Ant run, so it cannot be fixed upfront. So after the script stops with the error message above, I edited the java invocation in the "bin/asadmin" command (line is shortened) and added the "-Xmx64m" option:
   
   "$AS_JAVA"/bin/java -Xmx64m -Dcom.sun.aas.instanceName=server -Djava.library.path="$AS_NSS":"$AS_INSTALL/lib":"$AS_ICU_LIB" -Dcom.sun.aas.configRoot="$AS_CONFIG" ...
   

After making these adjustments to the various scripts, GlassFish could be started successfully, and it is running without problems so far. Note that these changes to the scripts do not affect the actual JVM settings that can be adjusted inside the GlassFish admin console.

Welcome to the Plugin Smithy Software weblog!

www.pluginsmithy.com is a small-business website that sells Eclipse plugins. It is running on Sun's Java application server Glassfish, using Magnolia Community Edition as the content management system and Apache Roller as the blog server software. The interactive pages are implemented with Java Server Faces. All this is running on an entry-level vServer.

With this combination I encountered some interesting questions and problems, and I will write about them here.

And of course I will also write from time to time about topics related to the Plugin Smithy Software products.