Compiling Apache 2.0 to support mod_rewrite

For a recent project, I downloaded the Apache 2.2.14 Unix source (http://apache.mirrors.tds.net/httpd) to assist in our automated BPEL/ANT compilations.

We added the following rewrite rules:

RewriteEngine on
RewriteRule ^(.*)?wsdl $1.wsdl [PT]

But when we started Apache, we received the following error:

oracle@oradev:/u01/httpd-2.2.14/bin> ./apachectl start

Syntax error on line 411 of /u01/httpd-2.2.14/conf/httpd.conf:
Invalid command 'RewriteEngine', perhaps misspelled or defined by a module not included in the server configuration

This is because when Apache was compiled using make, we forgot to include the appropriate flag.

This is the correct way to compile Apache to include the mod_rewrite module:

cd /u01
gtar -xzvf httpd-2.2.14.tar.gz
cd httpd-2.2.14
./configure --prefix=${PWD} --enable-rewrite=shared
make
make install

Afterwards, you will notice the mod_rewrite module being automatically configured to load in httpd.conf.

Now adding your rewrite rules will work.

Troubleshooting ORABPEL-05215 errors

The ORABPEL-05215 is annoying to troubleshoot. Advice online is pretty generic, and when I ran into this, I decided to once and for all get to the bottom of it.

My Scenario

1. I used ANT to compile the BPEL process.

ant --noconfig -buildfile build.xml compile

Compilation successful. The JAR file was successfully created in the ~/output directory.

HelloWorld/output/bpel_HelloWorld_1.0.jar

2. I used ANT to deploy the BPEL process.

ant --noconfig -buildfile build.xml deployProcess

Deployment failed. The output was as follows:

Buildfile: build.xml

deployProcess:
[echo]
[echo] --------------------------------------------------------------
[echo] | Deploying bpel process HelloWorld on sawftdev, port 7777
[echo] --------------------------------------------------------------
[echo]
[deployProcess] Deploying process /u01/deploy/code/SVN/HelloWorld/output/bpel_HelloWorld_1.0.jar

BUILD FAILED

/u01/deploy/code/SVN/HelloWorld/build.xml:124: A problem occured while connecting to server "sawftdev" using port "7777": bpel_HelloWorld_1.0.jar failed to deploy. Exception message is: ORABPEL-05215
Error while loading process.
The process domain encountered the following errors while loading the process "HelloWorld" (revision "1.0"): Failed to compile classes.
Failed to compile the generated BPEL classes for "HelloWorld".
.
If you have installed a patch to the server, please check that the bpelcClasspath domain property includes the patch classes.

Not enough information is provided unfortunately.

Deploying the JAR file manually via the BPEL Console produced the same result:

* bpel_AccSusResPortalProvABCSImpl_1.0.jar failed to deploy.
Error while loading process. The process domain encountered the following errors while loading the process "HelloWorld" (revision "1.0"): Failed to compile classes. Failed to compile the generated BPEL classes for "HelloWorld". . If you have installed a patch to the server, please check that the bpelcClasspath domain property includes the patch classes.

3. I restarted the OC4J_SOA container. No luck. I changed some of my JVM memory settings. No luck.

The Discovery

1. My server already had output and error logs separated for the OC4J_SOA container.

opmn.xml had the following configuration for the log files for the OC4J_SOA container:

<data id="oc4j-options" value="-out /u01/app/oracle/product/10.1.3/soa_1/opmn/logs/soa_group_1~oc4j_soa~soa_group_1~1.out -err /u01/app/oracle/product/10.1.3/soa_1/opmn/logs/soa_group_1~oc4j_soa~soa_group_1~1.err">

So basically, output and errors are contained in separate logs.

2. The soa_group_1~oc4j_soa~soa_group_1~1_2009_12_30_16_04_34.out log file showed:

<2009-12-30> Error while invoking bean "domain manager": [com.collaxa.cube.engine.deployment.ProcessLoaderException: Error while loading process.
The process domain encountered the following errors while loading the process "HelloWorld" (revision "1.0"): Failed to compile classes.
Failed to compile the generated BPEL classes for "HelloWorld".
.
If you have installed a patch to the server, please check that the bpelcClasspath domain property includes the patch classes.
]
ORABPEL-05215

Error while loading process.
The process domain encountered the following errors while loading the process "HelloWorld" (revision "1.0"): Failed to compile classes.
Failed to compile the generated BPEL classes for "HelloWrld".
.
If you have installed a patch to the server, please check that the bpelcClasspath domain property includes the patch classes.

This is similar to the output I received during deployment.

3. But the soa_group_1~oc4j_soa~soa_group_1~1_2009_12_30_16_04_34.err showed something different:

09/12/30 16:06:13 /u01/app/oracle/product/10.1.3/soa_1/bpel/domains/default/tmp/.bpel_HelloWorld_1.0_fc0bd0106b3c9d6a8abf672e863c8d72.tmp/BPEL-INF/src/bpel/helloworld/Execute6.java:3: Package com.sawft.util not found in import.
09/12/30 16:06:13 import org.w3c.dom.Element;import java.util.Properties;import java.io.FileInputStream;import com.sawft.util.*;import com.sawft.*;
09/12/30 16:06:13 ^
09/12/30 16:06:13 1 errors

Now we're getting somewhere.

This BPEL process apparently has embedded Java code and was importing packages that where not included in the bpelcClasspath on the server.

The .bpel file contained import statements such as:

<bpelx:exec import="com.sawft.*"></bpelx:exec&gt

My Solution

1. I added the necessary JARs to the classpath in $ORACLE_HOME/bpel/domains/default/config/domain.xml (the bpelcClasspath parameter).

What you should do

Leverage the OPMN logs when troubleshooting this issue. As a developer, you may need to reach out t the application administrator.

Interesting bug related to ANT deployments and importing JARs

This finding is related to the same issue I was troubleshooting in my next post.

Basically, if you use ANT to deploy a BPEL process to a domain that is not the default domain, and this BPEL process uses embedded Java and imports custom Java packages, unless those JARs are in the bpelcClasspath of the default domain as well, deployment will fail.

This has been tested on Oracle BPEL Process Manager 10g (10.1.3.3.1).

Scripted install for an Oracle SOA Suite 10g (10.1.3) cluster

Around a year ago, I wrote scripts that would allow you to silently install Oracle SOA Suite 10g as a 2-node cluster. Manual intervention is reduced by 95% (I would have reduced it by 100% if not for bugs in the silent installer).

For those who have installed Oracle SOA Suite in High Availability per the Oracle® Application Server
Enterprise Deployment Guide 10g Release 3 knows that the installation and configuration takes time (often a full day or more) and is loaded with endless manual steps.

My scripts takes away all the manual effort.

Completing the installation of a functioning 2-node SOA Suite cluster can be completed in 60 minutes.

Here is a small portion of the property file you must set up.

#----------------------------------------
# Parent directory of scripts
#----------------------------------------
SAWFT_INSTALL_DIRECTORY=/u01/temp2/soa_suite_cluster_install_scripts

#----------------------------------------
# Location of install software
#----------------------------------------
SAWFT_SOFTWARE_DIRECTORY=/u01/temp/software

#----------------------------------------
# User/Group owner
#----------------------------------------
SAWFT_INSTALL_USER=oracle
SAWFT_INSTALL_GROUP=oinstall

#----------------------------------------
# Hostname of this server
#----------------------------------------
SAWFT_HOSTNAME=oradev1.sawft.com
SAWFT_HOSTNAME_NODE1=oradev1.sawft.com
SAWFT_HOSTNAME_NODE2=oradev2.sawft.com

#----------------------------------------
# Used to create staticports file for installer
#----------------------------------------
SAWFT_J2EE_OPMN_REQUEST_PORT=6101
SAWFT_J2EE_OPMN_LOCAL_PORT=6201
SAWFT_J2EE_OPMN_REMOTE_PORT=6004

SAWFT_GTWY_OPMN_REQUEST_PORT=6102
SAWFT_GTWY_OPMN_LOCAL_PORT=6202
SAWFT_GTWY_OPMN_REMOTE_PORT=6007

SAWFT_OHS_OPMN_REQUEST_PORT=6100
SAWFT_OHS_OPMN_LOCAL_PORT=6200
SAWFT_OHS_OPMN_REMOTE_PORT=6001
.
.
.

If you would like a copy of these scripts, please contact me via the 'Contact me' link on http://thisisahmed.com.

A few things worth noting:

• The scripts work perfect, but there is redundant and/or messy code in some areas.
• Currently it installs a 2-node Oracle SOA Suite 10g 10.1.3.3.1 cluster on Red Hat Linux, but can be modified to support later versions or additional nodes.
• There are bugs in the Oracle Application Server and Oracle Web Services Manager silent installers that require the GUI installer to be used in a few steps (see http://blog.thisisahmed.com/2008/11/numerous-bugs-in-oracle-soa-suite-10g.html).

I am still doing some minor cleanup to them, so my response may be slow.

---

Update on 2009-12-01:
The scripts can be downloaded from here.

Bug in the SOA Suite 10g 10.1.3.5 patchset for Windows affects BPEL redeployments

My recommendation at this time is not to proceed with applying the Oracle SOA Suite 10g Patch Set 5 (10.1.3.5.0) for Windows. There is a bug that causes redeployment of BPEL processes to fail.

When redeploying an updated version of a BPEL process that is already deployed to the server, your ant or JDeveloper will return the following error:

* Error when deploying BUILD FAILED C:\data\HelloWorld\build.xml:79: A problem occured while connecting to server "localhost" using port "80": bpel_HelloWorld_v2009_06_01__55238.jar failed to deploy. Exception message is: Error deploying BPEL suitcase. An error occurred while attempting to deploy the BPEL suitcase file "C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\bpel_11747128.tmp"; the exception reported is: archive cannot rename C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\.bpel_HelloWorld_v2009_06_01__55238_a4ef0434bf12f22a3374aab6a0a942a1.tmp

Reviewing the domain.log reveals an ORABPEL-05250 error, essentially reporting the same thing:

<2009-10-12 09:39:30,281> <WARN> <default.collaxa.cube.engine.deployment> <DeploymentManager::deploySuitecase> Process bpel://localhost/default/HelloWorld~v2009_06_01__55238/ is being re-deployed (same version number) <2009-10-12 09:39:30,281> <ERROR> <default.collaxa.cube.engine.deployment> <DeploymentManager::deploySuitecase> archive cannot rename C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\.bpel_HelloWorld_v2009_06_01__55238_a4ef0434bf12f22a3374aab6a0a942a1.tmp <2009-10-12 09:39:30,312> <ERROR> <default.collaxa.cube> <BaseCubeSessionBean::logError> Error while invoking bean "domain manager": Error deploying BPEL suitcase. An error occurred while attempting to deploy the BPEL suitcase file "C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\bpel_11747128.tmp"; the exception reported is: archive cannot rename C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\.bpel_HelloWorld_v2009_06_01__55238_a4ef0434bf12f22a3374aab6a0a942a1.tmp ORABPEL-05250 Error deploying BPEL suitcase. An error occurred while attempting to deploy the BPEL suitcase file "C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\bpel_11747128.tmp"; the exception reported is: archive cannot rename C:\oracle\product\10.1.3\soa_1\bpel\domains\default\tmp\.bpel_HelloWorld_v2009_06_01__55238_a4ef0434bf12f22a3374aab6a0a942a1.tmp at com.collaxa.cube.engine.deployment.DeploymentManager.deploySuitcase(DeploymentManager.java:871) at com.collaxa.cube.ejb.impl.BPELDomainManagerBean.deploySuitcase(BPELDomainManagerBean.java:465) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Even if you remove the read-only attributes or set the Windows permissions and owner attributes to the Administrators group on the /domains folder, the behavior is the same.

The server deploys the project to a temporary directory in $ORACLE_HOME/bpel/domains/tmp and attempts to rename the existing project before replacing it with the new one. This is when the error occurs. Some unknown Java process within the BPEL server locks both the bpelclasses.jar file and one of your project WSDLs (e.g., _HelloWorld.wsdl), preventing the renaming of the parent directory.

To workaround this, you have 3 options:

1. Undeploy the process, shutdown SOA Suite, manually delete the project directory from the /tmp directory, start up SOA Suite, then redeploy your process.
   
2. Use a process unlocker utility to force remove the process lock on the JAR and WSDL files mentioned above, then redeploy your process.
3. Don't upgrade to 10.1.3.5 for Windows for now.

Unless there is a particular bug fix that you desperately need to push, avoid upgrading to 10.1.3.5 on Windows for the time being. As mentioned above, there are workarounds, but they are annoying and may disrupt operations.

Mini BPEL Console in Java

Here's a link a JSP page that lists all instances on your BPEL Console:

http://hugues.simonnet.free.fr/logiciels/blog/ngInstanceList.jsp

You can customize it to your needs (e.g., add input payload, etc.).

Installing AIA 2.2.1 on an Oracle SOA Suite cluster

A recent project required installing Oracle Application Integration Architecture (AIA) Foundation Pack 2 (2.2.1) on Red Hat Linux ES 4. AIA requires Oracle SOA Suite 10g and Oracle Service Registry. However, the AIA installer is designed to install on a single-node advanced SOA Suite installation, and not a clustered one. For those who have installed a clustered Oracle SOA Suite, you will understand how drastically different it is in terms of installation and configuration.

Below are 3 issues I have encountered during my setup and resolutions to each of them.

Issue #1: The 'htdocs' directory does not exist in the SOA ORACLE_HOME

A prerequisite step to installing AIA involves editing httpd.conf to enable directory browsing. The AIA installer detects this by creating a temporary directory under $ORACLE_HOME/Apache/Apache/htdocs (since that is one location where it copies the static EBOs, etc.).

An example of the changes to httpd.conf:

Old -> Options FollowSymLinks MultiViews
New -> Options Indexes FollowSymLinks MultiViews

Unfortunately, the AIA installer is not aware that your web server (Oracle HTTP Server in this case) may be installed in a separate ORACLE_HOME, and thus fails as it checks for the existence of 'htdocs' in the SOA ORACLE_HOME.

Solution:
Manually copy the 'htdocs' directory from the OHS ORACLE_HOME to $ORACLE_HOME/Apache/Apache on the SOA ORACLE_HOME.

Issue #2: AIA install fails with 'ESB_ERROR Destination Not Found Error'

The AIA installer tries to configure its ErrorHandlingMDB to listen to AS JMS (in-memory). When performing a clustered install of Oracle SOA Suite as per the instructions in the Enterprise Deployment Guide, the ESB_ERROR topic will be hosted by AQ and the resource adapter will be OracleOJMS, so that both nodes use a common version.

However, by design, the AIA Installer looks only for the in-memory OracleASjms (see Oracle Bug #6932261).

Solution:
To work around this known issue, a dummy error topic is created so that the AIA installation can proceed, and once the installation is complete, the MDB will be configured to use the AQ based JMS instead of this dummy topic.

Follow these steps to create the dummy error topic:

1. In Enterprise Manager, click on oc4j_soa -> Administration -> JMS Destinations.
2. On the 'JMS Destinations' screen, click the Create button.
3. Choose 'Type' as Topic.
4. For 'Destination Name', enter ESB_ERROR_DUMMY.
5. For the 'JNDI' entry, enter jms/topic/ESB_ERROR.
6. Click on OK.

Issue # 3: The Configuration Assistant fails

The AIA installer logs all messages to $AIA_HOME/Infrastructure/install/logs/FPInstall.log. You may receive an error similar to the following in this log:

Failed to read wsdl.

Error happened when reading wsdl at "/u01/app/oracle/product/10.1.3/soa/bpel/domains/default/tmp/.bpel_AIADemogetSMItemPriceProvABCSImpl_1.0_df91da2143861b6e7a295558f2ee44ba.tmp/_AIADemogetSMItemPriceProvABCSImpl.wsdl", because "Error reading import of file:/u01/app/oracle/product/10.1.3/soa/bpel/domains/default/tmp/.bpel_AIADemogetSMItemPriceProvABCSImpl_1.0_df91da2143861b6e7a295558f2ee44ba.tmp/_AIADemogetSMItemPriceProvABCSImpl.wsdl: Failed to read WSDL from http://rh-soa.itciss.com:7777/AIAComponents/UtilityArtifacts/RuntimeFault.wsdl:WSDL not found".

Make sure wsdl exists at that URL and is valid.

Solution:
The Configuration Assistant fails because the WSDL points to certain XSD’s which are supposed to be hosted by the web server.

Simply copy the contents of the htdocs from the SOA ORACLE_HOME to the OHS ORACLE_HOME’s htdocs directory and run the command:

/u01/app/oracle/product/10.1.3/soa/jdk/bin/java -jar /u01/app/oracle/product/2.2.1/aia/Infrastructure/install/lib/AIAUtils.jar -i /u01/app/oracle/product/2.2.1/aia -nopips

Good luck!

1 year anniversary!

Now that my blog (http://blog.thisisahmed.com) and my techie site (http://www.thisisahmed.com) have been up for just over a year, unique visits have been slowly growing.


This is a good sign. It means that there are others out there who are potentially benefiting from my notes and posts.

Thankfully, I am fortunate enough to personally have met individuals who have relied on several of my posts to resolve many of their technical issues surrounding various Oracle product sets. I also continue to receive private emails and do my best to respond (don't hesitate to contact me!).

My site is in the top 10 Google search results 93% of the time when querying for specific Oracle, Fusion Middleware, and SOA errors. The reason behind this is quite simple:

• If I troubleshoot an issue and no documentation or solution to my problem can be found... I post the solution once I figure it out.

• If a problem takes me longer than a few hours to figure out... I post the solution.
  

If I suffered through it, why should anyone else? That's why many of what you find on ThisIsAhmed.com are solutions to problems you won't find anywhere else.

Thanks to all those who visit, and no, I unfortunately don't make any money off this. :)

Check JNDIs in Oracle Application Server against deployed BPEL & ESB code

The links below provide scripts on how to automatically compare JNDIs configured on the Oracle Application Server against those used within your BPEL processes or ESB services.

http://www.thisisahmed.com/tia/bpel/checkjndibpel.html
http://www.thisisahmed.com/tia/esb/checkjndiesb.html

What is the importance of these scripts?

If BPEL or ESB code deployed to your server is using an invalid or non-existent JNDI, then it defaults to the MCF properties configured within that adapter.

This could be an issue in the following scenario.

Let's say that you have both a Dev and a Test environment, and the admins have instructed the developers to use the JNDI "eis/DB/LocalDBScott". On each of the environments, the JNDI is configured to point to a different database. Now, many developers rely on JDeveloper to configure their connections, often ignoring the value of the JNDI. This hardcodes the username/password/database connect information within the process itself (via MCF properties). If this same code is deployed to Test, when executed it would connect to your Dev database!

The usage behavior is as follows:
1. If the MCF properties are defined, the BPEL/ESB process uses them to connect.
2. If the MCF properties are defined but invalid, the BPEL/ESB process uses the JNDI to connect.
3. If the MCF properties are not defined, the BPEL/ESB process uses the JNDI to connect.

What are MCF properties?

MCF stands for "Managed Connection Factories".

When configuring a Database Adapter or AQ Adapter via the Adapter Configuration Wizard, a Database Connection is required to configure the adapter (this is already defined in your project). The Connection Information listed is actually pulled from your JDeveloper Database Connection information.

Once the adapter is created, a WSDL file is automatically created that includes the adapter definition. This includes the Managed Connection Factory (MCF) properties as well as the Java Naming and Directory Interface (JNDI) name, as shown below:

<service name="UpdateCustTable"> <port name="UpdateCustTable_pt" binding="tns:UpdateCustTable_binding"> <jca:address location="eis/DB/LocalDBScott" UIConnectionName="LocalDB" ManagedConnectionFactory="oracle.tip.adapter.db.DBManagedConnectionFactory" mcf.DriverClassName="oracle.jdbc.OracleDriver" mcf.PlatformClassName="oracle.toplink.platform.database.oracle.Oracle10Platform" mcf.ConnectionString="jdbc:oracle:thin:@//127.0.0.1:1521/orcl" mcf.UserName="scott" mcf.Password="9469F498BD53204FCC2CC246769B23C3" /> </port> </service>

Recommendation?

Always manually remove MCF properties from your code once development is complete.

Why does Linux report 100% memory usage all the time?

I figured this would be worth posting, even though I do have a note on it on my sister site.

http://www.thisisahmed.com/tia/linux/linuxmemory.html

Why does Linux report memory usage as being 99% used even though nothing is running on the server?

I'll start with the conclusion: If little swap is being used, memory usage is not impacting performance at all.

• Traditional Unix tools like top often report a surprisingly small amount of free memory after a system has been running for a while.

• The biggest place memory is being used is in the disk cache which is reported by top as "cached". Cached memory is essentially free, in that it can be replaced quickly if a running (or newly starting) program needs the memory.

• The reason Linux uses so much memory for disk cache is because the RAM is wasted if it isn't used. Keeping the cache means that if something needs the same data again, there's a good chance it will still be in the cache in memory. Fetching the information from there is around 1,000 times quicker than getting it from the hard disk. If it's not found in the cache, the hard disk needs to be read anyway, but in that case nothing has been lost in time.

Check for and remove empty nodes in XSLT

On one of our projects, we use AIA canonicals extensively, and for those familiar with them, their definition is quite large. At times, our payloads were over 1.5MB because all elements, including empty ones, were mapped to the final message.

One way to work around this is to create a transformation to remove the empty nodes.

For example, your process may have 5 input elements and 5 result elements. Your transformation may merely copy each input element to it's corresponding output element as shown here:

<xsl:template match="/"> <ns1:EmptyNodesTestProcessResponse> <ns1:result1> <xsl:value-of select="/ns1:EmptyNodesTestProcessRequest/ns1:input1"/> </ns1:result1> <ns1:result2> <xsl:value-of select="/ns1:EmptyNodesTestProcessRequest/ns1:input2"/> </ns1:result2> <ns1:result3> <xsl:value-of select="/ns1:EmptyNodesTestProcessRequest/ns1:input3"/> </ns1:result3> <ns1:result4> <xsl:value-of select="/ns1:EmptyNodesTestProcessRequest/ns1:input4"/> </ns1:result4> <ns1:result5> <xsl:value-of select="/ns1:EmptyNodesTestProcessRequest/ns1:input5"/> </ns1:result5> </ns1:EmptyNodesTestProcessResponse> </xsl:template>

However, during execution, if an input element is empty or non-existent, it will still map as 'blank' to the output element. For example, if there is no data for input3, the result is still returned as follows:

<ns1:result3/>

As XSDs are larger in size, this becomes more of a size concern.

To work around this, create a transformation that has a source and target variable of your output variable. In this, add the following:

<xsl:template match="@*|node()">
 <xsl:if test=". != '' or ./@* != ''">
   <xsl:copy>
     <xsl:apply-templates select="@*|node()" />
   </xsl:copy>
 </xsl:if>
</xsl:template>

As you can see from the execution below, the first transformation essentially copied all input elements to their corresponding output result element, including the blank element result3. However, by applying a second transformation, all empty nodes are removed.

Transient vs. durable BPEL processes

As a general practice, it is better to design your BPEL processes as transient instead of durable if performance is a concern. Note that this may not always be possible due to the nature of your process, but keep the following points in mind.

The dehydration store is uses to maintain long-running asynchronous BPEL instances storing state information as they wait for asynchronous callbacks. This ensures the reliability of these processes in the event of server or network loss. Oracle BPEL Process Manager supports two types of processes; transient and durable.

Transient Processes

Transient processes do not incur dehydration during their process execution. If an executing process experiences an unhandled fault or the server crashes, instances of a transient process do not leave a trace in the system. Thus, these instances cannot be saved in-flight regardless if they complete normally or abnormally. Transient processes are typically short-lived, request-response style processes. Synchronous processes are examples of transient processes.

Durable Processes

Durable processes incur one or more dehydration points in the database during execution. Dehydration is triggered by one of the following activities:

• Receive activity
• OnMessage branch in a pick activity
• OnAlarm branch in a pick activity
• Wait activity
• Reply activity
• checkPoint() within a <bpelx:exec> activity

Durable processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized.

What should you do?

• If the design of the process allows it, design your BPEL processes as short-lived, synchronous transactions.
• If the design of the process allows it, avoid the activities listed above.
  

Any time your process is dehydrated to the dehydration store, this naturally impacts the performance of the process, and becomes a concern particularly in high volume environments.

Invalid Usage of Element

Just ran into something annoying the past 10 minutes when working on a BPEL process (related to XSLT).

My transformation included the following (which I had manually entered in JDeveloper):

<ns0:QualificationDate> <xsl:if test="number(ns1:Highest/ns1:QualificationDate) > 0"> <xsl:value-of select="number(ns1:Highest/ns1:QualificationDate) + 190000"/> </xsl:if> </ns0:QualificationDate>

However, when I clicked on the "Design" tab of the mapper tool to switch from Source to Design mode, I received the following error:

Transformation_2.xsl
Error: Invalid Usage of <if> Element

Turns out, the correct usage of the <if> element is the following:

<xsl:if test="number(ns1:Highest/ns1:QualificationDate) > 0">
  <ns0:QualificationDate>
    <xsl:value-of select="number(ns1:Highest/ns1:QualificationDate) + 190000"/>
  </ns0:QualificationDate>
</xsl:if>

It's the little things that get you sometimes.

Querying the BPEL Process Manager instance table

When you have tens of thousands of instances logged and audited in your BPEL server, navigating the instances within BPEL Control (the BPEL console) becomes unrealistic.

You can use SQL to query the instance table in the ORABPEL schema in Oracle BPEL Process Manager 10g (10.1.3.4) to narrow down your search.

SELECT process_id, DECODE(TO_CHAR(state),'1','In-flight','5','Success','6','Warning','9','Stale','8','Error',TO_CHAR(state)) state, creation_date, revision_tag, modify_date-creation_date duration, SUBSTR(REGEXP_SUBSTR(title, '[^ ]+', 1, 2), 2) instance_id FROM cube_instance ORDER BY creation_date DESC;

Here is an example of the output generated by the query above:

PROCESS_ID STATE CREATION_DATE REVISION_TAG DURATION INSTANCE_ID ---------- ----- ---------------------------- ------------ -------------------------- ----------- HelloWorld Error 27-MAY-09 10.30.01.421000 PM 1.0 +000000000 00:00:00.016000 200003 HelloWorld Stale 27-MAY-09 10.27.20.437000 PM 1.0 +000000000 00:00:01.781000 200001 HelloWorld Success 26-MAY-09 10.46.08.593000 PM 1.0 +000000000 00:07:24.969000 190003

To explain a few things regarding the query:

• The STATE column stores values of 1 (In-flight), 5 (Success), 6 (Warning), 8 (Error), and 9 (Stale). Stale processes are those that have been undeployed.
• The DURATION column is calculated by substracting the CREATION_DATE from the MODIFY_DATE. The CREATION_DATE is when the process was instantiated, and the MODIFY_DATE is when the last activity in the process was executed.
• Per the SQL statement above, the INSTANCE_ID is actually parsed from the TITLE column.

With this query, you now have the power to easily access your BPEL instance information.

Ahmed presenting Oracle BAM at Collaborate 09

On May 7, 2009 in Orlando, Florida, I will be presenting at the Oracle Applications Users Group (OAUG) at 12:15pm. The title of my presentation is Bam! We Have Integration - Using SOA Suite and BAM to Monitor Your Integration Layer. The presentation is in W304C.

For more information, please check out the following links:
http://www.itconvergence.com/portal/page?_pageid=33,9942294&_dad=portal&_schema=PORTAL
http://itconvergence.blogspot.com/2009/03/presenting-oracle-bam-at-collaborate-09.html

How to setup dynamic routing of endpoints in Oracle SOA Suite 10g

Here we describe in detail how to create an ESB service to parse an incoming message, and based on the input, route to the appropriate endpoint which may not be defined during design time and which may also be updated dynamically as needed.

Why use dynamic routing? The reason for considering dynamic routing of endpoints is illustrated in the example below. For instance, you may have an ESB service designed as shown. Here, all customer creation requests are routed to some endpoint: In the future, you may decide to route U.S. customers to one server while routing European customers to another one (for the purpose of distributing load or handling them differently on the backends). As you can see on the left, the ESB project will have to be modified every time a new endpoint is added. Dynamic routing solves the problem by allowing you to add endpoints without having to update your ESB services.

The advantage of dynamic routing is that no modifications to the ESB service is needed if a new SOAP service is added which has the same WSDL but a different endpoint. The disadvantage is that it is not possible through ESB Control to show which of the endpoints the request was actually routed to.

The example detailed below uses the the country code in the input payload (in my example, stored in /imp1:Datas/imp1:Data/imp1:Application) to lookup the actual endpoint. The XSL mapper file will perform a DVM lookup via the orcl:lookup-dvm function, and store the result in the outbound header via the ehdr:setOutboundHeader function. The ehdr:setOutboundHeader function overrides the actual endpoint of the external SOAP service. As a result, although you have one external SOAP service defined in your ESB project, it can be overriden at runtime.

Step 1: Create a DVM

Create a DVM manually, or import the following XML through ESB Control:

<?xml version = '1.0' encoding = 'UTF-8'?> <dvm name="Endpoints" isNew="null"> <description>Dynamic endpoints for all systems</description> <columns> <column name="System"/> <column name="Endpoint"/> </columns> <rows> <row><cell>US</cell><cell>http://usaserver.itconvergence.com:7777/createCustomer</cell></row> <row><cell>EU</cell><cell>http://europeserver.itconvergence.com:80/createCustomer</cell></row> <row><cell>SA</cell><cell>http://southamericaserver.itconvergence.com/createCustomer</cell></row> </rows> </dvm>

This creates a DVM called Endpoints with 2 columns. The first column System denotes a region code (e.g., US), while the second column Endpoint denotes the actual endpoint URL. As new endpoints are defined, they can be dynamically added to the DVM at runtime without having to update the ESB code.

Step 2: Create the ESB service

Create a simple ESB service consisting of:
- Inbound file adapter
- Routing service
- Outbound SOAP service









Only one outbound external SOAP service is required, even though there are actually several endpoints. The WSDL of this SOAP service may have a hardcoded SOAP address location of any one of the external services. For example:

<soap:address location="http://southamericaserver.itconvergence.com/createCustomer"/>

This SOAP address location will be overriden at runtime though.

Step 3: Create mapper file

Create a new mapper file (i.e., XSL transformation) between the routing service and the SOAP service.

Step 4: Transform the outbound header

Utilize the header transformation extension function setOutboundHeader to update the outbound header and override the SOAP address location that is hardcoded in the WSDL. Click on "Source" to view the source of your XSL file just created, and immediately before the statement <xsl:template match="/">, add the following:

<xsl:variable name="LocationIn" select="orcl:lookup-dvm('Endpoints', 'System', /imp1:Datas/imp1:Data/imp1:Application, 'Endpoint','')"/> <xsl:variable name="LocationOut" select="ehdr:setOutboundHeader('/shdr:ESBHeader/shdr:location', $LocationIn, 'shdr=http://xmlns.oracle.com/esb;')"/>

What this does is use the country code /imp1:Datas/imp1:Data/imp1:Application to perform a lookup on the Endpoints DVM and returns the Endpoint column, storing it in the $LocationIn variable. This variable is then used by the ehdr:setOutboundHeader function to set the outbound location in the header. The design will look similar to the following:










That's it!

Creating a JMS queue for SOA consumption

This post describes how to create a JMS queue in the Oracle database to use within your BPEL processes and/or ESB services.

Create a JMS queue

Log on to the Oracle Database and create a queue table called JMS_ABC_QUEUETABLE. Afterwards, create a queue called JMS_ABC_QUEUE on this queue table. Finally, start the queue.

begin sys.dbms_aqadm.create_queue_table( queue_table => 'JMS_ABC_QUEUETABLE', queue_payload_type => 'SYS.AQ$_JMS_MESSAGE', sort_list => 'PRIORITY, ENQ_TIME', compatible => '10.0.0', primary_instance => 0, secondary_instance => 0, storage_clause => 'tablespace pctfree 10 initrans 1 maxtrans 255 storage ( initial 64K minextents 1 maxextents unlimited )'); end; / begin sys.dbms_aqadm.create_queue( queue_name => 'JMS_ABC_QUEUE', queue_table => 'JMS_ABC_QUEUETABLE', queue_type => sys.dbms_aqadm.normal_queue); end; / EXECUTE dbms_aqadm.start_queue (queue_name => 'JMS_ABC_QUEUE');

Configure the Oracle SOA Suite server

Edit the $ORACLE_HOME/j2ee/oc4j_soa/config/application.xml file and add the "myjms" resource provider.

<resource-provider class="oracle.jms.OjmsContext" name="myjms"> <description>oc4j-jms loop back resource provider</description> <property name="url" value="jdbc:oracle:thin:<user>/<password>@<dbhost>:<dbport>:<dbsid>"/> </resource-provider>"/>

Edit the $ORACLE_HOME/j2ee/oc4j_soa/application-deployments/default/JmsAdapter/oc4j-ra.xml file and add the "myconnfac" connector factory.

<connector-factory location="eis/Jms/myconnfac" connector-name="Jms Adapter"> <config-property name="connectionFactoryLocation" value="java:comp/resource/myjms/QueueConnectionFactories/myQCF"/> <config-property name="factoryProperties" value=""/> <config-property name="acknowledgeMode" value="AUTO_ACKNOWLEDGE"/> <config-property name="isTopic" value="false"/> <config-property name="isTransacted" value="false"/> <config-property name="username" value="<user>"/> <config-property name="password" value="<password>"/> <connection-pooling use="none"></connection-pooling> <security-config use="none"></security-config> </connector-factory>

Enqueing via PL/SQL

For testing purposes, you may decide to enqueue a payload via PL/SQL.

declare l_message clob := ''; v_enqueue_options dbms_aq.enqueue_options_t; v_msg_props dbms_aq.message_properties_t; v_msg_id RAW(16); v_message SYS.AQ$_JMS_MESSAGE := SYS.AQ$_JMS_MESSAGE.construct(DBMS_AQ.JMS_TEXT_MESSAGE); begin l_message := '<ns4:elements xmlns:ns4="http://ns.thisisahmed.com/MyMessage/20080702"><ns4:element><ns4:firstname>Ahmed</ns4:firstname><ns4:lastname>Naga</ns4:lastname></ns4:element></ns4:elements>'; v_message.set_text(xmltype(l_message).getstringval()); dbms_aq.enqueue(queue_name => 'JMS_ABC_QUEUE', enqueue_options => v_enqueue_options, message_properties => v_msg_props, payload => v_message, msgid => v_msg_id); commit; end; /

When configuring the JMS Adapter within JDeveloper, here are the settings you will want to use:

Service Name: (whatever name you choose; e.g., JMSQueue)
OEMS: Database
JNDI Name: (whatever your JNDI name is; e.g., eis/Jms/LocalDB)
Operation Type: Consume Message
Resource Provider: myjms
Destination Name: JMS_ABC_QUEUE
Schema Location: (whatever schema you want)

This is how my final BPEL process looked like.

Good luck.

Configuring a BPEL Process Manager cluster using TCP instead of UDP

When configuring multicast for an Oracle SOA Suite 10g (10.1.3.3) cluster, there are two sets of multicast addresses that are required; one for the Oracle Application Server and a completely separate one for BPEL Process Manager. This is to allow BPEL Process Manager to manage active/passive adapter state as well as propagate process deployment notifications in the cluster.

The TCP settings in jgroups-protocol.xml are designed for clustering across subnets, and Oracle seems to generally recommend using multicast otherwise. Unfortunately, the Enterprise Deployment Guide (10.1.3.3) does not describe how to configure TCP-based clustering for BPEL Process Manager.

When to use TCP for BPEL Process Manager Clustering

You should you use node-to-node (TCP) instead of multicast (UDP) when configuring a BPEL Process Manager cluster if a combination of the conditions below apply:
- You cannot configure multicast.
- Your BPEL Process Manager servers communicate across subnets.
- You have a very large number of nodes in your cluster.
- Performance of UDP is slow in your environment.

Configuring TCP-based clustering for BPEL Process Manager

Perform the following on all BPEL Process Manager servers in your cluster.

1. Comment out the UDP multicast configuration in jgroups-protocol.xml:

<-- <config> <UDP mcast_send_buf_size="32000" . . . </config> -->

2. Uncomment the TCP configuration in jgroups-protocol.xml and update the initial_hosts attribute with the hostnames in your node-to-node cluster, as well as the port you want to use (7900 in the example below):

<config> <TCP start_port="7900" loopback="true" send_buf_size="32000" recv_buf_size="64000"/> <TCPPING timeout="3000" initial_hosts="server1[7900],server2[7900]" port_range="3" num_initial_members="3"/> . . .

The configured port should be identical across all servers.

3. Restart OPMN.

If configured correctly, the log file SOA_GROUP~OC4J_SOA~SOA_GROUP~1.log should show it working with entries similar to:

INFO: server socket created on 192.168.28.210:7900 GMS: address is 192.168.28.210:7900 INFO: created socket to 192.168.28.207:7900 INFO: created socket to 192.168.28.210:7900

The log entries below shows the result of having incorrectly configured different ports:

WARNING: discarded message from different group "server2:8880" (our group is "server1:7777"). Sender was192.168.28.207:7900 WARNING: discarded message from different group "server2:8880" (our group is "server1:7777"). Sender was192.168.28.207:7900

Good luck.

Purge B2B runtime data from the command prompt for Oracle B2B 10g

Oracle B2B 10g (10.1.2.0.2) runtime data can grow at an alarming rate in a medium usage production environment. Therefore scripting the purging of older B2B runtime data should be considered. This can be done through the command prompt or from SQL*Plus. In my last project, the database grew at a rate of 10GB per week due to this runtime data and purging took approximately 1 minute for 10,000 records on a dual CPU server.

Purging B2B runtime data

1. Set the environment by using setenv.sh. You must use the fully qualified path when invoking it (do not type $ORACLE_HOME). For example:

. /u01/app/oracle/product/10.1.2/b2b_1/ip/install/setenv.sh

2. Change into the $ORACLE_HOME/ip/install directory before executing the command:

cd /u01/app/oracle/product/10.1.2/b2b_1/ip/install

Examples of purging Complete and Error instances are shown below:

java oracle.tip.repos.purge.PurgeManager purgeRuntime -start 01-JAN-2008 -end 10-JAN-2008 Complete java oracle.tip.repos.purge.PurgeManager purgeRuntime -start 01-JAN-2008 -end 10-SEP-2008 Error

3. Alternatively, you can SQL*Plus into the database and execute the purge_runtime procedure:

sqlplus b2b@orcl SQL> exec purge_runtime('21-JAN-2008', '07-FEB-2008', 'Complete');

You need to ensure that proper purging procedures are in place and are in conformance with your IT data retention policies.

MQSeries Adapter Get Reply/Reports is not supported in Oracle ESB 10g

In Oracle SOA Suite - New Features 10g (10.1.3.3) - E10381-03 - March 2008, the following is stated under the section titled MQSeries Adapter New Features:

Outbound Synchronous-Solicit-Request-Response Scenario:

The 10.1.3.3 MQSeries adapter supports the outbound synchronous-solicit-request-response scenario through use of several InteractionSpec properties. In this scenario, the adapter enqueues a normal/request message in a queue and expects the report/reply synchronously. The report/reply message arrives in the replyToQueueName of the normal/request message.

However, if you try to configure the MQSeries Adapter in JDeveloper for an ESB project, the radio button Send Message from MQ and Get Reply/Reports is grayed out. It is not grayed out in BPEL projects though.

Send Message to MQ and Get Reply/Reports creates 2 separate 1-way operations while the Get Message from MQ and Send Reply/Reports creates 1 synchronous 2-way operation.

The MQSeries Adapter supports the request-reply pattern both as a client and a server for BPEL. However, Send Message to MQ and Get Reply/Reports which sets up a reply queue is not supported in ESB. ESB is stateless and does not support asynchronous request-reply pattern out-of-the-box.