Showing posts with label wso2. Show all posts
Showing posts with label wso2. Show all posts

November 19, 2016

LDAP User Authentication

What is LDAP and LDAP Authentication ?

To get started on what and how LDAP works, take a look at [1].

Quoting from the above article, following is about how LDAP Authentication works :
To perform any of these LDAP operations, an LDAP client needs to establish a connection with an LDAP server. The LDAP protocol specifies the use of TCP/IP port number 389, although servers may run on other ports.
The LDAP protocol also defines a simple method for authentication. LDAP servers can be set up to restrict permissions to the directory. Before an LDAP client can perform an operation on an LDAP server, the client must authenticate itself to the server by supplying a distinguished name and password. If the user identified by the distinguished name does not have permission to perform the operation, the server does not execute the operation.
In order write a simple LDAP Authenticator using Java, refer to the article [2] first to get an idea.

As per the above quote and explanations, in order to perform an LDAP based user authentication, we validate the user's permission by trying to execute an operation on LDAP.

Java LDAP Authenticator

import javax.naming.Context;
import javax.naming.directory.DirContext;
import javax.naming.directory.InitialDirContext;
import java.util.Hashtable;
/**
 * Open a connection to the LDAP server with uid and password and authenticate the user.
 */
public class LDAPAuthenticator {
    private static String UID_FORMAT_STRING = "uid=%1s,%2s";
    private static String VDS_SERVER = "LDAPS://localhost:389";
    private static final String PEOPLE_OU = "ou=people,dc=local,dc=com";
    public static void main(String[] args) {
        Hashtable<String, String> authEnv = new Hashtable<String, String>();
        authEnv.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        authEnv.put(Context.PROVIDER_URL, VDS_SERVER);
        authEnv.put(Context.SECURITY_AUTHENTICATION, "simple"); //default authentication
        authEnv.put(Context.SECURITY_PRINCIPAL, String.format(UID_FORMAT_STRING, "uidName", PEOPLE_OU));
        authEnv.put(Context.SECURITY_CREDENTIALS, "password");
        try {
            DirContext ctx = new InitialDirContext(authEnv);
            ctx.close();
            System.out.println("User Authentication Successful");
        } catch (Exception e) {
            System.out.println("User Authentication Failed");
            e.printStackTrace();
        }
    }
}

In addition if you need perform attribute matching along with authentication use the following code as an example :

Attributes matchAttrs = new BasicAttributes(true); // ignore attribute name case
matchAttrs.put(new BasicAttribute("uid", "AARSupport"));

// Search for objects with those matching attributes
NamingEnumeration answer = ctx.search(PEOPLE_OU, matchAttrs);

WSO2 LDAP Connector

If you are using WSO2 ESB, you can use the WSO2 LDAP connector to perform LDAP authentication. You can download the connector from [6]. Documentation on using the LDAP connector for authentication can be found at [7].

Use the following configurations to call the Authenticate operation.

Create a local entry with authentication details as below.

<ldap.init xmlns="http://ws.apache.org/ns/synapse">
      <providerurl>LDAPS://localhost:389</providerurl>
      <securityprincipal>uid=UIDName,ou=people,dc=local,dc=com</securityprincipal>
      <securitycredentials>password</securitycredentials>
      <secureconnection>false</secureconnection>
      <disablesslcertificatechecking>false</disablesslcertificatechecking>
</ldap.init>


Or you can also create local entry as below without authentication details since they are again later provided at the authenticate operation.

<ldap.init xmlns="http://ws.apache.org/ns/synapse">
      <providerurl>LDAPS://localhost:389</providerurl>
      <securityprincipal></securityprincipal>
      <securitycredentials></securitycredentials>
      <secureconnection>false</secureconnection>
      <disablesslcertificatechecking>false</disablesslcertificatechecking>
</ldap.init>

In the mediation, you can call the LDAP connector's authenticate operation to perform authentication.

<ldap.authenticate configkey="LDAPConfig">
        <dn>uid=UIDName,ou=people,dc=local,dc=com</dn>
        <password>password</password>
</ldap.authenticate>


If the authentication is successful, you would be getting a Success response back.

References : 

  1. https://docs.oracle.com/cd/E19957-01/816-6402-10/ldap.htm
  2. http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ldap.html
  3. https://tools.ietf.org/html/rfc4519#section-2.39
  4. http://stackoverflow.com/questions/2522770/how-to-check-user-password-in-ldap-whith-java-with-given-ldapcontext
  5. http://stackoverflow.com/questions/7813868/whats-the-difference-in-using-distinguished-name-with-cn-or-uid-when-logging-in
  6. https://store.wso2.com/store/assets/esbconnector/details/4ecf8dde-60f3-4e91-ba22-5f49a4e302f4
  7. https://docs.wso2.com/display/ESBCONNECTORS/Working+with+User+Authentication+in+LDAP

October 30, 2016

Mocking Endpoint Behaviours for Troubleshooting in ESB


This article explains some common timeout and latency scenarios in endpoints and information on troubleshooting these issues. Following three scenarios will be discussed.
  1. Backend responds as expected but response time is fairly low
  2. Backend timeout
  3. 101503 Error : Backend connection refused
  4. Unknown Host exception
In order to explain the different scenarios, the easiest approach is to create a mock service for troubleshooting. For this article I am creating a mock service with SOAPUI. You may use any other tool to do the same. Refer [1] on more details for creating mock services using SOAPUI.

Once we create a mock service it is accessible with http://localhost:8080/mockservice.

Case 1 : Endpoint Responds with Low Response Time

This case is fairly straightforward. First we need to create the mock service and point to the mock service url in the ESB endpoint definition. Usually the cases to verify are the behaviour at different responses. To simulate this we can create multiple responses and attach to the mock service. This allows us to validate the behaviour for different responses.


Case 2 : Endpoint Timeout

Refer below endpoint timeout duration in the endpoint definition. We have set it to be 3000ms.


<?xml version="1.0" encoding="UTF-8"?>
<endpoint name="MockServiceEndpoint" xmlns="http://ws.apache.org/ns/synapse">
    <http method="post" uri-template="http//localhost:8080/mockservice">
        <timeout>
            <duration>3000</duration>
            <responseAction>fault</responseAction>
        </timeout>
        <suspendOnFailure>
            <errorCodes>-1</errorCodes>
            <initialDuration>0</initialDuration>
            <progressionFactor>1.0</progressionFactor>
            <maximumDuration>0</maximumDuration>
        </suspendOnFailure>
        <markForSuspension>
            <errorCodes>-1</errorCodes>
        </markForSuspension>
    </http>
</endpoint>

In this case, since endpoint timeout duration is 3000ms, we can verify four cases where;

  1. No backend latency
  2. 2000 < timeout_duration <  3000 (latency close to 3000)
  3. timeout_duration  > 4000 (latency much higher than 3000)
  4. timeout_duration = 3000

From above four cases, 3 and 4 cases should cause endpoint to timeout. Other scenarios should respond and the rest of the mediation flow should work correct. In order to add the response latency to mock service, add a sleep command on the onRequest script of the mock service window as below.



Notice the delay in response time for the request that has gone past 3000ms in the below screenshot (left bottom)


By changing the sleep duration we can test out the behaviour of response time at the above four scenarios listed above.

Case 3 : 101503 - Endpoint Connection Refused

When faced with this error, a latency of ~10000 ms will be observed from the backend. Considering the fact that the endpoint timeout was 3000ms and it seemed highly improbable to exceed this number.

In order to mock this scenario, you need to host the mock service in a different machine in the same network and point to it as the endpoint (referred to as the mock server hereafter). While requests were sent out, the ‘mock server’ was disconnected from the network and ~10000ms of latency was observed. When ‘mock server’ is reconnected to the network normal behaviour is observed again.

To explain further the latency of ~10000ms is caused by connection refused error where a hostname is valid by DNS but is no longer available. This is why the ‘mock server’ had to be disconnected from the network to mock this behaviour. This is different from Case 4 described below.

Another way to mock this behaviour is to use an IP address which timeouts when pinged as the hostname in the endpoint definition.

Explanation on the Observation from WSO2 ESB perspective

In ESB, whenever a request is made a callback is registered for the request. This callback is responsible for sending back the response and doing the required processing. When backend does not response, these callbacks have to be cleared. Clearing is done by a task (TimeoutHandler) which runs every 15000 ms (by default).  This time duration can be controlled by overriding the default time duration with “synapse.timeout_handler_interval” property in ‘synapse.properties’ file.

To further elaborate, what happens here is that TimeoutHandler is executed every 15s and the callbacks get cleared within a 15s timeframe. Thus the response time will be somewhere below 15s, yet above 3s (to allow initial endpoint timeout duration). Attached graph of response time at default timeout for the mock API we created further explains this fact.


Response time will go to a lower range when we reduce the timeout_handler_interval. However, this means a higher frequency of timeout handler task being executed and will in turn impact performance.

To confirm the above description please refer the following observations when timeout_handler_interval is reduced.

Timeout_handler_interval = 5000ms

Timeout_handler_interval = 1000ms

Case 4 : Unknown Host Exception

Another scenario that was verified was giving an invalid host name, which resulted in an ‘Unknown Host Exception’, but this is handled within the 3000ms timespan.

Apart from these, there are a number of scenarios related to troubleshooting issues in endpoints. Refer for possible error scenarios in endpoints in [2]

References

[1] https://www.soapui.org/soap-mocking/getting-started.html
[2] https://docs.wso2.com/display/ESB480/Error+Handling#ErrorHandling-codes

June 12, 2016

WSO2 Admin Services

WSO2 Admin Services

All WSO2 Products internally perform all of it's operations via admin services. From the management console, these admin services are called to facilitate the features.

You can find more information on this at : https://docs.wso2.com/display/Carbon420/Calling+Admin+Services+from+Apps

By default, admin services WSDLs are not exposed in a product. Therefore need to do the following to access the WSDLs.

  • Set the <HideAdminServiceWSDLs> element to false in <PRODUCT_HOME>/repository/conf/carbon.xml file and restart the server.
  • Additionally to access the OSGI console, start the server as <PRODUCT_HOME>/bin/wso2server.sh -DosgiConsole. By hitting enter on the console it will start the OSGI shell
  • osgi> listAdminServices will list all the admin services available in the product.


Invoking Admin Services

There are a few ways to invoke admin services.
  1. Invoke with SOAP UI using the WSDLs of the admin services
  2. Invoke via CURL
  3. Invoke programmatically via the service stubs.

Using SOAP UI to Invoke Admin Services

We can create a new SOAP UI project by using the WSDL. We can get the sample requests and fill up the necessary data to invoke the service operations.

Even if we are using CURL to invoke the admin services, we need to get the request format which is easiest to be done by importing the WSDL as a SOAP project to SOAP UI first.

Using CURL to Invoke Admin Services

As mentioned above, best approach is to get the request format from SOAPUI. Once you have the request format, we can invoke admin services as below.

1. First we need to authenticate. Therefore, invoke the AuthenticationAdmin service.

Request (auth.xml):

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://org.apache.synapse/xsd">
 <soapenv:Header/>
 <soapenv:Body>
      <aut:login xmlns:aut="http://authentication.services.core.carbon.wso2.org">
         <aut:username>admin</aut:username>
         <aut:password>admin</aut:password>
         <aut:remoteAddress>localhost</aut:remoteAddress>
      </aut:login>
   </soapenv:Body>
</soapenv:Envelope>

CURL command :

curl -k -v -H "SOAPAction: urn:login" -H "Content-Type: text/xml" -d @auth.xml https://localhost:9443/services/AuthenticationAdmin

Response :
*   Trying ::1...
* connect to ::1 port 9443 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: localhost
> POST /services/AuthenticationAdmin HTTP/1.1
> Host: localhost:9443
> User-Agent: curl/7.43.0
> Accept: */*
> SOAPAction: urn:login
> Content-Type: text/xml
> Content-Length: 467
* upload completely sent off: 467 out of 467 bytes
< HTTP/1.1 200 OK
< Set-Cookie: JSESSIONID=E74FD5AB56A283CD261E26A77B9854FB; Path=/; Secure; HttpOnly
< Content-Type: text/xml;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Sun, 12 Jun 2016 03:20:09 GMT
< Server: WSO2 Carbon Server
* Connection #0 to host localhost left intact

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Body>
      <ns:loginResponse xmlns:ns="http://authentication.services.core.carbon.wso2.org">
         <ns:return>true</ns:return>
      </ns:loginResponse>
   </soapenv:Body>
</soapenv:Envelope>

At this time you should also be able to see the following command in the ESB terminal (or the relevant product's terminal)

[2016-06-12 08:52:25,931]  INFO - CarbonAuthenticationUtil 'admin@carbon.super [-1234]' logged in at [2016-06-12 08:52:25,931+0530]

2. From the response, get the JSessionID. This will be used for later requests to authenticate.

JSESSIONID=E74FD5AB56A283CD261E26A77B9854FB

3. If you are trying this with ESB, we can try invoking TemplateAdminService as below.

Request (template.xml):
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://org.apache.synapse/xsd">
   <soapenv:Header/>
   <soapenv:Body>
      <xsd:getTemplates>
         <!--Optional:-->
         <xsd:pageNumber>0</xsd:pageNumber>
         <!--Optional:-->
         <xsd:templatePerPage>200</xsd:templatePerPage>
      </xsd:getTemplates>
   </soapenv:Body>
</soapenv:Envelope>


CURL Command

curl -k -v -H "Cookie: JSESSIONID=7D8798EB0A39E18194E778461F7E3E6D" -H "SOAPAction: urn:getTemplates" -H "Content-Type: text/xml" -d @template.xml https://localhost:9443/services/TemplateAdminService

Response :
*   Trying ::1...
* connect to ::1 port 9443 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: localhost
> POST /services/TemplateAdminService HTTP/1.1
> Host: localhost:9443
> User-Agent: curl/7.43.0
> Accept: */*
> Cookie: JSESSIONID=7D8798EB0A39E18194E778461F7E3E6D
> SOAPAction: urn:getTemplates
> Content-Type: text/xml
> Content-Length: 389
* upload completely sent off: 389 out of 389 bytes
< HTTP/1.1 200 OK
< Content-Type: text/xml;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Sun, 12 Jun 2016 03:33:22 GMT
< Server: WSO2 Carbon Server
* Connection #0 to host localhost left intact

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Body>
      <ns:getTemplatesResponse xmlns:ax2246="http://common.templates.mediation.carbon.wso2.org/xsd" xmlns:ns="http://org.apache.synapse/xsd">
         <ns:return xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ax2246:TemplateInfo">
            <ax2246:artifactContainerName xsi:nil="true"/>
            <ax2246:description xsi:nil="true"/>
            <ax2246:enableStatistics>false</ax2246:enableStatistics>
            <ax2246:enableTracing>false</ax2246:enableTracing>
            <ax2246:isEdited>false</ax2246:isEdited>
            <ax2246:name>getUserID</ax2246:name>
         </ns:return>
      </ns:getTemplatesResponse>
   </soapenv:Body>
</soapenv:Envelope>

Note : 

For all the requests, make sure to use the SOAP 1.1 which is xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" and not xmlns:soap="http://www.w3.org/2003/05/soap-envelope", since this will result in an error invoking the admin services with CURL as  : 

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
      <wsa:Action>http://www.w3.org/2005/08/addressing/soap/fault</wsa:Action>
   </soapenv:Header>
   <soapenv:Body>
      <soapenv:Fault>
         <faultcode>soapenv:VersionMismatch</faultcode>
         <faultstring>Transport level information does not match with SOAP Message namespace URI</faultstring>
         <detail/>
      </soapenv:Fault>
   </soapenv:Body>
</soapenv:Envelope>

Using Service Stubs to Invoke Admin Services

Following is the code segment to invoke to get the templates from ESB server by using the TemplateAdminService stub. 

First step is to authenticate : 

TemplateAdminServiceStub templateAdminServiceStub = new TemplateAdminServiceStub("https://localhost:9443/services/TemplateAdminService");
ServiceClient client = templateAdminServiceStub._getServiceClient();
Options client_options = client.getOptions();
HttpTransportProperties.Authenticator authenticator = new HttpTransportProperties.Authenticator();
authenticator.setUsername("admin");
authenticator.setPassword("admin");
authenticator.setPreemptiveAuthentication(true);
client_options.setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, authenticator);
client.setOptions(client_options);

Next we can invoke the service operation : 

OMElement templateOMElement = templateAdminServiceStub.getTemplate();

Invoking Admin Services in Tenant Mode

To invoke admin services in tenant mode, we only need to change the admin service URL with tenant URL and the username and password to the tenant.

E.g. :
Admin Service URL :  https://localhost:9443/services/t/maheeka.com/TemplateAdminService
Username : admin@maheeka.com
Password : admin123

Troubleshooting

1. Error when invoking admin service

[2016-05-25 16:41:36,054] ERROR - Class System failure.null
java.lang.NullPointerException
at org.wso2.carbon.server.admin.module.handler.AuthorizationHandler.doAuthorization(AuthorizationHandler.java:104)
at org.wso2.carbon.server.admin.module.handler.AuthorizationHandler.invoke(AuthorizationHandler.java:87)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:398)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:145)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-05-25 16:41:36,055] ERROR - AxisEngine System failure.
org.apache.axis2.AxisFault: System failure.
at org.wso2.carbon.server.admin.module.handler.AuthorizationHandler.invoke(AuthorizationHandler.java:93)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:398)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:145)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-05-25 16:41:36,057] ERROR - ServerWorker Error processing POST request for : /services/CarbonAppUploader
org.apache.axis2.AxisFault: System failure.
at org.wso2.carbon.server.admin.module.handler.AuthorizationHandler.invoke(AuthorizationHandler.java:93)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:398)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:145)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Solution : 
ESB or APIM has two types of transports - servlet and passthrough. The admin services are exposed through servlet transport. Therefore, we need to invoke admin services through transport port. E.g. : https://localhost:9443/services/AuthenticationAdmin. The common reason for this error is that the admin service WSDLs are exposed on passthrough port (8243).


July 13, 2015

WSO2 ESB with SAP in OSX

Configure WSO2 ESB for SAP

Refer below steps while following the official documentation [1] to do the configurations in OSX.

1. Download SAP JCO for OSX from http://service.sap.com/connectors

2. Copy the sapjco3.jar and sapidoc3.jar files to [ESB_HOME]/repository/components/lib folder

3. Setup classpath to SAP as follows : 
export LD_LIBRARY_PATH=/Users/maheeka/sapjco3/
export CLASSPATH=/Users/maheeka/sapjco3/sapjco3.jar
For additional details on configuring classpath refer [2]

4. Enable SAP transport receiver/sender and create *.dest and *.server files as mentioned in documentation [1] (will be referred to as SAP.dest and SAP.server below)

5. Start ESB with the following command
sh [ESB_HOME]bin/wso2server.sh -Djava.library.path=<path_sapjco3>
Give path to sapjco3 folder downloaded in step 1.

[1] https://docs.wso2.com/display/ESB481/SAP+Integration
[2] http://www.mobility-platform.com/en/documentation/administration/system-landscape-connector/sap-connector/sap-jco-installation/

Configure SAP with SAPGUI

SAPGUI is the client used for configuring SAP.

1. Download SAP GUI for OSX : PlatinGUI740_0-20012037.JAR and  (documentation) files from : http://scn.sap.com/community/gui/blog/2013/05/29/sap-gui-730-download
(SAP GUI 7.3.0 requires jdk 7 and SAP GUI 7.4.0 requires jdk 8 minimum)

2. Issue command : java -jar PlatinGUI740_0-20012037.JAR install
(Download PlatinManual_0-20008876.ZIP from the above location for the manual containing installation instructions)

3. Go through the installation wizard to install SAP GUI

4. Configure the following in SAP to connect to ESB via SAP GUI. First configure a connection and do the following steps.

  1. Create a TCP/IP connection and test the connection (RFC Destinations > TCP/IP Connections)
  2. Create a port (Port definition)
  3. Create a logical system (Maintaining Logical Systems)
  4. Create a partner agreement for the logical system (Partner Profiles)
  5. Create outbound and inbound parameters for partner agreement
You can also use transaction codes instead of using the user menu for this step. Transaction codes are available at : http://wiki.scn.sap.com/wiki/display/ABAP/List+of+Transaction+codes?original_fqdn=wiki.sdn.sap.com

Sample IDoc Sender

Use the following proxy as a sample IDOC sender.

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="IDOCProxy"
       transports="http"
       startOnLoad="true"
       trace="disable">
   <description/>
   <target>
      <inSequence>
         <xslt key="idoc.xslt"/>
         <property name="FORCE_SC_ACCEPT" value="true"/>
         <property name="OUT_ONLY" value="true"/>
         <send>
            <endpoint>
               <address uri="idoc:/SAP"/>
            </endpoint>
         </send>
      </inSequence>
   </target>
</proxy>


"idoc.xslt" will create a IDOC payload and the proxy will send the request to SAP.

Invoke the proxy with : curl -v -X POST "http://localhost:8280/services/IDOCProxy"
If the invoke is successful, you can view the response in IDOC List menu.


Troubleshooting

1. Enable tracing for SAP by setting the trace property in *.dest file as : jco.client.trace=1
2. When step (3) and (5) in "Configure SAP with WSO2 ESB" is not done correctly : 
FATAL - CarbonServerManager WSO2 Carbon initialization Failed
java.lang.ExceptionInInitializerError: JCo initialization failed with java.lang.UnsatisfiedLinkError: no sapjco3 in java.library.path
   at com.sap.conn.jco.rt.MiddlewareJavaRfc.<clinit>(MiddlewareJavaRfc.java:229)
   at com.sap.conn.jco.rt.DefaultJCoRuntime.initialize(DefaultJCoRuntime.java:98)
   at com.sap.conn.jco.rt.JCoRuntimeFactory.<clinit>(JCoRuntimeFactory.java:23)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:171)
   at com.sap.conn.jco.JCo.createJCo(JCo.java:52)
   at com.sap.conn.jco.JCo.<clinit>(JCo.java:26)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:171)
   at com.sap.conn.idoc.jco.JCoIDoc.<clinit>(JCoIDoc.java:140)
   at org.wso2.carbon.transports.sap.idoc.DefaultIDocXMLMapper.<init>(DefaultIDocXMLMapper.java:49)
   at org.wso2.carbon.transports.sap.SAPTransportSender.<init>(SAPTransportSender.java:62)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
   at java.lang.Class.newInstance0(Class.java:357)
   at java.lang.Class.newInstance(Class.java:310)
   at org.apache.axis2.deployment.AxisConfigBuilder.processTransportSenders(AxisConfigBuilder.java:688)
   at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:124)
   at org.wso2.carbon.core.CarbonAxisConfigurator.populateAxisConfiguration(CarbonAxisConfigurator.java:308)
   at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:188)
   at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64)
   at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65)
   at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:398)
   at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
   at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:77)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
   at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
   at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:347)
   at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
   at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
   at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
   at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
   at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
   at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
   at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
   at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
   at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
   at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
   at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
   at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
   at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
   at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
   at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
   at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
   at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
   at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1267)
   at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1186)
   at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1081)
   at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
   at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
   at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

March 2, 2015

WSO2 ESB - Running Integration Tests

Integration tests for WSO2 ESB is available in product-esb repository at https://github.com/wso2/product-esb.

You can clone this module and build ESB product with mvn clean install. The product will be created in [PRODUCT_ESB]/modules/distribution/target folder as wso2esb-[version].zip file.

This will also run all the integration tests for the product and you can find the surefire reports for the product at [PRODUCT_ESB]/modules/integration/tests-integration/[relevant_module_folder]/target/surefire-reports folder.

If you are executing tests by module, you can navigate to the relevant module and issue mvn clean install. However, before doing so, issue a mvn clean install -Dmaven.test.skip=true at product-esb level, to create the distribution pack. At test execution time, the test framework extracts this pack and starts the ESB server to deploy the required artifacts or make the required configuration changes for executing the tests.

Make sure you shutdown any running instances of a WSO2 product or a Axis2 Server instance or any other before running the tests, as it might interrupt with the servers that will be started and stopped during test execution.

Debugging Integration Tests

In order to debug integration tests, do the following :
  1. Navigate to required module or product-esb folder and issue mvn clean install -Dmaven.surefire.debug
  2. Notice the following in console :
    Listening for transport dt_socket at address: 5005
  3. Now click on debug button or Run>Debug in IntellijIDEA or Eclipse (Notice that you need to do a remote debugging here. Refer http://blog.maheeka.me/2014/11/remote-debugging-wso2-esb.html )
  4. You can apply breakpoints, watch expressions as you would in any normal debug scenario

Useful TestNG Tips

These integration tests use TestNG as the test engine.

Test suites and test cases to be run is included in the testng.xml. This testng.xml file can be found at [PRODUCT_ESB]/modules/integration/tests-integration/[relevant_module_folder]/src/tests/resources/testng.xml.

The tests can be included in package level or classes or even method level. Tests can also be excluded in debug Refer [1] to read up on TestNG. Following are some useful tips on TestNG.

1. To define a new test suite, add the following within <suite> tag of testng.xml :
   <test name="[TestSuite name]" preserve-order="true" verbose="2">
        <packages>
            <package name="[package containing tests]"/>
        </packages>
   </test>

set preserver-order to true, if you want to run the tests in the order specified in package or classes.

2. Instead of package, if you want to add classes :
   <test name="[TestSuite name]" preserve-order="true" verbose="2">
        <classes>
            <class name="[fully qualified class name]"/>
            <class name="[fully qualified class name]"/>
            <class name="[fully qualified class name]"/>
        </classes>
    </test>

3. If you want to add specific test methods of a class :
  <test name="[TestSuite name]" preserve-order="true" verbose="2">
        <classes>
            <class name="[fully qualified class name]"/>
                <methods>
                    <include name="[test method name]" />
                    <include name="[test method name]" />
                </methods>
            </class>
        </classes>
    </test>

4. If you want to exclude specific methods of a class :
  <test name="[TestSuite name]" preserve-order="true" verbose="2">
        <classes>
            <class name="[fully qualified class name]"/>
                <methods>
                    <exclude name="[test method name]" />
                    <exclude name="[test method name]" />
                </methods>
            </class>
        </classes>
    </test>

4. You can exclude or include all test methods by using ".*" as regex value for name :
    <exclude name=".*" />
    <include name=".*" />

5. To disable a test suite, use enabled property (true/false) :
    <test name="[TestSuite name]" preserve-order="true" verbose="2" enabled="false">
        ....
    </test>

6. The test suites run in the order specified in the testng.xml. However, the order of the surefire report is different to this

7. Test methods can be identified by @Test annotation in the Java class

8. You can also disable test methods in Java class
    @Test(groups = {"wso2.esb"}, description = "...", enabled = false)
    public void testMethod() {
        .....
    }
Refer next post for notes on writing ESB integration tests.

Reference : 
  1. http://testng.org/doc/documentation-main.html

November 13, 2014

Remote Debugging WSO2 ESB with IntellijIdea and Eclipse

This post assumes you are familiar with debugging concepts. The remote debugging will be explained using WSO2 ESB product.

What happens in remote debugging is that the debugging information of a running application is communicated with an IDE that has the source of the application. The application is not running from the source, and thus the term "remote" debugging. Similar to any normal debug scenario, where you would be running the application in debug mode from the IDE, you can have breakpoints, watch expressions, etc.

Read more about how remote debugging works in JVM at [1].

Let's proceed on how to debug WSO2 ESB.

Setup the Source for Debugging WSO2 ESB

There are a number of components in WSO2 stack that contribute to the complete WSO2-ESB product. Some of these components are : wso2-synapse, carbon-mediation, wso2-axis2-transport, carbon4-kernel, etc. You can download source of all these components at https://github.com/wso2.

Decide on the component(s) you need to debug, and import them to your preferred IDE (Eclipse or IntellijIdea). To do this :
  1. Clone the required component from the git repository : git clone <repository_url>
  2. Build the component : mvn clean install or mvn clean install -Dmaven.test.skip=true (to skip tests)
  3. To setup as an IDE project : mvn idea:idea (for IntellijIdea) or mvn eclipse:eclipse (for Eclipse)
  4. Add breakpoints as needed
  5. If debugging multiple components, 
    • Eclipse : Import as project to the same workspace and add breakpoints
    • Idea : Import as module to the same window and add breakpoints
    • Optionally, at the time of debug, when debugging takes you to the .class files, you can attach the relevant components as sources with "attach source" option and continue.
Note : 
If you want to debug message flow on ESB, add a breakpoint to receive(org.apache.axis2.context.MessageContext mc) method in org.apache.synapse.core.axis2.ProxyServiceMessageReceiver class in wso2-synapse. This is the start point to the message flow on ESB.

Remote Debugging WSO2-ESB

Step 1 : Start WSO2-ESB in debug mode

$ sh bin/wso2server.sh debug 5005
Please start the remote debugging client to continue...
JAVA_HOME environment variable is set to /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
CARBON_HOME environment variable is set to /Users/maheeka/wso2esb-4.8.1
Listening for transport dt_socket at address: 5005

We have started the ESB in debug mode, and it listens now on port 5005 for a remote debugging client. 

Step 2 : Start Remote Debugging Client (IDE)

IntelliJIdea
  1. Run > Edit Configurations
  2. On the left panel, click on "+" and add a new Remote Configurations. Notice the port to be same as you give when starting ESB.
  3. Then Run > Debug 'wso2-synapse' to start the listener
  4. The application will now run as normal and stop at the breakpoints you have added




Eclipse
  1. Run > Debug Configurations
  2. Add a new Remote Java Application Configuration. Notice the port number to be same as you give when starting ESB.
  3. Click on Debug to start debugging 

November 10, 2014

Building WSO2 ESB 4.8.1 from source in OSX

WSO2 products are all built on top of WSO2 Carbon Kernel. This is the common ground for all products. Turing is the platform release name for WSO2 Carbon 4.2.0 and WSO2 ESB 4.8.1 is based on turing. Not all products of the WSO2 platform does not get released at the same time. Therefore, these products are released in chunks. ESB 4.8.1 comes under chunk 7 of the turing platform. Refer the release matrix to get more info on this [2].
Prerequisites : 
1. Install Java 1.6 and Maven 3
       If you have only 1.7 installed, you will have to install 1.6 on OSX. Unfortunately, there is no direct installation for 1.6. But this sorted out for me  since jdk 1.6 was automatically installed when I installed IntellijIdea
       At the moment, all WSO2 products are compiled on Java 1.6 and is supported up to 1.7. This is why we need to build the source on 1.6.
2. Checkout Source (Orbit, Kernel, Platform)

Building the Source : 


Start building in order Orbit > Kernel > Platform (chunk 07) repos respectively.


To build ESB 4.8.1, you need to build wso2carbon-platform/product-releases/chunk-07 as Platform in above.


The distribution will be created in :

wso2carbon-platform/products/esb/4.8.1/modules/distribution/target/(esb.zip)

When building, since it will take a lot of time, and to avoid unit test failures from building the product, skip unit tests when building.


mvn clean install -Dmaven.test.skip=true


Build ESB only :

If you need to build only ESB, build at wso2carbon-platform/products/esb with mvn clean install -Dmaven.test.skip=true


Another option is to comment out other product modules in platform/trunk/pom.xml and rebuilding in Orbit > Kernel > Platform in order.


Encountered Errors : 
  1. WSS4 maven install fails > You need to compile in Java 1.6
  2. [WARNING] Error injecting: org.apache.maven.reporting.exec.DefaultMavenReportExecutor > Set maven-site plugin version to 3.3 in all places [4]
Environment :