Message Tracing with WSO2 API Manager

The latest version of WSO2 API Manager can be downloaded from here. We could deploy API Manager in a distributed manner by separating Gateway, KeyManager, Publisher and Store servers. All the API requests are first comes to the gateway server. Then gateway calls to the key-validation server to validate the key. Only if key validation is successful, gateway allows for the back-end call.

It's important to have message details logging capability in gateway and key-manager servers, so that we can track the request and response key validation details for all requests. And it should have a same activity_id for a particular message in the key-manager and gateway servers. So that we can co-relate the same message at the key-manager and gateway nodes when doing the analysis.

Message tracing feature has been included in the API-Manager-1.6.0 release. How this message tracing feature track the same message in different servers is, when the request comes into a particular server, it adds a http header called "activityID" if this header is already not there in the request. Same is doing for the outgoing response message as well.  It checks the incoming message context and if the "activityID" header is available, then same activityID is added to the outgoing message as well. So that the messages can be co-relate using this activityID header. This feature can be used in any product to track the message. As an example say message flow is ESB -> Application-Server -> Data-Services-Server, then we can install this feature into each server and track the message.

Coming back to the API-Manager scenario, with this message tracing feature, you could log the message info in gateway and key-manager servers and/or publish the message tracing info into Business Activity Monitoring server (BAM) as streams. Once you published message info into BAM server, those streams are saved in to Cassandra keyspace and we could use them for analysis.

How to enable message tracing to log details in gateway and key-manager servers ?

1. Login to the Management Console of Gateway server. ( eg: https://localhost:9443/carbon )
Go to "Configure -> Message Tracing". This will give you the configuration UI as shown in bellow and Select the "Enable Message Tracing" option.

2. Now go to the "Configure -> Logging". In the prompt UI, give the "APIManagerExtensionHandler" as input value to "Filter Loggers by" field and click on "Contains" button. This will give you the following filter result and change the "org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler" log level to "DEBUG" as shown.

3. Same way, go to the Management console of Key-Manager server and enable the "Massage Tracing" as we did in the step-1. Now go to the "Configure -> Logging" and set the log level to "DEBUG" for class "org.wso2.carbon.apimgt.keymgt.service.APIKeyValidationService" as we did in step-2.

Now, in each request call, we can see the message tracing logs in gateway and key-manager servers as shown bellow.

Gateway Server Logs:

Key-Manager Server Logs:

Note that same transaction_id has been printed in both gateway and key-manager servers for a given request.

We are done with logging. Further, if you need we could publish these tracing info in to BAM server as well.

How to configure message tracing to publish info into Business Activity Monitoring Server?

We need to have a remote BAM server running. You can download the latest version of BAM sever from here. Login to the management console of Gateway and Key-Manager severs and go to "Configure -> Message Tracing" and enable "Enable BAM Event Publishing" option and provide the BAM server details as shown bellow. (The following configuration is provided when BAM server is running with port offset 3).

Now, there will be streams publish by the gateway and key-manager servers to BAM server in each request. You can connect to the Cassandra server and check the published message info. Login to the management console of BAM server and go to "Tools > Cassandra Explorer > Connect to Cluster".

 Once you connect to the Cassandra server as mentioned above, you can see that there is a column family called "BAM_MESSAGE_TRACE" has been created. This column family contains all the message tracing data.

You can click on the "BAM_MESSAGE_TRACE" and explore the column family data.

Also, we can use "Activity Dashboard" comes with BAM server to analyse the data. Using this dashboard, we can filter the messages using certain criteria and it will group the messages that having same activity_id. This document describes how we can use the "Activity Dashboard".

How to change root context of the server redirect to API Store ?

WSO2 API Manager consists of two separate UIs for API Publishers and API Subscribers.  If the server hosted on a domain as apis.com on default ports, URLs of  store and publisher will be as follows.

API Store - https://apis.com:9443/store/
API Publisher - https://apis.com:9443/publisher/

If the user attempts to access https://apis.com:9443, it will redirect to the API Publisher URL by default. But if you need this root URL to be redirect to API Store instead of the API publisher, we could do it with the following changes.

In "{AM_HOME}/repository/components/plugins" directory contains a bundle called "org.wso2.am.styles_1.x.x.jar". Open this jar and find its component.xml resides in 'META-INF' directory. There, you could see 'default-context' has pointed to 'publisher'. You just need to change it to 'store' as follows and update the jar. Then restart the server.

<context>
       <context-id>default-context</context-id>
       <context-name>store</context-name>
       <protocol>http</protocol>
       <description>API Publisher Default Context</description>
</context>

Now, if you try to access the root URL of the server https://apis.com:9443, it will redirect to the API Store (https://apis.com:9443/store/).

Integrating WSO2 API Manager with BAM and CEP - (Real time data analyzing for WSO2 API Manager)

WSO2 API Manager provides complete solution for creating, publishing and managing all aspects of an API and its life cycle. One of the most common use case in API management is to statistic analyzing for APIs, like number of requests for a particular API within a given time period, number of subscribers for APIs, number of faulty requests happened for an API etc. At the same time, it's important to have real time data analyzing capability as well, like we may need to get a notification message if particular user exceed the throttling limit of an API etc.

WSO2 Complex Event Processing Server (WSO2CEP) is used to analyze the events and acts them on the real time. While WSO2 Business Activity Monitor (WSO2BAM server) is used to post process the events and store and analyze the data.

In this post, we are going to write a sample that will show how we can integrate the WSO2CEP with API Manager to do the real time event processing and same time we are going to integrate WSO2BAM as well into the same API Manager instance for statistic analysis.

Sample Scenario

API Manager will publish the events to CEP. CEP is doing two things.
1. Forward data to BAM to use for statistic calculations.
2. If same API invokes more than 5 times within 1 minute from the same application, this will send an email notification to a given email address.

Configure the Setup

First download the wso2am, wso2cep and wso2bam servers and extract them into somewhere in file system.

1. Configure WSO2 API Manager Server

The data-publisher in API Manager, publish events to CEP in each client request. Following configuration in api-manager.xml is used to do this. Usually when we configuring APIM with BAM, we provide BAM thrift port and url here. Instead of BAM server thrift port, we have provide CEP thrfit port/url here. (i.e 7613 ;because later in this sample, we are going to start the CEP server with offset 2).

i. Open the api-manager.xml (located in "{AM_HOME}/repository/conf/" directory) and modify the "<APIUsageTracking>" section as follows;

<APIUsageTracking>
 
        <Enabled>true<⁄Enabled>       
        <PublisherClass>
        org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher
        <⁄PublisherClass>

        <!--
            Thrift port of the remote CEP server.
        -->
        <ThriftPort>7613<⁄ThriftPort>

        <!--
            Server URL of the remote CEP server.
        -->
        <BAMServerURL>tcp:⁄⁄localhost:7613⁄<⁄BAMServerURL>

        <BAMUsername>admin<⁄BAMUsername>
        <BAMPassword>admin<⁄BAMPassword>
        <DataSourceName>jdbc⁄WSO2AM_STATS_DB<⁄DataSourceName>

        <GoogleAnalyticsTracking>
             <Enabled>false<⁄Enabled>
             <TrackingID>UA-XXXXXXXX-X<⁄TrackingID>
       <⁄GoogleAnalyticsTracking>

    <⁄APIUsageTracking>
 
        

ii. Open the master-datasources.xml file (located in "{AM_HOME}/repository/conf/datasources/" directory) and change the jdbc url path of WSO2AM_STATS_DB to point absolute path of BAM as given in following example. "WSO2AM_STATS_DB" is the data-source that pointed to BAM stats database. This is used for the statistic analyzing.

<datasource>
  <name>WSO2AM_STATS_DB<⁄name>
  <description>The datasource used for getting statistics to API Manager<⁄description>
  <jndiConfig>
        <name>jdbc⁄WSO2AM_STATS_DB<⁄name>
    <⁄jndiConfig>
  <definition type="RDBMS">
  <configuration>
         <url>jdbc:h2:⁄home⁄dinusha⁄BAM_CEP_AM⁄wso2bam-2.3.0⁄repository⁄database⁄APIMGTSTATS_DB;AUTO_SERVER=TRUE<⁄url>
         <username>wso2carbon<⁄username>
         <password>wso2carbon<⁄password>
         <driverClassName>org.h2.Driver<⁄driverClassName>
         <maxActive>50<⁄maxActive>
         <maxWait>60000<⁄maxWait>
         <testOnBorrow>true<⁄testOnBorrow>
         <validationQuery>SELECT 1<⁄validationQuery>
         <validationInterval>30000<⁄validationInterval>
    <⁄configuration>
 <⁄definition>
<⁄datasource>

We are done with the API Manager related configurations.

2. Configure WSO2 CEP Server

i. Open the carbon.xml file (located in {CEP_HOME}/repository/conf/ directory) and change the port offset parameter "<Offset>" to 2. So the CEP server is going to run in the 9445 https port.

ii. We are going to configure three brokers in CEP. A broker operates in between an event source and the CEP Engine, facilitating their connection. You can find the more details about CEP brokers from here. These brokers are going to use later by the buckets. Open the broker-manager-config.xml file (located in {CEP_HOME}/repository/conf/ directory) and add the following broker configurations.

<brokerConfiguraton name="externalAgentBroker" type="agent">
        <property name="receiverURL">tcp:⁄⁄localhost:7612<⁄property>
        <property name="authenticatorURL">ssl:⁄⁄localhost:7712<⁄property>
        <property name="username">admin<⁄property>
        <property name="password">admin<⁄property>
    <⁄brokerConfiguraton>

    <brokerConfiguraton name="emailBroker" type="e-mail">
        <property name="defaultSubject">Message from WSO2CEP<⁄property>
    <⁄brokerConfiguraton>

    <brokerConfiguraton name="APIM_Broker" type="agent">
        <property name="receiverURL">tcp:⁄⁄localhost:7613<⁄property>
        <property name="authenticatorURL">ssl:⁄⁄localhost:7713<⁄property>
        <property name="username">admin<⁄property>
        <property name="password">admin<⁄property>
    <⁄brokerConfiguraton>

iii. Total processing of received events and triggering of new events happens through the buckets. You can find more details about CEP buckets from here. We are going to create four buckets in CEP for this sample as follows.

APIMBucket.xml, APIMFaultBucket.xml, APIMResponseBucket.xml : These three buckets are used to stat publishing for BAM. Note that in all these three buckets have used "APIM_Broker" as the input broker which has pointed the broker in the CEP server and "externalAgentBroker" as the output broker which has pointed the broker in the BAM server.

APIMRealTimeBucket.xml : This bucket is use for show the real time data analyzing (Send an email). This bucket has used the "APIM_Broker" as the input broker and "emailBroker" as the output broker. This will read the some of the data from the request event stream ("org.wso2.apimgt.statistics.request/1.0.0") and do some calculations in the output and send an email using the "emailBroker".

You can download the bucket configurations from here and deploy them in  "{CEP_HOME}/repository/deployment/server/cepbuckets/" directory.


3. Configure WSO2 BAM Server

i. Open the carbon.xml file (located in {BAM_HOME}/repository/conf/ directory) and change the port offset parameter "<Offset>" to 1. So the BAM server is going to run in the 9444 https port.

ii. You can find the toolbox (API_Manager_Analytics.tbox) that used for statistic analysis in "{AM_HOME}/statistics/" directory. Copy this toolbox into the "{BAM_HOME}/repository/deployment/server/bam-toolbox/" directory.

iii. Open the master-datasources.xml file (located in "{BAM_HOME}/repository/conf/datasources/" directory) and add the following datasource configuration.

<datasource>
  <name>WSO2AM_STATS_DB<⁄name>
  <description>The datasource used for getting statistics to API Manager<⁄description>
  <jndiConfig>
        <name>jdbc⁄WSO2AM_STATS_DB<⁄name>
    <⁄jndiConfig>
  <definition type="RDBMS">
  <configuration>
         <url>jdbc:h2:repository⁄database⁄APIMGTSTATS_DB;AUTO_SERVER=TRUE<⁄url>
         <username>wso2carbon<⁄username>
         <password>wso2carbon<⁄password>
         <driverClassName>org.h2.Driver<⁄driverClassName>
         <maxActive>50<⁄maxActive>
         <maxWait>60000<⁄maxWait>
         <testOnBorrow>true<⁄testOnBorrow>
         <validationQuery>SELECT 1<⁄validationQuery>
         <validationInterval>30000<⁄validationInterval>
    <⁄configuration>
 <⁄definition>
<⁄datasource>

We are done with the all configuration changes. Now you can start the all three servers and verify the scenario. (There is a pre-configured setup with all these changes available to download here. If you are going to use it, please read the readme.txt provided there and change the data-sources and buckets to fit with our paths.)

Test and verify the setup

1. Login to the API Manager. Create and publish an API. Subscribe to it and invoke.
2. Stats should have shown in APIM stats page.
3. Invoke same API more than 5 times within 1 minute. Email notification should goes to email address defined in the APIMRealTimeBucket.xml.

How to configure API visibility level when multiple user stores are configured - WSO2 API Manager

WSO2 API Manger provides set of functionalities for API Publishers. API Visibility control functionality is one of them, where API visibility can be either 'Public' or 'Restricted'. If it is set to 'Public' then the API is visible through the API Store to all API subscribers and also anonymous users of the API store.  And, If it set to 'Restricted', then only the specified users who having the required role permission can see the API through the API Store.

We don't need to do any configuration changes to use this feature in normal scenarios (i.e standalone API Manager distribution). But, there is a some special case where we need to do some additional configurations to work this functionality; In a distributed setup where API publisher and API Store are separated and they have pointed to use different user-stores, then we need to do some configuration changes to work this  functionality.

In a production deployment of API Manager, depending on company security policies, you can either use same user-store for both API publisher and API store, or use two separate user-stores for them. Usually in most of production setups they tends to use internal user-store for API publisher and some external user-store for API Store. Even though, we use separate user-stores for API Publisher and API Store, we need to point them to a same permission store to work API visibility functionality. (Otherwise, even you have made the API visibility as public ,this API will not available in API Store, without login to it).  Following guide will explain how we can configure API Publisher and API Store nodes to use same permission store while they are pointed to use different user-stores.

User Manager has two types of storages. One is user store and the other is permission store. All resource permissions are stored in the permissions store while users are stored in user store. Usually when we pointed to two different user-stores, it will use separate permission stores as well, unless we have explicitly configured it to use same permission store. Because of that, when publisher grant permission on api registry resource, it will reflect only on the publisher permission store. Those permission will not available on the permission store of API Store.

So we need to share the same permission store used in Publisher with API Store as well. (Permission store contains only Role-Permission mapping. Sharing permission store will not share the the user details).

The global data-source defined in user-mgt.xml (if you open the user-mgt.xml, it can be found inside the top most <Configuration> section.) is used for permission store. When you don't have defined the <Property name="dataSource"> in JDBCUserStoreManager, the same data-source defined above for permission store will use for user store as well. But, you can defined some other data-source inside <UserStoreManager> for user store while using , something else for permission store.

You may need to do the following configuration changes to use same permission store of publisher within the API Store,

1. In 'master-datasources.xml' of API Store, add new data-source entry to use user-manager data-source of publisher.
eg: If the user-manager data source defined in Publisher is as follows, then add this data-source to 'master-datasources.xml' of API Store as well.

<datasource>
            <name>pubdb</name>
            <description>The datasource used for registry and user manager</description>
            <jndiConfig>
                <name>jdbc/pubdb</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://localhost:3306/pubdb?autoReconnect=true&amp;relaxAutoCommit=true&lt;/url>
                    <username>root</username>
                    <password>root</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>

2. In user-mgt.xml of API Store, change the data-source property(i.e <Property name="dataSource">) defined in '<Configuration>' section to use the above data source. (The data-source defined in this <Configuration> section, is used for permission store. By changing configuration as above we, point permission store of API store to permission store of publisher).

 eg:
<Configuration>
                <AdminRole>admin</AdminRole>
                <AdminUser>
                     <UserName>admin</UserName>
                     <Password>admin</Password>
                </AdminUser>
            <EveryOneRoleName>everyone</EveryOneRoleName> <!-- By default users in this role sees the registry root -->
<!-- <Property name="dataSource">jdbc/storedb</Property> -->
<Property name="dataSource">jdbc/pubdb</Property>
            <Property name="MultiTenantRealmConfigBuilder">org.wso2.carbon.user.core.config.multitenancy.SimpleRealmConfigBuilder</Property>
        </Configuration>

3. Add the, user-store data source of API Store inside the JDBCUserStoreManager configuration. (i.e <Property name="dataSource">jdbc/storedb</Property>)
eg:
<UserStoreManager class="org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager">
<Property name="dataSource">jdbc/storedb</Property>
<Property name="ReadOnly">false</Property>
            <Property name="MaxUserNameListLength">100</Property>
            <Property name="IsEmailUserName">false</Property>
            <Property name="DomainCalculation">default</Property>
            <Property name="PasswordDigest">SHA-256</Property>
            <Property name="StoreSaltedPassword">true</Property>
            <Property name="UserNameUniqueAcrossTenants">false</Property>
            <Property name="PasswordJavaRegEx">^[\S]{5,30}$</Property>
            <Property name="PasswordJavaScriptRegEx">^[\\S]{5,30}$</Property>
<Property name="UsernameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\&lt;&gt;,\'\"]{3,30}$</Property>
<Property name="UsernameJavaScriptRegEx">^[\\S]{3,30}$</Property>
<Property name="RolenameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\&lt;&gt;,\'\"]{3,30}$</Property>
<Property name="RolenameJavaScriptRegEx">^[\\S]{3,30}$</Property>
            <Property name="UserRolesCacheEnabled">true</Property>
<Property name="maxFailedLoginAttempt">0</Property>
        </UserStoreManager>

By doing above configuration changes, we will use same permission store for API Store and Publisher and you should be able to see the APIs that create with public visibility, without login to API Store.

Domain restrictions for API invocations with WSO2 API Manager

WSO2 API Manager provides OAuth2 bearer token as default authentication mechanism. But as the API providers, there can be situations where they need to restrict users from some domains being invoking their APIs and only allow users from the certain set of domains to invoke their APIs. "Domain restriction feature" added to the WSO2 API Manager provides this capability to API providers.

With the "Domain Restriction" feature, an application key will be associated with a set of domains, and requests from only those specified domains will be allowed through the gateway.

How to use domain restriction feature ?

• Access allowing domain/domains can be defined at the time, when you generate the application key. It can be either one domain or set of domains. If you are defining set of domains, they should provide as comma separated list in the 'Access Allowing Domains' field in the application key generation UI. If the APIs should not restrict for any domains, then no need to define the 'Access Allowing Domains' field.  

• Once you define the access allowing domains at the token generate time, later you can modify the domain list. 

• Client request message should contain a http header to inform his domain. Administrator can configure the header name, which is going to use for domain validation through the api-manager.xml (File located at {APIM_HOME}/repository/conf directory). There is a new element called '<ClientDomainHeader>' is added to the '</APIGateway>' section in the api-manager.xml. 

• If you define as, <ClientDomainHeader>domain</ClientDomainHeader>, then api invocation request should contain a http header called 'domain'. (Sending this header is not mandatory. It's required only if API access is allowed for certain set of domains).
• eg: curl -v -H "Authorization: Bearer xxx" -H "domain: wso2.com" http://localhost:8280/twitter/1.0.0/search.atom?q=cat

• Value of the domain header send with client request is validated against the set of domains defined by the API subscriber. API access will allow only if client domain matches with these domains. Otherwise client will be send a authentication failure message.

How to write a Custom Authentication Handler for WSO2 API Manger ?

WSO2 API Manager provide OAuth2 bearer token as its default authentication mechanism. But we can extend it to support any of the authentication mechanism other than the bearer token authentication.

This post explains, how we can write a custom authentication handler for WSO2 API Manager. 

Implementation of the default authentication handler used in WSO2 API Manger can be found here. As the same way, we can write our own authentication handler class by extending 'org.apache.synapse.rest.AbstractHandler' class.

In the authentication handler implementation class, we have to implement the 'handleRequest()' and 'handleResponse()' methods. See the sample 'CustomAPIAuthenticationHandler.java' class given bellow. 

 
package org.wso2.carbon.apimgt.gateway.handlers.security;

import org.apache.synapse.MessageContext;
import org.apache.synapse.core.axis2.Axis2MessageContext;
import org.apache.synapse.rest.AbstractHandler;

import java.util.Map;

public class CustomAPIAuthenticationHandler extends AbstractHandler {

    public boolean handleRequest(MessageContext messageContext) {
        try {
            if (authenticate(messageContext)) {
                return true;
            }
        } catch (APISecurityException e) {
            e.printStackTrace();
        }
        return false;
    }

    public boolean handleResponse(MessageContext messageContext) {
        return true;  
    }

    public boolean authenticate(MessageContext synCtx) throws APISecurityException {
        Map headers = getTransportHeaders(synCtx);
        String authHeader = getAuthorizationHeader(headers);
        if (authHeader.startsWith("userName")) {
            return true;
        }
        return false;
    }

    private String getAuthorizationHeader(Map headers) {
        return (String) headers.get("Authorization");
    }

    private Map getTransportHeaders(MessageContext messageContext) {
        return (Map) ((Axis2MessageContext) messageContext).getAxis2MessageContext().
                getProperty(org.apache.axis2.context.MessageContext.TRANSPORT_HEADERS);
    }
}

• Build the above class and copy the jar file to <AM_HOME>/repository/components/lib folder where <AM_HOME> is the root of the WSO2 API Manager distribution.
•  You can engage this handler to the API through the Management Console. Log in to the console and select 'Service Bus > Source View' in the 'Main' menu.
•  In the ESB configuration that opens, you can see following line as the first handler in the API, which is the current authentication handler used in API Manager. 

 
<handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>

Replace it with the one that we created.

  
<handler class="org.wso2.carbon.apimgt.gateway.handlers.security.CustomAPIAuthenticationHandler"/>