HTTPClient

java.lang.Object
- com.dkfqs.tools.http.HTTPClient

public class HTTPClient
extends java.lang.Object

A HTTP client which supports to measure detailed, far-reaching performance data. There are no static settings or static configurations so that multiple instances can run independently in parallel of each other.
Each single instance has an own connection pool and an own processing pool and is able to to send and receive multiple HTTP requests/responses in parallel to multiple Web servers.

Timeouts and Error Handling
This HTTP client will never hang forever, but there is no generic timeout implemented. Instead of this, the default and configurable timeouts are divided into 3 parts which correspond to the different protocol levels:

TCP/IP connect timeout
SSL Handshake timeout (for https connections only)
HTTP processing timeout

Alternatively, you can call the method setOverallTimeout(..) which calculates estimated timeout values for these 3 protocol levels.
If any of these timeouts occurs then a corresponding error exception is set in the HTTP request. Therefore, when receiving a HTTP response, you have first to check if no error exception has occurred at the HTTP request, before processing to the received HTTP response data.
Example: if (httpResponse.getHttpRequest().hasErrorException()) {...}.
Note that any received HTTP response error code such as '400 Bad Request' or '500 Internal Server Error' is part of a normal received HTTP response and will not be reflected as an request error exception. Contrariwise, if connecting to a dead HTTP server fails, then an error exception is set in the HTTP request (for example 'host not reachable exception' or 'unknown host exception').
Conclusion / Rule of Thumb: The error exception is set in the HTTP request if no HTTP response has received.

SSL Server Certificate Verification
The verification of the SSL server certificate(s) is disabled by default. Therefore the HTTPClient can also connect to HTTPS servers which contains an invalid or an expired SSL server certificate.
To override this behavior, you can set an own trust manager by using the method setTrustManager(TrustManager[]).

Other Features

Integrated automatic HTTP cookie handler
Support to set an outbound IP address
Support to set a SSL client authentication, for example extracted from a PKCS#12 client certificate
Support to limit the max. used SSL version
Getting the number of outstanding HTTP requests at realtime (in depth performance data)
Pluggable Log Adapter

Programming Examples
An instance of HTTPClient can be programmed either in a synchronous way or in an asynchronous way, but the two ways cannot be mixed in the same instance. If you set a HTTPClientAsyncResponseHandler, then the methods for getting synchronous HTTP responses are throwing a HTTPClientException.


      // initialization
      StdoutLogAdapter logAdapter = new StdoutLogAdapter();   // you may implement your own log adapter
      logAdapter.init(null);
      logAdapter.setLogLevel(LogAdapterInterface.LOG_DEBUG);
      HTTPClient httpClient = new HTTPClient(logAdapter);
      httpClient.setHTTPProcessingTimeout(30000);

      // --------------------

      // synchronous programming example 1
      HTTPRequest httpRequest = httpClient.newRequest("GET", "https://www.dkfqa.com");
      long requestId = httpClient.sendAsyncRequest(httpRequest);
      HTTPResponse httpResponse = httpClient.waitAndGetResponse(requestId);
      if (httpResponse.getHttpRequest().hasErrorException()) {
          System.out.println("HTTP request failed: " + httpResponse.getHttpRequest().getErrorException());
      }
      else {
          System.out.println(httpResponse.getHttpResponseContent().getContentAsString());
      }

     // synchronous programming example 2
     HTTPRequest httpRequest1 = httpClient.newRequest("GET", "https://www.dkfqa.com");
     httpClient.sendAsyncRequest(httpRequest1);
     HTTPRequest httpRequest2 = httpClient.newRequest("GET", "https://www.dkfqa.com/content/logo183x32.gif");
     httpClient.sendAsyncRequest(httpRequest2);
     httpClient.waitAllResponses();
     HTTPResponse httpResponse = httpClient.getNextResponse();
     while (httpResponse != null) {
          if (httpResponse.getHttpRequest().hasErrorException()) {
              System.out.println("HTTP request failed: " + httpResponse.getHttpRequest().getErrorException());
          }
          else {
              System.out.println(httpResponse.getHttpResponseContent().getContentAsString());
          }
          httpResponse = httpClient.getNextResponse();
     }

     // --------------------

     // asynchronous programming example

     // the (implemented) interface for instantly processing the received HTTP responses
     public class MyAsyncResponseHandler implements HTTPClientAsyncResponseHandler {
          public void onAsyncResponse(HTTPResponse httpResponse) throws Exception {
               HTTPRequest httpRequest = httpResponse.getHttpRequest();
               if (httpRequest.hasErrorException()) {
                    System.out.println("HTTP request failed for " + httpRequest.getUrl() + " | " + httpRequest.getErrorException());
               }
               else {
                    System.out.println("Receive HTTP response for request ID " + httpRequest.getId() + " = " + httpResponse.getHttpResponseHeader().getHttpStatusCode() + " " + httpResponse.getHttpResponseHeader().getReceivedHttpStatusText());
               }
          }
     }

     httpClient.setAsyncResponseHandler(new MyAsyncResponseHandler()); // set the (implemented) interface for instantly processing the received HTTP responses
     HTTPRequest httpRequestA = httpClient.newRequest("GET", "https://www.dkfqa.com");
     long requestIdA = httpClient.sendAsyncRequest(httpRequestA);
     HTTPRequest httpRequestB = httpClient.newRequest("GET", "https://www.dkfqa.com/content/logo183x32.gif");
     long requestIdB = httpClient.sendAsyncRequest(httpRequestB);
     httpClient.waitAllResponses();  // synchronisation point

     // --------------------

     // free all internal resources
     httpClient.closeAbort();

See Also:: setOverallTimeout(int), HTTPPerformanceData, setTrustManager(TrustManager[]), LogAdapterInterface, setAsyncResponseHandler(HTTPClientAsyncResponseHandler)

Constructor Summary

Constructors
Constructor and Description
`HTTPClient(LogAdapterInterface logAdapter)` Create a new instance with 4 processing threads.
`HTTPClient(LogAdapterInterface logAdapter, int numProcessingThreads)` Create a new instance with a specific number processing threads.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addClientAuthKeyManagers(java.lang.String serverHost, int serverPort, javax.net.ssl.KeyManager[] clientAuthKeyManagers)` Add SSL key managers for client authentication, for a particular HTTPS host and port.
`void`	`clearConnectionPool()` Close and remove all network connections from the connection pool.
`void`	`closeAbort()` Free all internal resources and close all connections of the HTTPClient.
`void`	`dumpConnectionPoolToStdout()` For debug purposes only, dump the network connection pool to stdout.
`void`	`dumpProcessingPoolToStdout()` For debug purposes only, dump the HTTP client processing pool to stdout.
`HTTPConnectionStatistic`	`getConnectionStatistic()` Performance data, get the statistic of the network connections used by the HTTPClient.
`HTTPCookieHandler`	`getCookieHandler()` Get access to the cookie handler of the HTTP client.
`int`	`getHTTPProcessingTimeout()` Get the default value for the processing timeout per URL at HTTP level.
`HTTPResponse`	`getNextResponse()` Get the next HTTP response of all completed HTTP requests.
`int`	`getNumOutstandingRequest()` Performance data, get the number of currently outstanding requests (= number of not yet completed, pending requests).
`HTTPResponse`	`getResponse(long httpRequestId)` Get the HTTP response for a particular HTTP request.
`int`	`getSSLHandshakeTimeout()` Get the SSL handshake timeout.
`int`	`getTCPConnectTimeout()` Get the TCP connect timeout.
`long`	`getTotalBytesReceived()` Get the total number of bytes which were received - counted over all ever executed HTTP/S calls.
`long`	`getTotalBytesSent()` Get the total number of bytes which were sent - counted over all ever executed HTTP/S calls.
`HTTPRequest`	`newRequest(java.lang.String httpRequestMethod, java.lang.String url)` Create a new HTTP request, but don't send it yet.
`void`	`removeAllClientAuthKeyManagers()` Remove all SSL key managers for client authentication.
`boolean`	`resumeSuspendedHTTPRequest(long httpRequestId)` Resume a suspended HTTP request.
`long`	`sendAsyncRequest(HTTPRequest httpRequest)` Send a HTTP request as an asynchronous operation.
`void`	`setAsyncResponseHandler(HTTPClientAsyncResponseHandler asyncResponseHandler)` Set the (implemented) interface for instantly processing the received HTTP responses.
`void`	`setClientDebugger(HTTPClientDebuggerInterface clientDebugger)` Set a client debugger.
`void`	`setCountTransmitBytesInterfaces(HTTPClientCountBytesSentInterface bytesSentInterface, HTTPClientCountBytesReceivedInterface bytesReceivedInterface)` Set callback interfaces which are called each time when the HTTP client has sent or received a data fragment to/from a server.
`void`	`setHTTPProcessingTimeout(int httpProcessingTimeoutMillis)` Set the default value for the processing timeout per URL at HTTP level.
`void`	`setLimitSSLVersion(java.lang.String limitSSLVersion)` Set/limit the maximal SSL version.
`void`	`setLocalOutboundAddress(java.net.InetAddress localOutboundAddress)` Set or unset a specific outbound IP address.
`void`	`setOverallTimeout(int overallTimeoutMillis)` Set a synthetically calculated overall timeout for the URL calls The 3 inner timeouts are calculated as follows: At first, the SSL Handshake Timeout is calculated: as 10% of the Overall Timeout but max.
`void`	`setSSLHandshakeTimeout(int sslHandshakeTimeoutMillis)` Set the SSL handshake timeout.
`void`	`setTCPConnectTimeout(int tcpConnectTimeoutMillis)` Set the TCP connect timeout.
`void`	`setTraceRequestContentTypesSet(java.util.HashSet<java.lang.String> traceRequestContentTypesSet)` Set a set of mime types for which the request content data are written to the log adapter at trace level.
`void`	`setTraceResponseContentTypesSet(java.util.HashSet<java.lang.String> traceResponseContentTypesSet)` Set a set of mime types for which the response content data are written to the log adapter at trace level.
`void`	`setTrustManager(javax.net.ssl.TrustManager[] trustManagers)` Set a specific SSL X509 trust manager, used to validate the certificate chain received from a HTTPS server during the SSL handshake.
`void`	`waitAllResponses()` Wait until all HTTP requests are completed.
`HTTPResponse`	`waitAndGetResponse(long httpRequestId)` Wait until a particular HTTP request is completed and get the corresponding HTTP response.
`void`	`waitResponse(long httpRequestId)` Wait until a particular HTTP request is completed.

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - HTTPClient
```
public HTTPClient(LogAdapterInterface logAdapter)
```
    Create a new instance with 4 processing threads.
    
    Parameters:
    
    logAdapter - the log adapter for this instance, or null if logging is disabled
  - HTTPClient
```
public HTTPClient(LogAdapterInterface logAdapter,
                  int numProcessingThreads)
```
    Create a new instance with a specific number processing threads.
    
    Parameters:
    
    logAdapter - the log adapter for this instance, or null if logging is disabled
    
    numProcessingThreads - the number processing threads, valid range = 1..64
    
    Throws:
    
    java.lang.IllegalArgumentException - if the number processing threads is out of the valid range
- Method Detail
  - setAsyncResponseHandler
```
public void setAsyncResponseHandler(HTTPClientAsyncResponseHandler asyncResponseHandler)
```
    Set the (implemented) interface for instantly processing the received HTTP responses.
    
    if such an (implemented) interface is set then the following methods of the HTTPClient instance are invalid and cannot called:
    - waitAndGetResponse
    - waitResponse
    - getResponse
    - getNextResponse
    But the following method can still called:
    - waitAllResponses
    Parameters:
    
    asyncResponseHandler - the (implemented) interface for instantly processing the received HTTP responses
    
    See Also:
    
    waitAndGetResponse(long), waitResponse(long), getResponse(long), getNextResponse(), waitAllResponses()
  - setLocalOutboundAddress
```
public void setLocalOutboundAddress(java.net.InetAddress localOutboundAddress)
```
    Set or unset a specific outbound IP address.
    Example: httpClient.setLocalOutboundAddress(InetAddress.getByName("192.168.0.100"));
    
    Parameters:
    
    localOutboundAddress - the specific outbound IP address, or null = use default outbound IP address
  - setLimitSSLVersion
```
public void setLimitSSLVersion(java.lang.String limitSSLVersion)
```
    Set/limit the maximal SSL version. Possible values are:
    - SSLv2Hello
    - SSLv3
    - TLSv1
    - TLSv1.1
    - TLSv1.2
    - TLSv1.3 (not supported by Oracle Java 8 JVM, requires to run on Java 9 JVM or newer)
    Parameters:
    
    limitSSLVersion - the maximal SSL version
  - setTrustManager
```
public void setTrustManager(javax.net.ssl.TrustManager[] trustManagers)
```
    Set a specific SSL X509 trust manager, used to validate the certificate chain received from a HTTPS server during the SSL handshake.
    Note: The verification of the SSL server certificate(s) is disabled by default if no specific SSL X509 trust manager is set.
    
    Parameters:
    
    trustManagers - the specific X509 trust manager
  - addClientAuthKeyManagers
```
public void addClientAuthKeyManagers(java.lang.String serverHost,
                                     int serverPort,
                                     javax.net.ssl.KeyManager[] clientAuthKeyManagers)
```
    Add SSL key managers for client authentication, for a particular HTTPS host and port. Note: This method can be called multiple times to support different SSL key managers for different HTTPS servers.
    
    Parameters:
    
    serverHost - the server name or IP address
    
    serverPort - the server TCP/IP port
    
    clientAuthKeyManagers - the key managers for SSL client authentication
    
    See Also:
    
    LoadPKCS12Certificate, LoadPKCS12Certificate.getKeyManagers()
  - removeAllClientAuthKeyManagers
```
public void removeAllClientAuthKeyManagers()
```
    Remove all SSL key managers for client authentication.
  - setOverallTimeout
```
public void setOverallTimeout(int overallTimeoutMillis)
```
    Set a synthetically calculated overall timeout for the URL calls
    The 3 inner timeouts are calculated as follows:
    1. At first, the SSL Handshake Timeout is calculated: as 10% of the Overall Timeout but max. limited to 5000 ms
    2. Then TCP Connect Timeout is calculated as 20% of the = (Overall Timeout - (minus) the SSL Handshake Timeout)
    3. Finally the URL Processing Timeout is calculated as 80% of the = (Overall Timeout - (minus) the SSL Handshake Timeout)
    Parameters:
    
    overallTimeoutMillis - the overall HTTP call timeout in milliseconds, not less than 10000 ms
    
    Throws:
    
    java.lang.IllegalArgumentException - if the overall HTTP call timeout is less than 10000 ms
    
    See Also:
    
    setTCPConnectTimeout(int), setSSLHandshakeTimeout(int), setHTTPProcessingTimeout(int)
  - setTCPConnectTimeout
```
public void setTCPConnectTimeout(int tcpConnectTimeoutMillis)
```
    Set the TCP connect timeout.
    
    Parameters:
    
    tcpConnectTimeoutMillis - the TCP connect timeout in milliseconds
  - getTCPConnectTimeout
```
public int getTCPConnectTimeout()
```
    Get the TCP connect timeout.
    
    Returns:
    
    the TCP connect timeout in milliseconds
  - setSSLHandshakeTimeout
```
public void setSSLHandshakeTimeout(int sslHandshakeTimeoutMillis)
```
    Set the SSL handshake timeout.
    
    Parameters:
    
    sslHandshakeTimeoutMillis - the SSL handshake timeout in milliseconds
  - getSSLHandshakeTimeout
```
public int getSSLHandshakeTimeout()
```
    Get the SSL handshake timeout.
    
    Returns:
    
    the SSL handshake timeout in milliseconds
  - setHTTPProcessingTimeout
```
public void setHTTPProcessingTimeout(int httpProcessingTimeoutMillis)
```
    Set the default value for the processing timeout per URL at HTTP level. Note: This timeout is applied after the TCP connection has opened, and after the (optional) SSL handshake has performed.
    
    Parameters:
    
    httpProcessingTimeoutMillis - the processing timeout per URL at HTTP level in milliseconds
    
    See Also:
    
    HTTPRequest.setRequestSpecificProcessingTimeoutMillis(long)
  - getHTTPProcessingTimeout
```
public int getHTTPProcessingTimeout()
```
    Get the default value for the processing timeout per URL at HTTP level. Note: This timeout is applied after the TCP connection has opened, and after the (optional) SSL handshake has performed.
    
    Returns:
    
    the processing timeout per URL at HTTP level in milliseconds
  - setCountTransmitBytesInterfaces
```
public void setCountTransmitBytesInterfaces(HTTPClientCountBytesSentInterface bytesSentInterface,
                                            HTTPClientCountBytesReceivedInterface bytesReceivedInterface)
```
    Set callback interfaces which are called each time when the HTTP client has sent or received a data fragment to/from a server.
    
    Parameters:
    
    bytesSentInterface - called each time when the HTTP client has sent a data fragment to a server
    
    bytesReceivedInterface - called each time when the HTTP client has received a data fragment from a server.
  - newRequest
```
public HTTPRequest newRequest(java.lang.String httpRequestMethod,
                              java.lang.String url)
                       throws java.net.MalformedURLException
```
    Create a new HTTP request, but don't send it yet. You can set additional HTTP request header fields and set the request content before sending the HTTP request.
    
    Note that each new created HTTP request has an unique request ID and can send only one time to the HTTP server. If you need to repeat the HTTP request, then you have to create another instance of a new HTTP request.
    
    Parameters:
    
    httpRequestMethod - the HTTP request method. Example: 'GET', 'POST', 'HEAD'
    
    url - the full URL. Example: 'https://www.dkfqa.com/test?param1=hello'
    
    Returns:
    
    the new HTTP request
    
    Throws:
    
    java.net.MalformedURLException - if the URL is invalid
    
    See Also:
    
    HTTPRequest.getId(), sendAsyncRequest(HTTPRequest)
  - sendAsyncRequest
```
public long sendAsyncRequest(HTTPRequest httpRequest)
                      throws java.lang.InterruptedException
```
    Send a HTTP request as an asynchronous operation.
    
    Parameters:
    
    httpRequest - the HTTP request to send
    
    Returns:
    
    the unique ID of the HTTP request
    
    Throws:
    
    HTTPClientException - if the request was already sent
    
    java.lang.InterruptedException - if somewhat fails
  - waitAndGetResponse
```
public HTTPResponse waitAndGetResponse(long httpRequestId)
```
    Wait until a particular HTTP request is completed and get the corresponding HTTP response. Note: This method returns null if the current thread is interrupted.
    
    Parameters:
    
    httpRequestId - the unique ID of the HTTP request
    
    Returns:
    
    the HTTP response for the particular HTTP request, or null if the current thread is interrupted
    
    Throws:
    
    HTTPClientException - if a HTTPClientAsyncResponseHandler is set.
  - waitResponse
```
public void waitResponse(long httpRequestId)
```
    Wait until a particular HTTP request is completed. Note: This method is aborted in case if the current thread is interrupted.
    
    Parameters:
    
    httpRequestId - the unique ID of the HTTP request
    
    Throws:
    
    HTTPClientException - if a HTTPClientAsyncResponseHandler is set.
  - getResponse
```
public HTTPResponse getResponse(long httpRequestId)
```
    Get the HTTP response for a particular HTTP request. Note: This method can be called only one time per unique HTTP request Id. Further calls with the same unique ID of the HTTP request will return a null value.
    
    Parameters:
    
    httpRequestId - the unique ID of the HTTP request
    
    Returns:
    
    the HTTP response for the particular HTTP request, or null if the HTTP request is still pending, and also null if a previous call of this method has already returned a valid HTTP response
    
    Throws:
    
    HTTPClientException - if a HTTPClientAsyncResponseHandler is set.
    
    See Also:
    
    waitResponse(long)
  - waitAllResponses
```
public void waitAllResponses()
```
    Wait until all HTTP requests are completed. Note: This method is aborted in case if the current thread is interrupted.
  - getNextResponse
```
public HTTPResponse getNextResponse()
```
    Get the next HTTP response of all completed HTTP requests.
    
    Returns:
    
    the next HTTP response of all completed HTTP requests, or null if no next HTTP response is available
    
    Throws:
    
    HTTPClientException - if a HTTPClientAsyncResponseHandler is set.
    
    See Also:
    
    waitAllResponses()
  - clearConnectionPool
```
public void clearConnectionPool()
```
    Close and remove all network connections from the connection pool.
  - closeAbort
```
public void closeAbort()
```
    Free all internal resources and close all connections of the HTTPClient. This will also abort any pending HTTP request.
  - getCookieHandler
```
public HTTPCookieHandler getCookieHandler()
```
    Get access to the cookie handler of the HTTP client.
    
    Returns:
    
    the cookie handler of the HTTP client
  - getNumOutstandingRequest
```
public int getNumOutstandingRequest()
```
    Performance data, get the number of currently outstanding requests (= number of not yet completed, pending requests).
    
    Returns:
    
    the number of currently outstanding requests
  - getConnectionStatistic
```
public HTTPConnectionStatistic getConnectionStatistic()
```
    Performance data, get the statistic of the network connections used by the HTTPClient.
    
    Returns:
    
    the statistic of the network connections used by the HTTPClient
  - dumpConnectionPoolToStdout
```
public void dumpConnectionPoolToStdout()
```
    For debug purposes only, dump the network connection pool to stdout.
  - dumpProcessingPoolToStdout
```
public void dumpProcessingPoolToStdout()
```
    For debug purposes only, dump the HTTP client processing pool to stdout.
  - setClientDebugger
```
public void setClientDebugger(HTTPClientDebuggerInterface clientDebugger)
```
    Set a client debugger.
    
    Parameters:
    
    clientDebugger - the client debugger
  - resumeSuspendedHTTPRequest
```
public boolean resumeSuspendedHTTPRequest(long httpRequestId)
```
    Resume a suspended HTTP request. A HTTP request can only come in suspended state if the HTTP Client Debugger Interface method onRequestBeforeSend(..) returns the boolean value true.
    
    Parameters:
    
    httpRequestId - the unique, internal HTTP request id
    
    Returns:
    
    true if the HTTP request was suspended and is now resumed.
    
    Throws:
    
    java.lang.NullPointerException - if no client debugger is set
    
    See Also:
    
    setClientDebugger(HTTPClientDebuggerInterface), HTTPClientDebuggerInterface.onRequestBeforeSend(HTTPRequest), HTTPRequest.getId()
  - setTraceRequestContentTypesSet
```
public void setTraceRequestContentTypesSet(java.util.HashSet<java.lang.String> traceRequestContentTypesSet)
```
    Set a set of mime types for which the request content data are written to the log adapter at trace level.
    
    Parameters:
    
    traceRequestContentTypesSet - a set of mime types for which the request content data are written to the log adapter at trace level
  - setTraceResponseContentTypesSet
```
public void setTraceResponseContentTypesSet(java.util.HashSet<java.lang.String> traceResponseContentTypesSet)
```
    Set a set of mime types for which the response content data are written to the log adapter at trace level.
    
    Parameters:
    
    traceResponseContentTypesSet - a set of mime types for which the response content data are written to the log adapter at trace leve
  - getTotalBytesSent
```
public long getTotalBytesSent()
```
    Get the total number of bytes which were sent - counted over all ever executed HTTP/S calls.
    
    Returns:
    
    the total number of bytes which were sent
  - getTotalBytesReceived
```
public long getTotalBytesReceived()
```
    Get the total number of bytes which were received - counted over all ever executed HTTP/S calls.
    
    Returns:
    
    the total number of bytes which were received

Class HTTPClient

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

HTTPClient

HTTPClient

Method Detail

setAsyncResponseHandler

setLocalOutboundAddress

setLimitSSLVersion

setTrustManager

addClientAuthKeyManagers

removeAllClientAuthKeyManagers

setOverallTimeout

setTCPConnectTimeout

getTCPConnectTimeout

setSSLHandshakeTimeout

getSSLHandshakeTimeout

setHTTPProcessingTimeout

getHTTPProcessingTimeout

setCountTransmitBytesInterfaces

newRequest

sendAsyncRequest

waitAndGetResponse

waitResponse

getResponse

waitAllResponses

getNextResponse

clearConnectionPool

closeAbort

getCookieHandler

getNumOutstandingRequest

getConnectionStatistic

dumpConnectionPoolToStdout

dumpProcessingPoolToStdout

setClientDebugger

resumeSuspendedHTTPRequest

setTraceRequestContentTypesSet

setTraceResponseContentTypesSet

getTotalBytesSent

getTotalBytesReceived