[CloudRAID] 4. Implementation (Continuation)

This post is a continuation of the blog series about the student research paper CloudRAID.

4. Implementation

4.2. RESTful API Endpoint Specifications

4.2.1. URL Mapping for API Endpoints

The default CloudRAID package contains a RESTful HTTP API. This is realized using Java Servlets. A Java web and application server will be used to deploy the servlet. To activate the servlet this is bound to a base path that will be required to be prefixed to the Uniform Resource Locator (URL) path. The actual functionality provided by this servlet will then be bound to sub paths as described in chapters 4.2.2 to 4.2.4 (see below). The basic concept of how the mapping between the sub paths and URL parameters is done, has been taken from the Django URL dispatcher.

The complete pattern and mapping implementation is done in the class RestApiUrlMapping within the package de.dhbw_mannheim.cloudraid.api.impl and is part of the CloudRAID-REST bundle. The RestApiUrlMapping constructor takes two to four parameter of various type as one can see in listing 9 below:

1. The first parameter must be the pattern that needs to be matched. This can either be a java.lang.String or a regular expression pattern of type java.util.regex.Pattern. If a String is passed, this is converted to a Pattern and hence must be a correct regular expression. Otherwise a IllegalArgumentException will be thrown.
2. The second, optional, parameter can be used to restrict a mapping to a specific HTTP request method. This allows a developer to have different callback functions for the same URL path, depending on the HTTP method.
3. The remaining parameters define the callback function which is called if the URL matches a HTTP request. This can either be a single parameter of type java.lang.reflect.Method or two parameters whereof the first must be of type Class<?> and the second of type java.lang.String. In the latter case, the static function findFunction() will be used to resolve both parameters to a Method. If the (resolved) method is null, this will result in a IllegalArgumentException. If the method needs to be resolved, any failure will raise as NoSuchMethodException or SecurityException.

package de.dhbw_mannheim.cloudraid.api.impl;

public class RestApiUrlMapping {
  public static Method findFunction(Class<?> klass,
    String function);

  public RestApiUrlMapping(Pattern pattern, Class<?> klass,
    String function);
  public RestApiUrlMapping(Pattern pattern, Method function);
  public RestApiUrlMapping(Pattern pattern, String method,
    Class<?> klass, String function);
  public RestApiUrlMapping(Pattern pattern, String method,
    Method function);
  public RestApiUrlMapping(String pattern, Class<?> klass,
    String function);
  public RestApiUrlMapping(String pattern, Method function);
  public RestApiUrlMapping(String pattern, String method,
    Class<?> klass, String function);
  public RestApiUrlMapping(String pattern, String method,
    Method function);
}

Listing 9: The URL mapping API

The actual mapping will be initialized in the constructor of the RestApiServlet as shown in the listing 10. The shipped REST API uses only the RestApiUrlMapping constructor defined in lines 17 and 18 of listing 9 because of its simplicity. But one can use this class to extend the API with own functions. Hence the other constructors are still in the code and will not be removed.

package de.dhbw_mannheim.cloudraid.api.impl;

public class RestApiServlet extends HttpServlet {
  private static ArrayList<RestApiUrlMapping> mappings =
    new ArrayList<RestApiUrlMapping>();

  public RestApiServlet() {
    RestApiServlet.mappings.add(new RestApiUrlMapping(
      "^/api/info/$", "GET", RestApiServlet.class,
      "apiInfo"));
    RestApiServlet.mappings.add(new RestApiUrlMapping(
      "^/file/([^/]+)/$", "DELETE", RestApiServlet.class,
      "fileDelete"));
    // Similar for other URL sub paths
  }
}

Listing 10: The URL mapping initialization

On each request to the RestApiServlet the list of available mappings will be checked for a matching mapping with the path information of the current request and its HTTP method. If a valid mapping exists, the regarding callback function will be executed. This callback function must accept exactly three arguments:

1. javax.servlet.http.HttpServletRequest req: The current request. This variable will contain session information, the HTTP headers and allows to access the request body.
2. de.dhbw_mannheim.cloudraid.api.responses.IRestApiResponse resp: The response object that will be returned to the client. It will contain a HTTP response code as defined by the specifications in the following chapters and a payload, depending on the request.
3. java.util.ArrayList<String> args: The URL patterns may contain matching groups, enclosed by ( and ). The content of each matching group will be added to this parameter and is therefore available to the callback function. The Create, Read, Update, Delete (CRUD) endpoints as defined in chapter 4.2.3 – File Related Function make use of a single matching group that contains the file name of the uploaded file.[/olist]

4.2.2. Version Information

The RESTful API already supports many features. But since the API might change someday, some kind of versioning is needed to tell the client side, which features are supported. This information, along with the version information of the Core-, Metadata- and Configuration-Services and the currently running RAID version and the versions of the activated storage connectors is provided via this URL.

RESTful API: API and service information

HTTP/1.1 200 OK
X-Powered-By: CloudRAID/0.2
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(6.1.x)

Core-Service:CloudRAID-Core v0.0.1.prealpha by cloudraid
RAID-Version:CloudRAID-RAID5 v0.0.2prealpha by cloudraid
Metadata-Service:CloudRAID-Metadata v0.0.1.prealpha by cloudraid
Configuration-Service:CloudRAID-Config v0.0.1.prealpha by cloudraid
API-Service:CloudRAID-RESTful v0.0.1.prealpha by cloudraid
Storage-Connector-0:CloudRAID-Dropbox v0.0.1.prealpha by cloudraid
Storage-Connector-1:CloudRAID-AmazonS3 v0.0.1.prealpha by cloudraid
Storage-Connector-2:CloudRAID-UbuntuOne v0.0.1.prealpha by cloudraid
API-Version:0.2

Listing 11: Example HTTP response: header elements and body

4.2.3. File Related Function

All CRUD functions are provided by the following four URLs. The endpoint for all of them is /file/([^/]+)/. Based on the HTTP method, a file is either being created, downloaded, removed or updated.

The Create and Update endpoints have to handle a huge amount of data. Therefore they use the POST and PUT methods respectively. In both cases the file will be trans- mitted synchronously to the CloudRAID service. All ongoing steps are going to be asynchronous. They do not influence the HTTP response to the upload.

On the other hand, the Delete and Download functions are completely synchronous. They use the DELETE and GET HTTP methods. The cloud storage connectors will trigger the deletion or get request to the cloud storage providers and are blocking up to their response or until the regarding network protocol throws a timeout error. From that follows, that a failure to the “outbound” connections from the CloudRAID service may block a client application, or at least the thread the connection runs in, for some time.

RESTful API: file deletion

RESTful API: file download

RESTful API: file upload of a new file

RESTful API: file upload of an existing file

While using a client application, the user needs to know, which files exist and can be downloaded and, if the system is under heavy usage, what the states of the uploads are. The following URL shows all files a user has access to and displays those information.

RESTful API: list all accessible file

4.2.4. User Related Functions

All connections, except the API version request, need some kind of authorization or provide and initiate it. The authentication is done on user base. The creation, authentication and de-authentication (logout) is performed on the /user/ endpoint namespace. Besides that, a user is able to change the login password, while user deletion requests are not allowed. This is basically due to the fact, that a deletion should trigger a deletion of all files belonging to the user from the cloud storage providers. But since this step is non-reversible and a full data loss will be the consequence, the decision to not implement this function has been made.

RESTful API: add new user

RESTful API: authenticate a user

RESTful API: change a user password

RESTful API: user logout