Generics in WCF REST JSON Communication

Overview

So I got a task in designing a scalable and modular Service Oriented application using JSON and WCF. I am designing a web service using WCF. The communication between the web service and the client will be using JSON. While the server is using .NET Framework, the client is using Android Java. I aim to design the service as modular as possible. For every web service request from the client, the server will return the response in some kind of “envelope”. The “envelope” contains the execution result, whether it is success or not, the text message from web service, and the attachment which may contains object of specific type as the result of the service call.

I want to develop the modularity in my service framework, where I can make the “envelope” object as generic object that accept any kind of attachment type. And I want to deserialize the JSON response from server in my Android directly into specified “envelope” and attachment type, so that I can use it fairly easy in my code.

Prerequisite

• .NET Framework and Windows Communication Foundation
• Android Java
• Jackson JSON Deserializer for Java (https://github.com/FasterXML/jackson)

Refresh Your Knowledge

Before we begin talking about what is in the title, let’s refresh about some terms.

Generics

Generic is an approach in programming to develop a code that is type-agnostic, i.e. you can use your code with many type of types without breaking the algorithm and type-safety. Type-safety is the way the compiler can ensure that when you perform type casting (converting a type to another type) is correct. In writing a method or class using generic, you specify a type as a variable to your method or class. So when the class is used or the method is called by other part of your program, you need to specify the type which will be used for the execution.

The easiest example is sorting algorithm below (taken from StackOverflow):

[csharp] public static void Main(string[] args)
{
int[] intArray = { 800, 11, 50, 771, 649, 770, 240, 9 };
BubbleSort(intArray);

string[] strArray = { "A", "C", "Z", "B", "H" };
BubbleSort(strArray);
}

public static void BubbleSort<T>(T[] arr) where T: IComparable<T>
{
T temp;

for (int write = 0; write < arr.Length; write++)
{
for (int sort = 0; sort < arr.Length – 1; sort++)
{
if (arr[sort].CompareTo(arr[sort + 1]) > 0)
{
temp = arr[sort + 1];
arr[sort + 1] = arr[sort];
arr[sort] = temp;
}
}
}
}
[/csharp]

You can use the BubbleSort method in any type that implements IComparable, so it can be compared to each other. Whether it is an integer or string, or your custom class.

WCF

Windows Communication Foundation, a framework to design a service-based application. The service is not limited to web service, but basically any client-server can be called a service-based application and developed using WCF. I developed a service-based application using WCF, and host the service as web service in IIS/ASP.NET. You don’t have to deal with how to design the communication between client and the server, because it is handled fully by WCF. So you can focus more on developing the service instead. This is a very powerful and modular framework that resembles the ISO OSI Network Stack in its design.

JSON

JavaScript Object Notation, the way to represent a structured data which is cross-compatible between system (which understands it). It is almost similar with XML, but JSON is quite powerful because it is smaller in terms of syntax, and can be executed directly on JavaScript engine to instantiate it as JavaScript object.

What I Designed

Service Design and Implementation

Here is the example of service design that is implemented in my server. I designed a simple service to get user information from database. The service is configured to be used using REST JSON call. I want to return this data to the caller of my service:

[csharp] [DataContract] public class UserInfo : AttachmentBase
{
[DataMember] public string FullName { get; set; }

[DataMember] public string NickName { get; set; }

[DataMember] public int UserID { get; set; }

[DataMember] public string PhotoUrl { get; set; }
}
[/csharp]

You can see that the data contract above inherits another class. AttachmentBase will be used in the “envelope” generics. While here’s the design of the “envelope”. The “envelope” is reusable across the service operation, and every operation must return the value using this “envelope”, and additional data must be stored inside in its attachment field.

[csharp] [DataContract] public class Envelope<T> where T : AttachmentBase
{
[DataMember] public bool Result { get; set; }

[DataMember] public string Message { get; set; }

[DataMember] public T Attachment { get; set; }
}

[DataContract] public abstract class AttachmentBase
{

}
[/csharp]

As you can see above, AttachmentBase is an empty abstract class. Its purpose is for specifying type in generics for the Envelope class. So only class that inherits AttachmentBase can be stored inside the “envelope”.

Moving on to the service implementation, check below for the design of my WCF service:

[csharp]

// Service Contract
[ServiceContract] public interface IUser
{
[OperationContract] [WebInvoke(
UriTemplate = "/GetUser",
BodyStyle = WebMessageBodyStyle.WrappedRequest,
RequestFormat = WebMessageFormat.Json,
ResponseFormat = WebMessageFormat.Json)] Envelope<UserInfo> GetUser(int userId);

[OperationContract] [WebInvoke(
UriTemplate = "/UploadProfilePicture",
BodyStyle = WebMessageBodyStyle.WrappedRequest,
RequestFormat = WebMessageFormat.Json,
ResponseFormat = WebMessageFormat.Json)] Envelope<PhotoInfo> UploadProfilePicture(string fileName, string photo);
}

// Service Implementation
public class User : IUser
{
public Envelope<UserInfo> GetUser(int userId)
{
// Do something
// Abrakadabra
// Buff, we got from the database
User u = Abrakadabra(userId);

return new Envelope<UserInfo>()
{
Result = true,
Message = "Success",
Attachment = new UserInfo()
{
FullName = u.FullName,
NickName = u.NickName,
// blablabla
}
};
}

// And some other more
}
[/csharp]

As straightforward as the code, I can just return the “envelope”, and attach the UserInfo object to the “envelope”. Just let the WCF serialize it into JSON. The client will receive the response in following format:

[js] {
"Attachment":{
"FullName":"Paijo",
"NickName":"Pi",
"PhotoUrl":"http://www.google.com/paijo.jpg",
"UserID":123
},
"Message":"Success",
"Result":true,
}
[/js]

What I Want in the Client

I want the client can deserialize the JSON into objects that is similar with the “envelope” design in the server. So I can directly operate the response from the server in straightforward manner in my code.

So, I want to have the similar Envelope and UserInfo class in my client:

[java] // Envelope.java
public class Envelope<T> {
@JsonProperty("Message")
private String message;

@JsonProperty("Result")
private boolean result;

@JsonProperty("Attachment")
private T attachment;
}

// UserInfo.java
public class UserInfo {
@JsonProperty("FullName")
private String fullName;

@JsonProperty("NickName")
private String nickName;

@JsonProperty("PlayerID")
private int playerID;

@JsonProperty("PhotoUrl")
private String photoUrl;
}
[/java]

And I want to encapsulate my web service client into a single class, so any code which needs to call the service just need to instantiate this class and call the service seamlessly. Below is the implementation of the client, including the mechanism to perform HTTP request to the service

[java] // ServiceClient.java
public class ServiceClient {

public Envelope<UserInfo> getUser(int userId) {
HashMap<String, Object> param = new HashMap<String, Object>();
param.put("userId", userId);
return this.callService(USERACCESS_GETUSER, param);
}

public Envelope<PhotoInfo> uploadPicture(String fileName, byte[] photo) {
HashMap<String, Object> param = new HashMap<String, Object>();
param.put("fileName", fileName);
param.put("photo", photo);
return this.callService(USERACCESS_UPLOADPICTURE, param);
}

/*
* CONSTANTS: Contains all the method name that is available on the server
*/

private final static String BASE_URL = "http://heliosky.";

private final static String USERACCESS_GETUSER = "User.svc/GetUser";
private final static String USERACCESS_UPLOADPICTURE = "User.svc/UploadProfilePhoto";

// Method to prepare service URL to call
private String prepareServiceUrl(String methodName) {
return BASE_URL + methodName;
}

// Execute to call service, by preparing the HttpPost request, and execute it
private <T> Envelope<T> callService(String methodName,
HashMap<String, Object> param) {
HttpPost request = prepareRequest(methodName, param);
Envelope<T> result = executeRequest(request);

return result;
}

// Prepare the request, resulting in HttpPost request object. The parameter
// is serialized into JSON
private HttpPost prepareRequest(String methodName,
HashMap<String, Object> param) {
// Prepare URL
String serviceUrl = this.prepareServiceUrl(methodName);

try {
// Prepare parameter
String jsonParam = "";
if (param != null) {
ObjectMapper mapper = new ObjectMapper();
jsonParam = mapper.writeValueAsString(param);
}

// Prepare HttpPost request object
HttpPost httpPost = new HttpPost(serviceUrl);
httpPost.setHeader("Content-Type", "application/json");

httpPost.setEntity(new StringEntity(jsonParam));

// Return the prepared request
return httpPost;
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}

// Exectue the prepared HttpPost request object, and deserialize
// the JSON result into local classes
private <T> Envelope<T> executeRequest(HttpPost request) {
HttpClient client = new DefaultHttpClient();

try {
// Execute request
HttpResponse httpResponse = client.execute(request);
int statusCode = httpResponse.getStatusLine().getStatusCode();
if (statusCode == 200) {

// Deserialize the response
// Check Jackson JSON for more info of this functionality
ObjectMapper mapper = new ObjectMapper();
return mapper.readValue(httpResponse.getEntity().getContent(),
new TypeReference<Envelope<T>>() {
});
} else {
throw new Exception("Unknown server error");
}
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}
}
[/java]

With that class, I am going to use it in my program which needs a service call towards my web service.

[java] public void getData() {
ServiceClient clt = new ServiceClient();
Envelope<UserInfo> res;

res = clt.getUser(123);

if(res.getResult()) {
UserInfo info = res.getAttachment();

String fullName = info.getFullName();
// Perform abrakadabra
}
}
[/java]

BUT, here comes the Exception! ClassCastException on the getAttachment method. Apparently the Jackson JSON deserialize the object into HashMap instead of the actual UserInfo instance. So when getAttachment tries to cast the class into UserInfo, it failed.

The Problem

Why it happened? Thank you very much for Java’s Type Erasure on its generic. What is that? So basically every type you specified on generic instantiation is not retained to the runtime. Generic only works on compile time checking by the compiler.

Jackson JSON relies on the reflection to detect the class structure, so that it can deserialize JSON value into correct field. In code, I specified Envelope<UserInfo> to Jackson, but in runtime, Jackson JSON sees it as Envelope<Object>.

[java] // Runtime class structure
public class Envelope {
@JsonProperty("Message")
private String message;

@JsonProperty("Result")
private boolean result;

@JsonProperty("Attachment")
private Object attachment;
}
[/java]

Yes, type erasure will make every T you specify in generics into Object in runtime. It will perform necessary downcasting from Object type back to the specified generic type during execution. It all happened dynamically. So when Jackson JSON tries to read the structure of the class using reflection, it will see that non-generic structure instead.

And when Jackson JSON encounters Object type field as serialization target, it will deserialize the JSON value into a Map instead, so you access the field using Map.get method instead. This is not what I want.

I want the Jackson JSON to read this structure instead. Jackson JSON can infer the UserInfo type and try to deserialize the JSON to fit with UserInfo class.

[java] // Runtime class structure that Jackson JSON need to infer from reflection
public class Envelope {
@JsonProperty("Message")
private String message;

@JsonProperty("Result")
private boolean result;

@JsonProperty("Attachment")
private UserInfo attachment;
}
[/java]

And I want to achieve that using generic. So I can reuse Envelope class with other service call and other attachment type without having a separate Envelope class for each attachment type. But to achieve that, I cannot only rely with the Java generic system because of its limitation.

Solution

I traversed around the Jackson JSON documentation, and look for method that I can use to deserialize the JSON dynamically.I am using ObjectMapper class form Jackson JSON, and I found interesting method that I think I can use: convertValue(Object fromValue, Class<T> toValueType). This method will convert an object into another object, which works by serializing the object fromValue, then deserialize it into target type specified in toValueType.

That method will require class structure information from reflection. Once again because I cannot rely on the generic, I have to specify the class name explicitly on the runtime. To help specifying the class name on the runtime, the best way is to modify a little bit of my service response, to include the type of attachment in the response.

And because I am going to perform custom deserialization of the JSON data, I need Jackson JSON to include my implementation on deserialization process. Jackson JSON is quite extensible. I can specify @JsonCreator annotation on the constructor of the Envelope class, so Jackson JSON will instantiate the class using the specified constructor. So I will perform additional deserialization, which converts the HashMap from JSON deserialization into the actual mapped class.

Service Modification

This is some simple modification on the Envelope class on the server, which adds AttachmentType property. The program will automatically populate the property by reading from class information using reflection.

[csharp] [DataContract] public class Envelope<T> where T : AttachmentBase
{
public ActionResult()
{
// Get AttachmentType name using reflection
this.AttachmentType = typeof(T).Name;
}

[DataMember] public bool Result { get; set; }

[DataMember] public string Message { get; set; }

[DataMember] public T Attachment { get; set; }

[DataMember] public string AttachmentType { get; set; }
}
[/csharp]

Magical, isn’t it? Why in C# I can infer the generic T type information? Because C# generic does not erase the type during runtime. So the reflection will be able to infer what type is specified in the generic. This is why .NET is superior in Java in terms of generic programming, where in .NET, generic infrastructure is implemented in runtime, while Java only in compile time.

So from that, below is the JSON response from the service will be. No additional modification on the server.

[js] {
"Attachment":{
"FullName":"Paijo",
"NickName":"Pi",
"PhotoUrl":"http://www.google.com/paijo.jpg",
"UserID":123
},
"AttachmentType":"UserInfo",
"Message":"Success",
"Result":true,
}
[/js]

Client Modification

As previously described, I need to create a constructor that will be used by Jackson JSON to instantiate the object during deserialization.

[java] // Envelope.java
public class Envelope<T> {
private String message;
private boolean result;
private T attachment;

@JsonCreator
public Envelope(@JsonProperty("Message") String message,
@JsonProperty("Result") boolean result,
@JsonProperty("Attachment") Object attachment,
@JsonProperty("AttachmentType") String attachmentType) {
try {
this.message = message;
this.result = result;

// If the attachment is not null, remap to the actual object
if (attachment != null) {
ObjectMapper map = new ObjectMapper();

// This is to tackle the stupid type erasure of Java! Load the
// class definition dynamically
this.attachment = (T) map.convertValue(attachment,
Class.forName(attachmentType));
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
[/java]

From Jackson JSON documentation, if we specified JsonCreator in a class, it will use its constructor or static method instead of inferring the structure automatically. So we omit the @JsonProperty annotations in the class’ private fields definition, and put it in the constructor or method parameter instead.

Method Class.forName is used to load class information from a name. We specified the name in AttachmentType in JSON response. So, Java can explicitly know what class to load, and provided to Jackson JSON so it can perform deserialization. Additionally, we have to cast the return value from convertValue method into T type so it can be stored in private field.

Jackson JSON will work as expected, and deserialize the attachment into specified mapped object, the desired operation in getData() method above can be executed without ClassCastException.

Final Words

Is this elegant? Not so much. There will be a big error if the client and the server does not agree with what to put in AttachmentType field. The client assumes that the server will give correct AttachmentType, and the class with the name specified is available. If not, it will raise ClassDefNotFoundException, because it cannot find the specified class name. So you can say that this mechanism based on trust between the client and server.

Many systems actually built on top of this trust system, such as internet routing system. You may have heard that Google DNS traffic was routed to invalid endpoint by Indonesian ISP, which makes the Google DNS service inaccessible for some time.

One other note, don’t forget that you have to specify the fully qualified class name in Class.forName method. So if your class is inside a Java Package, don’t forget to specify the package name.