Use ProtoBuf.Net to improve serialization performance

Our application relied on data that did not change often. It had therefore implemented a solution that cached this data as xml files. These files were written to the web server on the first request and subsequently read whenever they were needed.
Some of the problems that arose due to this was:
a. Since the application was load balanced, a web server servicing a web request was not guaranteed to serve the next web request. This lead to the data in the various web servers quickly getting out of sync.
b. Our application was very slow. Profiling the application showed that showed that reading and writing(i.e serialization and de-serialization) the XML cache files was one of the biggest bottleneck in the application.
XML serialization is one of the slowest serializationde-serialization formats in the .Net framework. See the excellent article where Maxim Novak compares the various types of serialization in the .Net framework.

We therefore decided to change the serialization format from XML to ProtoBuf.Net. Protobuf-net is a contract based serializer for .NET code, that writes data in the “protocol buffers” serialization format engineered by Google. This framework component is written by Marc Gravell, considered as a .Net guru.
We also decided to change the application to write the cache files to a single file server and create the cache files in a background thread. I will write a separate article for the same in the future.
Changing the application to ProtoBuf.Net protocol was pretty straightforward.
a. Use Nuget to install ProtoBuf.Net framework component.

Install-Package protobuf-net

b. Add [ProtoContract] attribute to each class that needs to be serialized and [ProtoMember] Attribute with a unique integer to identify each member that needs to be serialized.

using ProtoBuf;
class Person 
    public int Id {get;set;}
    public string Name {get;set:}

c. Serialize and deserialize your data

using (var file = File.Create("person.bin")) 
    Serializer.Serialize(file, person);
Person person;
using (var file = File.OpenRead("person.bin")) 
    person = Serializer.Deserialize<Person>(file);

As you can see from the charts below, the change from
a. XML to ProtoBuf.Net serialization
b. caching the files in a single file server instead of multiple file servers
c. offloading the work of generating the cache files to a background thread
was a huge success.




Using ProtoBuf.Net Serialization: The .Net profiler shows “HotSpots” due to serialization and de-serialization are much reduced. File sizes are also much reduced.



Using XML Serialization: The .Net profiler shows high CPU usage i.e “HotSpots” due to XML serialization and de-serialization. File sizes are also much larger.



Continue Reading