PrevUpHomeNext

Serialization

Standard C++ types
User-defined types
Binary data
Portability
File-based serialization

Serialization is a fundamental part of every remote call. Remote call arguments need to be serialized into binary form so they can be transmitted across the network, and once received, they need to be deserialized from binary form back into regular C++ objects.

RCF provides a serialization framework of its own, SF, which handles serialization of common C++ types automatically, and can be customized to serialize arbitrary user-defined C++ types.

Serialization of fundamental C++ types (char, int, double, etc) is handled automatically by RCF. Serialization of a number of other common and standard C++ types requires inclusion of the relevant header file:

Type

Which header to include

std::string, std::wstring, std::basic_string<>

#include <SF/string.hpp>

std::vector<>

#include <SF/vector.hpp>

std::list<>

#include <SF/list.hpp>

std::deque<>

#include <SF/deque.hpp>

std::set<>

#include <SF/set.hpp>

std::map<>

#include <SF/map.hpp>

std::pair<>

#include <SF/utility.hpp>

std::bitset<>

#include <SF/bitset.hpp>

std::auto_ptr<>

#include <SF/auto_ptr.hpp>

std::unique_ptr<>

#include <SF/unique_ptr.hpp>

boost::shared_ptr<>

#include <SF/shared_ptr.hpp>

std::tr1::shared_ptr<>

#include <SF/shared_ptr_tr1.hpp>

std::shared_ptr<>

#include <SF/shared_ptr_std.hpp>

boost::scoped_ptr<>

#include <SF/scoped_ptr.hpp>

boost::intrusive_ptr<>

#include <SF/intrusive_ptr.hpp>

boost::any

#include <SF/any.hpp>

boost::tuple<>

#include <SF/tuple.hpp>

std::tr1::tuple<>

#include <SF/tuple_tr1.hpp>

std::tuple<>

#include <SF/tuple_std.hpp>

boost::variant<>

#include <SF/variant.hpp>

boost::array<>

#include <SF/array.hpp>

std::tr1::array<>

#include <SF/array_tr1.hpp>

std::array<>

#include <SF/array_std.hpp>

std::tr1::unordered_map<>, std::tr1::unordered_multimap<>

#include <SF/unordered_map.hpp>

std::tr1::unordered_set<>, std::tr1::unordered_multiset<>

#include <SF/unordered_set.hpp>

stdext::hash_map<>, stdext::hash_multimap<>

#include <SF/hash_map.hpp>

stdext::hash_set<>, stdext::hash_multiset<>

#include <SF/hash_set.hpp>

In general, for a standard C++ type defined in <xyz>, or a Boost type defined in <boost/xyz.hpp>, the corresponding serialization definitions are located in <SF/xyz.hpp> .

C++ enums are serialized automatically, as integers. Serialization of C++11 enum classes requires the use of a helper macro:

// Legacy C++ enum. Automatically serialized as 'int'.
enum Suit
{
    Heart = 1,
    Diamond = 2,
    Club = 2,
    Spade = 2
};

// C++11 enum class with custom base type (8 bit integer).
enum class Colors : std::int8_t 
{ 
    Red = 1, 
    Green = 2, 
    Blue = 3 
};

// Use SF_SERIALIZE_ENUM_CLASS() to specify the base type of the enum class.
SF_SERIALIZE_ENUM_CLASS(Colors, std::int8_t)

By composing containers, you can build up structures of considerable complexity that can be used in RCF interfaces, without having to write any serialization code at all:

#include <boost/tuple/tuple.hpp>

#include <RCF/Idl.hpp>
#include <RCF/IpServerTransport.hpp>
#include <RCF/RcfServer.hpp>
#include <RCF/TcpEndpoint.hpp>

#include <SF/map.hpp>
#include <SF/list.hpp>
#include <SF/tuple.hpp>
#include <SF/vector.hpp>

typedef 
std::map<
    std::string, 
    std::list< 
        std::pair<
            int, 
            std::string> > >    MyMap;

typedef std::string                                             Name;
typedef std::string                                             Street;
typedef unsigned int                                            Zip;
typedef std::string                                             Suburb;
typedef std::string                                             State;
typedef std::string                                             Country;

typedef boost::tuple<Name, Street, Zip, Suburb, State, Country> Address;
typedef std::vector<Address>                                    Addresses;


RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(MyMap, Echo, const MyMap &)
    RCF_METHOD_R1(Addresses, Echo, const Addresses &)
RCF_END(I_Echo)

class EchoImpl
{
public:
    MyMap Echo(const MyMap &myMap)
    {
        return myMap;
    }

    Addresses Echo(const Addresses &addresses)
    {
        return addresses;
    }
};


int main()
{
    EchoImpl echoImpl;
    RCF::RcfServer server( RCF::TcpEndpoint(0));
    server.bind<I_Echo>(echoImpl);
    server.start();

    int port = server.getIpServerTransport().getPort();

    MyMap myMap;
    myMap["1-3"].push_back( std::make_pair(1, "one"));
    myMap["1-3"].push_back( std::make_pair(2, "two"));
    myMap["1-3"].push_back( std::make_pair(3, "three"));
    myMap["4-6"].push_back( std::make_pair(4, "four"));
    myMap["4-6"].push_back( std::make_pair(5, "five"));
    myMap["4-6"].push_back( std::make_pair(6, "six"));

    RcfClient<I_Echo> client(( RCF::TcpEndpoint(port) ));

    // Echo a MyMap object.
    MyMap myMap2 = client.Echo(myMap);

    Addresses addresses;
    addresses.push_back( Address("", "", 123, "", "", ""));
    addresses.push_back( Address("", "", 456, "", "", ""));
    addresses.push_back( Address("", "", 789, "", "", ""));

    // Echo a Addresses object.
    Addresses addresses2 = client.Echo(addresses);

    return 0;
}

If you take a class of your own, and use it in an RCF interface:

class Point3D
{
public:
    double mX;
    double mY;
    double mZ;
};

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(Point3D, Echo, const Point3D &)
RCF_END(I_Echo)

, you'll end up with an compiler error similar to this one:

..\..\..\..\..\include\SF\Serializer.hpp(324) : error C2039: 'serialize' : is not a member of 'Point3D'
        C6.cpp(13) : see declaration of 'Point3D'
        ..\..\..\..\..\include\SF\Serializer.hpp(336) : see reference to function template instantiation 'void SF::serializeInternal<T>(SF::Archive &,T &)' being compiled
        with
        [
            T=U
        ]
        <snip>

The compiler is telling us that it couldn't find any serialization code for the class Point3D. We need to provide a serialize() function, either as a member function:

class Point3D
{
public:
    double mX;
    double mY;
    double mZ;

    // Internal serialization.
    void serialize(SF::Archive &ar)
    {
        ar & mX & mY & mZ;
    }
};

, or as a free function in either the same namespace as Point3D, or in the SF namespace:

// External serialization.
void serialize(SF::Archive &ar, Point3D &point)
{
    ar & point.mX & point.mY & point.mY;
}

The code in the serialize() function specifies which members to serialize.

The serialize() function is used both for serialization and deserialization. In some cases, you may want to use different logic, depending on whether you are serializing or deserializing. For example, the following snippet implements serialization of the boost::gregorian::date class, by representing it as a string:

// Serialization code for boost::gregorian::date can be placed either in
// the boost::gregorian::date namespace (where the path class is defined), or 
// in the SF namespace. Here we've chosen the SF namespace.
namespace SF {

    void serialize(SF::Archive & ar, boost::gregorian::date & dt)
    {
        if (ar.isWrite())
        {
            // Code for serializing.
            std::ostringstream os;
            os << dt;
            std::string s = os.str();
            ar & s;
        }
        else
        {
            // Code for deserializing.
            std::string s;
            ar & s;
            std::istringstream is(s);
            is >> dt;
        }
    }

}


RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(boost::gregorian::date, Echo, const boost::gregorian::date &)
RCF_END(I_Echo)

To send a chunk of binary data, you can use the RCF::ByteBuffer class:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(RCF::ByteBuffer, Echo, RCF::ByteBuffer)
RCF_END(I_Echo)

class EchoImpl
{
public:
    RCF::ByteBuffer Echo(RCF::ByteBuffer byteBuffer)
    {
        return byteBuffer;
    }
};


int main()
{
    // Set max message length to 600 kb.
    RCF::setDefaultMaxMessageLength(600*1024);

    EchoImpl echoImpl;
    RCF::RcfServer server( RCF::TcpEndpoint(0));
    server.bind<I_Echo>(echoImpl);
    server.start();

    int port = server.getIpServerTransport().getPort();

    // Create and fill a 500 kb byte buffer.
    RCF::ByteBuffer byteBuffer(500*1024);
    for (std::size_t i=0; i<byteBuffer.getLength(); ++i)
    {
        byteBuffer.getPtr()[i] = char(i % 256);
    }

    RcfClient<I_Echo> client(( RCF::TcpEndpoint(port) ));

    // Echo it.
    RCF::ByteBuffer byteBuffer2 = client.Echo(byteBuffer);

    return 0;
}

std::string or std::vector<char> could be used for the same purpose. However, serialization and marshaling of RCF::ByteBuffer is significantly more efficient (see Performance). In particular, with RCF::ByteBuffer, no copies at all will be made of the data, on either end of the wire.

Because C++ does not prescribe the sizes of its fundamental types, there is potential for serialization errors when servers and clients are deployed on different platforms. For example, consider the following interface:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::size_t, Echo, std::size_t)
RCF_END(I_Echo)

The Echo() method uses std::size_t as a parameter and return value. Unfortunately, std::size_t has different meanings on different platforms. For example, the 32 bit Visual C++ compiler considers std::size_t to be a 32 bit type, while the 64 bit Visual C++ compiler considers std::size_t to be a 64 bit type. If a remote call is made from a 32 bit client to a 64 bit server, with std::size_t arguments, a runtime serialization error will be raised.

The same issue arises when using the type long with 32 and 64 bit versions of gcc. 32-bit gcc considers long to be a 32 bit type, while 64-bit gcc considers long to be a 64-bit type.

The correct approach in these situations is to use typedefs for arithmetic types whose bit sizes are guaranteed. In particular, <boost/cstdint.hpp> provides a number of useful typedefs, including the following.

Table 1. Portable integral types

Type

Description

boost::int16_t

16 bit signed integer

boost::uint16_t

16 bit unsigned integer

boost::int32_t

32 bit signed integer

boost::uint32_t

32 bit unsigned integer

boost::int64_t

64 bit signed integer

boost::uint64_t

64 bit unsigned integer


To ensure portability, the example with std::size_t, above, should be rewritten as:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(boost::uint32_t, Echo, boost::uint32_t)
    RCF_METHOD_R1(boost::uint64_t, Echo, boost::uint64_t)
RCF_END(I_Echo)

SF can be used independently of RCF, for instance to serialize objects to and from files. To do so, use SF::OBinaryStream and SF::IBinaryStream:

std::string filename = "data.bin";

X x1;
X x2;

// Write x1 to a file.
{
    std::ofstream fout(filename.c_str(), std::ios::binary);
    SF::OBinaryStream os(fout);
    os << x1;
}

// Read x2 from a file.
{
    
    std::ifstream fin(filename.c_str(), std::ios::binary);
    SF::IBinaryStream is(fin);
    is >> x2;
}

For more advanced topics see Advanced Serialization.


PrevUpHomeNext