Syncing app data: Sync Realm with server-side database like MySQL

Over the last couple of months, Realm became the go-to mobile database solution. It’s a mobile, offline-first database solution to store your app’s data without the need of working with lower level solutions like SQLite or Core Data. And most importantly: It’s platform independent and works on Android and iOS. But what happens if you want to sync your local database with a remote server which uses a database like MySQL?

Since most apps are connected to a backend-service as well, things start to get more complicated. Your app should always work seamlessly without an internet connection but still needs to sync data to a remote server. Realm’s Mobile Platform could be a solution because it syncs the data automatically for you, but you also have to depend on an extra service that costs a lot of money.

Today, we will demonstrate a more generic, easier solution by adding an automatic sync service to your app without touching your existing logic. The goal is an easy-to-setup sync service class that manages the complexity of sending your data changes to your REST service automatically when the local database changes. The REST service could then e.g. manifest those changes in a database like MySQL on the server. Which one you choose is up to you.

Making data-syncing less complex

Syncing data with a remote server leads to a more complex architecture, often consisting of different services managing your local database and your network service:

To avoid this added complecety we developed a technique that we call Database-Driven Architecture. The database will be the main component of the architecture. The other components derive their actions by observing changes in the database:

We propose to use Realm’s reactive feature for live updating data objects to build an easy and robust solution. In combination with Swift 4’s addition of `Codable`, we can make our lightweight solution with around 100 lines of code and still be able to manage all the complexity of detecting changes and sending them to our server.

Our sample app

You can start by downloading our sample project consisting of a simple app that shows a list of users with a timestamp of their last update. You can swipe cells to delete a user and add a new one by clicking the plus button. Our app already uses Realm to store the user objects in a database. All the logic is currently in the ViewController class which we will not touch at all.

Next up, we take a look at our User class. As always, it’s a subclass of Realm’s Object class so that it can be managed by its database:

The User object consists of an id, a username and an updatedDate property to store its last update.

Define the needed protocols and update your models

To make our app sync the user data to the server we need to know how the API is structured. Our REST API consists of

1. a POST /users endpoint to add a new user
2. a PUT /users/:id endpoint to update users
3. a DELETE /users/:id endpoint to delete user

Via this API database changes are propagated to the server-side database. However, since the REST service is decoupled from a specific implementation this could be a MySQL database but it just as well could be a NoSQL database like CouchDB.

Next up, we define a new protocol called Uploadable. It defines all the information we need to upload an object of that type to our endpoint. Thanks to Swift’s Codable protocol our definition is super easy.

We only need our Model class to conform to the Uploadable-protocol to convert it into a JSON representation for the API-requests. We do this by adding the computed property resouceURL:

Detecting changes in the database

Now we are able to go one step further and add some helper methods to our Uploadable protocol. The getId() method returns the String representation of the object’s id. That doesn’t necessarily mean your object’s primary key must be a String, it can also be an Int. But all of our managed objects need to have a primary key set to make them identifiable. Additionally, we add the helper method encoded to convert our models to their JSON representation:

Finally, we get to the part of detecting changes in our database. We define a static registerNotificationObserver method that registers a Realm observer for a specific model class. We collect the inserted, modified and deleted objects and return them in form of an Update struct that contains all of these objects. Realm does not allow us to access the deleted objects after we received their change notification. That’s why we store the objects’ ids in the database instead and use them in the Update struct:

Writing the SyncService

Now that we know how to observe changes for a specific model type, we have to define an object which actually defines the classes being managed. We want it to be easily initialized in a single line of code with the most basic configuration possible, for example:

We pass an array of classes to be synced to our initializer. Optionally we pass in the realm instance. Otherwise, one with the default configuration will be created. To give the SyncService a lifetime without the need of additional start and stop methods, we use Automatic Reference Counting (ARC). As long as we have at least one strong reference to our SyncService instance, it keeps syncing the object changes. That makes it a perfect fit for the AppDelegate class. We can keep the SyncService sync changes to the server for the whole runtime of the app by adding a single of code to the application-didFinishLaunchingWithOptions method:

Since we now know how we want our interface to look like we need to implement our SyncService class. We create the class with its initializer to inject realm as a dependency and to define the synced model types.

We create database change observers for each element in modelTypes. We store them in an instance variable to bind their lifetime to the lifetime of our SyncService class instance. Finally, we use a static handleUpdate method for reacting to database changes:

The handleUpdate implementation is quite simple. It calls our method for uploading new, updated or deleted objects to the remote server. We use the same method for uploading new objects and updating existing objects because for both of them we need to send the whole JSON representation of our model to the server. The only difference will be the API endpoint and the HTTP Method used, therefore we pass a boolean property update to the upload method.

Sending updates to the server

Let’s implement our upload and delete methods now:

Ok, this code speaks for itself. We define all the properties needed to perform the network request, like the HTTP method, endpoint and body data. Finally, we call a performRequest method that handles the actual network request.

Before we write the actual network code we write a dummy implementation. It just logs everything to the console to test the functionality of our SyncService more easily:

If we open the app and click the + button we can see a first log statement:

POST: /api/users
{"id":720983718,"username":"720983718","updatedDate":549106474.768875}

Our ViewController code added a new User to the database and our SyncService detected the changes. If we now click on a cell, we update the User object. We can see a log statement that looks like this:

PUT: /api/users/720983718
{"id":720983718,"username":"720983718","updatedDate":549106504.51612198}

We can see that we used the PUT method now and changed the API endpoint to include the object’s id. Finally, our object has a new updatedDate.

Last but not least we swipe over a cell to delete it and see the following log statement:

DELETE: /api/users/720983718

Wrap up

Wow, that was easy. Our SyncService class does all the magic for us. It detects changes in the database and it performs the network requests for us. It does all of that without touching our existing logic code in the ViewController. To make a second model automatically sync, we only need it to conform to our Uploadable protocol and add it to the array of synced model types.
Thanks to Swift 4’s Codable addition we were able to create a JSON representation of our models without writing code. If our JSON API wraps multiple models in one parent model we can do that easily: We just don’t add the child model to the modelTypes array. That means our sync service doesn’t observe the database for this child model at all, but if the parents model changes we sync the whole parent object with its child objects.
You can download the full source code of the sample project containing all the changes from here.

Unlike Realm’s Mobile Platform which enforces Realm also on the server-side, the REST service is not tied to any specific database implementation. We can choose MySQL but we could just as well choose Firebase or Cloudkit. That prevents a vendor lock-in in the future, keeping us flexibel.
Finally, we see that our idea of a Database-Driven Architecture makes our code cleaner and easier to read. In the current version, our only real dependency is Realm and we can work with it in whatever way we want. Since CoreData and SQLite can also offer you notifications when data changed we can implement the same feature for them, but it’s just a little bit more effort.

All of this is just the first step. There is still a lot of work left to make this solution even more powerful and more generic:

• implement retry mechanism if the internet connection is not stable
• fetch data from the service
• detect changes and download only minimal necessary changesets
• use swift models on the server to autogenerate REST service for you
• build a bridge to push changes from server to the client and vice versa without delay
• support other database systems

This could easily be realized in a framework and/or toolset implementation to bring all of these ideas to life. You can expect more from us to come in the following weeks and months. Stay tuned! 😎

Example-project: State before, State after