Open Social Is Evolving: The First Phase of Fully Decoupling

Open Social recently started working on an exciting and challenging project evolution: decoupling Open Social’s online community software. But what is decoupling, why did we decide to take on this challenge and what will the process entail? This blogpost will give a technical overview of Open Social’s decoupling process, and serve as a useful resource to anyone else interested in creating a fully decoupled Drupal or CMS (Content Management System) system.

Over the coming months we will redevelop Open Social’s front-end with React and ReScript and switch to an API-first system through which a redesigned front-end can communicate with the backend. Keep reading below to learn more.

What is decoupling?

A decoupled system allows changes to be made to any one part of the system without affecting the other parts. A decoupled system thus consists of interrelated parts of a system that are technically independent – parts that can communicate with one another without altering each other.

Open Social uses the Drupal CMS to build and deploy online community platforms for its clients. Websites or web applications that rely on CMSes like WordPress, Joomla, or Drupal are traditionally coupled. In this type of architecture, the backend data-storage and the frontend or user-interface is part of the same technical system and inseparable.

To decouple Open Social, we thus need to move from the current setup to a setup in which the backend and frontend are technically separate systems.

Why Open Social decided to decouple

Despite a number of benefits that a coupled system offers, a drawback of the current setup is that Drupal’s coupled frontend model is centered largely around web pages. These web pages create a system of separate page navigation instead of allowing for dynamic content that can change within a page through in-line and page-based navigation. We thus realized that this system’s coupled frontend creates a disruptive user experience as a page needs to refresh each time a user wants to view different ‘on-page’ content.

Open Social’s platform, which relies on different types of content populating a single page, can thus be enhanced through moving to a decoupled setup. A decoupled setup will allow users to interact with different content on a single page without having to navigate away through page-based navigation. Enabling this kind of interaction will only be possible with more complex JavaScript solutions than what Drupal’s Twig and JQuery frontend currently provides.

In addition to improved user-experience and interactivity, a decoupled system also has other advantages, including:

1. It allows independent development: We can focus on either the front-end or back-end at any given time, developing specific features for either without having to work on both of them. This will save time and resources.
2. It reduces barriers to the adoption of new technologies: Without having the frontend coupled with our backend, it will be easier to adapt to the newest technologies.
3. It opens us up to more integrations and external channels: With data being available over APIs we can serve a much wider range of channels and integrations.
4. It makes it easier to recruit new talent: Switching from a Drupal-only platform to other, more popular, frameworks such as React will allow us to more easily recruit developers in the future as we expand and scale-up.

What will a decoupled Open Social look like?

The design for the decoupled Open Social will be based on our existing frontend, adopting the sleek Sky layout theme from Drupal. But of course, we will be making the improvements to the frontend design that we’ve been itching to make, but were unable to implement due to limitations in the coupled setup! The goal of decoupling Open Social is, after all, to develop a high-quality web front-end that enables us to create highly engaging experiences through enjoyable interactions that are accessible to all.

We will build a new frontend that is ready for the future. The decoupled system will allow us to create an Open Social that looks and feels more like a modern social media platform with real-time content-streams all taking place on a single page. Users will be able to dynamically load more content, use real-time on-page data-streams and post chat-messages or comment without having to navigate between separate pages. This is a core focus for us, as it will allow users to interact with Open Social in ways that they are familiar and comfortable with. This setup will also open up development of future features such as real-time chat, video players and content previews for internal linking.

The decoupling roadmap

It is great to have a forward-looking vision of what a decoupled Open Social will look and feel like. But underlying everything written above is the technically complex development process of moving Open Social from a coupled to a decoupled codebase. An outline of the decoupling roadmap looks like:

1. Researching decoupling methods, technologies and technical requirements

Preparation is key. Taking on such a complex project requires adequate research and informed decision making.

Until now, most of the work that we’ve done has been theoretical: making comparisons of frameworks and languages to decide which tools we will use. You can read more about it below in this blog post.

2. Validating our theoretical work

We have recently moved to the second phase of the decoupling process. We are now validating the theoretical work that we’ve done so far by prototyping the new frontend set-up. This will provide us with insights into any gaps we may not have covered yet. Preparation and research only get you so far: you have to test your assumptions before you implement them completely.

3. Moving to the actual development phase

After the validation of assumptions is completed and the infrastructure is set-up, we will be able to begin developing the actual application.

As with any new project we will need to make sure our development infrastructure is up to standards to ensure that we can achieve the high velocity of development that we’re after. This includes ensuring we deliver a performant and accessible new front-end.

Throughout the development process, we will also be working on making sure we can provide a seamless transition for all of our current and future customers.

4. Roll-out

Once the foundation for a decoupled Open Social is built, we will be able to roll out the features as they become available to new and existing customers. This will include releasing new extensions or upgrading existing features with improved user-experience. To give our customers a new and improved platform, without creating any inconvenience for them or their users, is the end-goal of this process.

Phase 1: Research

The first step in the decoupling roadmap has been researching and selecting the specific technological and technical decisions that we need to take. These include:

1. Choosing a decoupled approach

There are different decoupling models. The first step in decoupling is to decide which model to follow: You can either progressively decouple or fully decouple.

When adopting new front-end technologies such as React you can take the output of a system such as Drupal and progressively enhance it with scripts. This works well for self-contained features. One area where we’ve previously applied this is in the dynamic search suggestions.

We decided that we need to implement another method, namely: fully decouple. We decided that we need to build a completely new front-end application that uses new technologies. The existing application is thus modified to provide the data it contains through an API.

Progressive decoupling usually requires executing JavaScript on the client to fetch the needed data before something is displayed. This makes the content difficult to access for automated systems. To combat this we could run the JavaScript on the server but this is very difficult in our current Drupal application. It’s much easier in a separate application that can handle JavaScript. Our new fully decoupled front-end application using ReScript would handle this scenario well.

2. Switching to API-first 

The primary focus of an API-first project is to provide the data it contains in an easily consumable format for other (automated) systems.

In the context of Open Social’s decoupled project, the front-end will be headless. It will fetch data that it needs from the API-first Open Social instance and display that data to the user. Any data it receives from a user will not be stored in the front-end application but will be passed on to the Open Social installation, using the API for storage.

The development of an API for Open Social works towards positioning the Open Social product as a platform. This will also mean that in the future this API can be easily used by for example the mobile app.

Additionally, by building the API with consumers outside of Open Social in mind, the API can be exposed to customers to let their own applications talk to Open Social. This could lead to novel ways of integrating Open Social into our customers processes.

3. Choosing ReasonML for our front-end

In today’s web, interactivity requires the use of JavaScript. However, JavaScript itself is a weakly typed language. This means that information about data you’re manipulating is known to the program that executes the JavaScript, but is not taken into account when writing a program. This is problematic because trying to execute a function on an incorrect type can cause errors.

After a comparison of plain JavaScript, TypeScript, Flow and ReasonML, we have selected ReasonML as our front-end language. We believe the language provides developers with better tools to create efficiënt and stable solutions.

ReasonML lets you write simple, fast and high-quality type-safe code while leveraging JavaScript. In preparation for this choice, Open Social developer Alexander Varwijk did in-depth research comparing the different front-end languages.

4. Choosing our back-end API format

With the direction for the front-end decided upon. We had to decide on how we would provide it with data. Two popular options exist for this: GraphQL and JSON:API. After evaluating both we have decided to adopt GraphQL.

While Drupal core has chosen to integrate the JSON:API module, the GraphQL module is still in active development in the space of contributed modules. This makes either option viable from Drupal’s perspective.

We’ve noted that the JavaScript (and by extension the ReasonML) as well as the React communities have a preference for GraphQL. GraphQL makes it very easy for the front-end to indicate what data is needed. It leaves the difficult work of bringing together all the requested data to the server. This is in contrast with JSON:API where the server decides how data is exposed, which tasks the front-end with bringing together related data.

We believe that for this project either technology could work. For us, it makes sense to utilize as much existing front-end tools as possible and invest in expanding the GraphQL tools in Drupal.

The reward at the end of the road

We have only recently started out with the decoupling process, with a lot of work still ahead. But throughout our research, we have already become convinced that decoupling will result in an Open Social that is faster, leaner, more flexible and easier to continuously update and improve.

Most importantly, however, as a SaaS company, we will have a product that will give our clients new features, capabilities and a future proof product. This will help them build more engaged and active online communities using Open Social. At the end of the day, we are taking on the challenge to become the best online community platform service available.

We will be publishing updates on the decoupling process as we go along. If you want to follow the development of the new Open Social platform, feel free to subscribe to this blog or follow us on Twitter and LinkedIn where we will post the updates.