Performance Gains Using BigPipe for Open Social

This article investigates how feasible it is to implement the Drupal 8 BigPipe core module for Open Social. We explore potential caveats and give recommendations.

Introduction to BigPipe

Facebook invented the BigPipe technique which splits an HTML page in “pagelets” which then are simultaneously build by the server and served by the browser. The technique does not require updates of the browser but relies purely on javascript. The illustration below shows how the people with a slow connection can start using the website in the first seconds before the full page is loaded which drastically improves the perceived performance. BigPipe is especially significant for people that have a slow internet connection which is likely a majority of the 3 billion internet users worldwide. The ability to use new technologies like BigPipe helps Open Social to be competitive against closed-source alternatives.

BigPipe and Drupal 8

The BigPipe module was introduced as experiential module in Drupal 8.1 core. At the keynote in DrupalCon Dublin Dries BuyTaert – the founder and project lead of Drupal – mentioned that it is important for experimental modules to become stable within a year or they likely be removed from core. The BigPipe module currently is in beta and should become stable in Drupal 8.3. It seems that there is one known major issue left to be tackled before the module will be stable.

Experimental enabling BigPipe in Open Social

To determine how well BigPipe works in Open Social we conduct an experiment on a clean installation of the distribution. We first enable the big_pipe and dynamic_page_cache modules and disable render cache and advagg aggregation. To see what is going on under the hood we use Xdebug. In the video below you could see a demonstration.

We see that a response is being created by the “HtmlResponseBigPipeSubscriber” object. This event subscriber allows objects to hook into the response process of Drupal 8. In the initial response the BigPipe javascript and the BigPipe placeholders are added. Then the javascript in “big_pipe.js” loops until all the placeholders are replaced. During the loop it executes an empty Ajax call which allows the server to send the placeholder replacements.

To see how it works we will see in what order blocks are being loaded for the homepage of a logged in user in Open Social.

1. The initial page contains the header and footer and the menu on the left. The drop down in the explore menu is then already working.
2. Afterwards the user menu in the header on the right is loaded. Including the notification centre and the add content button.
3. Then the full activity stream view block is loaded including the pager except that some images will be loaded afterwards
4. At the final step the view blocks on the right are rendered.

When looking at the initial BigPipe placeholders IDs we see that the activity stream block has a single placeholder while the blocks have separate placeholders for each block. This is not ideal because the activity stream can be split into multiple pagelets as well based on the cards that are already there.

Conclusions and recommendation

The default behavior of the BigPipe module seems to be stable enough to be used in production environments. However, for some important pages like the activity streams some additional work in Open Social is necessary to create extra placeholders inside the view blocks. It could provide some value to already enable the BigPipe module when your Open Social project is used in regions where people have slow internet connections.

For most cases I would recommend to wait for the module to be labeled stable which probably is the case in the Drupal 8.3 release. All help is welcome to get the module to this stage and I would recommend using the drupal.org issue queue for this. We use this Open Social issue for improvements in the distribution related to the BigPipe module.

Contributing to Open Social or Drupal

So how to go about contributing to Open Social and/or Drupal in general? For Open Social a good starting point is the Open Social Documentation Guide. If you want a starting point for helping Drupal to grow you can find many different ways on the ‘ways to get involved’-page on Drupal.org.