A migration sync tool for the New York Post: A spotlight

At Alley, we don’t just deliver websites – we pride ourselves on strong, long-term partnerships with our clients. One of our longest ones, at over eight years, is with New York Post and WordPress VIP

New York Post is one of the largest sites hosted on WordPress VIP and includes three separate sites: New York Post, Page Six, and Decider. Combined together, the sites have several million pieces of content. In early 2020, Alley worked with New York Post and WordPress VIP to move all three sites from the legacy WordPress.com VIP hosting platform to the new VIP Go platform. 

When migrating sites to a new hosting platform, the simplest procedure is to pause updating the site, back up the data, and then move the data and media assets to the new hosting, then switch all traffic to the new site and start publishing content again. Due to the size of the New York Post sites, copying all the data from one platform to another took several hours, plus additional time for indexing the content with ElasticSearch, which is used to make the site blazing fast. However, since the news cycle never stops for even a few minutes, this method was not an option. We needed to develop a way to continuously sync content between the two hosting platforms so that the switch could happen with very minimal downtime for The Post staff and zero downtime for the site visitors.

The synchronization tool needed to meet the following requirements:

  • Any new or updated content on the existing platform, including uploaded files, would be synced to the new platform.
  • All URLs and unique identifiers would be the same on both platforms, as this data is shared out to many other platforms.
  • Any items changed during the data copying process would need to be queued up for synchronization after the initial copy was done.

The logical vehicle for this synchronization process was the WordPress REST API. The REST API allows for submitting data to WordPress in a structured format. It can be extended to allow for custom functionality and has built-in support for data validation. The communication happens using standard web technologies, so no special setup was required between the two platforms.

On the new hosting, we added a custom API endpoint and secured it with Application Passwords (which was available as a plugin at the time and is now part of WordPress core). The endpoint received data from the live site and would create or update all necessary content and also synchronize all metadata. On the live site, we added custom code that ran any time content was saved. This code would either mark content for later synchronization if the copy process was ongoing or synchronize it immediately.

Once the synchronization tool was ready to go, we were able to work with WordPress VIP to perform the time-consuming copy process several days before the final switchover. All changes during the copy process were queued up and then synchronized once the new site was ready. Once the queued changes were synchronized, the content on both hosting platforms stayed identical until we were ready to switch over. We made the final switch early in the morning a few days later, and The Post staff only had to pause their work for a matter of minutes as the DNS switched over.

Now that the New York Post is running on the VIP Go Platform, the site is faster and more stable. The move to VIP Go has also allowed us to transition to the WordPress Block Editor, which provides an intuitive editing interface that the New York Post editorial staff loves. The Block Editor and WordPress VIP platform also gave us a foundation to build an extremely powerful curation tool which we will cover in a future post. Stay tuned!

If you’re interested in learning more about our ongoing work with the New York Post, definitely make sure to check out our case study on the project, with loads more details on the work that we’ve done for them over the years.