Refactoring 10 Years of Legacy

In my time at Agworld, I embarked on a long-lasting project to pay back part of the technical, and design-debt that had built up in its core web product over a decade. The focus was to look at the way we handled styling, with a view to improve performance, legibility, accessibility and overall developer experience.

Agworld takes its technology very seriously, so this was seen as an extremely important project that I was well-suited to tackle. I led this project over the course of 3 cycles lasting 6 weeks each; with the assistance from members of a team dedicated to technical improvements.

Four Bootstrap-sized Problems

The Agworld codebase at the time was a study of its history; filled with interesting artefacts and styling decisions in an environment constantly in flux. Over the years - tastes, personnel, technology and "best practices" evolve, and these were represented prominently, which made it a significantly challenging landscape to work in.

The main problems and pain points that I identified here were:

• Every page could use a different layout; so while things appeared very similar, the underlying code was actually very different, which posed a maintainence issue
• Every page bundled up and served 4 (FOUR) versions of the popular styling library Bootstrap. This make the bundle size to download enormous, and would cost the business a great deal in hosting costs
• All Javascript and CSS resources blocked rendering, leading to a very large First Contentful Paint (FCP) time
• There were no clearly defined performance budgets, so we didn't know just how poorly we were doing, or where to focus

One Bite at a Time

for each desired change, make the change easy (warning: this may be hard), then make the easy change - Kent Beck

A colleague of mine mentioned this quote around the time that this project was kicking off, and it's certainly far easier than it sounds! It was extremely difficult to get a broad understanding of the breadth of the change that I needed to make, so I approached it by dividing work into high-level themes, and then those themes into steps and stages that could be shipped incrementally.

Defer-loading Resources

Initially, identifying key performance budgets and metrics was my top priority for this project. Understanding how we were performing, and targeting changes specifically to improve these scores provided a great baseline for work. I focused primarily on First Contentful Paint (FCP) with this work, with an eye to improving Time to Interactive (TTI)

Certainly, the largest problem here was the size of the payload that needed to be served, but the order that it was loaded was also something that needed to be rectified.

The first change I decided to make here was to make sure that these resources were defer-loaded, or loaded asynchronously, so that it didn't block the critical rendering path.

From the figures above, the measurable FCP score saw improvements of ~50% on some pages, and from not measurable to well below 3s on others - which is a significant improvement, and allowed some of the other more impactful refactoring to take place.

Splitting the Bundle and Removing CSS Dead Wood

Agworld's engineering team employs a high shipping cadence, and has spent significant resources improving this over the time I worked there. At one stage we - as a team - went from shipping a large update every 2-4 months, to shipping several times per day, and dozens of times per week. This exposed an issue with the core product's architecture, where it forced a refreshed asset payload on a large number of those deployments. When the payload is as large as this was, that could be extremely impactful on users - particularly those located in remote areas; so it was extremely important that these assets were cut down to size.

The great majority of this payload was actually unused - meaning it had literally no impact on the final product. On some of the most heavily used pages of the site, over 95% of the served CSS was unused. This was due to the fact that all resources were bundled together, even if they weren't relevant to that particular page. So one page may be using Bootstrap 3, another with Bootstrap 4, and all of it was combined and served together.

The solution I embarked on was to split these resources and only serve what was required for a particular page.

In real-world numbers, the initial size of the bundle was ~430kb. Spltting this up into what was required for different pages meant potentially some only needed to download ~131kb, and others ~162kb. Any reduction here is welcome, but this represented one of the most signifcant, technical savings to the company in the running of the core product. This work was also volatile, so I - along with help from the technical team - put in significant testing time and effort to ensure high quality.

Consolidating layout

The final step in this refactoring project was to consolidate the three, vastly different layouts into one.

This work would unblock further initiatives I would go onto undertake around making Agworld's core product responsive; but also dramatically reduce the cognitive load and complexity facing developers working on new and existing features. A real win for the developer experience at Agworld.

Sandbox Resources

Initially, the first problem that I faced here was determining how to ensure that legacy content and experiences still behaved as desired with a single layout. It was important to ensure that features that people were using out in the field were not disrupted.

I decided to use a sandboxing strategy to isolate resources, layout, and content based on the different layout "contexts" (my words) identified in the audit shown above. Each of these served namespaced layout, resets, and resources required for content within it - effectively shielding it from other, potentially damaging influences.

Completing this meant that legacy content could safely be maintained, regardless of other work that had been undertaken. It's difficult to overstate just how important this is from a maintainability perspective. Being able to utilise the same markup in any context leads to a far more consistent developer experience, which in turn can make maintenance a lot more straightforward.

Improving Experience - Perceived performance

Improving performance metrics is certainly a great way to improve the overall experience that someone has with your product, but there are times when doing this can unintentionally create a poor experience for the person using the product. Deferring the loading of styling assets - in this instance - created quite a significant Flash of Unstyled Content (FOUC). This is where the structure of a page loads in a raw, unstyled format before styling is loaded and applied.

While the product loaded significantly faster than it did before, the overall experience felt worse because of this FOUC; so I created a flexible "loading experience" for developers to opt into and customise to the experience that they wanted to achieve. This could be as simple as a loading spinner, or as complex as an animated skeleton, or loading screen.

Like all things, making sensible defaults and opt-in experiences is important to improving the overall developer experience.

Clearing the Path Forward

This was one of the most impactful technological projects I undertook during my time at Agworld; working in the context of 10 years of legacy front-end code; and achieving some significant outcomes:

• The complexity of the developer experience was significantly reduced
• Performance metrics improved by orders of magnitude (by over 50% in some cases)
• Split the asset payload into several bundles rather than a single one, reducing it by almost 70% in some cases
• Drastically improved real and perceived performance
• Solidified overall user experience to create a cohesive, consitent product