How we built the new BBC Homepage
In this technical retrospective (2015), @achalkley shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
Core Principles for Independent Builders:
Defaulting to simpler architectures (such as monolithic frameworks, SQLite, or managed VPS) prevents early engineering bankruptcy.
Evaluating true compute and egress costs reveals that high cloud bills can often be slashed by 80%+ through judicious re-platforming.
Code you write at 2 AM as a solo engineer must be easy to debug six months later without complex microservice dependencies.
Community Debate Themes:
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
CATEGORY Frugal Engineering →
HISTORICAL ERA Cloud & Mobile Era (2014–2019)
COMMUNITY SCORE ★ 154 points
TOTAL DISCUSSIONS 94 comments
Hmm. While it's nice to push rendering out to clients for saving your server CPU, that can lead to a suboptimal mobile experience, requiring more CPU and battery power on the mobile device to work on the javascript and render the page. There are other factors, for sure, but you want to be keeping client side javascript down to a minimum.
Taking a quick spin through yslow in the mobile browser suggests they've got a number of areas to improve on to make the time to screen significantly better on mobile devices (even on a fast connection here it took several seconds to even start showing me content, and several more before it had finally loaded everything)
Given the world wide reach of the BBC, expecting high speed and low latency networks seems like a bad idea. In the US, 3G & 4G typically see 90-100ms latency per request. Mobile Yslow is reporting that they've got 21 javascript scripts alone on the page. IIRC The android browser will limit itself to 4 threads retrieving content typically so that's (21/4 * 100ms) 525ms just lost in latency requesting the javascript, let alone actually downloading it and the overhead of the javascript renderer. It's also pulling in content from 21 different websites, so at the bare minimum that's 21 DNS calls being made (with the same latency penalty!) A bunch of those are being done just to load a single piece of content too, which is a little crazy.
Don't get me wrong, the site looks good.. it's just for a 'mobile-first' experience, they seem to be missing the all important time-to-screen and giving the mobile user a lot of work to do.
A useful tool from google for analysing the site for both mobile and desktop: https://developers.google.com/speed/pagespeed/insights/?url=http%3A%2F%2Fm.bbc.co.uk%2F
and a good talk from last year's Google I/O conference on optimising the mobile experience: https://www.youtube.com/watch?v=WrA85a4ZIaM
I do love when companies share this stuff. Thanks.
I don't doubt this is true, but its worth noting you can get good performance out of a "blocking" framework too. Node.js does better than others in some situations, but in this its not a snowflake, and is in some regards worse.
But I will criticize priorities: I think this is too much fad, not enough practical. the experience is notably worse than the old site, and it seems like they just threw buzzwords at their problems instead of really crafting a solid solution.
I think where they're coming from, maybe this makes sense - they needed to overshoot from their previous platform. But I think they'll find it problematic in the long run and change some of their approaches and release a new site, and that will be a good platform and serve their needs for a good while.
Most of their tests take half a second each to run? Some examples: "The first story displays an image and not an alt text attr if one is defined. (793ms)" "The module banner color is BBC News Red (552ms)"
The 13 tests shown in the screenshot take an average of 577ms each, a total of over 7.5 seconds. 13 simple tests with wild variance in execution time. Checking a module banner color? 42ms. Checking a module background color? 552ms. So a 12x increase for checking a different color within a module?
Those tests are going to rot because of the expense of executing them and they'll be discarded. Over a 500ms average per test just isn't sustainable, particularly at the scale required for checking every conceivable kind of background color.
It's like they monitored the front-page of HN everyday for a year, made a list of the top mentioned technologies and then gave that list to their management.
Their manager said "Your goal is to write a blog post mentioning each one of these technologies, and add links so it appears in dark-bold-blue text. If we don't have a project using a trendy tech-stack... you bust your ass and get something up and running."
The engineers balked... "but why?"
Then the boss said, "I'm tired of being ignored in 'Who's Hiring' on Hacker News. We make this article and they'll all come begging us for jobs. At the BBC we don't wait for news to come to us, we make the news. And THAT'S what we call journalism kids."
This mentions using Node/Express to help with concurrency. Ok. But this also mentions using 'isomorphic' javascript with React, and also not completely reliant on cache. React.renderToString (and all other isomorphic options -- Rendr, etc) are synchronous and slow. (rendering a large page in React w/ static data would take ~600ms; whereas the same page with with Mustache in Go for example would take ~30ms). End result, decreasing concurrency, and hurting overall performance.
OP - can you shed any light on how this is actually impacting your performance? Or maybe things that you had to do to get around the problem (ex - details of 'module level' cache with Redis etc).