Roundtable: reducing the environmental impact of data centres

Industry experts discussion at CIBSE roundtable

Roundtable sponsor

With AI driving the need for more powerful chips, data centres are having to handle escalating rack heat and load densities.

The resulting demand for cooling is having a major impact on energy and water consumption: data centres’ share of UK electricity is expected to grow from 2.5% to 9% by 2030 and water consumption is expected to grow from 214 billion to 359 billion litres by 2031.’

The drive for data centre efficiency is the subject of our second article on the recent CIBSE Journal roundtable sponsored by Tate. Leading data centre engineers gathered in a London hotel to discuss concerns over resource use and what they thought would happen to the rapidly changing sector over the next five years.

Ethos’s Gavin Murphy MCIBSE said making smart decisions on the efficiency of cooling and power systems as early as possible in the design stage is essential. This means ‘selecting the right plant, minimising distribution losses, maximising opportunities for free cooling, setting sensible temperatures in the data hall – avoiding unnecessary overcooling – and tightening up controls’, he said.

By carrying out power usage effectiveness (PUE) assessments early in Stage 2, Ethos has visibility of where the big loads sit and where design effort will have the most impact, Murphy said: ‘As the design moves through Stages 3 and 4, we keep revisiting those numbers, and explore every realistic avenue to drive those loads down and improve overall efficiency.’

Seddik Daoud, of FoundDigital DS, said his team runs PUE and water use effectiveness (WUE) calculations on each site, and compares against the embedded carbon and water in the local grid. He added that the figures are designed to comply with regulations such as the Energy Efficiency Directive (EED) and emerging PUE targets, and with voluntary frameworks such as the Climate Neutral Data Centre Pact, which includes WUE targets.

Aidan Jones, Felix Cox and Chris Dunn

Daoud also carries out water-stress index assessments. ‘Where water is genuinely abundant and the local network has capacity, using it can reduce grid burden and overall energy demand,’ he said.

Other measures deployed by Daoud’s team are rainwater harvesting, high-efficiency water treatment with wash water recovery, and cooling tower blowdown recovery. High-efficiency water treatment with wash water recovery is an advanced purification process that captures, cleans and reuses the dirty water generated during filter cleaning to maximise water savings. Cooling tower blowdown recovery is the process of treating and recycling the mineral-rich wastewater drained from a cooling system so it can be reused.

The biggest single lever for minimising both energy and water is running the IT at higher temperatures, says Daoud. New-generation, liquid-cooled graphics processing units (GPUs) can accept supply temperatures up to around 45°C, which extends free cooling hours significantly and removes the need for mechanical cooling for much of the year in many climates.

Chris Hayward and Seddik Daoud

Arup’s Matt Butler said there is a ‘potential sustainability argument’ for running chips cooler, even if they are more efficient when run hot. ‘The cooler you run the chips, the longer they can last for,’ he said. ‘We can run them warmer and get a bit of an energy saving, but then you’re potentially throwing away chips more quickly.’

Aecom’s Felix Cox, a member of CIBSE’s data centre group, is looking at how to apply the TM65 methodology for assessing the embodied carbon of building services equipment. The guide won’t address the embodied carbon of the IT housed in the data centres, he said, but it shouldn’t be discounted entirely.

For example, with a direct air (cooling) system  servers might be changed once every six months. ‘That’s a monumental carbon footprint,’ added Cox.

Recovering waste heat from data centres is one way of potentially mitigating carbon footprints. If the vast amounts of excess heat they generate can be recovered, it can be piped via heat networks for a variety of uses, such as heating hospitals and apartment blocks.

This idea is great in theory, and popular among policy-makers and regulators. As MiCiM’s Sophia Flucker says: ‘You’re generating heat, so why are you wasting it?’

However, uptake of heat waste recovery has been limited in practice because of the cost, the roundtable heard.

Gavin Murphy and Barbara Smitten

Just laying the pipework to connect a data centre to a heat network can cost £1m to £2m, Butler noted, recalling a project for a data centre campus in Scotland. The installation of a heat network had not made sense economically or environmentally in terms of the embodied carbon that digging up nearby roads would have entailed, he added.

The heat network must be integrated into a development from the very start, at the masterplanning stage, added Butler. ‘When you try to plug in afterwards, it really struggles on environment, cost and everything else.’

Cost isn’t the only barrier to heat waste recovery solutions, according to Daoud. Pointing to west London’s Old Oak and Park Royal regeneration scheme – to which many providers of nearby data centres have committed to supply heat – he said: ‘It’s a nightmare to get the district heating pipes to the site. That requires cooperation at municipal level.’

But this isn’t the case everywhere, such as in Scandinavia, said Cox.

‘In the Nordics, there are tens of major data centre waste heat recovery schemes, which are operational and heating the local housing stock,’ he said, adding that lack of progress on waste heat recovery is ultimately a “cultural” issue.

‘It’s political will,’ he said, adding that waste heat recovery feasibility assessments are becoming a ‘default requirement’ throughout Europe. But the lack of progress in the UK on recovering waste heat recovery from data centres is ‘a missed opportunity’, said Cox: ‘It would be such a great story for our industry.’

Facing the future

The roundtable discussed what the future holds for data centres and the technologies used to cool them. IT densities will continue to increase, which will have ramifications on the cooling infrastructure that data centres require.

‘I wonder if the industry is fully settled on direct-to-chip cooling,’ said Cox, ‘or whether we may end up revisiting some immersion cooling technologies when the densities increase further.’

Immersion cooling is where entire servers and their electronic components are submerged in a bath of dielectric (electrically non-conductive) liquid. It is considered the ultimate thermal management solution because fluids transfer heat significantly better than air or partial liquid loops, allowing data centres to handle the extreme heat densities of next-generation hardware.

Chris Dunn and Simon Steed

Arup’s Chris Dunn predicted that there will be a return to immersion cooling once fluid materials improve. ‘It [immersion cooling] was a great idea, but premature from a materials perspective,’ he said.

Availability of Grid and power will still be a ‘major choke point’ for data centre development in five years, said Tate’s James Maher: ‘That’s really going to be the delay to developments in Europe,’ he added – and the huge sums going into the data centres increases the pressure to deploy them swiftly.

‘Moving forward, the design phase will reduce significantly,’ he added, ‘driven by the unprecedented revenue generated by the
likes of Nvidia’s server racks.’ This means multidisciplinary teams of M&E engineers and structural engineers will be working together around the clock to meet clients’ expectations.

The industry’s skills shortages, which were also discussed by the roundtable, are unlikely to be solved in five years – or even 10 years, said Flucker: ‘It’s a bit of a gold rush at the moment, because there’s so much money around and so many people wanting to capitalise on AI.

‘That has a knock-on effect on data centres and the whole ecosystem around it – the service providers and the product developers.’

Gavin Murphy said data centre engineering and the challenge of making it more sustainable should be an attractive proposition for engineers. A number of engineers at Ethos had moved into data centres from commercial and residential roles, he added.

‘Some of them were quite sceptical at the start, but now they’re really enjoying it,’ he said, noting that the enormity of the challenge of reinventing data centres will attract more into the industry.

‘The extent of innovation in the industry is phenomenal, and that really excites me – and, hopefully, it’s something that could entice engineers. What can be more exciting than a whole new industry to design?’

Data centres give exposure to the full holistic design of the building, Murphy continued, describing it as a ‘fantastic experience.

‘Engineers are working closely with teams who acutely understand the services – the operations team, clients and SMEs – and they’re all working together to improve the product and delivery of  amazing projects,’ he said.

Dunn believes the data centre sector may not be growing at its current breakneck rate in five years’ time because of limits on the amount of data that people need, ‘There has to be a peak to the industry,’ he said.

PM Group’s Simon Steed agreed. ‘We should have a debate about whether we’re using too much data in the first place and what we are actually going to do with it all,’ he said.

A slowdown in data centre development may be good news, said Tim Hale from Meit. ‘It will give us stability in the market,’ he added. ‘To invest, you need stability, so the rate of development has to slow. The rate of change we’re seeing at the moment isn’t sustainable.’

Gavin Murphy and Christopher Leahy

The huge giga-scale data centre developments may not prove sustainable in the long run, Steed added. ‘The whole playing field will be very different in five years’ time. I’m not convinced the hyperscale developments we have seen will be continued,’ he said.

‘Where we will be in five years’ time will be nothing like what we see at the moment. It could be a completely different arrangement – more spoke and hub. It’s just not going to happen. You’ll have far smaller facilities dotted around.’

While larger data centres will continue to be required for applications such as cloud computing and AI machine learning, smaller 10MW to 20MW sites will be required in urban environments where speed is of the essence – such as for helping autonomous vehicles to navigate, believes Steed.

Hale suggested the rate of growth in IT racks’ computing density means they could be rolled out as unitary, modular developments that could be paired with a generator and a cooling source. Cabinet-sized racks with mini compressors installed on the back for cooling could be housed in a shed in every park, he added.

At the other extreme, Maher suggested that Elon Musk’s proposal for data centres in space may happen within the next five years.

However, the data centre sector as a whole needs to become ‘less dependent’ on cooling, said Deern’s Christopher Leahy: ‘That’s the next thing that has to happen. We have to reduce that heat generation inside the data centre: it can’t increase and increase.’

Flucker added: ‘There’ll be problems we haven’t thought of yet – but hopefully there will be solutions to some of the problems we do already know about.’