Demand for AI "Surging"

[edison]

US tech giants investing $500bn in AI infrastructure. https://www.bbc.co.uk/news/articles/cy4m84d2xz2o

The announcement of the new open source AI model from DeepSeek in China has rattled Silicon Valley. It's performance is supposedly broadly comparable with western models from the likes of OpenAI etc. but at a tiny fraction of the cost of required GPU hardware and compute.

https://www.theregister.com/2025/01/...eek_r1_ai_cot/

Shares of NVIDIA and related companies like ASML are down c.10% in pre-market trading. Will the $500B investment announced be obsolete before the new data centres are rolled out? What does this mean for the supposed 'surge' in AI demand this year?

https://financialpost.com/pmn/busine...-stock-selloff

[willendure]

The announcement of the new open source AI model from DeepSeek in China has rattled Silicon Valley. It's performance is supposedly broadly comparable with western models from the likes of OpenAI etc. but at a tiny fraction of the cost of required GPU hardware and compute.

https://www.theregister.com/2025/01/...eek_r1_ai_cot/

Shares of NVIDIA and related companies like ASML are down c.10% in pre-market trading. Will the $500B investment announced be obsolete before the new data centres are rolled out? What does this mean for the supposed 'surge' in AI demand this year?

https://financialpost.com/pmn/busine...-stock-selloff

The full R1 model is 671 billion parameters. Assuming that is FP16, that is approx 1300 GB to run the full model. A typical nVidia 4090 card has 24GB of memory so you would need around 50 of them to run it! Of course, the data centre GPU models have more memory than that, nearly 240GB for the largest ones I think - you would still need 6 of them and each one probably costs as much as a luxury car.

But DeepSeek have applied the techniques to derive quantized and smaller models, and also made it all open sourced. Which is cool. Just bear in mind that these smaller models are significantly less capable that the full model.

I do find it interesting and surprising that whatever OpenAI does is reverse engineered and open sourced pretty quickly. Stable diffusion is a better image generator than Dale ever was for example. LLama has been great for enabling developers to experiment with LLMs on their own hardware.

I would say, its like the old USA vs USSR space race. Now that there is some competition, if anything we will see the USA using its wealth to ensure it wins.

[edison]

I was in Cambridge this week and had a bit of time to kill so went for a wander in the area around the main station. Apple has a large office opposite the station which focuses on AI/ML research and is eventually going to have 900 people based there.

Almost directly opposite is Microsoft's research lab which has c.200 research engineers. Just a little further down the same road is Graphcore which could have been the UK's answer to NVIDIA but sadly has been split up and sold to Meta and Softbank. I noticed they are recruiting quite a lot of ML engineers, AI research scientists etc.

I think these are the kinds of roles that are still likely to be the main AI ones needed in the next year or two whilst tech companies work out how to bring more AI products to the market that other companies will pay money for.

My career is winding down now but if I had my time again, I think retraining in an area like human-computer interaction/product management for AI would have been an interesting career switch.

[willendure]

I was in Cambridge this week and had a bit of time to kill so went for a wander in the area around the main station. Apple has a large office opposite the station which focuses on AI/ML research and is eventually going to have 900 people based there.

Almost directly opposite is Microsoft's research lab which has c.200 research engineers. Just a little further down the same road is Graphcore which could have been the UK's answer to NVIDIA but sadly has been split up and sold to Meta and Softbank. I noticed they are recruiting quite a lot of ML engineers, AI research scientists etc.

I think these are the kinds of roles that are still likely to be the main AI ones needed in the next year or two whilst tech companies work out how to bring more AI products to the market that other companies will pay money for.

My career is winding down now but if I had my time again, I think retraining in an area like human-computer interaction/product management for AI would have been an interesting career switch.

Did my CS degree there graduating in 1997. At that time the main CS lab was being rebuilt out on the west side of town with funding from Microsoft. So the one near the station must be more of a commercial lab, I think? And the one out to the west is the universities teaching and research lab.

I bought pre-ipo shares in Graphcore. When it was bought out by softbank, they paid enough to repay all its debt with zero left to the equity portion. So I lost the entire investment, which was not huge, and certainly a lot less than the founders! I also have some pre-ipo shares in Cerebras, which might still pull off an IPO - fingers crossed on that one. If it does I will make back a nice profit overall so not too hung up on the Graphcore loss.

In some ways the two chips were similar, both highly configurable dataflow machines. Cerebras just went big with their wafer scale architecture in a way that Graphcore did not. Another issue with Graphcore is that their chip did not shine with LLMs, where 8-bit floating point formats can really help pack in all those billions of parameters and crunch them faster. nVidia with more income and a tighter chip lifecycle was able to adapt to this trend much quicker and by the time that happened the Graphcore chip, amazing as it is, was kind of irrelevant.

Interestingly some of the key figures in Graphcore were involved in the transputer revolution in the late 90s. Transputers could run at a much higher throughput than Intel CPUs. In the end it did not matter, Intel had the entire world as its customer, and the steady flood of money and tick-tock development pace pushed everything else to the sidelines. Moral of the story - start your chip business in the USA, because you are going to need a lot of money!

I graduated from my AI masters in 1998. I have always been way ahead of the curve. But that is not something that does one any favours. There is a right time for everything and being early or late means you miss out. Most of my career has been writing boring enterprise software in Java. But what I would say is that things like this take at least 20 years to go from research idea to mainstream. I think for the last 20 years AI has been a research field, and the kind of people needed are PhDs, who would work in an accademic setting or a commercial research lab like the ones we are talking about here. But AI has certainly reached the mainstream now - seems like every youtuber is installing R1 and spouting off whatever nonsense gets viewers to their channel.

Businesses are interested in doing something with AI and need specialists who do know what they are talking about. So I do see a growing opportunity for contractors with the right skillset to get involved. I have been monitoring AI jobs on jobserve and I definitely am seeing several per day that ask for python, tensorflow, langchain and so on.

[edison]

I graduated from my AI masters in 1998. I have always been way ahead of the curve. But that is not something that does one any favours. There is a right time for everything and being early or late means you miss out. Most of my career has been writing boring enterprise software in Java. But what I would say is that things like this take at least 20 years to go from research idea to mainstream. I think for the last 20 years AI has been a research field, and the kind of people needed are PhDs, who would work in an accademic setting or a commercial research lab like the ones we are talking about here. But AI has certainly reached the mainstream now - seems like every youtuber is installing R1 and spouting off whatever nonsense gets viewers to their channel.

Businesses are interested in doing something with AI and need specialists who do know what they are talking about. So I do see a growing opportunity for contractors with the right skillset to get involved. I have been monitoring AI jobs on jobserve and I definitely am seeing several per day that ask for python, tensorflow, langchain and so on.

I think you're right, AI has been very research based and to some extent still is. I looked at some sites for internship vacancies and it was interesting to see what type of organisations are recruiting students interested in AI/ML (undergrads up to PhD).

Nearly all were hedge funds/IBs and large tech companies.

[willendure]

I think you're right, AI has been very research based and to some extent still is. I looked at some sites for internship vacancies and it was interesting to see what type of organisations are recruiting students interested in AI/ML (undergrads up to PhD).

Nearly all were hedge funds/IBs and large tech companies.

Interesting that you see hedge funds and IBs recruiting. Presumably these are more applied positions as opposed to research, which I would expect to be happening more on the tech company side. That is, they want people that can hook up their AI systems to the data and figure out how to make them even more money, regardless of whether its advancing the research agenda.

[edison]

Interesting that you see hedge funds and IBs recruiting. Presumably these are more applied positions as opposed to research, which I would expect to be happening more on the tech company side. That is, they want people that can hook up their AI systems to the data and figure out how to make them even more money, regardless of whether its advancing the research agenda.

Actually in some cases it seems to be both applied and research. JP Morgan are recruiting both - the applied side in their ML Centre of Excellence and the research side in their Research Team.

Although I suspect the latter aren't going to be publishing many research papers at the top AI conferences like the big tech boys do. Jane Street for example specifically says academic publication won't be allowed.

[willendure]

Actually in some cases it seems to be both applied and research. JP Morgan are recruiting both - the applied side in their ML Centre of Excellence and the research side in their Research Team.

Although I suspect the latter aren't going to be publishing many research papers at the top AI conferences like the big tech boys do. Jane Street for example specifically says academic publication won't be allowed.

Interesting. Where is the JP Morgan ML Centre of Excellence? Was in Glasgow recently and saw JPMC have a whole building there now with their logo in big writing on the side. Worked for them over a decade ago when they just had a couple of floors in a different building.

[ladymuck]

Well that's encouraging: https://www.theregister.com/2025/06/...stings_comptia

Want a job? Just put 'AI skills' on your resume

Job postings for people with AI skills reached an all-time high last month, the training and certification company said, with some 125,000 open jobs in the tech sector mentioning the need. Those user-level skills, CompTIA noted, could involve anything from a marketer using ChatGPT to help develop new language, to a web dev using just a bit of AI to help them generate or debug some code.

In short, there's no reason to go the software engineer and LLM expertise route, because the AI jobs can be found in the "summarize my boss' emails for me" and "create some marketing campaign mockups with X, Y and Z elements that I couldn't get a human to do" spaces of expertise, not the "build me a new LLM" and "connect this chatbot to this unformatted data and work me a miracle" one. You know - "can you use Copilot?" not "build me a new copilot."

CompTIA didn't make clear what specific AI products businesses may be looking for expertise in, but the trend is clear: AI skills aren't the new "learn to code;" they're the next iteration of "Proficient with MS Word."

[jamesbrown]

Well that's encouraging: https://www.theregister.com/2025/06/...stings_comptia

Want a job? Just put 'AI skills' on your resume

Job postings for people with AI skills reached an all-time high last month, the training and certification company said, with some 125,000 open jobs in the tech sector mentioning the need. Those user-level skills, CompTIA noted, could involve anything from a marketer using ChatGPT to help develop new language, to a web dev using just a bit of AI to help them generate or debug some code.

In short, there's no reason to go the software engineer and LLM expertise route, because the AI jobs can be found in the "summarize my boss' emails for me" and "create some marketing campaign mockups with X, Y and Z elements that I couldn't get a human to do" spaces of expertise, not the "build me a new LLM" and "connect this chatbot to this unformatted data and work me a miracle" one. You know - "can you use Copilot?" not "build me a new copilot."

CompTIA didn't make clear what specific AI products businesses may be looking for expertise in, but the trend is clear: AI skills aren't the new "learn to code;" they're the next iteration of "Proficient with MS Word."

Well, it's both, but the former will gradually become expected/required, in the same way it's expected that you can operate a spreadsheet or word processor, i.e., it may discriminate applicants for now, but it won't as the technology progresses and becomes integral. There is plenty of work at the other end of the spectrum too, but that isn't merely (or even primarily) a coding skillset.