Demand for AI "Surging"

[dsc]

Generative AI or chatbotting is a tiny subset of AI, though, it's at the party trick end of the spectrum. There are, OTOH, may applications in science, engineering and medicine where AI techniques are proving to be absolutely transformative. Not that they are a recent invention, but the practicality of actually using them has surged, as well as efforts to improve the underlying models.

Sure, but I have a feeling that a lot of "hype" is driven by the chatbotting functionality currently. And fair enough it's the most "relatable" feature as I doubt a lot of people are interested how AI is used more heavily in engineering / science, but as you say it's more of a party trick, hence the silly AI features pushed to phones where you can generate bloody pokemons riding on dragons and chuckle.

[jamesbrown]

Sure, but I have a feeling that a lot of "hype" is driven by the chatbotting functionality currently. And fair enough it's the most "relatable" feature as I doubt a lot of people are interested how AI is used more heavily in engineering / science, but as you say it's more of a party trick, hence the silly AI features pushed to phones where you can generate bloody pokemons riding on dragons and chuckle.

Transformative technologies are very often overhyped when they first take off. The basic ideas have been around for decades, but recent progress in computing, the availability of massive datasets and, ultimately, interest and investment has led to an explosion of applications. I agree that the chatbots are massively overhyped, largely because they are very relatable to a non tech audience and seem like “magic”. OTOH, in many areas of science, engineering and medicine, AI is leading to a step change in the skill of mathematical models, pattern recognition etc. and that is not being overhyped so much.

[willendure]

OTOH, in many areas of science, engineering and medicine, AI is leading to a step change in the skill of mathematical models, pattern recognition etc. and that is not being overhyped so much.

I think the masses have no idea what that all means and its not something they can relate to. The chat thing and image generators are more relatable to ordinary folk, so have gained a lot of hype regardless of whether or not they are more important innovations in the long run.

Generative is a genuine innovation in AI and uses different models to previous AIs. Earlier AIs were more about prediction and pattern recognition. Generative is about immitation and creating novel instances of the data that satisfy some set of desirable properties - such as looking "human". But also generative could create new drugs with the desirable property of being non-toxic and effective. So even beneath the hype, it is an important innovation with serious applications and one that we are really just at the start of exploring.

Generative also seems to frequently incorporate a cross-over with natural language. So image generators actually start with a randomized field of pixels, then apply successive passes that massage that towards an image that looks like one out of their training set. Then that entire process is trained using image labels so that the randomized field essentially becomes a big bag of switches that control the generated image and the switches get set through natural language - that is what the so-called CLIP does in stable diffusion. The idea is now that the human-in-the-loop can supply an idea in natural language, and the AI can generate it.

This kind of cross-entropy between more natural forms of control and generative AI is also a fascinating innovation. For example, I saw a paper on a system nVidia built for natural control of a computer game fighting figure. The human just uses a joystick or mouse but the generative AI system translates that into moving the figure around its environment and controlling its fighting moves, all trained on motion capture models to look natural.

I think this is potentially an important pathway to general AI. Especially if you consider the swiss army knife theory of the brain. The idea is that over evolution brains first evolved specialist "swiss army knife" skills - like the ability to judge distance and throw a stone. Then over time those skills get crossed over with everything else the brain does and become a fully integrated skill - and one that can for example be described/requested/acted upon with language. The saying "people who live in glass houses shouldn't throw stones", shows how stone throwing can completely cross over onto a conceptual level and become a word play.

All this cross-entropy generative stuff is starting to help us stitch together disparate domains of AI and get our first idea of what an AI brain might look like and might do.

[jamesbrown]

It isn’t that AI content creation aka generation has no useful scientific application, on the contrary, but generative AI more easily captures the public imagination when compared to predictive AI and hence receives disproportionate attention. However, predictive AI has far more expansive and immediate (but often mundane) applications across a broad set of scientific disciplines because prediction is so central to modern scientific endeavour. In the long run, it’s impossible to say what branch of AI will have a greater impact, and it’s not even a very interesting question, but predictive AIs very easily plug into existing problems and frameworks, such as forecasting high dimensional/non-linear systems, and those impacts/benefits are already here and are really quite impressive, delivering improved skill for much lower cost and hence reaching a very broad audience (think laptop rather than HPC).

[willendure]

I don't even really think you should think of predictive and generative as 2 separate branches. The new techniques invented for generative will filter back in to the other. Generative has predictive aspects also - literally all LLMs do is predict the next word anyway.

[jamesbrown]

I don't even really think you should think of predictive and generative as 2 separate branches. The new techniques invented for generative will filter back in to the other. Generative has predictive aspects also - literally all LLMs do is predict the next word anyway.

It's you that made this distinction above, "earlier AIs were more about prediction and pattern recognition"

The underlying mathematical techniques are largely the same. For example, deep learning techniques are used for chatbot content creation and weather forecasting, alike. However, prediction and content creation are subtly different problems. For example, prediction is largely about estimating conditional probability distributions, i.e., "given X=x, Y=y, and Z=z, what is the set of possible outcomes of Q and their associated probabilities?", whereas generation is more about understanding all possible combinations, aka joint probability, and sampling novel outcomes from joint distributions.

Anyway, my point remains, namely that chatbots are at the party trick end of the AI spectrum, not in terms of the methods used, but in the way they are being applied. Chatbots are an application of a technology. Prediction remains central to scientific applications of AI where there's typically a set of (e.g., observed) initial conditions that constrain the problem.

[willendure]

It's you that made this distinction above, "earlier AIs were more about prediction and pattern recognition"

Yes, but I see them as a forward evolution of the earlier models, not as 2 parallel branches.

[willendure]

Look at something like Cerebras which sold its systems to G42 healthcare - they want AI to predict how drugs will behave and also to invent or help invent novel drugs - thats generative right there. So while chatbots are at the party trick end, similar generative AI technology is being used for serious applications. Plenty things being developed out there that are way too complex and involving hard science and engineering for most people to appreciate.

[jamesbrown]

Yes, but I see them as a forward evolution of the earlier models, not as 2 parallel branches.

Who said anything about parallel branches?

[jamesbrown]

Look at something like Cerebras which sold its systems to G42 healthcare - they want AI to predict how drugs will behave and also to invent or help invent novel drugs - thats generative right there. So while chatbots are at the party trick end, similar generative AI technology is being used for serious applications. Plenty things being developed out there that are way too complex and involving hard science and engineering for most people to appreciate.

I think I said the same thing above . I agree, the underlying models span many worthwhile applications, but chatbots capture the public consciousness because "oh, look at the clever robot!"