Google TurboQuant tech could save us from the RAMpocalypse
Google may have saved everyone from the RAMpocalypse
Finally, we have some good news from the AI space. Google has announced new tech that has sent the stock prices of memory companies lower. Why? Google’s new “TurboQuant” tech promises to reduce AI’s memory usage by 6x. This tech could cause the AI industry’s demand for memory to plummet, alleviating the world’s memory shortage.
If Google’s “TurboQuant” tech delivers on its promised improvements, AI hardware will be able to do much more work with much less memory. AI memory footprint can decrease by 6x, while inferencing performance can increase by as much as 8x. Google claims that this is without a loss in AI model accuracy. If this is true, the AI industry could become much less memory-hungry. However, it is worth noting that big tech could also use this advantage to construct more advanced AI models with larger datasets. In other words, AI’s memory demands aren’t suddenly going to decrease by 6x. Regardless, this change is a good thing for everyone.
How does TurboQuant work?
Google’s TurboQuant technology “optimally addresses the challenge of memory overhead in vector quantisation” and “achieves a high reduction in model size with zero accuracy loss.” In other words, it acts as a vector compression technique that doesn’t result in accuracy losses like other methods.
Instead of using traditional vectors, Google convers inputs into a “shorthand” for storage and processing. This lowers memory use and can accelerate AI throughput (as smaller data can be faster to work with). Using this tech, Google can accelerate the runtimes of AI models and lower their memory utilisation. This can make future AI servers much less memory-hungry and much less expensive.
TurboQuant combines PolarQuant, which simplifies data coordinates to lower memory overhead, and Quantised Johnson-Lindenstrauss (QJL), which acts as a 1-bit mathematical error-checker that enables a high level of accuracy. Together, this tech is TurboQuant.
Will TurboQuant end the RAMpocalypse
In effect, Google’s tech should allow AI companies to do more with less. This means that AI datacenters should be less memory-hungry, which could alleviate the world’s memory shortage.
The memory shortage, known by many as the “RAMpocalypse”, has caused the price of consumer technology to skyrocket. Everything that uses DRAM is now more expensive, which is bad news for consumers. Now that AI companies have a way to make their technology more memory-efficient, there should be more DRAM available for non-AI use. However, given that there isn’t currently enough memory to meet demand, this technology will only have a limited impact. AI’s reduced requirements could be used to build larger, more memory-intensive AI models. Furthermore, it will take a while for this technology to be widely adopted.
What TurboQuant proves is that the AI market’s insane demand for memory isn’t guaranteed. It also means the AI industry’s huge DRAM stockpiles could be pushed much further than expected. This softens AI’s expected future demand for DRAM, which is good news for consumers. However, this doesn’t mean that the RAMpocalypse is over.
The AI market is entering tough times
With the Iran War causing global fuel/material shortages and rises in electricity costs, the AI market is in for a challenging time. In recent weeks, OpenAI has already closed its SORA AI video generator and abandoned video generation tools entirely. OpenAI has also dramatically cut its planned AI infrastructure spend, and it remains unclear whether it can raise the funding to meet those commitments.
Will the “AI bubble” burst? Who knows. One thing is for certain: many companies have spent a lot of money on memory they might have no use for once their data centres are built. If TurboQuant works as intended, AI companies will need to find a way to effectively use this freed-up memory. If not, they have spent a tonne of money on memory that won’t be effectively utilised.
You can join the discussion on Google’s TurboQuant technology on the OC3D Forums.
