Deep generative models can offer the most promising developments in AI

Missed a session at the Data Summit? View here on demand.

This article is contributed by Rick Hao, lead deep tech partner at pan-European VC Speedinvest.

With an annual growth rate of 44%, the AI and machine learning market is attracting continued interest from business leaders in every industry. With some projections estimating that AI will increase the GDP of some local economies by 26% by 2030, it’s easy to see the reason for the investment and hype.

Among AI researchers and data scientists, one of the most important steps to ensure AI delivers on the promise of enhanced growth and productivity is to expand the reach and capabilities of models available for use by organizations. And at the top of the agenda is the development, training and implementation of Deep Generative Models (DGMs) – which I consider to be some of the most exciting models for use in the industry. But why?

What are DGMs?

You’ve probably already seen the results of a DGM in action – it’s basically the same type of AI models that produce deepfakes or impressionist art. DGMs have long thrilled academics and researchers in computer labs for bringing together two very important techniques that represent the confluence of deep learning and probabilistic modeling: the generative modeling paradigm and neural networks.

A generative model is one of the two main categories of AI models and, as the name suggests, it is a model that can take a dataset and generate new data points based on the input it has received so far. This is in contrast to the more commonly used – and much easier to develop – discriminatory models, which look at a data point in a data set and then label or classify it.

The “D” in “DGM” refers to the fact that, in addition to generative models, they use deep neural networks. Neural networks are computer architectures that give programs the ability to learn new patterns over time – what makes a neural network “deep” is an increased level of complexity provided by multiple hidden “layers” of inferences between the inputs. of a model and the output of a model. This depth gives deep neural networks the ability to work with extremely complex data sets with many variables at play.

In summary, this means that DGMs are models that can generate new data points based on entered data, and that can handle particularly complex data sets and topics.

The opportunities of DGMs

As mentioned above, DGMs already have some notable creative and imaginative uses, such as deepfakes or art generation. However, the potential full range of commercial and industrial applications for DGMs is huge and promises to improve many sectors.

Consider, for example, the issue of protein folding. Protein folding – discovering the 3D structure of proteins – allows us to find out which drugs and compounds interact with different types of human tissue, and how. This is essential for drug discovery and medical innovation, but it is very difficult to discover how proteins fold because scientists have to dissolve and crystallize proteins before analyzing them, meaning the entire process for a single protein takes weeks or months. can last. Traditional deep learning models are also insufficient to address the protein folding problem, as their primary focus is on classifying existing datasets rather than on being able to generate their own output.

In contrast, last year the DeepMind team’s AlphaFold model managed to reliably anticipate how proteins would fold based solely on data about their chemical makeup. By being able to generate results in hours or minutes, AlphaFold has the potential to save months of lab work and vastly accelerate research in just about any area of biology.

We also see DGMs popping up in other domains. Last month, DeepMind released AlphaCode, a code-generating AI model that successfully outperformed the average developer in trials. And the applicability of DGMs can be seen in fields as far as physics, financial modeling or logistics: by tacitly learning subtle and complex patterns that humans and other deep learning networks fail to recognize, DGMs promise to be able to create surprising and insightful generate results in just about every area.

The challenges

DGMs face some notable technical challenges, such as the difficulty of training them optimally (especially with limited data sets) and ensuring they can deliver consistently accurate results in real-world applications. This is an important driver for the need for further investment to ensure that DGMs can be deployed on a large scale in production environments and thus deliver on their economic and social promises.

Beyond the technical hurdles, however, a major challenge for DGMs is ethics and compliance. Due to their complexity, the decision-making process is very difficult for DGMs to understand or explain, especially by those who do not understand their architecture or operations. This lack of explainability can pose the risk of an AI model developing unwarranted or unethical biases without the operators’ knowledge, in turn generating outputs that are inaccurate or discriminatory.

In addition, the fact that DGMs operate at such high complexity means that there is a risk that it will be difficult to reproduce their results. This difficulty with reproducibility can make it difficult for researchers, regulators, or the general public to have confidence in a model’s results.

Ultimately, to mitigate the risks associated with explainability and reproducibility, devops teams and data scientists seeking to leverage DGMs must ensure that they use best practices in formatting their models and that they use recognized accountability tools in their implementations.

While DGMs are only just beginning to enter production environments on a large scale, they represent some of the most promising developments in the AI world. By looking at some of the most subtle and fundamental patterns in society and nature, these models will eventually prove transformative in just about every industry. And despite the challenges of ensuring compliance and transparency, there is every reason to be optimistic and excited about the future DGMs that promise technology, our economy and society at large.

Rick Hao is lead deep tech partner at pan-European VC Speedinvest.

DataDecision makers

Welcome to the VentureBeat Community!

DataDecisionMakers is where experts, including the technical people who do data work, can share data-related insights and innovation.

If you want to read about the latest ideas and up-to-date information, best practices and the future of data and data technology, join us at DataDecisionMakers.

You might even consider contributing an article yourself!