OpenAI Gets Into the Text-to-3D Game with Point-E

Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.

📝 Editorial 

Created with: DALL-E

I would like to start today’s editorial by wishing you a very blessed holiday season. 2022 has been a difficult year in tech in general but has had many excitments in AI including the explosion of innovations in generative AI which is today’s topic.

The generative AI space continue pushing the boundaries of imagination in the deep learning space. Language and text-to-image models have been the areas to show the most progress but 3D is quickly surfacing as the next frontier. Generating 3D objects has resulted very challenging in deep learning given the lack of training datasets as well as the computational costs. Pretrained diffusion models removed some of these boundaries by being able to go from text to an image and then to the 3D object. Google recently unveiled some of their work in this area with the DreamFusion model and Stability AI has been making steady progress extending Stable Diffusion to 3D. Last week, OpenAI joined the race with the release of Point-E, a generative model that can produce 3D objects from language inputs.

Point-E takes a very unique approach to the text-to-3D problem. Instead of generating a complete 3D object, Point-E generates a discrete set of data points that represents the 3D shape which is known as point clouds. From a computational standpoint, point clouds are way easier to synthesize. Point-E is based on two fundamental submodels: a text-to-image model based on diffusion methods and an image-to-3D model that generates the point cloud. OpenAI extended this architecture by adding the capability of coloring the point cloud resembling a complete 3D object. This area still has flaws. In addition to the research paper, OpenAI release an open source version of the model and is already included in HuggingFace.

Language, images, video, 3D, the generative AI race is nothing short of fascinating. Point-E is certainly a great contribution and might be incorporated into a new version of DALL-E.

🗓 Next week in TheSequence Edge:

Edge#255: Our series about ML interpretability continues by discussing the accumulated local effects(ALE) technique. The research section looks into OpenAI’s Microscope neuron visuation technnique and we discuss the IBM AI Explainability 360 stack .

Edge#254: We deep dive into the architecture powering the famous ChatGPT.

🔎 ML Research

Point-E

OpenAI published a paper detailing Point-E, a new language-to-3D generative model —> Read more.

CALM

Google Brain published a paper detailing confident adaptive language modeling(CALM), a method for improving the efficiency of large language models at inference time —> Read more.

CoCoA-MT

Amazon Science published a paper and open source dataset that improves formality control in large language models —> Read more.

🤖 Cool AI Tech Releases

Jasper Chat

Generative AI startup Japer released Jasper Chat, a conversational interface to assist with the different business tasks automated in the platform —> Read more.

Quora Poe

Quora announced Poe, a conversational interface to interact with chatbots a la ChatGPT and receive instant answers —> Read more.

New TensorFlow Models

TensorFlow added new state-of-the-art quantized models to its Model Garden repository —> Read more.

🛠 Real World ML 

Scaling ViT

PyTorch discusses how to scale the vision transformer(ViT) model to 120 billion parameters. —> Read more.

Auto Machine Translation at Amazon

Amazon Science detailed the machine translation architecture used to translate the popular Dive Into Deep Learning textbook —> Read more.

💸 Money in AI

• Autonomous driving startup Helm.ai raised a $31 million series C.

• Reliance acquired $23.3  million of AI robotics startup Exyn.

• Digital photography startup Imagen AI raised $30 million for product and M&A expansion.

• AI pharma startup Quris raised a $9 million seed round.

• Business communication startups Diapad announced its AI labs initiative with a $50 million investment.