New AI Agent Learns to Use CAD to Create 3D Objects From Sketches

Researchers at MIT have unveiled a groundbreaking new artificial intelligence agent capable of using professional Computer-Aided Design (CAD) software just like a human engineer. The new system can take a simple hand-drawn sketch and automatically generate a fully functional, editable 3D model by executing the necessary clicks, mouse drags, and menu selections within the software.

This development, announced by an MIT engineering team, represents a major leap forward in design automation. Unlike previous "text-to-3D" generators that produce static, uneditable blobs (meshes), this AI agent actually operates the CAD interface to build the object step-by-step.

"VideoCAD": Learning by Watching
The secret to the agent's success lies in a massive new training dataset called VideoCAD.

The researchers realized that to teach an AI how to design, it needed to watch humans do it first. They compiled a library of over 41,000 videos of real design sessions. By analyzing these recordings, the AI learned the visual language of the software—understanding that drawing a circle and then selecting the "extrude" tool turns a 2D shape into a 3D cylinder.

"Real-world CAD models are built through a sequence of specific operations," said one of the lead researchers. "Our agent learns to mimic those operations, effectively acting as a digital apprentice that watches over your shoulder and then does the work for you."

A "Co-Pilot" for Engineers
The implications for the engineering and manufacturing industries are significant. Professional CAD software (like SolidWorks, AutoCAD, or Onshape) is notoriously difficult to learn, often requiring years of training to master.

This new AI agent aims to flatten that learning curve. A user could simply sketch a bracket, a gear, or a piece of furniture on a napkin, scan it, and have the AI build the precise 3D file in minutes. Because the AI builds the model using standard CAD commands, the final file preserves the "design history," allowing human engineers to easily go back and tweak dimensions or features—something impossible with current generative 3D art tools.

The team envisions the technology evolving into a sophisticated "CAD Co-Pilot," allowing engineers to focus on high-level creativity while the AI handles the tedious, repetitive tasks of clicking and modeling.