As part of a technical experiment, Nvidia demonstrated an impressive result: the accuracy of the Cosmos 3 Nano model on a four-option multiple-choice test increased from 54.41% to 93.35% in less than 24 hours. A key feature was that all stages of fine-tuning were performed by the Codex AI agent, operating according to predefined instructions within the TAO tool ecosystem.

The experiment was conducted on the specialized Woven Traffic Safety dataset from Woven by Toyota, which contains video recordings of road situations and four-option multiple-choice questions. Over 8,000 examples were used for training and validation.

Without additional adaptation, the Cosmos 3 Nano model demonstrated only 54.41% correct answers. The developers instructed the Codex agent to evaluate baseline performance and perform fine-tuning using the Low-Rank Adaptation (LoRA) method. The agent independently selected the appropriate Cosmos-reason instruction, analyzed the dataset annotations, and identified the missing frame rate parameter. After correcting the configuration, it loaded the model weights, prepared the settings, and launched the training container.

https://youtu.be/9AQkVbx3fKA?si=6y1p3qZHRgeqzuEq

The LoRA method, unlike full fine-tuning, does not modify all model parameters but instead trains small additional adapters. According to Nvidia's calculations, this approach required approximately seven times fewer GPU hours than fully tuning Cosmos 3 Nano. One LoRA run on eight NVIDIA A100 accelerators with 80 GB of memory took about 30 minutes, increasing accuracy to 87.14%.

In the second stage, the developers launched TAO AutoML for automatic hyperparameter tuning: learning rate, batch size, and LoRA parameters. The system conducted 43 parallel trials, and the best result with Bayesian optimization reached 93.35%. This stage took 19.5 hours on several A100 nodes in Oracle Cloud Infrastructure.

It is important to emphasize that the Codex agent did not simply analyze results — it selected the workflow, checked data, corrected configurations, launched containers, and compared metrics. Although its autonomy was limited by pre-prepared instructions, this level of automation opens new horizons for scaling the fine-tuning of AI models.

However, it is worth noting that the 93.35% figure reflects accuracy on the validation portion of a single research dataset. It is not a measure of autonomous driving safety and does not confirm the model's ability to make real-time decisions. This is merely a demonstration of the effectiveness of an automated approach to fine-tuning.

Earlier, researchers from Nvidia, Carnegie Mellon University, and the University of California, Berkeley presented the ENPIRE framework, which allows AI agents for programming to improve robot control policies on real hardware.

Analytical Commentary: This experiment is an important step toward democratizing the fine-tuning of large models. Automating processes that previously required deep knowledge and manual labor from engineers could significantly accelerate the adoption of AI in specialized fields such as autonomous driving. However, the key challenge remains the generalizability of such results to real, uncontrolled scenarios.