Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

In AI, is bigger always better?

As generative AI models grow larger and more powerful, some scientists advocate for leaner, more energy-efficient systems.
  1. Anil Ananthaswamy

    1. Anil Ananthaswamy is a freelance science writer based in California.

    View author publications

    You can also search for this author in PubMed  Google Scholar

Cartoon of a scientist in front of a large computer with three consecutive displays getting larger each with more complex data.

Illustration by Fabio Buonocore

Artificial-intelligence systems that can churn out fluent text, such as OpenAI’s ChatGPT, are the newest darlings of the technology industry. But when faced with mathematical queries that require reasoning to answer, these large language models (LLMs) often stumble. Take, for instance, this algebra problem:

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 615, 202-205 (2023)

doi: https://doi.org/10.1038/d41586-023-00641-w

Updates & Corrections

  • Clarification 10 March 2023: This article has been modified to clarify that Mark Sellke, a statistician at Stanford University in California, was the only other person involved in introducing the robustness law.

References

  1. Frieder, S. et al. Preprint at https://arxiv.org/abs/2301.13867 (2023).

  2. Lewkowycz, A. et al. Preprint at https://arxiv.org/abs/2206.14858 (2022).

  3. Chowdhery, A. et al. Preprint at https://arxiv.org/abs/2204.02311 (2022).

  4. Kaplan, J. et al. Preprint at https://arxiv.org/abs/2001.08361 (2020).

  5. Hoffmann, J. et al. Preprint at https://arxiv.org/abs/2203.15556 (2022).

  6. Caballero, E. et al. Preprint at https://arxiv.org/abs/2210.14891 (2022).

  7. Bubeck, S. et al. Preprint at https://arxiv.org/abs/2105.12806 (2021).

  8. Le Scao, T. et al. Preprint at https://arxiv.org/abs/2211.05100 (2022).

  9. Luccioni, A. S. & Hernandez-Garcia, A. Preprint at https://arxiv.org/abs/2302.08476 (2023).

  10. Luccioni, A. S., Viguier, S. & Ligozat, A.-L. Preprint at https://arxiv.org/abs/2211.02001 (2022).

  11. Yin, B. et al. Nature Mach. Intell. 3, 905–913 (2021).

    Article  Google Scholar 

  12. Borgeaud, S. et al. Preprint at https://arxiv.org/abs/2112.04426 (2021).

  13. Du, N. et al. Preprint at https://arxiv.org/abs/2112.06905 (2021).

Download references

Subjects

Latest on:

Nature Careers

Jobs

  • Postdoctoral Position

    We are seeking highly motivated and skilled candidates for postdoctoral fellow positions

    Boston, Massachusetts (US)

    Boston Children's Hospital (BCH)

  • Qiushi Chair Professor

    Distinguished scholars with notable achievements and extensive international influence.

    Hangzhou, Zhejiang, China

    Zhejiang University

  • ZJU 100 Young Professor

    Promising young scholars who can independently establish and develop a research direction.

    Hangzhou, Zhejiang, China

    Zhejiang University

  • Head of the Thrust of Robotics and Autonomous Systems

    Reporting to the Dean of Systems Hub, the Head of ROAS is an executive assuming overall responsibility for the academic, student, human resources...

    Guangzhou, Guangdong, China

    The Hong Kong University of Science and Technology (Guangzhou)

  • Head of Biology, Bio-island

    Head of Biology to lead the discovery biology group.

    Guangzhou, Guangdong, China

    BeiGene Ltd.

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links