AI-assisted coding: Experiments with GPT-4

04/25/2023
by   Russell A. Poldrack, et al.
5

Artificial intelligence (AI) tools based on large language models have acheived human-level performance on some computer programming tasks. We report several experiments using GPT-4 to generate computer code. These experiments demonstrate that AI code generation using the current generation of tools, while powerful, requires substantial human validation to ensure accurate performance. We also demonstrate that GPT-4 refactoring of existing code can significantly improve that code along several established metrics for code quality, and we show that GPT-4 can generate tests with substantial coverage, but that many of the tests fail when applied to the associated code. These findings suggest that while AI coding tools are very powerful, they still require humans in the loop to ensure validity and accuracy of the results.

READ FULL TEXT

page 2

page 4

page 5

research
04/19/2023

How Secure is Code Generated by ChatGPT?

In recent years, large language models have been responsible for great a...
research
07/10/2023

Unmasking the giant: A comprehensive evaluation of ChatGPT's proficiency in coding algorithms and data structures

The transformative influence of Large Language Models (LLMs) is profound...
research
10/18/2020

The Convergence of AI code and Cortical Functioning – a Commentary

Neural nets, one of the oldest architectures for AI programming, are loo...
research
05/17/2023

Scratch Copilot Evaluation: Assessing AI-Assisted Creative Coding for Families

How can AI enhance creative coding experiences for families? This study ...
research
03/21/2022

A Policy Driven AI-Assisted PoW Framework

Proof of Work (PoW) based cyberdefense systems require incoming network ...
research
05/20/2023

CodeCompose: A Large-Scale Industrial Deployment of AI-assisted Code Authoring

The rise of large language models (LLMs) has unlocked various applicatio...

Please sign up or login with your details

Forgot password? Click here to reset