A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages

08/08/2023
by   Alessio Buscemi, et al.
0

Large Language Models (LLMs) are advanced Artificial Intelligence (AI) systems that have undergone extensive training using large datasets in order to understand and produce language that closely resembles that of humans. These models have reached a level of proficiency where they are capable of successfully completing university exams across several disciplines and generating functional code to handle novel problems. This research investigates the coding proficiency of ChatGPT 3.5, a LLM released by OpenAI in November 2022, which has gained significant recognition for its impressive text generating and code creation capabilities. The skill of the model in creating code snippets is evaluated across 10 various programming languages and 4 different software domains. Based on the findings derived from this research, major unexpected behaviors and limitations of the model have been identified. This study aims to identify potential areas for development and examine the ramifications of automated code generation on the evolution of programming languages and on the tech industry.

READ FULL TEXT

page 7

page 8

research
11/27/2019

FSE/CACM Rebuttal^2: Correcting A Large-Scale Study of Programming Languages and Code Quality in GitHub

Ray, Devanbu and Filkov issued a rebuttal of our TOPLAS paper "On the Im...
research
05/09/2023

The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

We present The Vault, an open-source, large-scale code-text dataset desi...
research
03/21/2023

ChatGPT for Programming Numerical Methods

ChatGPT is a large language model recently released by the OpenAI compan...
research
05/25/2023

ChatGPT for PLC/DCS Control Logic Generation

Large language models (LLMs) providing generative AI have become popular...
research
01/27/2021

Evolution of artificial intelligence languages, a systematic literature review

The field of Artificial Intelligence (AI) has undoubtedly received signi...
research
09/12/2023

Unveiling the potential of large language models in generating semantic and cross-language clones

Semantic and Cross-language code clone generation may be useful for code...
research
05/16/2023

A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions

This paper evaluates the capability of two state-of-the-art artificial i...

Please sign up or login with your details

Forgot password? Click here to reset