AI Code Assistants and Cybersecurity Risk: 3 Recent Findings

By now everyone knows the statistic: according to GitHub data 9 out of 10 developers are using generative AI code assistants*. That was reported as early as June 2023 and this category is still showing explosive growth.

‍

But the cybersecurity risk looming behind those figures is getting much less attention. Here we’re going to discuss 2 vulnerabilities we have uncovered at Lasso Security, and another that’s fairly well known. These range in severity from weird to serious. We’ll also talk about what organizations can do to allow their developers to make full use of these powerful tools, while keeping the door securely shut on looming cybersecurity risks.

‍

_{* GitHub survey: 92% of developers use A.I. coding tools | Fortune}

Generative AI Code Assistants: Major Players

Software developers are under increasing pressure to scale their output. The demand is for more speed and more accuracy which is almost impossible to meet with traditional methods. AI code assistants make this possible driving the hunger for these tools and the rapid growth in the category.

‍

Currently, the market is dominated by a small number of large players.

GitHub Copilot 

GitHub Copilot, powered by OpenAI's Codex, has set a high bar for AI-driven coding assistance. Leveraging an extensive corpus of public code Copilot offers real-time code suggestions, providing contextually relevant snippets, whole functions and even documentation. Its integration into popular IDEs like Visual Studio Code amplifies its appeal, making Copilot an indispensable tool for many developers.

Google Duet AI

Duet AI has firmly placed Google on the map for code assistance. Its integration directly into Google Cloud operations and products offers developers a crucial advantage to streamline work and enhance productivity.

Amazon CodeWhisperer 

With CodeWhisperer Amazon now competes directly with GitHub Copilot by offering a similar proposition: real-time code recommendations and insights driven by machine learning. Integrated within AWS's ecosystem CodeWhisperer differentiates itself by emphasizing security and compliance features, making it particularly appealing for enterprises concerned with maintaining code quality and adhering to regulatory standards. 

Failures and Flaws in AI Code Assistants: 3 Case Studies

Now that we are past the initial excitement, and the dust is settling, more and more organizations are waking up to the reality that these tools bring their own cybersecurity risks. CISOs are increasingly aware that developers are using code assistants, even if this goes against policy, and that they need to address their growing shadow LLM.

‍

Here are 3 recent findings that highlight these risks.

Dangerous or Harmful Outputs

Here’s what happened when our team asked for some help in the kitchen:

‍

‍

Of course, AI code assistants’ main function is to perform coding-related tasks. Our query clearly falls outside that scope, this mismatch is at least partly responsible for the nonsensical output. 

‍

But it does demonstrate potential risks that developers need to be aware of. Firstly, it underscores the need to avoid overreliance on any content that an AI tool generates. If you’re using Copilot (or any other AI code assistant), you need to critically evaluate whatever it gives you. This incident also highlights the need for stringent content filtering and behavioral guidelines. If these are not present in the AI model, the results can be inappropriate, unethical, or as we see here, potentially harmful.

Reliability and Malicious Use

This one is more serious, and if the attention it garnered on social media is anything to go by, people are starting to take note.

‍

‍

‍

This incident highlights a number of cybersecurity risks and issues:

‍

Data Breach Risk

API keys are sensitive credentials that authenticate and authorize access to external services and data. If an attacker gains access to an API key they can perform actions on behalf of the actual owner of the key. These could include accessing sensitive data, or abusing the services that the API key provides access to.

‍

Training Data Privacy

The fact that the tool could offer a live API key suggests it might have been trained on codebases containing real, sensitive data. This raises concerns about the privacy and security practices surrounding the training data for AI models.

‍

An important find made by Lasso Security engineers - Github Copilot provides outdated advice

‍

Like many developers I also work with Github Copilot Chat and recently I encountered an interesting problem.

‍

Upon installing packages of an existing Node.js project, I faced a challenge and asked for assistance from Copilot. The tool recommended downgrading from the current Node.js 20, which is the latest LTS version available, to Node.js version to 16.

‍

This immediately raised a red flag, because we are very familiar with the risks of using outdated versions. An older version may be prone to security vulnerabilities that had been addressed in the newer releases (from Node.js 16 to Node.js 20). 

‍

As a result, this reliance on obsolete data could potentially leave systems vulnerable to known security threats fixed in the later Node.js versions.

‍

‍

I was surprised by GitHub Copilot Chat’s answer, and turned Gemini, entered the following query:

‍

“What are the security enhancements implemented between Node.js 16 to 20?”

‍

Here is Google's answer:

‍

Node.js 20 introduced several security fixes compared to Node.js 16. These fixes address various vulnerabilities that could potentially be exploited by malicious actors to compromise Node.js applications. Here are some of the notable security fixes included in Node.js 20:

‍

• CVE-2023-30586: This vulnerability allows loading arbitrary OpenSSL engines when the experimental permission model is enabled, potentially bypassing or disabling the permission model altogether.
  
  
• CVE-2023-30588: When an invalid public key is used to create an x509 certificate using the crypto.X509Certificate() API, a non-expected termination occurs, making it susceptible to denial-of-service (DoS) attacks.
  
  
• CVE-2023-30589: The llhttp parser in the http module in Node.js does not strictly use the CRLF sequence to delimit HTTP requests, which can lead to HTTP Request Smuggling (HRS) attacks.
  
  
• CVE-2023-30590: The generateKeys() API function returned from crypto.createDiffieHellman() only generates missing (or outdated) keys, not regenerating keys after setting a private key. This could lead to security issues if the private key is compromised.

‍

By addressing these vulnerabilities, Node.js 20 enhances the overall security posture of Node.js applications, making them less susceptible to attacks and more resilient to malicious code.

‍

Here's a table summarizing the security fixes introduced in Node.js 20:

‍

CVE ID

Description

CVE-2023-30586

Loading arbitrary OpenSSL engines bypasses permission model

CVE-2023-30588

Process interruption due to invalid Public Key information in x509 certificates

CVE-2023-30589

HTTP Request Smuggling via Empty headers separated by CR

CVE-2023-30590

DiffieHellman do not generate keys after setting a private key

‍

A less security-minded developer may have unwittingly exposed themselves to a major vulnerability by uncritically accepting Copilot’s recommendation to downgrade. In addition to overreliance, this incident highlights the need for increased awareness and training for developers. 

Securing Your Shadow LLM with Lasso Security

According to a recent survey, 70% of developers reported that AI code assistants made them more productive, and a worrying 55% of those said that they bypassed organizational security protocols to use them. This is exactly the kind of tradeoff between productivity and security that Lasso Security helps organizations to avoid.

‍

Lasso's platform is specifically designed to help users of AI code assistants and other LLM tools to proceed responsibly and securely. For developers, this translates into the freedom to leverage the very best tools available, with complete peace of mind. 

‍

Contact our team to learn more about how secure your organization’s shadow LLM and stay at the forefront of AI-supported code production. 

‍