Tech leaders are eager to reap the benefits of generative AI, but are they really production ready and able to be used in a secure way yet? According to Randall Degges, head of developer relations and community at Snyk, no.
He joined us on the most recent episode of our weekly podcast, Get With IT, to talk about the security issues these tools have. He also shared the results of the company’s recent AI Readiness report.
Here is an abridged and edited version of that conversation:
Dave Rubinstein, editor-in-chief of ITOps Times: How does the C-suite feel about using AI coding tools versus the AppSec practitioners?
RD: We recently conducted this AI Readiness study at Snyk specifically to get more in-depth information on how different people in these organizations perceive AI readiness. And what we found was really interesting. So first of all, we surveyed like 400 different organizations. And these are people who are not Snyk customers at all, they’re just random companies that have more than 1000 employees who responded to our survey request. And we interviewed four different personas in this report: there’s developers, application security teams, chief information security officers — so people responsible for holding the line on security — and then finally, chief technology officers.
And with that being said, I think the single most interesting take away from the report is that, in general, C-suite executives dramatically underestimate the risk of using AI tools. They also dramatically overestimate their organization’s readiness to start leveraging these things in a meaningful way. And I think that’s really interesting because basically application security teams are saying, we’re not ready to use these things securely, there’s a lot of gaps. Developers are saying, we don’t even have a lot of the policies in place and things like that to properly leverage these tools. And the C-suite executives are like, everything’s going great, we’re scaling up, AI is saving us a ton of money, we’re more productive than ever. And so that’s where the biggest disconnect lies today.
DR: Sure, because I think they have to report to either their shareholders or whoever is involved in all this and say “look, we’re on the AI train, we’re moving forward, we’re modern, we’re keeping pace with the rest of the industry,” when they’re clearly — in many cases — not prepared to do that.
RD: Exactly. And I mean, I totally get it, right? From a leadership perspective, you want to motivate your teams, you want to motivate investors, you want people to be excited. However, I think the reality is that the technology is still pretty young, and there’s a lot of barriers to using it in production. We found that C-suite execs were five times more likely to rate AI coding tools as not risky at all, as compared to application security engineers, which is a very big difference.
DR: Well, they’re seeing that AI in many cases is just picking up code that already has vulnerabilities in it, and they’re not capable at this point — and maybe they will be in the future — but they can’t discern what is a vulnerability and what isn’t. So those are the security risks that AI coding is bringing in. And so one of the things I want to talk to you about is, so who’s reviewing the code? If artificial intelligence is writing the code, are the developers now simply peer reviewers who are checking things to make sure these vulnerabilities don’t get released? And is that a “higher level function” than just writing the code? That’s what people always talk about: “Well, developers won’t be replaced. They’ll just be doing higher function things.” So what are your thoughts on that?
RD: The way large language models work is they’re kind of like autocomplete. So they take what you’ve given them and they add some extra context to it and spit out what they think is the statistical next best thing. And that’s how they work. That’s how they generate all this information. When you’re talking about using AI to help solve coding problems, the same thing is true. And the reason I think that’s important to know is because let’s say you go into ChatGPT and you type a prompt that says something along the lines of “please create a Python web application that allows users to store notes securely in their browser. This application should use a SQLite database. It should be perfectly secure and be ready for production use.”
Now, based on that prompt, you would expect, because I explicitly stated in there, it has to be perfectly secure, it has to be ready for production use, you would expect that it’s going to generate something pretty secure, right? Well, you’d probably be shocked if you took the output that it gave you, and dumped that into a security tool to analyze your code, because you’ll still find a bunch of security issues there.
And this is something I actually do regularly when I’m doing demos for engineers and other technical practitioners. I’ll literally ask them to help me think of the most secure possible prompt you could get to help output some secure code. And then every single time I copy and paste that back into an analysis tool, you can see that there’s a lot of problems.
Why is that? Well, one reason is because when large language models are trained, they’re slurping in code examples from all over the internet, like Stack Overflow, open source projects on GitHub, white papers, books, etc. And not all code samples are created equal.
Additionally, security is one of those things where context really matters. If you take an example of a code block that looks totally innocuous on Stack Overflow, and you copy and paste it into your codebase. Maybe it’s something where you want to open this file and read the contents, something really, really basic. Large language models will show you that example, and it looks great theoretically, however, if you put that into the same context of a web application, unless you’re specifically restricting where the files can come from, where the path is, what the names are, and all these other things, you might end up with path traversal vulnerabilities.
And so, context is extremely difficult to figure out in security, even without LLMs. And so when you put LLMs into the mix, it’s like security vulnerabilities on steroids.
You may also like…
Balancing AI’s role in sustainability: A dual-edged sword
Securing client-facing apps in a hostile, risk-filled 21st century
