Potential risks and mitigation strategies

June 22, 2023hacker news

Are generative AI apps keeping you awake? You’re not alone and you’re not wrong. According to Astrix Security Research Group, the medium-sized organization already has an average of 54 Generative AI integrations integrated into its core systems such as Slack, GitHub, and Google Workspace, and this number is expected to continue to grow. To understand potential risks and how to minimize them, read on.

Book a Generative-AI Discovery session with an Astrix Security expert (free – no strings attached – agentless and frictionless)

“Dear ChatGPT, please review and optimize your source code.”

“Jasper.ai, please generate an email summary of all net new customers for this quarter.”

“Hey Otter.ai, summarize the Zoom board meeting.”

In this era of financial turmoil, businesses and employees alike are automating work processes by connecting third-party apps to core business systems such as Google workspace, Slack, and GitHub via API keys, OAuth tokens, and services. We are always looking for tools to improve our efficiency and productivity. account etc. The rise of Generative-AI apps and his GPT services has exacerbated this problem, allowing employees in all departments to add the latest and greatest AI apps to their productivity arsenal without the security team’s knowledge. adding rapidly.

From engineering apps like code review and optimization to marketing, design and sales apps like content and video creation, image creation and email automation apps. With ChatGPT becoming the fastest growing app of all time, with 1506% more AI-powered app downloads than last year, the security risks of connecting these unvetted apps to business core systems are worse. In particular, we have already had sleepless nights due to security reasons. leaders.

Connectivity between your organization’s apps

Risks of Gen-AI apps

AI-based apps present two main concerns for security leaders.

1. Data sharing via apps such as ChatGPT: The power of AI is in its data, but this very strength can be its weakness if mismanaged. An employee may unintentionally share sensitive, business-critical information, including a customer’s PII, code, and other intellectual property. Such leaks can expose organizations to data breaches, competitive disadvantages, and compliance violations. And this is no fable – ask Samsung.

Samsung and ChatGPT Leaks – Beware

reported by Samsung Exfiltration of 3 different confidential information by 3 employees using ChatGPT for productivity purposes. One of her employees shares sensitive source code to check for errors, another shares code for code optimization, and a third to turn it into meeting notes for presentations. I have shared the recording of the meeting. All this information was used by ChatGPT to train her AI model and is now available for sharing on the web.

2. Unverified Generative AI App: Not all generative AI apps come from verified sources. A recent Astrix study found that employees are increasingly connecting these AI-based apps (which typically have highly privileged access) to their core systems like GitHub and Salesforce, raising significant security concerns. was found to occur.

Huge number of generative AI apps

Book a Generative-AI Discovery session with an Astrix Security expert (free – no strings attached – agentless and frictionless)

Real-life example of dangerous Gen-AI integration:

In the image below, you can see the Astrix platform detailing the risky Gen-AI integration that connects to an organization’s Google Workspace environment.

This integration, the Google Workspace integration “GPT For Gmail”, was developed by an untrusted developer and gives your organization’s Gmail account high privileges.

The scope of permissions granted to the integration includes “mail.all”. This allows third-party apps to read, compose, send and delete emails. This is a very sensitive privilege.

Information about untrusted integration suppliers:

how Astrix Helps minimize AI risks

To safely navigate the exciting and complex landscape of AI, security teams need visibility into the third-party services their employees connect to, control permissions, and properly assess potential security risks. requires robust non-human identity management. With Astrix you can:

Astrix Connectivity Map

• Get a complete inventory of all AI tools your employees use and access core systems, and understand the risks associated with them.
• Eliminate security bottlenecks with automated security guardrails. Understand the business value of each non-human connection, including usage level (frequency, last maintenance, usage), connection owner, internal personnel using the integration, and marketplace information.
• Reduce attack surface – Give least privileged access to all AI-based non-human identities accessing core systems, removing unused connections and untrusted app vendors.
• Detect anomalous activity and remediate risks. Astrix analyzes and detects malicious behavior such as token theft, internal app abuse, and untrusted vendors in real-time through IP, user-agent, and access data anomalies.
• Faster remediation: Astrix reduces the burden on security teams with automated remediation workflows and directs end users to individually resolve security issues.